UNIT 2:

Semiotic and Multimodal

Approaches
SEMIOTIC

Semiotics (also called semiotic

studies) is the study of sign
process (semiosis). It includes
the study of signs and sign
processes, indication,
designation, likeness, analogy,
allegory, metonymy, metaphor,
symbolism, signification, and
communication.
SEMIOTIC

The semiotic tradition explores

the study of signs and symbols
as a significant part of
communications. Different from
linguistics, semiotics also studies
non-linguistic sign systems.
SEMIOTIC

In media, semiotics are

important as they help us to
read text analytically and think
of a deeper meaning behind
signs and codes as we
understand signs by
differentiating them to other
things.
SEMIOTIC

Semiotics can be applied to

anything which can be seen
as signifying something – in
other words, to everything
which has meaning within a
culture.
 SEMIOTIC
Signs can be used to help individuals
understand their environment. For
instance, public signs have been
designed to enable literate and illiterate
individuals to easily identify places and
things, such as rest rooms and
telephones. Symbols can show actual
objects or they can represent a concept.
Visual symbols move from the concrete
object to the abstract idea. There are
many different types of signs and
symbols that communicate visual
information.
 METAPHOR
Orientational metaphors primarily
relating to spatial organization
(up/down, in/out, front/back, on/off,
near/far, deep/shallow and
central/peripheral);
Ontological metaphors which
associate activities, emotions and
ideas with entities and substances
(most obviously, metaphors
involving personification);
 METAPHOR

Structural metaphors: overarching

metaphors (building on the other
two types) which allow us to
structure one concept in terms of
another (e.g. rational argument is
war or time is a resource).
What are metaphors and
symbols?
"A metaphor is a statement that
means something different, or
more, than its literal meaning. A
symbol has complex meaning; it
has not only literal, but also
additional meaning(s) beyond
the literal.
Why is metaphor
different from a symbol?
They're related, but generally a
metaphor is used to draw a
comparison between two
distinct objects, whereas a
symbol is used a stand-in for a
much more complex, and
generally more abstract, idea.
Are metaphors and
symbols different?
A metaphor and a symbol can
be included in many forms of art
and literature. The difference
between a metaphor and a
symbol is that a metaphor is a
direct substitution of one idea
or object for another, while a
symbol is used to imply
something else.
What are the examples of
metaphor?
The snow is a white blanket.
He is a shining star.
Her long hair was a flowing golden
river.
Tom's eyes were ice as he stared at her.
The children were flowers grown in
concrete gardens.
Kisses are the flowers of affection.
The falling snowflakes are dancers.
The calm lake was a mirror.
Does a metaphor contain
symbolism?
A metaphor is a figure of speech
that uses symbolism. It
compares two things that aren't
similar and proves they actually
have something in common. In a
metaphor, there is an additional
meaning to a word. This makes
it an example of symbolism.
Examples of Common
Metaphors
Anger bottled up inside
He was a Lion on the battlefield
Apple of my eye
Home was prison
Homework is a breeze
Sea of knowledge
Battle of egos
 CODES

Is a set of conventions or sub-codes

currently in used to communicate
meaning. The most common is one's
spoken language, but the term can
also be used to refer to any narrative
form : consider the color scheme of
an image (e.g red, for danger), or the
rules of a board game (e.g the
military signifiers in chess).
CODES
Code, in communications, an
unvarying rule for replacing a
piece of information such as a
letter, word, or phrase with an
arbitrarily selected equivalent.
The term has been frequently
misapplied and used as a
synonym for cipher, which is a
method for transforming a
message according to a rule to
conceal its meaning.
CODES
Code, in communications, an
unvarying rule for replacing a
piece of information such as a
letter, word, or phrase with an
arbitrarily selected equivalent.
The term has been frequently
misapplied and used as a
synonym for cipher, which is a
method for transforming a
message according to a rule to
conceal its meaning.
CODES, KEYS AND
CIPHERS
1. The Caesar shift.
2. Alberti's disk.
3. The Vigenère square.
4. The Shugborough inscription.
5. The Voynich manuscript.
6. Hieroglyphs.
7. The Enigma machine.
8. Kryptos.
The Caesar shift
Named after Julius Caesar, who used it
to encode his military messages, the
Caesar shift is as simple as a cipher gets.
The Caesar shift/cipher is one of the
earliest known and simplest ciphers. It is a
type of substitution cipher in which each
letter in the plaintext is 'shifted' a certain
number of places down the alphabet. To
pass an encrypted message from one
person to another, it is first necessary that
both parties have the 'key' for the cipher,
so that the sender may encrypt it and the
receiver may decrypt it.
For example:
The text we will encrypt is the
plaintext, 'defend the east wall of the
castle', with a shift (key) of 1. So the
message or text that will be sent to
cipher or for the receiver to decrypt
is, ‘ efgfoe uif fbtu xbmm pg uif
dbtumf’. It is easy to see how each
character in the plaintext is shifted
up the alphabet. Decryption is just as
easy, by using an offset of -1.
Alberti's disk
Alberti cipher disk, sometimes called
a formula disk is a cipher (known as the 
Alberti cipher) was described by 
Leon Batista Alberti in his treatise "De
Cifris" (1467), it is a very early example of 
polyalphabetic substitution. It is a tool to
decrypt/encrypt Alberti. The Alberti cipher
is a polyalphabetic cipher system that uses
two mobile concentric disks which can
rotate. With Alberti cipher, encryption uses
a disk with two alphabets, one fixed
(stabilis) one moving (mobilis). By rotating
a disk, it shifts an alphabet to the next
letter.
Example
Encrypt DCODE with the parameters:
initial shift: 1, periodic increment: 2, period:
3.
Alphabets are such
as ABCDEFGHIJKLMNOPQRSTUVWXYZ is
aligned with bcdefghijklmnopqrstuvwxyza.
The period begins, D is coded
by e, C by d, O by p, the period (length 3)
ends, the disk is rotated by 2 letters.
Alphabets are now aligned like this:
ABCDEFGHIJKLMNOPQRSTUVWXYZ with
defghijklmnopqrstuvwxyzabc the new
period begins, etc. The encrypted message
is edpgh.
Example
The Vigenère square
The Vigenère cipher is an example
of a polyalphabetic substitution
cipher. A polyalphabetic substitution
cipher is similar to a monoalphabetic
substitution except that the cipher
alphabet is changed periodically
while enciphering the message. This
makes the cipher less vulnerable to
cryptanalysis using letter frequencies.
The Vigenère square
Blaise de Vigenère developed
what is now called the Vigenère
cipher in 1585. He used a table
known as the Vigenère square, to
encipher messages. This page
discusses two different versions
of the Vigenère cipher, the
autokey method and the
keyword method.
Example
The Shugborough inscription

The Shugborough Inscription is a

sequence of letters – O U O S V A V V,
between the letters D M – carved on
the 18th-century Shepherd's
Monument in the grounds of
Shugborough Hall in Staffordshire,
England, below a mirror image of
Nicolas Poussin's painting the
Shepherds of Arcadia. It has never
been satisfactorily explained, and has
been called one of the world's top
uncracked ciphertexts.
The Shugborough inscription

There are numerous hypotheses that

have been carefully constructed to explain
the Shugborough inscriptions. Three
theories, both acrostic (matching the first
letters to a word) and non-acrostic, are
perhaps most popular:
1. The inscriptions are an ancient love
letter which glorifies the Holy Grail.
2. They are George Anson’s declaration of
mourning for his deceased wife.
3. The monument is a huge prank, acting
as a red herring to tantalize whoever tries
to find meaning in the inscriptions.
The Shugborough inscription
The Voynich manuscript
 
Many call the fifteenth-century
codex, commonly known as the
“Voynich Manuscript,” the world’s
most mysterious book. Written in an
unknown script by an unknown
author, the manuscript has no clearer
purpose now than when it was
rediscovered in 1912 by rare books
dealer Wilfrid Voynich.
The Voynich manuscript
 
The manuscript appears and
disappears throughout history, from
the library of the Holy Roman
Emperor Rudolf II to a secret sale of
books in 1903 by the Society of Jesus
in Rome. The book’s language has
eluded decipherment, and its
elaborate illustrations remain as
baffling as they are beautiful.
The Voynich manuscript
 After looking at the so-called code for a
while, Gibbs realized he was seeing a common
form of medieval Latin abbreviations, often used
in medical treatises about herbs. "From the
herbarium incorporated into the Voynich
manuscript, a standard pattern of abbreviations
and ligatures emerged from each plant entry," he
wrote. "The abbreviations correspond to the
standard pattern of words used in the Herbarium
Apuleius Platonicus – aq = aqua (water), dq =
decoque / decoctio (decoction), con = confundo
(mix), ris = radacis / radix (root), s aiij = seminis
ana iij (3 grains each), etc." So this wasn't a code
at all; it was just shorthand. The text would have
been very familiar to anyone at the time who was
interested in medicine.
The Voynich manuscript
Hieroglyphs
 The word hieroglyph literally
means "sacred carvings". The
Egyptians first used hieroglyphs
 exclusively for inscriptions carved or
painted on temple walls. This form of
pictorial writing was also used on 
tombs, sheets of papyrus, wooden
boards covered with a stucco wash,
potsherds and fragments of
limestone.
Hieroglyphs
Hieroglyphs are written in
columns or in horizontal lines. They
are generally read from right to left
and from top to bottom. Sometimes,
the script is read from left to right.
The reader can determine the
orientation by looking at the animal
and human figures -- they face
towards the beginning of the text.
For example: if a figure faces right,
the text should be read from right to
left.
Hieroglyphs
Words and names written in
hieroglyphs were believed to have 
magical powers. For this reason,
funeral texts and the names of the
deceased were written on coffins and
tomb walls. This meant that the gods
would hear the prayers and the
individuals would be protected from
harm. A name written in hieroglyphs
embodied a person's identity. If it
was obliterated, the person's identity
was lost, along with his or her means
to continue living in the afterworld.
Hieroglyphs
The Enigma machine
An Enigma machine is a famous 
encryption machine used by the
Germans during WWII to transmit
coded messages. An Enigma machine
allows for billions and billions of
ways to encode
The Enigma machine a message, making it
incredibly difficult for other nations
to crack German codes during the
war — for a time the code seemed
unbreakable.
The Enigma machine
An Enigma machine is made up of
several parts including a keyboard, a
lamp board, rotors, and internal
electronic circuitry. Some machines,
such as the ones used by the military,
have additional features such as a
plug board.
Encoded messages would be a
particular scramble of letters on a
given day that would translate to a
comprehendible sentence when
unscrambled.
The Enigma machine
Kryptos
Kryptos is a sculpture by the American
 artist Jim Sanborn located on the grounds of
the Central Intelligence Agency (CIA) in 
Langley, Virginia. Since its dedication on
November 3, 1990, there has been much
speculation about the meaning of the four 
encrypted messages it bears. Of these four
messages, the first three have been solved,
while the fourth message remains one of the
most famous unsolved codes in the world.
The sculpture continues to be of interest to 
cryptanalysts, both amateur and professional,
who are attempting to decipher the fourth
passage. The artist has so far given four clues
to this passage.
Kryptos
WHAT ARE THE TYPES OF
CODES IN MEDIA?
 MULTIMODAL ANALYSIS
Multimodality is a theory of
communication and socialsemiotics.
Multimodality describes
communication practices in terms of
the textual, aural, linguistic, spatial,
and visual resources - or modes -
used to compose messages. Where
media are concerned, multimodality
is the use of several modes (media)
to create a single artifact.
 MULTIMODAL ANALYSIS
Multimodality - “the combination of
different semiotic modes”
*For example language and music in
a communicative artifact or event.
Multimodal discourse -to analyze
how several or all of the different
semiotic modes intertwined together
to create a unified text or
communicative event.
 WHAT IS A MULTIMODAL
TEXT?
A text may be defined as multimodal
when it combines two or more
semiotic system.

Linguistic, Visual, Audio, Gestural,

and Spatial.
 SEMIOTIC SYSTEM
Linguistic: vocabulary, structure, grammar
of oral/written language
Visual: color, vectors and viewpoint in still
and moving images
Audio: volume, pitch and rhythm of music
and sound effects
Gestural: movement, facial expression and
body language
Spatial: proximity, direction, position of
layout, organization of objects in space.
Multimodal text can be printed, physical or
digital
LEVELS OF MEANING

1. Preiconographic- the denotative

level, focused on the interpretation
of the elementary or natural
meaning, which involves in the
identification of visual materials such
as the objects known from daily
experience.
LEVELS OF MEANING

2. Iconographic-the connotative
level, mainly focusing on the
interpretation of secondary or
traditional meanings, and during the
interpretative process, the viewers
learn to move beyond the natural or
surface meaning of the image to
consider their knowledge and
experiences.
LEVELS OF MEANING

3. Iconological- mainly focused on

the interpretation of ideological
meanings of an image structured in
particular social, cultural political,
and historical contexts
INTERPRETATIVE STRATEGIES

Meta-interpretative strategies- the

readers must have the awareness of
multimodal meaning constructing.
They must bear in mind that visual
images are no longer ornamental, or
subsidiary, and are increasingly
employed to make sense of the
world, often overshadow the once
dominant mode of written language.
INTERPRETATIVE STRATEGIES
Perceptual Strategy - what the reader
has noticed: the visual and design
elements like pattern, shape, line,
color, topography, and presented in
the multimodal texts. The readers
may create an inventory of visual
representation, and devise a
vocabulary for enumerating and
depicting these various elements,
which is an important aspect of
visual meaning and interpretation.
INTERPRETATIVE STRATEGIES
Analytical Strategy -After noticing what
is in the visual images and the design
elements of a multimodal texts, what
these objects and elements mean is an
important and key aspect of the
following comprehension process.
e.g. How various objects are organized
and located in the visual image and how
they interact and coordinate with other
elements, relative to the objects and
participants in the image, is the viewer
position very close or far away and so
on
INTERPRETATIVE STRATEGIES
Socio-cultural Strategy- “pictures are
not merely analogues to visual
perception but symbolic artifacts
constructed from the conventions of
a particular culture”.
EXAMPLE OF SEMIOTIC ANALYSIS
EXAMPLE OF SEMIOTIC ANALYSIS
EXAMPLE OF SEMIOTIC ANALYSIS
EXAMPLE OF SEMIOTIC ANALYSIS
EXAMPLE OF SEMIOTIC ANALYSIS
EXAMPLE OF SEMIOTIC ANALYSIS
EXAMPLE OF SEMIOTIC ANALYSIS
EXAMPLE OF SEMIOTIC ANALYSIS
EXAMPLE OF MULTIMODAL ANALYSIS
EXAMPLE OF MULTIMODAL ANALYSIS
EXAMPLE OF MULTIMODAL ANALYSIS
EXAMPLE OF MULTIMODAL ANALYSIS
 MARXISM
Marxism is a theory and method of
working-class self-emancipation. As a
theory, it relies on a method of
socioeconomic analysis that views
class relations and social conflict
using a materialist interpretation of
historical development and takes a
dialectical view of social
transformation. It originates from the
works of 19th-century German
philosophers Karl Marx and Friedrich
Engels.
 MARXISM
Marxism uses a methodology, now
known as historical materialism, to
analyze and critique the development
of class society and especially of
capitalism as well as the role of class
struggles in systemic economic,
social, and political change.
 MARXISM
According to Marxist theory, in
capitalist societies, class conflict arises
due to contradictions between the
material interests of the oppressed and
exploited proletariat—a class of wage
labourers employed to produce goods
and services—and the bourgeoisie—the
ruling class that owns the means of
production and extracts its wealth
through appropriation of the surplus
product produced by the proletariat in
the form of profit.
 PSYCHOANALYSIS
The basic assumption is that the
personal unconscious is a potent part
— probably the more active part —
of the normal human psyche.
Reliable communication between the
conscious and unconscious parts of
the psyche is necessary for
wholeness.
 PSYCHOANALYSIS
Also crucial is the belief that dreams
show ideas, beliefs, and feelings that
individuals are not readily aware of
but need to be, and that such
material is expressed in a
personalized vocabulary of visual
metaphors. Things 'known but
unknown' are in the unconscious,
and dreams are one of the main
vehicles for the unconscious to
express them.
 ARCHETYPE
In Jung's psychological framework,
archetypes are innate, universal
prototypes for ideas and may be used
to interpret observations. A group of
memories and interpretations
associated with an archetype is a
complex, e.g. a mother complex
associated with the mother archetype.
Archetypes are collective as well as
individual, and can grow on their own
and present themselves in a variety of
creative ways.
 ARCHETYPE
Jungian archetypes are defined
as images and themes that derive
from the collective unconscious,
as proposed by Carl Jung.

Archetypes have universal

meanings across cultures and
may show up in dreams,
literature, art or religion.
 SELF-REALIZATION AND
NEUROTICISM
An innate need for self-
realization leads people to
explore and integrate these
disowned parts of themselves.
This natural process is called
individuation, or the process of
becoming an individual.
 SELF-REALIZATION AND
NEUROTICISM
According to Jung, self-realization is
attained through individuation. His is
an adult psychology, divided into
two distinct tiers. In the first half of
our lives, we separate from
humanity. We attempt to create our
own identities (“I”, “myself”). This is
why there is such a need for young
men to be destructive, and can be
expressed as animosity from teens
directed at their parents.
 SELF-REALIZATION AND
NEUROTICISM
Jung also said we have a sort of
"second puberty" that occurs
between ages 35 and 40: outlook
shifts from emphasis on
materialism, sexuality, and
having children to concerns
about community and
spirituality.
 SELF-REALIZATION AND
NEUROTICISM
In the second half of our lives,
humans reunite with the human race.
They become part of the collective
once again. This is when adults start
to contribute to humanity (volunteer
time, build, garden, create art, etc.)
rather than destroy. They are also
more likely to pay attention to their
unconscious and conscious feelings.
 SELF-REALIZATION AND
NEUROTICISM
Young men rarely say "I feel angry"
or "I feel sad." This is because they
have not yet rejoined the human
collective experience, commonly
reestablished in their older, wiser
years, according to Jung. A common
theme is for young rebels to "search"
for their true selves and realize that a
contribution to humanity is
essentially a necessity for a whole
self.
 SELF-REALIZATION AND
NEUROTICISM
Jung proposes that the ultimate goal
of the collective unconscious and
self-realization is to pull us to the
highest experience. This, of course, is
spiritual. If a person does not
proceed toward self-knowledge,
neurotic symptoms may arise.
Symptoms are widely defined,
including, for instance, phobias,
psychosis, and depression.
 SHADOW

The shadow is an unconscious

complex defined as the repressed,
suppressed or disowned qualities of
the conscious self. According to Jung,
the human being deals with the
reality of the shadow in four ways:
denial, projection, integration and/or
transmutation.
 SHADOW

According to analytical psychology, a

person's shadow may have both
constructive and destructive aspects.
In its more destructive aspects, the
shadow can represent those things
people do not accept about
themselves. For instance, the shadow
of someone who identifies as being
kind may be harsh or unkind.
 SHADOW

Conversely, the shadow of a

person who perceives himself to
be brutal may be gentle. In its
more constructive aspects, a
person's shadow may represent
hidden positive qualities. This
has been referred to as the "gold
in the shadow".
 ANIMA AND ANIMUS
Jung identified the anima as being the
unconscious feminine component of men
and the animus as the unconscious
masculine component in women. However,
this is rarely taken as a literal definition:
many modern-day Jungian practitioners
believe that every person has both an
anima and an animus. Jung stated that the
anima and animus act as guides to the
unconscious unified Self, and that forming
an awareness and a connection with the
anima or animus is one of the most
difficult and rewarding steps in
psychological growth.
 ANIMA AND ANIMUS
Often, when people ignore the anima or
animus complexes, the anima or animus
vies for attention by projecting itself on
others. This explains, according to Jung,
why we are sometimes immediately
attracted to certain strangers: we see our
anima or animus in them. Love at first
sight is an example of anima and animus
projection. Moreover, people who strongly
identify with their gender role (e.g. a man
who acts aggressively and never cries)
have not actively recognized or engaged
their anima or animus.
 ANIMA AND ANIMUS
Jung attributes human rational
thought to be the male nature, while
the irrational aspect is considered to
be natural female (rational being
defined as involving judgment,
irrational being defined as involving
perceptions). Consequently, irrational
moods are the progenies of the male
anima shadow and irrational
opinions of the female animus
shadow.
 PSYCHOLOGICAL TYPES
Analytical psychology distinguishes
several psychological types or
temperaments.

• Extravert

• Introvert
 PSYCHOLOGICAL TYPES
According to Jung, the
psyche is an apparatus for
adaptation and
orientation, and consists of
a number of different
psychic functions.
Among these he distinguishes
four basic functions:
• Sensation – Perception by means of the
sense organs
• Intuition – Perceiving in unconscious way
or perception of unconscious contents
• Thinking – Function of intellectual
cognition; the forming of logical
conclusions
• Feeling – Function of subjective
estimation
Thinking and feeling functions are rational,
while the sensation and intuition functions
are irrational.
 COMPLEXES
Jung seemed to see complexes as
quite autonomous parts of
psychological life. It is almost as if
Jung were describing separate
personalities within what is
considered a single individual, but to
equate Jung's use of complexes with
something along the lines of multiple
personality disorder would be a step
out of bounds.
Ten of the most common
psychological complexes:
• Oedipus/Electra Complex
• Madonna/Whore
• God Complex
• Persecution Complex
• Martyr Complex
• Inferiority Complex
• Superiority Complex
• Guilt Complex
• Don Juan Complex
• Hero Complex
 SOCIOLOGY
Sociology is the study of society,
patterns of social relationships, social
interaction and culture of everyday
life using the principles of
psychology neuroscience and
network science. It is a social science
that uses various methods of
empirical investigation and critical
analysis to develop a body of
knowledge about social order,
acceptance, and change or social
evolution.
 BASIC
What’s in the image? or, What do we
see?

What’s surprising?

What’s communicated in the image

and text?
 INFORMATION
Where has the information in the image
come from?
What information has been included and
what information has been left out?
What proportion of the image could be
inaccurate?
What information presented is
factual/manipulated/framed?
What is the relationship between the
image and any text?
What impact does the size of images
within the picture have?
 INFORMATION
What people are depicted in the image
(even if there are now actual people in the
image, whose culture or experiences are
being shown?
Who created the image and for what
purpose?
Who is the intended audience for the
image?
Whose point of view does the image take?