Resume

The Role of Gestures and Facial Signs in Second Language

Listening Comprehension

By: Zainul Arifin

Background of the study

This study was conducted by Ayano Sueyoshi and Debra M. Hardison of

Michigan State University. The reseachers investigated gestures and facial signs

contribution to ESL learners’ listening comprehension from a videotaped lectured by a

native English speaker. Based on the other reseacher who believes that communications

used gestures, body movements, head nods, hand gestures, facial expressions, eye gaze,

posture, and interpersonal distance (Kellerman, 1992) are very helpful in listeners’

comprehension. And so the other researchers who are very sure that facial speech has a

power to send the message from the speaker to the listener.

This article shows the findings of a study designed to:

a. Evaluate gestures and facial signs contribution to listening comprehension by low-

intermediate and advance learners of ESL.

b. Study the ESL learners’s opinions about visual signs and face to face communication.

The fact that each culture has its own gesture, makes the study has more

challenges. Non-verbal communication send the message from the speaker to the listener.

Korean’s facial expressions in showing a gentleness are different with Western’s. For the

new Asian learners who study abroad like USA, they will find a little difficulty in the

expressions. In USA, eye contact is common when we do communication, but it may be

1
rude in Asia such as in Japan. The USA students think that smiling face is smarter that

neutral face, but Japanese students don’t.

There are some terms in this study those the researchers used.

1. Hand gestures shows an interactive element during communication.

2. Iconic gestures are assosiated with meaning and used while speaker is describing

specific things.

3. Beat gestures are often used when the speaker controls the speed of speech. It doesn’t

related with image of things.

4. Metaphoric gestures like iconic gestures but the gestures describe abstract ideas.

5. Representational gestures (iconics and metaphoric) are used more when the speaker

can be seen.

6. Diectics are pointing gestures for spesific objects, abstract object in non spesific time

or location.

Some studies of native speakers show that gestures help to send messages to both

speaker and listener in communication. Gestures help speakers to remember the words

faster. Gestures also effective to send the message to the listener when the surrounding is

noisy enough to reduce the speaker’s voice. Gestures are also can be functioned as an

indicator of the language development. The lower language proficiency learners mainly

use diectic such as waving and clapping. As their language proficiency develops they

begin to use iconics and beats.

The lip movements in each language have diffecent perceptions. Therefore, the

ESL learners are helped to increase the identification accuracy from auditoy-visual input

better than auditory-only in sound training like in /r/ and /l/. The participants of the

2
research who had lived in USA for 7 weeks as the time study began did more accurate in

audio-video pretest than in audio-only pretest. Lip movements are very important as

source of facial sign to speech. A survey by Hattori (1987) revealed that Japanese students

who lived in the United States for more than 2 years reported that they looked more at the

faces of the speaker since it allow them to use visual information to help comprehension.

Introducing gestures in language learning also improves the social pragmatic

competence of L2 learners. The process of listening becomes more active when

accompanied by visual motions, and the nonverbal aspect of speech is an integral part of

the whole communication process.

Some studies which was focusing on gesture use by L2 learners have found that

the learners who learn language in naturalistic setting have greater advantage than L2

learners who learn language as a foreign language. Those learners also produce more

gesture when they speak in their L2 than speak in their L1.

A study in Primary school by Cabrera and Martines (2001) shows a possitive

effect of gestures. The students comprehended more in storytelling with more gestures. It

indicated that gestures have possitive contribution in language comprehension.

Research Questions and Objectives

The first question was equipped with the comprehension task, however the second

and the third questions was equipped with questionare.

1. Does the use of visual signs such as gestures and lip movements improve ESL

students’ listening comprehension?

Hypothesis: AV-gesture-face group in the present study would show better listening

comprehension scores for the higher and lower proficiency levels because of the

3
 presence of both facial and gestural signs, followed by the AV-face groups, and then

the A-only. This was based on previous research demonstrating the contribution of

facial cues to perceptual accuracy and word identification (Hardison, 1999, 2003,

2005b, 2005c) and studies suggesting that gestures in speech contain meaningful

information that help comprehension of content.

2. Does proficiency level affect the learners’ preference for visual signs in

communication and their choice of activities for the development of listening and

speaking skills and vocabulary?

3. Does proficiency level affect the perception of gestures in general and participants’

own gesture use with L1 and L2 speech?

Hypothesis: Learners in both proficiency levels would feel convinience with the

presence of additional visual signs to help communication and skill development, but

the higher proficiency learners might consider facial signs more informative and report

more interested due to their language experience.

METHOD

The participants consisted of 42 ESL learners. They were from 18 to 27 years old.

There were 29 females and 13 males. Most of them were Korean (35), Chinese (1),

Japan(3), Thailand (1), Italian (1), and unknown (1). The learners were either in Intensive

English Program (IEP) or English for Academic Purposes Program (EAP) at Midwestern

University in USA. Then from the IEP there were 21 lower proficiency level learners and

4 higher proficency learners. From the EAP got 21 higher proficiency level learners and

4 lower foficiency learners. Both participants levels of proficiency were randomly placed

4
to one of the stimulus conditions gestures: AV-gestures-face, AV-face, and A-only. There

were six groups in this research design, each of group consisted of 7 participants. The

lenght of residence (LOR) of students in the USA are 6 months or less.

Materials

The material for each group was a video-recorded of a female graduate teaching

assistant whose English as her native language. The video-recorded was about “Ceramics

for Beginners”. This topic was chosen to avoid any influence of the learners’ prior

experience (based on questionaire results) and to ensure many gestures use. The

researchers were helped by some people to make sure the quality of the video-recorded.

The video-recording process was done in two sessions. The duration is 20 minutes for

each video. The video recorder units were chosen to produce the best quality of the

recording. One showed the lecturer’s upper body in order to capture gesture use, and the

other was focused on her face (shoulders and above).

The speakers were asked to speak as if they spoke in front of many people. The

video editor used in this research was iMovie, a video editor for Mac computer. The

recording were edited into five clips to reduce the memory used. A multiple choice

comprehension task was used in listening task (A-only). All of the questions in listening

task were designed can be answered without looking at visual signs. The participants were

given the same questions with four multiple-choice questions, each with four options. The

questionaire asked about participants’ background; such as their native language, LOR in

USA, experience with ceramics and use of English.

Procedure

5
 The AV-gesture-face and AV-face groups were presented via computer and

projected onto a screen at the front of the classroom. The audio was played through the

speaker. For A-only group, only the audio track was presented. Each participant was

given a response booklet with written instruction in front of it. The participants were

asked to complete the questionaire which was included in the booklet. They could ask if

they didn’t understand the question. The duration of each session was 30 minutes.

Result and Discussion

Gesture Types

The order of the gestures those used by the participants were as follows: Beats (38%),

Iconics (31%), Metaphorics (23%), Diectics (8%)

The following words and phrases were accompanied by gestures.

- Triangle and circle, Hollowed out (form a sign similar to ‘OK’), Slide the clay over

to ... (two hands forming a circle as if holding an object and moving it to the left)

The following word was accompanied by metaphoric.

- Nothing (moving the hand from side to side)

The following words and phrases were accompanied by diectics.

- Pot (palm down) or clay (palm up), Function rather than form.

The beats were noted almost in the lecture but happened most in the important

informations.

Listening Comprehension Task

6
It was conducted to answer the research question: Does the use of visual signs such as

gestures and lip movements improve ESL students’ listening comprehension? The

findings confirmed the hypothesis that the more visual information available to the

participants, the better the comprehension. The table shows that The higher proficiency

learners’ highest score was AV-face stimulus, followed by AV-gesture-face and then A-

only. However, the tabulated data for the lower proficiency learners shows that the AV-

gesture-face condition produced the best scores.

Questionaire

The results of the questionaire here are presented in order. Number 1 about “Use of

English” is the first rank is the learners used English in class, followed by they used

English to do homework or talk to friends. In number 2 about “Improve listening” the

learners practice to improve their listening skill by talking to Americans, followed by

watching TV. Number 3 shows that the learners “improve speaking” by talking to

Americans, followed by watching TV. Number 4 in “build vocabulary”, the learners build

vocabulary by reading , then followed by doing their homework.

General Discussion

The result that the AV-face and AV-gesture-face groups achieve better on

comprehension task than those in the A-only groups in higher proficiency groups shows

the need of the class of more gesture in listening task. But, The lower proficiency groups

achieved better in AV-gesture-face conditions. So, these different achievements need to

be discussed. Maybe, the lower proficiency level learners achived better in AV-gesture-

face condition because they think seeing the speaker’s gesture was more informative than

7
their facial sign only. The questionaire reported that gesture and lip-movement

contributed to the learners’ comprehension.

There were significant differences in result on the comparison questionaire

between lower and higher proficency levels. The higher proficency thought that AV-face

condition helped their comprehension alot. It may be as the result of their longer

experience in interaction with Americans. But the lower proficiency level groups thought

say that they were so confused in AV-face condition.

However, the difference was not significant for both proficiency levels. This was

because of the relation between gesture and semantic compared with the relation between

lip movements with phonology.

Conclusions

This research that showed the effectiveness of visual signs in L2 listening

comprehension recomended the need of visual sign in the second language acquisition

learning, but it needs further investigation, because the study only limited the

investigation in the ceramics topic. It may have different result if it was done with

different topic, condition and place. And the differrent participants from other country

may lead to different results. Linguistic and cultural experience, proficiency level too, are

factors which affecting the value of the information of a language.

It also needs further investigation to study how to control speech and gesture with

visual signs, as it shows positive result to help the English second language learners

comphehension and raise the knowledge of English.

8
Comments

The Role of Gestures and Facial Signs in Second Language

Listening Comprehension

The way the speakers say is influenced by their cultural background. The study

explains that people from Asia and Western show different gestures and facial signs to

send the message to the listener. That is why, when people with different culture come to

different country with different culture, sometimes they can get a missperception about

what someone says. This study reveals some findings about what gestures and visual signs

can influence by comparing the results of experiment groups that watch a video-taped

with different gestures and visual signs and other groups with less gestures and audio only

recording.

This research analized some gestures of hand gestures, iconic, beat, metaphoric,

representational, and diectics. These gestures are believed affected the comprehension of

the listening activities. The more gestures speakers produce, the more listening

comprehensions the listeners get. But the findings do not make me surprised because I

think it is not an uncommon study. Everybody can guess what the results are. So maybe

the researchers should find new interesting variables those are new in language study.

This kind of research also needs more effort to conduct, because we should prepare more

advanced technology to facilitate it. But, if we want to conduct the same study with

different equipments, we may get different results due to lower quality of the equipments.

The results may have different findings if it is done in an environment whose

English is not the native language. The English environment could support the

9
enhanchement of the language proficiency. The need of speaking in L2 could give extra

motivation to always add someones’ language proficiency.

In this research, the ceramics making process is the only topic. Because of the

limited topic, the validity could be questioned. It could have less correct result since it

doesn’t have more topics to compare.

But the way the reseachers presented the research in simple language makes me

enjoy reading the article. The article rarely uses too uncommon words, it means that the

researchers want the readers to understand the content of the article easily. I believe that

they realize that many of the readers are not from countries whose English is their native

language. This journal gives me another knowledge of a research, how the research

conducted and what can be influenced by the research. The reseachers presented the

article systematically and easy to follow.

The researchers also try to present the research as good as they can by providing

the best equipments of advanced technology. Considering the quality of the equipments,

I am sure that the quality of the audio video and audio are very good. It proves how serious

the researchers conducted the research. They did not want to get the data from easy way,

so they hired some people and equipments to support their research. Eventhough it was a

quite simple research but it is a very good one.

This journal gives me another knowledge of a research, how the research

conducted and what can be influenced by the research. The reseachers presented the

article systematically and easy to follow. This journal also gives me some ideas of doing

the similar research with different topic and conditions. The topic will be adjusted with

the situations in the research conducted.

10