Understanding Narrative of Audiovisual without and with Sound

Understanding Narrative of Audiovisual without and with Sound Ashay Sahasrabuddhe, Research Scholar, Dept. Of Communication & Journalism, [email protected] Abstract: Films are audiovisual in nature. Both film’s sounds as well as visuals are source of information required to understand the film narrative. When persons with hearing impairment view film content, they cannot access the sound of the film. Hence it is possible that persons with hearing disabilities may not be able to understand the film narrative at par with the people who don’t have any such disability. This experimental research study is aimed at finding the difference between the understanding of the narrative of AV clip when it is watched with sound and without sound. As a pre-test a trailer of the film without any sound was shown to people without audiovisual disabilities. And then the same trailer will be shown with sound to the same sample audience. Responses about viewers understanding of the narrative of the trailer will be collected in the form of questionnaire. This research study is a small part of more extensive research aimed at finding how persons with disabilities understand the film narrative and what methods can be used to improve accessibility of audiovisual media so that PWDs understand film narrative at par with persons without disabilities. Key words: Hearing impairment, audiovisuals, film trailer, narrative, cognition, cinema Hence Introduction: Entertainment industry in India is predicted to grow at 14.2% every year as per recent KPMG report (FICCI; KPMG, 2014). In 2018, the entertainment industry size is expected to be INR 1786 billion. Television, film, animation and visual effects, gaming industries are predominantly audiovisual in nature. Even when persons with visual impairment (VI) or hearing impairment (HI) view any audiovisual content, they can access only one aspect of the audiovisual content i.e. audio or visuals respectively. Thus the understanding of the narrative of that particular audiovisual content among persons with VI or HI is based only audio or video. some amount of content produced for digital advertising is in audiovisual format. This gives the idea about how much Understanding & cognition: audiovisual content gets produced every It is believed that human beings year and is available for audiences to view. perceive and understand things by means Among all various forms of audiovisual of sense. The information about the world media content, feature films are the most in transmitted through senses into human widely accessed and popular form of mind (Bickhard). Hence it can be argued entertainment across all sections of the that senses act as media to provide society in India. Various regional film information to mind for processing to industries along with Hindi film industry understand it. In case of persons with produce hundreds of films. According to hearing Film Federation of India, in year 2013-14 communicated through ears in the form of Central Board for Film Certification various (CBFC) certified almost 2000 films. Eventually mind cannot process that According to Census 2011 more than 50.7 impairment sounds the cannot information reach mind. information because it doesn’t receive it. lacs people living in India have hearing Understanding begins with the impairment. At the same time, more than process of cognition. In simple words, 50.3 lacs people living in India have visual cognition impairment. Hence audiovisual disability ‘knowledge’. Knowledge can be thought and inaccessibility of media content of prevents them from enjoying feature films manipulation and assimilation of raw at par with persons without audiovisual input, perceived via our sense of sight, disability. hearing, taste, touch and smell (Kellet, as means memories ‘to formed know’ from or the 2008). The concept of cognition has been an understanding of that world, because used frequently in the study of psychology sound is an important factor in the physical as well as study of philosophy. world. To understand a language, and for The process of cognition begins with the activation of senses in order to perceive the events happening in the physical world. Human senses – the sight, sound and touch – observe the events and development of vocabulary/speech, one must be able to hear the sounds. Children with deafness cannot hear sound; hence they cannot learn the language spontaneously. information is passed on to motor, visual Hence to access all the information and auditory systems. Whenever the communicated by audiovisual content already perceived the viewers should be able to view the content reaction to them in driven not only by the as well as hear it. But with persons with input but also from memory. The cognitive audiovisual disability, they cannot access ability of human being enables him/her to information remember, understand, apply, analyze, visuals or the sound. Thus their knowledge evaluate and create. about the audiovisual’s narrative will not events repeat, Multi-faceted process of cognition is the coordinated performance of the several language and non-language tasks. communicated by either be at par with knowledge among persons without disablity about the narrative of same audiovisual. These tasks include perception, memory, concept formation, mental imagery, learning language, problem solving, Misc-en-scene: academic achievement and control of Misc-en-scene is a french term which everyday life (Sternberg, 1989). means ‘placed in the scene’ or ‘onstage’. The cognitive development of a person has been explained by the Theory of Mind (Peterson & Slaughter, 2006). Cognitive Development is the ability to understand oneself and other people which develop as children mature. Hearing ability Elements like actors, lights, set, costumes, make up and other features of the image collectively form misc-en-scene. each of these elements play important role in meaning making (Corrigan & White, 2008). helps to link the physical world with the a. Settings and sets – These are the sounds of that physical world, leading to most fundamental elements of the misc-en-scene. Settings and sets indicates the relationship between inform viewers about the place characters in the misc-en-scene. where actions and events of the d. Costumes and make-up – In films, film occur. Importantaly, settings costumes and make-up help to and sets are used to create sccenic create realism by reproducing the realism and scenic atmoshpere for clothing and facial featues of the making the picture truthful to the people living that space and time. extent of making it appear realistic. e. Lighting – It is misc-en-scene’s the Scenic realism plays crucial role in most subtle and yet important creative cinematic realism while element. watching a film. Physical, cultural perspectives of the viewer about a and historical accuracies of the person ot a thing by evoking backgrounds, objects, and other feelings. figures enhance the realism of the film. b. Props – Props are objects that are part of the sets or tools used by the actors. Very often props are used to express emotions or thoughts of the Lighting affects All these factors contribute together as misc-en-scene to make the film look realistic. It help to viewers to draw the boundaries of time and space to set the film in. characters. It gives them significant importance in the scheme of the film. Props are either Soundtrack: used according to their common fuctions Film is an audio-visual medium. For the or used for giving a different, narrative of the film visuals are considered unexpected meaning to them. to be primary. It does not mean audio has c. Actors – Actors’ facial expressions, no role to play. Very often it is assumed gestures and movements help to that visuals are important than audio or bring the characters to life and sound in audio-visual medium. Visuals communicate important nuances of communicate the message more efficiently the characters. Facial expressoins with viewers. In reality, soundtrack has and the equal importance along with visuals in any the film. Fundamentally, soundtrack consists gestures emotions accentuate through which character is going. Particularly the character movements or blocking of three types. other 1. Dialogues or speech situation 2. Music and songs 3. Background characters on present screen. in the Types of subjective sound are Inner Voice, score and sound effects Spoken Writing, Remembered Sound, Imagined Sound and Distorted Sound. Raskin c. Personal Narration – It is a narrative (1992), depending on the how the sound voice that can be heard by viewers. has been used in the film, film sound is Viewer will come to know that the classified as follows: character is in different time/space A. Diegetic Sound – If a sound of piano setting than that of the setting of According to Richard playing is heard and a character is playing the piano on screen, it is a Diegetic Sound. Here the sound narration, if the character is shown on screen. It is considered to be actual sound. source is established a part of the story B. Non-diegetic Sound – It is a sound space. Also, a sound playing in the which is not part of the scene and characters mind is also considered as hence cannot be heard by any of the Diegetic Sound. This sound is either characters in it. Also the source of character’s such sounds does not belong to scene own voice or other character’s voice that a character hears as she or he is remembering or thinking. be heard by anyone present in that situation of the film according to the premises of the scene. Some types are On-Screen Sound, a. Impersonal Narration – It is a spoken narration from a voice which does not a. Actual Sound – It is a sound that can – created. Sound, Transitional Off-Screen Sound and Differentially Audible Sound. b. Subjective Sound – It is a sound which occurs inside the mind of a particular character in the situation. This sound usually cannot be heard by belong to any character established in the scene or to any narrator. b. Non-Diegetic Music – If music is being played in the background which can be heard only by audiences and not by the characters in the scene, it is Non-Diegetic music. Here the source of the sound i.e. music is not part of the scene created on screen. All these types of sound are important to event by using a particular sound the film and its effect on viewers. Every phrases repeatatively. This type of sound in a film communicates with viewer technique which associates a sound in to give some or other information which memory adds up to overall experience of the film. symbolizes the character and the past – known as leitmotiv, or future existence of the event. German composer Richard Wagner Importance of sound in film: developed and refined the concept of Each of these fundamental types of sound performs certain functions with respect to the film. This study discusses the importance of music, sound effects and dialogues with respect to audio-visual content. leitmotifs. Whenever a particular thematic sound is heard by viewer, viewers expects to see corresponding character or event on screens. Roger Hickman states ‘one must observe a clear and consistent relationship between a musical idea and its a. Musical sounds – Musical sounds onscreen counterpart’(2006). create mood of a film (Lipscomb & Tolchinsky). When viewers listen to such sounds which correspond to various emotions and moods, they can feel the emotion that is being felt by the characters on screen. Music can convey scale of the film which can range from a story about some person to an epic drama. Music can match and enhance the percieved energy that drives the story. Music also has the power to convey the message that the director of the film wants to convey to its viewers. Another study argues that According to Rasa Theory from Bharata’s Natyashastra, various rasas pre-exist in every human being, i.e. Sthayee Bhaava. An external component, here music, stimulates the Sthayee Bhava in human. Rasa Thoery refers to this stimulus as Vibhaava. Vibhavaa brings life to the Sthayee Bhaava that exists in viewer. Hence viewer can feel that emotion, i.e. viewer can have the Anubhaava (experience) of that particular Sthayee Bhaava. meaning of the film can be influenced b. Sound effects and ambient sound – by the kind of music used in it Sound effects and ambient sound is (Gorbman, 1987). Music as a theme also equally important for the overall can be paired with a character or an experience of the film. Diegetic sounds effects enhance the authenticity of Gorman’s model onscreen situation in films. Very often, underlines principles of invisibility and ambient sounds are used to create the inaudibility which are related to the role atmosphere of a particular location and functions of the film sound. Principle where the scene is happening. of invisibility deals with the instrument (Gorbman, 1987) c. Dialogues – Like every other sound associated with the non-diegetic sound in dialogues play very crucial role in the film. Principle of inaudibility deals films. dialogues are with the film music which is not be communicating the consciously Primarily, involved in heard. Music works as narrative of the film. Dialogues are signifier of emotion. Music provides medium to conduct a conversation referential and narrative cues as well as among the characters of the film. Apart rhythmic and formal continuity. It helps to from the conversation between two or construct formal and narrative unity. more characters, dialogues also convey much more about the characters involved. Film or any audiovisual media consist of various texts in the form of visuals and sound. The meaning of such audiovisual Dialogue illuminates character faster than content any narrative because dialogue is a meanings derived from individual texts function of character (Sartor, 2008). In which are part of that content (Baldry & some cases, dialogues also help in Thibault, 2006). Hence for a person establishing between without audiovisual disability both visuals characters. Some dialogues take the story and sound take part in meaning making of the film further. process. the relationship is a composite product of Some theorists refer to audio of film as film music which includes musical scores, ambient sound, dialogues, sound effects Accessibility of audiovisual content: and silence (Lipscomb & Tolchinsky). The Accessibility is the degree to which a functions of these elements often interact product, device, service, or environment is and overlap with each other. Music adds available to as many people as possible. the third dimension to the two-dimensional Accessibility can be viewed as the "ability film screen. to access" and benefit from some system or entity. The article 9 of United Nations Convention on Rights of Persons with Disabilities (UNCRPD), talks about accessibility. without any sound. In order to collect responses about the information that they Accessibility features like sign language interpretation, subtitling, captioning are available for making audio elements of audiovisual content accessible for persons with HI. But in India, due to lack of got by viewing this clip was collected in questionnaire form. Later the same clip was shown to same viewers with sound and same questionnaire form was given to viewers to record their responses. awareness about disability, rights of PWDs Convinient sampling was used to collect and accessibility very little content gets responses from 36 viewers. produced keeping in mind needs of PWDs. Hence all the aural clues and information communicated through sound in the film In the questionnaire respondents were asked following 6 important questions: cannot be accessed by person with hearing 1. What is the genre of the film? impairment. In such case how do persons 2. What is the period of the film? with 3. Where is the film happening? hearing impairment view and understand the audiovisual content without 4. According complainer what case is it? any access to sound? 5. According detective what case is it? This experimental research study is a comparison between understanding of 6. Is it a love story between male and female leads? narrative of audiovisual content viewed These questions were asked because without any sound and the same viewed knowing their with without understanding of the narrative considering audiovisual impairment were selected as the suspense thiller genre of the film. And respondents for this study. just visuals of the trailer may not give sound. Hence persons answers is important for conclusive answers of these questions. Methodology: For this experimental research study, 153 Observations: seconds long trailer of upcoming Hindi All 36 respondents responded correctly to feature film ‘Detective Byomkesh Bakshy’ the genre, period and place of the film was selected. As pre-test of the experiment when trailer was viewed without sound as this trailer clip was shown to viewers well as with sound. Observations for remaining questions are Chart 2.1 – Responses to trailer viewed as follows: without sound 1. Responses to question - According 28 complainer what case is it? 15 0 14 3 4 1 4 2 1 Chart 2.2 – Responses to trailer viewed with sound Chart 1.1 – Responses to trailer viewed without sound 3. Responses to question – Is it a love story between male and female leads? 23 13 14 10 0 9 3 0 Yes No Inconclusive Chart 1.2 – Responses to trailer viewed Chart 3.1 – Responses to trailer viewed with sound with sound 2. Responses to question - According 24 detective what case are it? 14 12 14 5 0 2 1 Yes No Inconclusive Chart 3.2 – Responses to trailer viewed with sound Discussions: All the 36 respondents responded correct between them suggest that it is not love story between the two (Chart 3.2). to the genre, period and place of the film When respondents viewed the trailer after watching trailer without sound as without any sound they gave mixed and well as with sound. equal responses. Some thought it is love The discussion between complainer and detective unfolds the answers to questions regarding their point of views about the case. story, some said it isn’t and some could not conclude anything from trailer (Chart 3.1). But when respondents viewed the trailer with sound none of them said it is a love story. 24 respondents said it is not a Complainer thinks it is a case of missing love story and rest could not conclude person. But detective thinks it is a case of anything. murder. Based only visuals i.e. without sound majority of respondents said according to Conclusion: complainer it is a case of murder or it is Based on comparative analysis of all the inconclusive (Chart 1.1). But when trailer questions it can be concluded that visuals was viewed with sound majority of them of the audiovisual content are not enough understood that complainer thinks it is a to case of missing person (Chart 1.2). regarding the narrative of the audiovisual. Based only visuals i.e. without sound We can conclude that visuals i.e. misc-en- majority of respondents said according to scenes of the trailer are sufficient to know detective it is a case of murder or it is about details like genre, period and place inconclusive (Chart 2.1). But when trailer of the film. But at the same time visuals was viewed with sound majority of them are insufficient to know nuances important understood that complainer thinks it is a for case of missing person (Chart 2.2). audiovisual. The visuals of the trailer do not give Hence conclusive idea about the relationship impairment to understand the narrative of between the male and female leads of the the film they must get information film (Chart 3.1). But the dialogues communicated through sound in visual know the important understanding for the persons information narrative with of hearing form. Thus audiovisual content should be made available to them in accessible formats. In further cope of study it is important to find out what a person would understand through only sound and without any visuals of audiovisual content. Kellet, G. (2008, September 17). What is Cognition? Retrieved July 22, 2013, from Lumiosity blog: http://www.lumosity.com/blog/what-iscognition/ Raskin, R. (1992, April). Dept of Aestetics and Communication. Retrieved August 8, 2013, from University of Aarhus: http://imv.au.dk/~pba/Homepagematerial/ MMproduktionmateriale/Raskin%20Soun d%20%20Paper.pdf References: Baldry, A., & Thibault, P. J. (2006). Multimodal Transcription and Text Analysis. A multimedia toolkit and coursebook. London & Oakville: Equinox. Sartor, L. A. (2008, july 10). The Importance of Great Dialogue. Retrieved july 03, 2013, from Five Scribes blog: http://fivescribes.blogspot.in/2008/07/imp ortance-of-great-dialogue.html Corrigan, T., & White, P. (2008). Film Experience: An Introduction (2nd ed.). Bedford: St. Martin's. Sternberg, R. (1989). The Triarchic Mind: A New Theory of Human Intelligence. New Tork: Penguine Books. FICCI; KPMG. (2014). The Stage Is Set Indian Media and Entertainment Industry Report. KPMG. UNCRPD. (n.d.). Accessibility. Retrieved from United Nations enable: http://www.un.org/disabilities/default.asp? id=269 Frankl, V. (1963). Man's Search for Meaning. Boston: Beacon Press. Gorbman, C. (1987). Unheard Melodies: Narrative Film Music. Bloomington, IN: Indiana University Press. Hickman, R. (2006). Reel Music: Exploring 100 Years of Film Music. New York: W W Norton & Company Inc. Williams, R. (2005). Cognitive Theory. In Handbook of Visual Communication (pp. 193-210). New Jersey: Lawrence Erlbaum Associates. Name of the author: Ashay Sahasrabuddhe E-mail id: [email protected] Contact no.: +91 9821818882 Address: 103, Surabhi soc., Green Road, Louiswadi, Thane (W) - 400604