Expanding Analytical Perspectives on Children's Picturebook Apps

Article Expanding Analytical Perspectives on Children’s Picturebook Apps Literacy Research: Theory, Method, and Practice 1-15 ª The Author(s) 2016 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/2381336916661516 lrx.sagepub.com Earl Aguilera1, Dani Kachorsky1, Elisabeth Gee1, and Frank Serafini1 Abstract Research on the nature and impact of book apps or e-reading in general is still limited and informed by diverse assumptions about the nature of these new ‘‘texts,’’ the varied forms of engagement and meaning-making associated with them, and their implications for understanding literacy and learning in the digital age. The purpose of this article is to explore the affordances and constraints inherent in an examination of children’s picturebook apps through multiple analytical frameworks—in this case drawn from social semiotics, film analysis, and game studies. After outlining these frameworks in the context of our evolving new media landscape, we move on to more detailed analyses of the children’s picturebook app The Fantastic Flying Books of Mr. Morris Lessmore from each of these perspectives. We conclude with lessons that might be learned from juxtaposing these analytical frameworks and suggest implications for literacy education, research, and practice. Keywords children’s literature, picture books, educational technology, mobile apps, multimodality 1 Arizona State University, Tempe, AZ, USA Corresponding Author: Earl Aguilera, Mary Lou Fulton Teachers College, 1050 S Forest Mall, Tempe, AZ 85281, USA. Email: [email protected] 2 Literacy Research: Theory, Method, and Practice As reading on digital reading devices begins to take up a larger share of the literacy engagement time of children, parents, and educators, along with a growing number of scholars have argued for a need to better understand the nature of e-reading and the learning and engagement potential associated with interactive book apps (Meyers, Zaminpaima, & Frederico, 2014; Sargeant, 2015). Picturebook apps have received particular attention, in part because scholarship on picturebooks draws on theoretical constructs such as multimodality, intertextuality, and visual literacy that lend themselves to the analysis of digital texts. These apps are also of interest due to their significance in the early literacy experiences of many children (Meyers et al., 2014). Research on the nature and impact of picturebook apps is still emerging and informed by diverse assumptions about the nature of these new texts, the varied forms of engagement and meaning-making associated with them, and their implications for understanding literacy and learning in a digital age. We believe that research of picturebook apps could be enhanced by juxtaposing understandings from varying fields or areas of study that address multimodality, digital media, and interactive technologies. To understand children’s engagements with digital texts, it is important to understand how they work and the semiotic potential of these multimodal ensembles. The purpose of this article is to explore the value of multiple analytical frameworks—in this case drawn from the perspectives of social semiotics, film analysis, and game studies—for understanding the meaning potentials and constraints of picturebook apps. Several interrelated questions served as the focus of this qualitative content analysis (Krippendorff, 2004). Those questions were: What can be learned about the nature of picturebook apps by expanding multimodal analytical frameworks to include other perspectives such as film analysis and game studies? What might a juxtaposition of these analytical frameworks reveal about children’s picturebook apps that single analytical approaches might not? And finally, what limitations, challenges, and open questions might such an expanded analysis have to account for in future research? We begin by defining picturebook apps and their relationship to multimodal analytical frameworks. We continue by describing three analytical frameworks, social semiotics, film studies, and video game, studies that are relevant to understanding children’s picturebook apps. After outlining these frameworks, we move on to more detailed analyses of a single children’s picturebook app entitled The Fantastic Flying Books of Mr. Morris Lessmore (Moonbot Studios, 2011), from each of these perspectives. Finally, we will revisit our central questions by highlighting insights learned from examining the app across these analytical frameworks and suggest implications for literacy education, research, and practice. Defining Children’s Picturebook Apps The contemporary picturebook remains one of the most ubiquitous multimodal ensembles in elementary classrooms. The interplay of visual images, design features, and written language requires researchers to consider analytical perspectives beyond Aguilera et al. 3 linguistic analyses to understand the semiotic potential of this format (Serafini, 2013). It is important to adopt research methods based on theories of visual grammar (Kress & van Leeuwen, 1996), visual discourse analysis (Albers, 2008), and multimodal analysis (Machin, 2007) to address the affordances and transmedial features of the various instantiations of print-based (picturebooks) and digital platforms (ebooks and apps). At its core, a picturebook app is a type of software application that consists of picturebook content in a digital form that users navigate using a touch screen interface, such as on an iPad or Kindle Fire (Serafini, Kachorsky, & Aguilera, 2015). These apps are uploaded by publishers to online application marketplaces, commonly known as ‘‘app stores,’’ or are available on individual publishers’ websites, and can then be downloaded by individual users. Unlike the prior generation of e-books, which are essentially digitized versions of traditional print books, picturebook apps are typically distinguished by interactive features that may alter the experiences of a reader in significant ways (Sargeant, 2015). For example, through the addition of sound effects, animations, or background music. Along with the traditional features of e-books, such as—digitally displayed text and images, navigational buttons, and home screens—picturebook apps offer interactional features that expand the options, potential reading paths, and experiences of the reader. Many picturebook apps also include voice over narration, sound effects, animation, and even game-like features, thus creating an experience that sometimes more resembles ‘‘play’’ than ‘‘reading,’’ in a traditional sense. Children’s Picturebook Apps and Multimodality We ground our analyses in the notion that children’s picturebook apps, much like their printed predecessors, are multimodal ensembles that draw upon different modalities (visual image, written language, animation, music) in the rendering of a narrative (Duncum, 2004; Jewitt, 2009; Kress, 2003; Serafini, 2013). Such an approach takes as a central understanding that communication and representation are about more than language (Jewitt, 2009). We argue that the key assumptions underpinning the field of multimodal analysis align well with the digital and multimodal features associated with children’s picturebook apps (Jewitt, 2009; Norris, 2004; Painter, Martin, & Unsworth, 2013). Children’s picturebooks typically draw on visual images, design elements, and written language as modes of meaning-making. While children’s picturebook apps expand the types of modalities presented by including animation, sound effects, music, and embodied user interactions to orchestrate complex ensembles of meaning in addition to text and image (Kress, 2009; Rowsell, McLean, & Hamilton, 2012; Serafini, 2013). Multimodality also assumes that various semiotic resources—the ‘‘building blocks’’ for meaning-making—are socially shaped over time by their materiality, meaning potentials, and the contexts in which they are used (van Leeuwen, 2005). The transition of picturebooks from printed codes into a digital, interactive 4 Literacy Research: Theory, Method, and Practice medium reflects the shaping of semiotic resources to accommodate emerging ways of constructing meaning in the digital age. Finally, multimodality assumes that people orchestrate meaning through the ways they select and configure modes, thus highlighting the importance of both the interaction between modes and the motivations and interests of people in a specific social context (Jewitt, 2009). Children’s picturebook apps, like any multimodal artifact, are best understood not as ‘‘timeless’’ or monolithic but rather as ideologically laden with the intentions of the designers as well as the readers of these apps. In aligning picturebook apps with the broader research agenda of multimodal analytical frameworks, we find ourselves faced with the question of which analytic approach might be most appropriate for interrogating the meaning potentials associated with picturebook apps. Several approaches to multimodal analysis, for example, visual discourse analysis (Albers, 2008), multimodal discourse analysis (O’Halloran, 2004), and multimodal interaction analysis (Norris, 2004), have each contributed to our understanding of multimodal phenomena across a variety of contexts. Our aim in this article is not to imply the diminished effectiveness of these approaches, rather to suggest that a richer understanding could be revealed through the juxtaposition of perspectives drawn from different disciplines that have varied conceptual tools and frameworks. In the following sections, we briefly outline three analytical perspectives—social semiotic analysis, film analysis, and game studies—before applying each of them to a single picturebook app to illustrate what such a multifaceted approach might offer. Perspective 1: Social Semiotic Analysis Social semiotic approaches to multimodal phenomena have historically been characterized by three broad aims: (a) the collection, documentation, and systematic cataloguing of semiotic resources; (b) the investigation of how these resources are used in specific cultural, historical, and institutional contexts; and (c) the discovery and development of new semiotic resources and new uses of existing semiotic resources (Jewitt & Rumiko, 2001; van Leeuwen, 2005). Drawing upon various multimodal analytical approaches in the specific case of a children’s picturebook app, this would involve the description of the various semiotic resources used in an app, an examination of these resources in specific contexts of use, and understanding how these resources are used to render the visual and textual narrative. Detailed analyses of the multimodal elements of the picturebook app often discounts the semiotic resources used in the production of the various narrative instantiations and reader–viewers’ reception of these narratives. However, a focused analysis of the multimodal ensemble itself revealed visual and textual structures that may go unnoticed by narrative analysis or reader response approaches. The selections made by the app designers are not disinterested processes, rather designers and publishers select from the various modalities available to tell their story in ways that fit their interests and the sociocultural contexts of their production and reception. The semiotic Aguilera et al. 5 resources available to designers, authors, and app publishers offer different potentials in the rendering of the narrative and need to be considered as a visual object, a multimodal ensemble, and as a cultural artifact (Serafini, 2015). Multimodal analysis should include all three sites of analysis (Rose, 2012) as they try and understand the meaning potential of the multimodal ensemble under consideration. However, for the purpose of our analyses, the site of text—namely, the multimodal features instantiated through the design of a particular app—will be our primary focus. The Morris Lessmore app draws upon and offers modalities in addition to written language and visual images, including animation, sound effects, and background music that are not available in the print-based picturebook. These additional modalities expand the meaning potential of the narrative beyond the traditional interplay of text, image, and design (Nikolajeva & Scott, 2001). We recommend that researchers investigate the ways these additional modalities complement or possibly distract the reader from attending to the narrative. Investigations of how background music complements the text and visual images, as well as the meaning potentials offered by the animated sequences, require more attention to the multimodal elements and how these might work to cohere the presentation of the narrative. Perspective 2: Film Analysis Film analysis offers us another existing, formalized framework for examining multimodal texts, although it evolved well outside the communities of traditional multimodal research. Broadly speaking, film analysis is a way of reading film that considers the choices made by filmmakers in constructing film narrative, form, and style (Geiger & Rutsky, 2013). Narrative is ‘‘a chain of events linked by cause and effect and occurring in time and space’’ (Thompson & Bordwell, 2012, p. 79). The aspects of film that most consumers are familiar with are the plot, the narration, the chronology, and the causality. Form describes the ways in which parts of a film work together to create an overall effect on the audience (Thompson & Bordwell, 2012). In film analysis, form is fundamentally structural. It includes motifs and variations in structural patterns that create a sense of unity or disunity across the film. For those who are familiar with the film Groundhog Day, the repeated sequences that start Bill Murray’s day are an example of structural patterns. Style is considered the organization of chosen techniques to create the overall look and feel of a film (Thompson & Bordwell, 2012). This includes the mise-en-scene, cinematography, editing, and sound. In working within the film medium, filmmakers make a number of decisions. However, those decisions are linked to film style, including what lighting to use in a particular shot, what music to use in particular scene, which editing cuts to make in a particular sequence that historically have been the most unique to the medium. Each of the core modes of film style—mise-en-scene, cinematography, editing, and sound— relies on a range of semiotic resources (Metz, 1974) and functions according to a preexisting set of conventions or grammar. 6 Literacy Research: Theory, Method, and Practice In mise-en-scene, sets are designed and lighting is selected in order to convey specific moods. For example, a foggy street at night might be selected if the intention is to convey a sense of mystery or danger (Geiger & Rutsky, 2013). In cinematography, camera angle, level, height, and distance are all considered when composing a shot. So too are photograph tonalities and camera motion. For example, a shaking camera like that used by Spielberg in Saving Private Ryan might be used to convey a sense of realism. In film editing, different types of edits are used to achieve different goals. Typically, a standard cut is designed to be invisible (Spadoni, 2014). This means that when an audience sees a fade, they understand that time is passing. However, when an audience sees a wipe, as made popular by the Star Wars franchise, they recognize that a scene or setting change is occurring. Sound in film is a combination of music, dialogue, sound effects, and narration. The volume of dialogue and sound effects can create a sense of space and distance. The choice of music can impact the level of tension or excitement. For example, fast-paced music is a convention of the action sequence, while a rapid succession of screeching violins is associated with danger in a horror film. These conventions are grounded in the tradition of classic Hollywood cinema, meaning that they have been used over and over again by the Hollywood cinematic machine to the point of standardization. The conventions of these modes are intentionally unobtrusive (Kolker, 1998). They are meant to disappear in order to draw the audience as far as possible into the film experience. Finally, all these modes work together to create a complete, coherent, multimodal ensemble. In each case, a film production team makes choices grounded in the historical, traditional, practical, financial, and technical (Bazalgette & Buckingham, 2013). As such, film analysis typically considers film as both a text and a product. As picturebook apps evolve from static digitized imitations of printed picturebooks, they begin to incorporate conventions of film style. For instance, The Fantastic Flying Books of Mr. Morris Lessmore picturebook app (Moonbot Studios, 2011) incorporates short film sequences which utilize all the aspects of film style discussed earlier. In relying on film style, picturebook app designers call on an established set of grammars and their affiliated cultural and social associations. As such, approaching children’s picturebook apps from film analysis perspective helps to illuminate semiotic features and meaning potentials not otherwise emphasized in frameworks emphasizing the static elements of visual and textual representation. Rather, film analysis helps viewers consider the roles motion and sound play in these texts (Bazalgette & Buckingham, 2013). Perspective 3: Game Studies Game studies is relatively new as an academic field of study, attracting scholars primarily in tandem with the rapid growth of the video game industry beginning in the 1980s. These scholars have come from a variety of academic disciplines across the humanities, social sciences, and computer science, resulting in the juxtaposition of Aguilera et al. 7 diverse concepts and perspectives. While there continues to be considerable debate over the defining attributes of games and how they might best be understood, interactivity has perhaps been most commonly singled out as a defining feature of games. The feature of interactivity is an attribute that distinguishes games from other forms of media, such as books or film. Of course, all forms of media require some form of active cognitive and emotional engagement on the part of the audience, and other types of digital media, including apps, can require physical engagement. But perhaps no other medium has drawn such attention to the role of the player/audience in creating meaning and experience through their own choices and actions. Indeed, scholars have claimed that video games incorporate a unique mode of meaningmaking in the mechanics of play itself (Aarseth, 1997; Holmes, 2013; Juul, 2005). Salen and Zimmerman (2003), for example, argue that meaning arises from the player’s interactions with the game, and combined with feedback from the system, these interactions inform ongoing cycles of action and reflection (Gee, 2013). Interactivity in itself is a broad and vague term, and some scholars have come to criticize its common and at times simplistic application to games (Egenfeldt-Nielsen & Smith, 2006; Garite, 2003). At the most basic level, interactivity suggests that both game and player are changed through game play. In other words, players’ actions have consequences, both for the game and for the player. This general conceptualization, however, does not offer much insight into the nature of such interactions or their effects, particularly in relation to the quality of game play or the meaningfulness of the player’s experience. Other scholars have acknowledged the many potential kinds of interactivity and attempted to tease out the most important aspects of interactivity in relation to games. Salen and Zimmerman (2003) provide a useful conceptualization of interactivity in games. They describe four broad types of interactivity in relation to a narrative experience (including, according to their definition, games): (a) cognitive interactivity, or the ‘‘in-the-head’’ interpretations of the ‘‘content’’ of a game, (b) functional interactivity, or interactions with a game that are utilitarian, such as how you use the controller, (c) explicit interactivity, consisting of ‘‘participation with design choices and procedures’’ (Zimmerman, 2004, p. 159) or what the player actually does in a game, and (d) meta-interactivity, going beyond interactions with a single game to how players might discuss game play with others, critically analyze game content, and so forth. Salen and Zimmerman note that these types of interactivity are not mutually exclusive and can take place simultaneously and are common in some form to all of our media experiences, not just within games. Most game scholars continue to focus on explicit interactivity as the type of interactivity most specific to games (Carlquist, 2013; de Mul, 2005). Salen and Zimmerman (2003) argue that games can be narrative systems in ways that other media cannot, through the unfolding of player experience as she or he interacts with the complex combination of game elements. Meaningful play arises from the interaction of players with a game’s rules and goals and from the relationship between player actions and outcomes in the game. To use a nondigital example, kicking a soccer ball 8 Literacy Research: Theory, Method, and Practice is given meaning in the context of the game rules and goals; the result can be a goal, a penalty, or some other result depending on the particular game. One implication of this perspective is that interactions should have consequences that make sense, given the logic of the game rules; as they progress in a game, players will strive to understand the underlying game system (Gee, 2007) and will be frustrated when their actions lead to results that are unpredictable or inconsistent. Another way to view interactivity in games is in terms of the effects on players. Using the metaphor of a conversation, games ‘‘talk back’’ to players in response to their actions. In order to be successful, players adapt to the game’s rules and goals. While this can be viewed in a positive light as a form of complex, situated learning (Gee, 2007), scholars such as Garite (2003) argue that video games ‘‘play the player’’ (n.p.); that is, players internalize the rules and underlying ideological structures of the game. Whether this should be a cause for concern is another issue, but the salient point is the idea that interactivity requires the player to more directly participate in ideological worlds (Squire, 2006) than books or film. Thus, the meaning of players’ actions and the game’s responses can be viewed beyond the immediate context of the game, as reflecting dominant beliefs and associated practices. The analytical perspectives of social semiotics, film analysis, and game studies have served as foundations from which to approach the core textual artifacts of focus within distinct disciplinary traditions. Over time, each of these frameworks have shaped and been shaped by the texts and contexts to which they are applied. However, the evolution of ‘‘hybrid’’ media forms such as the children’s picturebook app appear to blend and remix established conventions of meaning-making and have opened up new possibilities for exploring analytical frameworks across interdisciplinary perspectives. To illustrate the insights that these divergent perspectives might provide, we now turn to an analysis of a specific children’s picturebook app The Fantastic Flying Books of Morris Lessmore (Moonbot Studios, 2011) from each of these perspectives. Analyzing Morris Lessmore Across Interdisciplinary Perspectives The Fantastic Flying Books of Mr. Morris Lessmore (Moonbot Studios, 2011) is a story about a man who loses his home during a hurricane and is guided to another world where he becomes the curator of books in a magical library. He takes care of the books in the library and shares them with other people in order to make their lives more fulfilling. Morris Lessmore writes the story of his life in a journal and passes this book along to future curators as his life draws to a close. Our selection of this particular story for analysis was, in part, influenced by unique aspects of its production and release. First envisioned by author William Joyce, the story was released as a short film, a printed book, and picturebook app, with each instantiation sharing the core content of the story but diverging in their use of meaning-making affordances of each medium of telling. Aguilera et al. 9 Previous research on examining Morris Lessmore as a picturebook app exemplar has analyzed this narrative across different platforms, focusing on how the app ‘‘defines, represents, or shapes the meaning of reading itself’’ (Hateley, 2013, p. 2). Differences in readers’ experiences transacting with narratives in a picturebook, app, and film formats (Schwebs, 2014), and how picturebook theories need to be reconceptualized in light of the affordances and limitations of emerging digital platforms (Al-Yaqout & Nikolajeva, 2015). However, our intent here is to examine a single instantiation—the picturebook app—across multiple frameworks of analysis. These analyses are not meant to be exhaustive catalogues of all multimodal elements of meaning potentials inherent in the design of the app; rather, we seek to illuminate the possibilities for deeper understanding afforded by such an approach as well as highlight areas of limitation and further examination. Morris Lessmore Through the Lens of Social Semiotics Approaching the Morris Lessmore app from a social semiotic perspective, we considered the various semiotic resources that have been included in the narrative and how they are incorporated into the picturebook app. We might examine, for example, how visual images, animated sequences, design elements, tableaux features, written and spoken language, and transitional features add to the narrative in ways that differ from the printed picturebook. For example, the opening tableaux, where Morris Lessmore is sitting on a porch reading, offers different meaning potentials from the picturebook because the animated sequence of wind blowing across the scene, the voice-over narration, and the movement of books across the tableaux are different than the static printed image and text. The sound effects of the wind add to the drama of the scene and suggest a storm is brewing, rather than simply stating this in the written text. Again, rather than examining each of these multimodal elements as separate from one another, a social semiotic perspective foregrounds the importance of how these features orchestrate meaning across the entire multimodal ensemble. To deepen the analysis of the opening tableaux, we considered how text, image, and sound begin in a complementary relationship where the meaning potentials of the image, text, and sound build upon one another. This begins to shift, however, to a divergence of image and sound from the text, when the scene begins to transition to the coming storm that would literally turn Lessmore’s life upside down. Thus, we can see a dissonance between multimodal elements echoing the chaotic events of the story to come. Analyzing these features as separate semiotic resources—that is, assuming that image, sound, or text operate independently of one another—would miss the meaning potential when viewed as complementary aspects of a more coherent whole. Examining this scene from a social semiotic perspective allows for the investigation of the complex ways that meaning can be designed into the children’s picturebook app and experienced by readers in different ways. 10 Literacy Research: Theory, Method, and Practice Morris Lessmore Through the Lens of Film Analysis Applying a framework of film analysis to look at the opening sequence to Morris Lessmore, one might begin to notice the role of other design elements in the orchestration of meaning. From the very beginning, for instance, the title page, or ‘‘home screen’’ of the app is paired with sound and music. While in appearance this home screen is designed to resemble the printed book, we note that the transition from title page to opening tableau starts off more like a film. Techniques associated with cinematography draw the reader into the book by mimicking use of camera movement, immediately distinguishing the picturebook app from the static picturebook. In examining the mise-en-scene, it can be noted that the setting in the first tableau resembles the French Quarter of New Orleans and Lessmore’s clothes are reminiscent of the 1920s and 1930s. More specifically, he wears a porkpie hat and carries a cane which cause him to look distinctly like Buster Keaton, a comedic silent film star. Although these elements are certainly present in the images of the setting in the printed picturebook, they were viewed as still, static images. In the app, the movement of the camera adds a dimension of embodied space around the viewer, as they are figuratively pulled into the app. As the sequence continued, one might further note that the lighting in the mise-enscene began bright and sunny connoting happiness and safety, but changed within the image to dark, windy skies without the need for a page turn. This animation, paired with ominous music, established a sense of foreboding and danger. A common film convention of intertextuality is triggered, as the viewer might have recognized the hurricane scene as being reminiscent of The Wizard of Oz. One might argue here that intertextuality, as its name suggests, is also a common feature of many other kinds of texts, including traditional print. However, a key difference is in the nature of the intertextuality across media forms. While reading and comprehending an intertextual reference in print may assume a degree of conceptual background knowledge on the part of the reader, filmic intertextuality may depend more on perceptual recognition of the reference (such as ‘‘Hey, I’ve seen that before!’’). Editing does not appear in the first two tableaux of the picturebook app, but fast paced cuts begin in the third tableau. Thus, one can observe how the app blends elements from the traditional printed picturebook with techniques borrowed from film, creating an experience that is at once a hybrid of these approaches, as well as a unique media form altogether. Morris Lessmore Through the Lens of Game Studies A game studies perspective prompts us to foreground the importance of ‘‘designed interactions’’ as a resource for meaning-making in the Morris Lessmore app. We drew on Salen and Zimmerman’s (2003) broad framework to identify the role of four types of interactivity in the app. On one level, we examined the nature of cognitive interactivity. Just as in any kind of literacy experience, readers interpret the content of Aguilera et al. 11 Morris Lessmore, but this interpretation was moderated by reader’s decisions to access optionally accessible content—through hotspots, manipulatives, and other features—leading to a higher variation in reader experience of this content. On another level, we also noted the functional dimension of interactivity (Salen & Zimmerman, 2003), both as a typical design consideration of picturebook apps in general, and its specific instantiation in Morris Lessmore. Besides the common user inputs of tapping, swiping, and dragging on the touch screen interface, we also considered how the app structures and scaffolds these interactions. Not every element of every tableaux is interactive, for example, and designed into each tableaux are visual cues that differentiate interactives from background imagery. In certain cases, such as the opening tableau, the image of an arrow subtly flashes across the screen to invite the reader to drag a finger across the screen to transform the scene. The ability to interpret cues and interest in engaging with interactives is important to the reader experience. Considering the explicit interactivity designed into Morris Lessmore might shift the focus of analysis to how readers are actively engaged in shaping how the story itself unfolds. Among game designers, there is no standard for how much control a player should have over what happens in a game, and many games are quite linear in their narrative. Rather, a game studies perspective led us to ask: ‘‘How are the reader’s actions and the corresponding results meaningful, in relation to the narrative or ‘world’ created through the app?’’ A more traditional view of reading as a ‘‘linear’’ process might view interactive elements as a kind of distraction from the core content of the story. While a game studies perspective understands that such playful, creative experiences may serve to deepen a reader’s experience through the invitation to personalize the content and create a story experience of their own. Finally, the nature of meta-interactivity outside the app itself would not be possible without examination of such a text in use. However, by examining certain design features, we inferred ideological claims about the nature of reading and what it means to be a reader in the modern world. For example, despite the digital and multimodal nature of the app, much of the visual and interactional design appears to mimic, and in a sense privilege the features of the printed book. This similarity occurred in the visual style of the text display echoing a 19th-century schoolbook to the ‘‘page-corners’’ of the screen reminiscent of physical page turning in a printed book. Thus, we might conclude that reading in the context of the Morris Lessmore app is still something portrayed as linear in nature, that stories have a definite beginning, ending, and structure. To be a ‘‘reader’’ of this particular app, however, is more than just being able to pronounce words on a page; in fact, with the built-in narration features, it could be argued that decoding skills would serve as less of a barrier to participation than a traditional printed book. Instead, to be a reader of the Morris Lessmore app also involves navigating, discovering, and playing with interactive elements unique to this emerging multimodal experience. An analysis of the Morris Lessmore app grounded in game studies foregrounds elements of interaction in children’s picturebook apps and allows to reconsider ‘‘optional’’ or ‘‘distracting’’ interactional features as instead promoting a stance of 12 Literacy Research: Theory, Method, and Practice discovery, creation, and nonlinear navigation as part of this new multimodal experience. Coupled with social semiotic analyses of visual meaning and film analysis, this perspective provides a deeper understanding of the complex meaning potentials designed into interactive multimodal experiences like the children’s picturebook app. Combining these perspectives is not without its challenges, however, and will require additional conversation, theoretical development, and testing of assumptions through research. Concluding Thoughts and Future Directions The purpose of this article is to explore the affordances and constraints inherent in an examination of a picturebook app through multiple analytical frameworks. Rather than argue for the application of single analytical approach to the investigation of a multimodal ensemble or propose a unified framework that might try to account for all of the complexities of a particular multimodal experience, our research drew from the perspectives of social semiotics, film analysis, and game studies. We assert that in using these different frameworks for examining picturebook apps, new ways of talking about and subsequently analyzing these media artifacts become available. Our focus on analysis at the site of text also highlights the importance of further research to examine emerging multimodal experiences such as children’s picturebook apps at the site of production, as well as the site of reception (Rose, 2012). All three perspectives described in this article assume some ideological motivation on the part of the producer of such texts. Further research in this area might seek to examine the ideational, creative, or commercial processes that might underlie the development of these apps. Whether we foreground the importance of the sociocultural context, audience co-construction, or user interaction, additional research at the site of reception must examine the role of these apps as part of broadening the literacy experiences in our modern world. Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Funding The author(s) received no financial support for the research, authorship, and/or publication of this article. References Aarseth, E. J. (1997). Cybertext: Perspectives on ergodic literature. Baltimore, MD: John Hopkins University Press. Albers, P. (2008). Theorizing visual representation in children’s literature. Journal of Literacy Research, 40, 163–200. Aguilera et al. 13 Al-Yaqout, G., & Nikolajeva, M. (2015). Re-conceptualising picturebook theory in the digital age. BLFT—Nordic Journal of ChildLit Aesthetics, 6. doi:10.3402/blft.v6.26971 Bazalgette, C., & Buckingham, D. (2013). Literacy, media and multimodality: A critical response. Literacy Discussion, 47, 95–102. doi:10.1111/j.1741-4369.2012.00666.x Carlquist, J. (2013). Playing the story: Computer games as a narrative genre. Human IT: Journal for Information Technology Studies as a Human Science, 6. Retrieved from https://ojs.bib. hb.se/index.php/humanit/article/viewFile/144/148 de Mul, J. (2005). The game of life: Narrative and ludic identity formation in computer games. In J. Raessens & J. Goldstein (Eds.), Handbook of computer game studies (pp. 251–266). Cambridge, MA: MIT Press. Duncum, P. (2004). Visual culture isn’t just visual: Multiliteracy, multimodality and meaning. Studies in Art Education, 45, 252–264. Egenfeldt-Nielsen, S., & Smith, J. H. (2006). Computer games, media and interactivity [Web post]. Retrieved from http://game-research.com/index.php/articles/computer-gamesmedia-and-interactivity/ Garite, M. (2003). The ideology of interactivity (or, video games and taylorization of leisure). In M. Copier & J. Raessens (Eds.), Level up: Proceedings of the 2003 digital games research association conference (vol. 2). Utrecht. Retrieved from http://www.digra.org/dl/db/05150. 15436.pdf Gee, J. P. (2007). What video games have to teach us about learning and literacy (2nd ed.). New York, NY: St. Martin’s Press. Gee, J. P. (2013). The anti-education era: Creating smarter students through digital learning. London, England: Palgrave Macmillan. Geiger, J., & Rutsky, R. L. (2013). Film analysis: A Norton reader. New York, NY: W. W. Norton. Hateley, E. (2013). The value of books: The Fantastic Flying Books of Mr. Morris Lessmore and the social hieroglyphic of reading. Paper presented at the 40th Annual Children’s Literature Association Conference, Biloxi, MS. Holmes, J. (2013). How to look at videogames: A visual analysis toolkit. In Proceedings of the Games þ Learning þ Society Conference 9.0. Pittsburgh, PA: ETC Press. Retrieved from http://press.etc.cmu.edu/files/GLS9.0-Proceedings-2013-web.pdf Jewitt, C. (2009). An introduction to multimodality. In C. Jewitt (Ed.), The Routledge handbook of multimodal analysis (pp. 14–27). New York, NY: Routledge. Jewitt, C., & Rumiko, O. (2001). Visual meaning: A social semiotic approach. In T. van Leeuwen & C. Jewitt (Eds.), The handbook of visual analysis (pp. 134–156). London, England: Sage. Juul, J. (2005). Half-real: Video games between real rules and fictional worlds. Cambridge, MA: MIT Press. Kolker, R. P. (1998). The film text and film form. In J. Hill & P. Church Gibson (Eds.), The Oxford guide to film studies (pp. 11–29). Oxford, England: Oxford University Press. Kress, G. (2003). Literacy in the new media age. London, England: Routledge. Kress, G. (2009). Multimodality: A social semiotic approach to contemporary communication. London, England: Routledge. 14 Literacy Research: Theory, Method, and Practice Kress, G., & van Leeuwen, T. (1996). Reading images: The grammar of visual design. London, England: Psychology Press. Krippendorff, K. (2004). Content analysis: An introduction to its methodology. Thousand Oaks, CA: Sage. Machin, D. (2007). Introduction to multimodal analysis. Oxford, England: Oxford University Press. Metz, C. (1974). Film language: A semiotics of the cinema. Chicago, IL: University of Chicago Press. Meyers, E. M., Zaminpaima, E., & Frederico, A. (2014). The future of children’s texts: Evaluating book apps as multimodal reading experiences. In iConference 2014 Proceedings. Retrieved from https://www.ideals.illinois.edu/handle/2142/47386 Moonbot Studios. (2011). The Fantastic Flying Books of Mr. Morris Lessmore (Version 1.4.5). Retrieved from https://itunes.apple.com/us/app/fantastic-flying-books-mr./id438052647? mt¼8 Nikolajeva, M., & Scott, C. (2001). How picturebooks work. New York, NY: Routledge. Norris, S. (2004). Analyzing multimodal interaction: A methodological framework. London, England: Routledge. O’Halloran, K. (Ed.). (2004). Multimodal discourse analysis: Systemic functional perspectives. London, England: A&C Black. Painter, C., Martin, J. R., & Unsworth, L. (2013). Reading visual narratives: Image analysis of children’s picturebooks. Sheffield, England: Equinox. Rose, G. (2012). Visual methodologies: An introduction to researching with visual materials. London, England: Sage. Rowsell, J., McLean, C., & Hamilton, M. (2012). Visual literacy as a classroom approach. Journal of Adolescent & Adult Literacy, 55, 444–447. Salen, K., & Zimmerman, E. (2003). Rules of play: Game design fundamentals. Cambridge, MA: MIT Press. Sargeant, B. (2015). What is an ebook? What is a book app? And why should we care? An analysis of contemporary digital picturebooks. Children’s Literature in Education, 46, 454–466. Schwebs, T. (2014). Affordances of an app—A reading of The Fantastic Flying Books of Mr. Morris Lessmore. BLFT – Nordic Journal of ChildLit Aesthetics, 5. Retrieved from http:// www.childlitaesthetics.net/index.php/blft/article/view/24169 Serafini, F. (2013). Reading the visual: An introduction to teaching multimodal literacies. New York, NY: Teachers College Press. Serafini, F. (2015). Paths to interpretation: Developing students’ interpretive repertoires. Language and Literacy, 15, 118–133. Serafini, F., Kachorsky, D., & Aguilera, E. (2015). Picturebooks 2.0: Transmedial features across narrative platforms. Journal of Children’s Literature, 41, 16–24. Spadoni, R. (2014). A pocket guide to analyzing films. Oakland, CA: University of California Press. Squire, K. (2006). From content to context: Videogames as designed experience. Educational Researcher, 35, 19–29. Aguilera et al. 15 Thompson, K., & Bordwell, D. (2012). Film art: An introduction. New York, NY: McGraw-Hill. van Leeuwen, T. (2005). Introducing social semiotics. London, England: Routledge. Zimmerman, E. (2004). Narrative, interactivity, play, and games: Four naughty concepts in need of discipline. In P. Harrigan & N. Wardup-Fruin (Eds.), First person: New media as story, performance, and game (pp. 154–164). Boston, MA: MIT Press. Author Biographies Earl Aguilera is a doctoral student in the Learning, Literacies, and Technologies program at Arizona State University. His current scholarly interests focus on literacy development as it occurs across digital and face-to-face learning contexts. Dani Kachorsky is a third year doctoral student in the Learning, Literacies, and Technologies program at Arizona State University. Her research interests include visual and multimodal literacy. Specifically, she is interested in the literacy practices that develop around comic books and films in secondary classrooms and afterschool programs. Elisabeth Gee is professor and Delbert & Jewell Lewis Chair in Reading & Literacy at Mary Lou Fulton Teachers College at Arizona State University. Her current scholarly interests include intergenerational learning and literacies associated with new digital media, particularly video games. Frank Serafini is a professor of Literacy Education and Children’s Literature in the Mary Lou Fulton Teachers College at Arizona State University. His current research investigates the nature of digital picturebooks, the role of multimodality in literacy education, and approaches for analyzing multimodal phenomenon.