Human + agent

Human + Agent: Creating Recombinant Information Andruid Kerne, Vikram Sundaram, Jin Wang, Madhur Khandelwal, J. Michael Mistrot Interface Ecology Lab | Center for Digital Libraries | Texas A&M Department of Computer Science College Station. TX, 77843. USA | +1 979.862.3217 {andruid, vikrams, jin, madhur, mistrot}@cs.tamu.edu ABSTRACT combinFormation is a tool that enables browsing and collecting information elements in a generative space. By generative, we mean that the tool is an agent that automatically retrieves information elements and visually composes them. A combinFormation session presents a dynamic, evolving recombination of information elements from different sources. The elements are manipulable in the information space. Recombination is the process of taking previously unconnected elements, and combining them to create new configurations. One purpose of this space is to support the formation of ideas, through more and less focused processes of foraging. While ideas are forming, the criteria that underlie information foraging activities may not be well defined. Collecting the specific subset of related information elements is challenging. Cognitive scientists have established that combinations of images and textual elements are examples of preinventive structures that can lead to the emergence of new ideas. These preinventive structures often combine existing representations. Our program generates recombinant visualizations that develop interrelationships between the information elements. The generative visualization is based on a procedural model of the information, and the user’s interests. The user model reflects interactions in which s/he explicitly expresses interest. The agent retrieves information based on the evolving model. The visual composition is also developed to emphasize the user’s evolving sense of what is important. This involves solving problems in the dynamic visualization of dynamic, heterogeneous collections. In our novel interaction model, the human being shares control of the evolving information space with the agent. The user can express interest in information elements as they stream in, and design the visual space, using interactive tools. Expressions feed back through the model to drive the program’s retrieval and visual composition decisions. Categories and Subject Descriptors H.5.2 [User Interfaces]: Navigation; H.5.4 [Hypermedia]: User issues. General Terms: design, human factors. Keywords: recombinant information, generative hypertext, granularity of browsing, procedural visual composition 0. DEVELOPMENT PATH combinFormation [http://csdl.tamu.edu/combinformation] is the next generation of the program formerly known as CollageMachine [4-7]. The new information model incorporates textual attributes, derives metadata for images, and synthesizes search engine queries to satisfy users’ expressions of interest. The visual composer utilizes stroked text and alpha masking to enhance the sensation of recombination. A new interface enables greater articulation of expressions of interest and design, by affording independent control of these tool settings. 1. GRANULARITY OF BROWSING Granularity refers to the size of the fundamental units that browsers present. combinFormation shifts the granularity of browsing down from documents to smaller units. It makes information elements essential. In the current implementation, supported media elements consist of images and chunks of text. A study by Schraefel et al [16] confirms that users regularly need to deal with these finer grains. combinFormation translates hypermedia documents into containers of media elements and references to other containers. Together, the media elements, their containers, and hyperlinks comprise the set of hypermedia information elements, on which the program conducts its operations. Documents are automatically downloaded every few seconds; information elements stream into the visual space once per second.1 Temporally, the granularity of browsing is extended from a single static view that settles and waits for the user to click a hyperlink, to a dynamic one that evolves continuously. 2. RECOMBINANT INFORMATION Recombinant information [10] combines existing elements to create new visual and semiotic relationships, and facilitate readings of these amalgams. Readers are drawn to make their own interpretations. Recombinant compositions are examples of preinventive structures that are often necessary for the emergence of new ideas [3]. Distinct elements are recombined and re-cognized as a connected entity. One difficulty is that until discovery does occur, we may not know what the necessary elements are, or how to specify their attributes. When we do encounter the right combinations, they help us breakthrough to creative innovations [3]. Recombinant information techniques have been developed by collage artists such as Ernst, montage filmmakers such as Eisenstein, and musicians such as Public Enemy [10]. The art involves specificities of selection, juxtaposition, and compositing. 3. FUNCTIONAL MODULES combinFormation is based on the functional integration of three essential modules. The first models information and user interests, and conducts retrieval and selection operations. The second composes the visualization. The third enables expressive interaction with the information space. Retrieval / Selection of Desirable Information combinFormation engages in adaptive automatic retrieval of information based on the evolving model of user interests in conjunction with content structure. The model drives operations 1 Copyright is held by the author/owner(s). MM'03, November 2-8, 2003, Berkeley, California, USA. ACM 1-58113-722-2/03/0011. These are the default; the user can halt content streaming, and control its rate (See below). It is also interrupted automatically by interactive media element drag operations. such as document retrieval (the downloading of web pages, either by crawling hyperlinks or initiating new search engine queries), and information element selection. The model includes one component that utilizes hypermedia graph structure [5], and another component that utilizes Salton’s vector space model of text [10], adapted for web content [1]. Expressions of interest are pumped through containing documents, hyperlinks, and terms using damped spreading activation [9, 5]. As in the work of Page and Brin [8], the anchor texts of hyperlinks play an important role. Usability studies have shown [4] that when the user expresses positive interest, there is an onus on the program to display related information. Thus, the program assesses its stock of related information; if the set of information elements already on hand does not contain sufficient related information, the program will synthesize a search engine query that expands the information space. Procedural Visual Composition An information space on research at the Texas A&M Center for Digital Libraries. interactive expression are made independent. This enables finer control of the expression of both aspects. A future paper will describe this scenario in detail. The goals of document element set visualization include legibility, layering, representation of structural relationships and visual integration [12]. Unlike the prior work, we are dealing with dynamically formed, heterogeneous corpora. The agent is required to compose the space procedurally, as its contents are defined. This introduces new challenges to the visualization problem. Our visual composition also adds new goals. These include creating recombinant information effects, and layering that evolves with the flow of information elements over time. We achieve recombinant visual integration through the use of typographic techniques such as text stroking, and image processing techniques such as alpha blending. Piles of text elements with opaque backgrounds obscure each other quickly. If they have completely transparent backgrounds, they become illegible. The use of text stroking promotes legibility and layering of overlapping text elements, while facilitating recombinant integration. Visual attributes of text elements such as stroke colors (saturation and value) and stroke width are mapped to hypermedia structural relationships. For example, all text elements from pages and sites with the same internet domain are rendered with the same hue. The visualization also reflects age-wear through the use of history-enriched and interest-enriched media elements [3]. Text elements fade away, with transparency increasing over time. Image elements are gradually desaturated; they grow duller and recede. The result is a cycling of foreground/background relationships. Age-wear is suspended for elements that the user expresses positive interest in. Interaction Model Our paradigm for interaction addresses issues that arise in the development of interface agents and support for creativity. The agent and the user work together in realtime in a single design space, as if they were two people collaborating. The program includes the user’s explicit design decisions in its model for deciding where to place new elements in the space. The user also communicates her/his interests to the program. S/he uses tools with explicit semantics for expressing positive or negative interest in an element, and its associated attributes. In our latest version of the interface, the design and interest aspects of 4. REFERENCES [1] Balabanovic, M. Learning to Surf: Multiagent Systems for Adaptive Web Page Recommnd. Stanford Ph.D. Dist, 1998. [2] Finke, R., Ward, T., Smith, S. Creative Cognition. Cambridge MA., MIT Press, 1992. [3] Hill, W., Hollan, J.. "History-Enriched Digital Objects", Proc ACM Computers, Freedom and Privacy, 1993. [4] Karadkar, U.P., Kerne, A., Furuta, R., Francisco-Revilla, L., Shipman, F., Wang, J., Connecting Interface Metaphors to Support Creation of Hypermedia Collections, Proc European Conference on Digital Libraries, 2003. [5] Kerne, A. CollageMachine: A Model of Interface Ecology. NYU Ph.D. Dissertation. 2001. [6] Kerne, A. Concept-Context-Design: A Creative Model for the Development of Interactivity, Proc ACM Creativity and Cognition 4, 2002. [7] Kerne, A., Sundaram, V. A Recombinant Information Space. Proc Cosign 2003. [8] Page, L., Brin, S., Motwani, R., Winograd, T. The PageRank Citation Ranking: Bringing Order to the Web. http://dbpubs.stanford.edu:8090/pub/1999-66. 1998. [9] Salton, G. and Buckley, C. On the Use of Spreading Activation Methods in Automatic Information Retrieval. Proc SIGIR 1988. 147-160. [10] Salton, G., and McGill, M.J., Introduction to Modern Information Retrieval, New York, McGraw-Hill, 1983. [11] Schraefel, M. et al. Hunter Gatherer: Interaction Support for the Creation and Management of Within-Web-Page Collections. Proc WWW 2002. [12] Small, D.L., Rethinking the Book, MIT Media Lab Ph.D. Dissertation, 1999.