Interactive Paper

We present a general model and information server for the digital annotation of printed documents. The resulting annotation framework supports both informal and structured annotations as well as context-dependent services. A demonstrator application for mammography that features both enhanced writing and reading activities is described.

Despite the increasing availability of various forms of digital maps and guides, paper still prevails as the main information medium used by tourists during city visits. The authors describe how recent technologies for digitally augmented paper maps can be used to develop interactive paper maps that provide value-added services for tourists through digital overlays. An initial investigation into the use of these maps to support visitors to the Edinburgh festivals is also presented.

Over the last few years, there has been a significant increase in the number of researchers dealing with the integration of paper and digital information or services. While recent technological developments enable new forms of paper-digital integration and interaction, some of the original research on interactive paper dates back almost twenty years. We give a brief overview of the most relevant past and current interactive paper developments. Then, based on our experience in developing a wide variety of interactive paper solutions over the last decade, as well as the results of other research groups, we outline future directions and challenges for the realisation of innovative interactive paper solutions. Further, we propose the definition of common data formats and interactive paper design patterns to ensure future cross-application and framework interoperability.

PowerPoint is one of the most frequently used tools to present multimedia for educational purposes. Nevertheless, little is known about the users' needs when using PowerPoint during lecturing. Our study focused on the presenter's needs in this context of use. We interviewed nine university lecturers from the computer science department by means of a questionnaire. The results show that users require features that are not yet realised with PowerPoint. For example, the control and use of other media should be adequately integrated. Or the navigation within the slide collection should be improved to better meet the users' needs. Based on our findings, we outline required system features and suggest solutions in form of a prototype.

Overview of ongoing research in the CISA group (WISE research lab) at the Vrije Universiteit Brussel. The group is investigating cross-media information spaces and architectures, interactive paper and augmented reality solutions as well as multimodal and multi-touch interaction frameworks. Thereby, we are doing research on new document formats for representing information across different types of media as well as fluid user interfaces for interacting with the resulting cross-media information spaces. A significant part of our research is based on applying and extending the resource-selector-link (RSL) hypermedia metamodel.

More information: https://beatsigner.com/cross-media-information-spaces-and-architectures.html

While there have been dramatic increases in the use of digital technologies for information storage, processing and delivery over the last twenty years, the affordances of paper have ensured its retention as a key information medium. In this book we review a wide variety of projects and technological developments for bridging the paper-digital divide. We present our information-centric approach for a tight integration of paper and digital information that is based on a general cross-media information platform. Different innovative augmented paper applications that have been developed based on our interactive paper platform and Anoto Digital Pen and Paper technology are introduced. For example, these applications include a mobile interactive paper-based tourist information system (EdFest) and a paper-digital presentation tool (PaperPoint). Challenges and solutions for new forms of interactive paper and cross-media publishing are discussed. The book is targeted at developers and researchers in information systems, hypermedia and human computer interaction, professionals from the printing and publishing industry as well as readers with a general interest in the future of paper.

Buy from Amazon: http://www.amazon.co.uk/Fundamental-Concepts-Interactive-Cross-Media-Information/dp/3837027139

Despite the increasing availability of various forms of digital maps and guides, paper still prevails as the main information medium used by tourists during city visits. The authors describe how recent technologies for digitally augmented paper maps can be used to develop interactive paper maps that provide value-added services for tourists through digital overlays. An initial investigation into the use of these maps to support visitors to the Edinburgh festivals is also presented.

Research paper: https://beatsigner.com/publications/overlaying-paper-maps-with-digital-information-services-for-tourists.pdf

"While there have been dramatic increases in the use of digital technologies for information storage, processing and delivery over the last twenty years, the affordances of paper have ensured its retention as a key information medium. Despite predictions of the paperless office, paper is ever more present in our daily work as reflected by the continuously increasing worldwide paper consumption.

Many researchers have argued for the retention of paper as an information resource and its integration into cross-media environments as opposed to its replacement. This has resulted in a wide variety of projects and technological developments for digitally augmented paper documents over the past decade. However, the majority of the realised projects focus on technical advances in terms of hardware but pay less attention to the very fundamental information integration and cross-media information management issues.

Our information-centric approach for a tight integration of paper and digital information is based on extending an object-oriented database management system with functionality for cross-media information management. The resulting iServer platform introduces fundamental link concepts at an abstract level. The iServer’s core link management functionality is available across different multimedia resources. Only the media-specific portion of these general concepts, for example the specification of a link’s source anchor, has to be implemented in the form of a plug-in to support new resource types. This resource plug-in mechanism results in a flexible and extensible system where new types of digital as well as physical resources can easily be integrated and, more importantly, cross-linked to the growing set of supported multimedia resources. In addition to the associative linking of information, our solution allows for the integration of semantic metadata and supports multiple classification of information units. iServer can, not only link between various static information entities, but also link to active content and this has proven to be very effective in enabling more complex interaction design.

As part of the European project Paper++, under the Disappearing Computer Programme, an iServer plug-in for interactive paper has been implemented to fully integrate paper and digital media, thereby gaining the best of the physical and the digital worlds. It not only supports linking from physical paper to digital information, but also enables links from digital content to physical paper or even paper to paper links. This multi-mode user interface results in highly interactive systems where users can easily switch back and forth between paper and digital information. The definition of an abstract input device interface further provides flexibility for supporting emerging technologies for paper link definition in addition to the hardware solutions for paper link definition and activation that were developed within the Paper++ project.

We introduce different approaches for cross-media information authoring where information is either compiled by established publishers with an expertise in a specific domain or by individuals who produce their own cross-media information environments. Preauthored information can be combined with personally aggregated information. A distributed peer-to-peer version of the iServer platform supports collaborative authoring and the sharing of link knowledge within a community of users.

The associations between different types of resources as well as other application-specific information can be visualised on different output channels. Universal access to the iServer’s information space is granted using the eXtensible Information Management Architecture (XIMA), our publishing platform for multi-channel access.

Our fundamental concepts for interactive paper and cross-media information management have been designed independently of particular hardware solutions and modes of interaction which enables the iServer platform to easily adapt to both new technologies and applications. Finally, the information infrastructure that we have developed has great potential as an experimental platform for the investigation of emerging multimedia resources in general and interactive paper with its possible applications in particular."

Recent developments in digital pen and paper solutions enable, not only the digital capture of handwriting, but also paper to be used as an interactive medium that links to digital information and services. We present a tool that builds on technologies for interactive paper to enable PowerPoint presentations to be controlled from printed slide handouts. Furthermore, slides can be easily annotated during presentations by simply drawing on the printed version of the slide. As well as discussing the advantages of such a paper-based interface and initial findings on its use, we describe how we were also able to exploit it to provide a general prototyping tool for interactive paper applications.

In this paper, we describe the process of using user needs, collected through case communities, to design concepts that link print products and digital services. One of the example cases was an online community, while the other was a traditional off-line community. Users often, perhaps subconsciously, know the strengths of print and web and use the media suitable for their needs accordingly. With a user-centered design approach, it's possible to create new and enticing hybrid media products for user communities with very different backgrounds.

Over the last decade, there has been an increasing interest in paper-digital systems that allow regular paper documents to be augmented or integrated with digital information and services. Although a wide variety of technical solutions and applications have been proposed, they all rely on some means of specifying links from areas within paper pages to digital services where these areas correspond to elements of the document's artwork. Although various frameworks and tools are available to support the development of paper-digital applications, these tend to either require some programming skills or focus on specific application domains. We present an advanced publishing solution that is based on an authoring rather than programming approach to the production of interactive paper documents. Our solution is fully general and we describe how it uses concepts of templates and variable content elements to reduce redundancies and increase the flexibility in developing paper-digital applications.

Paper persists as an important medium for documents and this has motivated the development of new technologies for interactive paper that enable actions on paper to be linked to digital actions. A major issue that remains is how to integrate these technologies into the document life cycle and, specifically, how to facilitate the authoring of links between printed documents and digital documents and services. We describe how we have extended a general web publishing framework to support the production of interactive paper documents, thereby integrating paper as a new web channel in a platform for multi-channel access.

Recent approaches for linking paper and digital information or services tend to be based on a one-time publishing of digital information where changes to the printed document become isolated from its digital source. Structural information which is available when authoring a digital document is lost during the printing process making it difficult to map interactions within the physical document to the corresponding elements of the digital document. We identify the necessary requirements for an integrated digital and paper-based document lifecycle and present our solution which supports a seamless transition between digital and physical document instances. PaperProof is presented as a paper-based proof-editing application that exploits our new approach for mapping pen-based interactions with paper documents to the corresponding operations in the digital document instance.

Citations form the basis for a web of scientific publications. Search engines, embedded hyperlinks and digital libraries all simplify the task of finding publications of interest on the web and navigating to cited publications or web sites. However the actual reading of publications often takes place on paper and frequently on the move. We present a system Print-n-Link that uses technologies for interactive paper to enhance the reading process by enabling users to access digital information and/or searches for cited documents from a printed version of a publication using a digital pen for interaction. A special virtual printer driver automatically generates links from paper to digital services during the printing process based on an analysis of PDF documents. Depending on the user setting and interaction gesture, the system may retrieve metadata about the citation and inform the user through an audio channel or directly display the cited document on the user’s screen.

A video about the system can be found at: https://www.academia.edu/11583686/Print-n-Link_Weaving_the_Paper_Web

Pen and paper support the rapid production of sketches. However, the paper interface is not always optimal for collaborative sketching as seen in brainstorming sessions where multiple parties would often like to communicate and participate in the sketching synchronously. Novel interactive paper solutions may provide the answer by bridging the paper-digital divide and allowing users to sketch on paper simultaneously while capturing the actions digitally. We present an analysis of collaborative sketching activities in working environments with remote participation. After highlighting the importance of paper for natural interaction in these settings, we introduce PaperSketch, an interactive paper-digital tool for collaborative remote sketching. We discuss the collaborative development of ideas based on the prototype and outline how important feedback issues have been addressed by utilising spatial constraints and multimodal features.

In the area of highly interactive systems, the use of object databases has significantly grown in the past few years due to the fact that one can, not only persistently store data in the form of objects, but also provide additional functionality in terms of methods defined on these objects. However, a limitation of such a tight coupling of objects and their methods is that parts of the application logic cannot be reused without also having instances of these objects in the new application database. Based on our experience of designing multiple interactive cross-media applications, we propose an approach where we distinguish between regular database objects containing the data and so-called active components storing metadata about specific services. Active components are first class objects which, at activation time, can perform some operations on the server as well as on the client side. Since active components are standalone lightweight components, they can be dynamically bound to single objects or semantically grouped sets of objects and be automatically invoked by different forms of database interactions. The database-driven development of arbitrary client and server-side application functionality not only simplifies the design of highly interactive systems, but also improves the reuse of existing components across different systems.

DelosDLMS is a prototype of a next-generation Digital Library (DL) management system. It is the result of integrating various specialized DL services provided by partners of the DELOS network of excellence into the OSIRIS platform. OSIRIS is a middleware environment for the reliable and scalable distributed execution of processes. Processes, in turn, are DL applications that are built from the specialized services available in the integrated system. DelosDLMS provides support for content-based retrieval in image, audio, video, and 3D collections and combination of any of these media types with keyword queries. It allows annotating retrieved information, provides a rich set of advanced graphical user interfaces to browse and explore large collections, and supports users in interacting with the system using a speech interface and interactive paper. Finally, it can even be applied in settings where DL content is available in a distributed P2P system.

This paper presents a discussion of work combining paper maps with electronic information on handheld devices for use in a mobile context. We consider the benefits of paper, electronic media, and mixing the two in relation to the domain of group navigation. A prototype design is described that attempts to utilize these benefits toward providing a lightweight, ad hoc group navigation support system. Of particular interest is the extent to which an augmented paper map can be used as a shared display, to enhance communication and awareness of the activities of group members while navigating together.

Paper persists as an integral component of active reading and other knowledge-worker tasks because it provides ease of use unmatched by digital alternatives. Paper documents are light to carry, easy to annotate, rapid to navigate, flexible to manipulate, and robust to use in varied environments. Interactions with paper documents create rich webs of annotation, cross reference, and spatial organization. Unfortunately, the resulting webs are confined to the physical world of paper and, as they accumulate, become increasingly difficult to store, search, and access. XLibris ] and similar systems address these difficulties by simulating paper with tablet PCs. While this approach is promising, it suffers not only from limitations of current tablet computers (e.g., limited screen space) but also from loss of invaluable paper affordances.

We present a general web-based information infrastructure capable of supporting the rapid development of highlyinteractive information environments that cater for widely varying requirements across application domains and all forms of fixed and mobile client devices. In particular, we describe how this infrastructure has been extended to support digitally augmented paper through a special transformation component that can map active areas of document pages to information objects so that user- and contextdependent interaction can be supported. Our infrastructure is sufficiently general and flexible to adapt to, not only emerging and even unanticipated technologies in the area of interactive paper, but also the rapidly expanding interaction sphere of hypermedia.

The first steps towards bridging the paper-digital divide have been achieved with the development of a range of technologies that allow printed documents to be linked to digital content and services. However, the static nature of paper and limited structural information encoded in classical paginated formats make it difficult to map between parts of a printed instance of a document and logical elements of a digital instance of the same document, especially taking document revisions into account. We present a solution to this problem based on a model that combines metadata of the digital and printed instances to enable a seamless mapping between digital documents and their physical counterparts on paper. We also describe how the model was used to develop iDoc, a framework that supports the authoring and publishing of interactive paper documents.

Using a participatory design process, we created three prototype augmented laboratory notebooks that provide the missing link between paper, physical artifacts and on-line data. The final a-book combines a graphics tablet and a PDA. The tablet captures writing on the paper notebook and the PDA acts as an "interaction lens" or window between physical and electronic documents. Our approach is document-centered, with a software architecture based on layers of physical and electronic information.

The present study reports on the use of a linguistic category "interrogative," which has been traditionally associated with the act of questioning, and its use in argument talk in Japanese. Based on the observation that interrogative utterances in argument data are regularly followed by non-answers, it is argued that interrogative utterances in argument sequences may not be designed/interpreted as doing questioning. Such use of interrogatives can become an orderly practice to which participants orient themselves in social activities recognizable as arguments. However, though an answer is not expected, the recipient invariably provides some form of response, or the initial speaker seeks such a response when none is provided. Thus the nature of interrogatives as a grammatical category seems to reside in the basic structural unit of social interaction [recipient-oriented action]- [response]. In general, this study is intended to show the dynamically interlocking relationship between grammar and interaction by exploring the intricate interplay between a local action for which interrogative grammar is employed, and the sequential environment and activity framework in which the action takes place.

PACER is a gesture-based interactive paper system that supports fine-grained paper document content manipulation through the touch screen of a cameraphone. Using the phone's camera, PACER links a paper document to its digital version based on visual features. It adopts camerabased phone motion detection for embodied gestures (e.g. marquees, underlines and lassos), with which users can flexibly select and interact with document details (e.g. individual words, symbols and pixels). The touch input is incorporated to facilitate target selection at fine granularity, and to address some limitations of the embodied interaction, such as hand jitter and low input sampling rate. This hybrid interaction is coupled with other techniques such as semi-real time document tracking and loose physical-digital document registration, offering a gesturebased command system. We demonstrate the use of PACER in various scenarios including work-related reading, maps and music score playing. A preliminary user study on the design has produced encouraging user feedback, and suggested future research for better understanding of embodied vs. touch interaction and one vs. two handed interaction.

While popular digital maps support an unprecedented number of use cases, new reference map customization tools have been created for purposes for which those maps fall short. With the goal of informing the design of this new class of cartographic tools, we present the first study of naturalistic (" bespoke ") map customization behavior. Through a mixed methods and mixed-media approach involving a survey, the analysis of a corpus of customized maps, and an interview with a power user, we find that bespoke map customization is a relatively common activity and identify frequent use cases as well as map customization strategies. We discuss these use cases and strategies in detail, and propose design implications for future customization tools, such as the use of templates for common use cases, adaptability for various customization styles and the support of multimedia interaction.

The current environment used by French air traffic controllers mixes digital visualization such as radar screens and tangible artifacts such as paper strips. Tangible artifacts do not allow controllers to update the system with the instructions they give to pilots. Previous attempts at replacing them in France failed to prove efficient. This paper is an engineering paper that describes Strip'TIC, a novel system for ATC that mixes augmented paper and digital pen, vision-based tracking and augmented rear and front projection. The system is now working and has enabled us to run workshops with actual controllers to study the role of writing and tangibility in ATC. We describe the system and solutions to technical challenges due to mixing competing technologies.