Mobile Application Model for the Blind
Jaime Sánchez, Mauricio Sáenz, and Nelson Baloian
Department of Computer Science
University of Chile
{jsanchez,msaenz,nbaloian}@dcc.uchile.cl
Abstract. This study presents a model to design and implement mobile
applications to support the displacement and dynamic decision making of users
with visual disabilities. To identify the real added value of using mobile
technologies as support aids for decision making in dynamic contexts for users
with visual disabilities, we provide an application case. By using a graph to
represent the computer model of a real school for blind children, for whom a
system was already developed using our model, we provide a real example
application of this model. This provided enough input to enrich, improve and
redesign the model; ending up with a usable mobile application model to assist
the mobility and orientation of blind users.
Keywords: Mobile, learning, model, software, blind learners.
1 Introduction
The massive use of mobile tools in every life has motivated researchers and
implementers to design software applications in order to provide users with high
demand data processing services always available including useful and upgraded
information. Diverse efforts have been made to give accessibility to these applications
for users with visual disabilities.
Diverse applications have offered critical improvements in accessibility [7], [20]
and learning [5], [9], [14], [11], [17] for users with visual disabilities. However, these
applications have been designed for users in a rather static working context without
integrating dynamic and upgraded information about the surrounding area. The world
around us changes constantly with irregular situations that modify and alter it through
time. Sighted people can easily order the environment through the use of the visual
sense, but people with visual disability can hardly make decisions when unexpected
situations occur. Thus, the development of technology for users with visual
disabilities should imply user-centered methodologies, identifying clearly interaction
modes and their immediate consequences in the user’s performance.
Educational software for people with visual disabilities usually lacks of critical
interface elements commonly present in software for sighted children. Most software
does not include things such as explicit model knowledge and skills learners to be
enhanced when using the software, explicit learner model, and appropriate feedback
to improve the learners' performance. To many authors, designers of educational
C. Stephanidis (Ed.): Universal Access in HCI, Part I, HCII 2007, LNCS 4554, pp. 527–536, 2007.
© Springer-Verlag Berlin Heidelberg 2007
528
J. Sánchez, M. Sáenz, and N. Baloian
software for children with disabilities conceive the software with interaction
constrains in their minds, fixing the interaction modes from the very beginning.
Educational software for learners with visual disabilities should design without
considering from the beginning the users' disabilities. Rather, they should take into
consideration aspects such as a model representing existing or artificial interacting
world, a model for representing the knowledge to be learned, and model for the
learner. The learner’s capabilities and disabilities should be considered when mapping
inputs and outputs of the model into an interface.
Since educational software development depends on people, tools, and
methodologies involved, and considering there is no a clear methodology to carry out
this process for children with visual disabilities, the results mainly depends on the
skills of the involved people. This situation can cause many drawbacks typical of a
handcrafted process. Rather, software engineering uses methodologies to help to
reduce the craftsmanship level of software development by using the best
methodological practices.
The navigation of blind users in real environments exposes them to higher risks
than sighted users because they cannot use the physical cane to help them to identify
objects in the space. For this reason, some cues should be provided to users with
visual disabilities to get a more reliable mobility, allowing them to access to richer
information from the environment [4],[5],[6]. Then, it is necessary to extend the
model to mobile contexts, to allow designer to create educational software for mobile
devices.
We present a model that extend the process model already described in Sánchez
et al. [10],[13], by instantiating it for mobile contexts. This extension involves users
with visual disabilities in contexts and situations behaving autonomously. Thus,
through the support of digital technology they can define strategies to solve problems
that sighted users can solve rapidly by using their vision input.
2 Related Work
Burgstahler [1] proposes a way technology can be used by learners with visual
disabilities, allowing them independence, productivity, and participation in academic
activities and everyday life, proposing the following ways technology can provide to
people: (a) Gain access to the full range of educational options, (b) Participate in
experiences not otherwise possible, (c) Succeed in work-based experiences,
(d) Secure high levels of independent living and (e) Work side-by-side with peers.
Roschelle et al. [8] recommend avoiding a process of software development
centered on just software professional by including multidisciplinary teams in the
design and development, concentrating in specific domain aspects. The main idea is
to produce specific domain software components to better manage the expected
results of educational software.
We have described elsewhere a model for educational software for people with
visual disabilities [13]. It is mainly based on tasks to be solved by the user. The model
consists of the stages of analysis, design, development, and validation (see Figure 1).
Analysis. Cognitive goals to be achieved by the learner are designed as well as the
definition of software requirements. Cognitive evaluations define procedures and
Mobile Application Model for the Blind
529
functions to evaluate the achievement of cognitive goals. Design. A metaphor is
defined for a virtual environment or scenario where the learner constructs knowledge
through interaction within the virtual world. Normally, this is gaming software so the
playing rules are defined. All of this leads to define the model of the virtual world and
the knowledge to be constructed. Development. This stage implies three subprocesses: first, the computational implementation of models of the world and learner,
second, the implementation of the evaluation process and feedback to the student, and
third, the projection of the models. It is important to implement these actions after
setting the model to avoid constraining from the beginning the software design.
Validation. This stage consists of two sub-processes. First, we develop usability tests
to gather data about how well the system fit our objectives in order to attain the
cognitive goals set at the beginning. In this stage, the emphasis is on the analysis of
some elements of human-computer interaction. Second, we analyze these results and
study how the metaphor, models, and the projection of input/output variables can be
improved. An error in the integrity of the system for learning can imply to review the
metaphor and models used. Usability issues can lead to review the projection.
Fig. 1. Schema of model for educational software [10]
Diverse intents have been made to adapt mobile devices to the requirements of
users with visual disabilities, obtaining hardware prototypes oriented to specific users.
For instance, a pocketPC specially targeted for blind people contains screen readers
with buttons instead of a tactile screen [3]. Wobbrock proposes the use of specific
hardware to complement the features of standard pocketPC. However, this adaptation
has two major constrains: the high cost and the loss of mobility when using the
mobile device.
Sánchez and Aguayo [12] emphasize the development of two interface modules to
allow interaction with a pocketPC by users with visual disabilities. The first is the
input module consisting in a virtual keyboard of nine buttons placed on the tactile
screen of the device, letting users to write in the same way as they do with their
530
J. Sánchez, M. Sáenz, and N. Baloian
cellular phone without needing external devices. The second is the output module
based on a text-to-speech engine adapted to the user needs.
AudioStoryTeller [15] is a pocketPC application to support the development of
reading and writing skills in learners with visual disabilities through storytelling,
providing diverse evaluation tools to measure these skills. This software application
has been implemented through auditory interfaces to support actions and provide
feedback.
Software for a pocketPC, mBN [16], contains a metaphor that represents a
simulation of a subway travel through a wagon. Travels are developed in a logical
way without considering spatial representations of virtual spaces. The metaphor
considers notions of consecutive stations, transfer stations, and terminal stations.
Interaction is achieved by using the corners of the pocketPC screen, joining the
adjacent points. Hence, the software watches, analyzes and interprets the movements
of the pointer without special devices.
The system PGS (Personal Guidance System) proponed by Loomis [2] was
developed as an outdoor navigation tool depending completely on a GPS. The system
provides instructions to mobilize from one point to the other through verbalized
instructions and descriptions by using dual earphones.
The navigation and guidance system NOPPA [19] is oriented for blind user to
travel through the city without interruptions by using buses, trains, and walking. It is
based on a client-server architecture accessing to information through internet from a
terminal. Information is communicated through voice synthesizers.
In [2] is presented PINS (Personal Indoor Navigation System), an indoor
navigation system to provide an independent and efficient navigation to users with
visual disabilities. The system allows to solve navigation tasks and route plans, but is
does not include obstacles avoidance. PINS use a positioning and orientation system,
a spatial data base and a user interface. The input of information is through audio and
a Braille keyboard, allowing the downloading of navigation maps when entering to a
place.
Other aids include tactile maps that through talked voices provide directions. These
maps have low resolution, hard to get and use while the user is moving, so they don’t
solve users’ orientation problems [20].
Finally, a related work was done by Sasaki [18], presenting how mobile technology
can support mobility and orientation of visually impaired users when utilizing public
transportation buses.
3 A Model for Developing Mobile Applications for Blind People
The need for models to develop educational software has already been recognized in
order to generalize and replicate good practices. We have designed a model for
developing software for mobile devices targeting people with visual disabilities when
learning how to mobilize more efficiently to accomplish certain tasks in a known area
(see Figure 2). This model is based on the initial model presented in chapter 2 [11]
aiming to develop educational software for people with visual disabilities. Since our
goals are focused on software for the mobile scenarios we can instantiated the model
for mobile purposes and make a detailed description of its parts.
Mobile Application Model for the Blind
531
Fig. 2. Mobile Application Model
The model itself is a workflow consisting of four principal phases:
Analysis: This phase takes as input the learning goal of the application. This learning
goal will define the cognitive tasks the learner should acquire by using the system.
Based on these cognitive tasks, we can specify the concrete tasks the user has to
perform with the help of the system in order to reach the learning goals. The cognitive
goals established in this phase will be later used to produce the metrics in order to
evaluate the correct achievement of these tasks and will serve for validating the
application in its effectiveness.
Modeling: In this phase the tasks supported and the environment are modeled. The
main product of this task is the Model. It is the computer’s internal representation of
the knowledge the user has to acquire. For the mobile case, this is the real world
including the environment in which the user should move and all the relevant
information needed to accomplish the tasks defined in the previous phase. A good
way to computationally represent such an environment is using a graph. In such a
graph the nodes are called spots, and represent only the relevant objects of the
environment the user can recognize (a corner, a booth, or even a trash bin if relevant)
which can be used by the user to recognize the place. The links of the graph are called
ways, including relevant information about the characteristics of shortcuts paths to go
from one spot to another. The type and amount of information of ways depends on the
kind of the cognitive task the system is assisting the user to accomplish. In some
cases, relevant information may be the distance between spots measured in meters, or
532
J. Sánchez, M. Sáenz, and N. Baloian
the average time a user would take to go from one spot to the other. In other cases a
number identifying the user’s degree of confidence to cover the distance or the safety
degree of walking through that way may be useful. (see figure 3).
Fig. 3. An example of a graph representing spots and ways. The values l, d, p, t represent the
evaluation of different characteristics of the way, such as the time needed to cover it, the
distance, the degree of safety, etc.
Having a graph representing the real environment where the user has to accomplish
tasks makes it easy for the computer to calculate all possible paths (collection of
ordered ways) for moving from one spot to another by applying well known
algorithms over graphs like Dijksta’s. Applying different values of the ways as input
for the algorithm, we can find different kind of paths between two spots. For example,
using the distance value we can find the shortest path, using the time value we can
find the faster path. Moreover, the system can find the best path for a combination of
safety and speed by giving weights to these numbers.
As already described in [2] a very important issue of the modeling phase is to
establish the state, input, and output variables. State variables are those describing the
state where the user is. A very important state variable of the model is the user’s
current position, which can be on a spot or on a way. The mechanism to update this
variable depends on the environment where the user moves and the resources
available. Whenever the distances are long enough, a GPS mechanism can be used.
The user must update this variable otherwise. Input variables are those received from
the user and/or the environment to update the state variable. Output variables are
those used by the system to provide relevant information to the user.
The model component should also include a user model. A user model is the
information the system uses to compute what the user knows, does not know, and if
the user has wrong knowledge about something. With this information the system
should be able to assist the user providing the right information at the right time to
accomplish the task. For the mobile case, the simpler, yet most effective way to
implement a model of use is using the overlay technique. In this case, an overlay user
model will be a graph containing the collection of spots and ways he or she already
knows. Figure 4 shows the graph representing the computer model of a real school for
blind children in Santiago, Chile, for which a system was already developed using this
Mobile Application Model for the Blind
533
model. The ways and spots belonging to both, the user’s model graph and the
computer’s knowledge representation graph, are shown in white color. The ways
belonging to the computer’s graph are shown in black
Fig. 4. The figure shows a scale model of the school for blind children and both graphs for
implementing the computer’s knowledge model (all nodes and lines) and the user’s model (only
white lines and nodes)
The user’s model can be initialized with the graph representing the spots and ways
the user usually uses to move from one point to another. It should also be updated
when the user uses a new way between two spots. This represents a learning step of
the user. By overlaying the user model’s graph to the one representing the computer’s
knowledge, it easy for the system to decide whether at a certain point it can provide
useful information to the user. The system computes the most convenient path from
that point to the target arrival point. If the first way in the path is not on the user
model, the system can provide this information to him or her.
Development: Graphs along with algorithms and procedures are implemented. The
methodologies and procedures for evaluating the user’s performance are also
developed. The evaluation methodologies will depend on the cognitive tasks
supported by the system. For example, if the original task is to find the faster way
between two places, the evaluation should consider the time invested by the user and
compare it against the “normal” time an average user will need when using the fastest
path. If the task is to find the shortest path, then the system will have to consider the
length of the path used and contrast it with the shortest path the system can compute
using the algorithms. The projection component of the development phase is the
process of mapping input and output variables on proper input and output devices of
the mobile computing device. Since mobile devices have more restricted possibilities
for good input and output, especially for people with visual disabilities, in most cases,
the input variables will be mapped on haptic devices (such as buttons or keys) and the
output devices on audio signals. Considerations that should be regarded for
534
J. Sánchez, M. Sáenz, and N. Baloian
implementing these projections are described in [1] for the non-mobile scenario. In
the mobile scenario we should also consider that audio output must be even clearer
and concise because of the presence of environmental noise.
Validation: During the validation phase the usability and effectiveness of the
software is evaluated. The usability is for the user’s acceptance and how well he or
she interacts with the model. This will depend on how good input and output variables
of the model have been projected on the input and output possibilities available for a
mobile case. The effectiveness is evaluated by finding out the user’s performance
with and without the application. The first step is it to define the tasks evaluation
strategies and methodologies in accordance with the tasks defined in the analysis
phase. These will in turn serve as input to develop the evaluation methodologies for
the user’s performance and at the same time, to evaluate whether the system makes a
contribution to the performance of the user.
4 Conclusions
This study introduces a model for implementing mobile applications for users with
visual disabilities. We have also provided an application example by using a graph to
represent the computer model of a real school for blind children, for which a system
was already developed using our model.
We are implementing a research study to applying this model to real mobile cases
such as the subway, neighborhood, and the school. The result of this new study will
validate more fully our model for designing mobile applications for decision making
in dynamic contexts.
The use of mobile applications should not be reduced to sighted users even though
the whole interface is thought for their mental models. There is no need for producing
devices specially tailored for them that have a high cost because of the limited target
population. The design of interfaces to provide efficient input/output access and
interaction is sufficient to exploit the potential of these devices in the everyday life of
blind users.
Software applications should be implemented specially tailored for users with
visual disabilities. It is not enough to adapt existing software; rather there is a huge
need for software specially tailored for the needs and mental models of users with
visual disabilities. This will allow these users to take advantage of the benefits of
pocketPC devices closing the gap between the unique features of mobile technologies
and the actual use by these users.
The model proposed here allows designing diverse applications to help users with
visual disability to improve their mobility and orientation skills in everyday contexts.
The availability of a mobile aid that provides information about shortcuts and
efficient ways and routes, helps them to mobilize efficiently and autonomously in
different scenarios and thus helping them to become more socially included.
Acknowledgements. This report was funded by the Chilean National Fund of Science
and Technology, Fondecyt, Project 1060797.
Mobile Application Model for the Blind
535
References
1. Burgstahler, S.: The role of technology in preparing youth with disabilities for
postsecondary education and employment. A white paper developed for the PostOutcomes Network of the National Center on Secondary Education and Transition
(NCSET) at the University of Hawaii at Manoa. (2002) Available on-line at
http://www.ncset.hawaii.edu/publications/pdf/role_of_technology.pdf
2. Cheung, S., De Ridder, S., Fishman, K., Francle, L., Patterson, J.: A Personal Indoor
Navigation System (PINS) for people who are blind. Last Access (18/07/06) Available online at http://vision.psych.umn.edu/ gellab/5051/prev_projects/pgs1.pdf
3. Geekzone: Last Access (29 January, 2007), (2006) http://www.geekzone.co.nz/
content.asp?contentid=2976
4. Koruda, T., Sasaki, H., Tateishi, T., Maeda, K., Yasumuro, Y., Manabe, Y., Chihara, K.:
Walking aids based on wearable/ubiquitous computing – aiming at pedestrian’s intelligent
transport systems. In: Proceedings of the IV International Conference Disability, Virtual
Reality & Associated Technologies, ICDVRAT 2002. Veszprém, Hungary, pp. 117–122
(2002)
5. Lahav, O., Mioduser, D.: Blind Persons’ Acquisition of Spatial Cognitive Mapping and
Orientation Skills Supported by Virtual Environment. In: Proceedings of the 5th
International Conference on Disability, Virtual Reality and Associated Technologies,
ICDVRAT 2004, pp. 131–138 Oxford, UK (2004)
6. Lahav, O., Mioduser, D.: Multisensory virtual environment for supporting blind persons’
acquisition of spatial cognitive mapping, orientation, and mobility skills. In: Proceedings
of the 4th International Conference on Disability, Virtual Reality and Associated
Technologies, ICDVRAT 2002, Veszprém, Hungary pp. 213–220 (2002)
7. Plamon, O., Oxman, R., Shahar, M., Weiss, P.L.: Virtual environment as an aid to the
design and evaluation of home and work settings for people with physical disabilities. In:
Proceedings of The 5th International Conference on Disability, Virtual Reality and
Associated Technologies, ICDVRAT 2004, Oxford, United Kingdom pp. 183–189
(September 20–22, 2004)
8. Roschelle, J., DiGiano, C., Koutlis, M., Repenning, A., Jackiw, N., Suthers, D.:
Developing educational software components. IEEE Compute 32(9), 50–58 (1999)
9. Sánchez, J., Flores, H.: AudioMath: Blind Children Learning Mathematics through Audio.
In: Proceedings of The 5th International Conference on Disability, Virtual Reality and
Associated Technologies, ICDVRAT 2004, Oxford, United Kingdom, pp. 183–189
(September 20–22, 2004)
10. Sánchez, J., Baloian, N.: Modeling 3D interactive environments for learners with visual
disabilities. In: Miesenberger, K., Klaus, J., Zagler, W., Karshmer, A.I. (eds.) ICCHP
2006. LNCS, vol. 4061, pp. 1326–1333. Springer, Berlin Heidelberg (2006)
11. Sánchez, J.: Interactive Environments for Blind Children: Computing, Usability, and
Cognition. In: Proceedings of the International Conference on New Technologies in
Science Education (I), Aveiro, Portugal, pp. 17–27 (July 4-6, 2001)
12. Sánchez, J., Aguayo, F.: Mobile Messenger for the Blind. In: Stephanidis, C., Pieper, M.
(eds.) EP 2007. LNCS, vol. 4397, pp. 369–385. Springer, Berlin Heidelberg (2007)
13. Sánchez, J., Baloian, N.: Modeling Audio-Based Virtual Environments for Children with
Visual Disabilities. In: Proceedings of the World Conference on Educational Multimedia,
Hypermedia & Telecommunications ED-MEDIA 2005. Montreal, Canadá. pp. 1652–1659
(June 27 - July 2, 2005)
536
J. Sánchez, M. Sáenz, and N. Baloian
14. Sánchez, J., Flores, H.: Training Blind Children to Develop Mathematics Skills Through
Audio. In: Proceedings of the Cybertherapy 2005, Basel, Switzerland, pp. 123–124 (June
6-10, 2005)
15. Sánchez, J., Galaz, I.: AudioStoryTeller: Enforcing Blind Children Reading Skills. Lecture
Notes in Computer Science, LNCS (2007) (in press)
16. Sánchez, J., Maureira, E.: Subway Mobility Assistance Tools for Blind Users. In:
Stephanidis, C., Pieper, M. (eds.) EP 2007. LNCS, vol. 4397, pp. 386–404. Springer,
Berlin Heidelberg (2007)
17. Sánchez, J., Sáenz, M.: 3D Sound Interactive Environments for Problem Solving. In:
Proceedings of The Seventh International ACM SIGACCESS Conference on Computers
and Accessibility, Assets 2005, Baltimore, Maryland, USA, pp. 173–178 (October 9-12,
2005)
18. Sasaki, H., Tateishi, T., Kuroda, T., Manabe, Y., Chihara, K.: Wearable computer for the
blind–aiming at a pedestrian’s intelligent transport system. In: Proceedings of the 3rd
International Conference on Disability, Virtual Reality and Associated Technologies,
ICDVRAT 2000, pp. 235–241 (2000)
19. Virtanen, A., Koskinen, S.: NOPPA: Navigation and Guidance System for the Blind. Last
Access (30/06/06) Available on-line at http://virtual.vtt.fi/noppa/noppa%20eng_long.pdf
20. Vogel, S.: A PDA-Based Navigation System for the Blind. M.S. Integrative Paper.
(Retrieved April 6, 2006), from http://www.cs.unc.edu/ vogel/IP/IP/IP_versions/
IPfinal_SuzanneVogel_Spring2003.pdf
21. Wobbrock, J., Myers, B., Aung, H.: Writing with a joystick: a comparison of date stamp,
selection keyboard, and EdgeWrite. In: Proceedings of the 2004 conference on Graphics
interface GI ’04, Canadian Human-Computer Communications Society (2004)