Running Head:

PUSHING THE ENVELOPE

Pushing the envelope: How advances in technology are allowing

us to expand our senses beyond our natural limitations.
Bryan Card
Psychology 335, Theories of Personality
University of Maryland, University College

2
PUSHING THE ENVELOPE
Sensation and perception.

Two concepts we take largely for

granted because, like many other complicated yet autonomic

bodily functions, we do not have to spend much effort or
conscious thought to experience our world.

From our eyes and

ears to our tongue and skin, our bodies come equipped with an
array of sophisticated sensors that provide our brain the
impulses it uses to construct our subjective reality (Foley,
2010).

What happens then, when circumstance leaves us without

one of these senses?

Or even more provocative, are the five

senses: sight, hearing, taste, touch, and smell, the only senses
our brains are capable of understanding?

Advances in technology

permit us not only to overcome sensory handicaps, but have the

potential to allow us to expand our senses into previously
unavailable directions.
In his book Sensation & Perception, Hugh Foley defines
sensation and perception as the functioning of our sensory
system, and the interpretation of those sensations, giving
them meaning and organization respectively (Foley, 2010).

In

the field of psychology, experts recognize the poorly defined

boundaries between the two, going so far as to hold that while
sensation is a key component of perception, another and perhaps
the more important component comes from our perceptual
experiences.

3
PUSHING THE ENVELOPE
The reality an organism is capable of perceiving is known
in science as the umwelt, a German word for the surrounding
world.

For most of us, our umwelt consists of everything we can

take in using our five combined senses.

Individuals who have

sensory handicaps, therefore, are denied access to a significant

portion of the human umwelt.

Recent technological advances have

provided the means for us to intervene in those instances where

individuals are denied the experience of an entire sense,
restoring or in some cases granting new access to that missing
piece of their umwelt.
As recently as the year 2000, many scientists questioned
the possibility of marrying technology with the complex
neurochemical system our brain uses to decipher sensory input.
They doubted the capability of our brain to translate signals
from digital media into the electrochemical language of
perception.

The reality is that though the data is expressed

differently, over time, the brain learns to interpret these

signals to provide a sensory experience (Eagleman, 2015).
Take the experience of deafness experienced by 2.1 percent
of Americans in 2012 (Harrington, 2014).

In many of these

cases, both conductive and sensorineural hearing loss can be

corrected with the help of hearing aids.

These devices have

become progressively sophisticated over the years and are widely

4
PUSHING THE ENVELOPE
used to restore the auditory experience of individuals suffering
from a wide variety of hearing loss.
In other instances, the damage is so extensive that the
individual may require a cochlear implant, a device that
consists of a microphone, a signal processor, and a host of
microelectrodes placed throughout the cochlea (Foley, 2010).
These electrodes stimulate the neurons related to the
frequencies present in the sounds being experienced.

While

current cochlear devices only work in instances where the

auditory nerve is undamaged, future implants have the potential
to bypass the nerve altogether and transmit data directly to the
brain (Foley, 2010).
Another example of the technologys capacity to restore a
sense are the ongoing experiments with bionic eyes widely seen
in the news as recently as February.

Officially known as the

Argus II retinal prosthesis, this retinal implant restores

limited vision in the functionally blind (Gross, 2014).

This

particular device is designed to correct issues related to

retinitis pigmentosa, a hereditary ocular condition that effects
more than 1.5 million worldwide (Retinitis pigmentosa, 2012).
Consisting primarily of two elements, an eyeglass mounted
camera and processor and a retinal implant which houses 60
electrodes, this device stimulates the patients retina which,
in turn, sends the resulting information to the brain via the

5
PUSHING THE ENVELOPE
optic nerves.

Though this implant has proven incapable of

restoring sight above the threshold for legal blindness, it has

been observed to improve both mobility and special orientation.
These examples demonstrate the ability of technology to
restore sensation to those individuals whose sensory data
cables are intact; but what of those who are not so lucky?
What of those individuals who by birth or circumstance do not
possess a viable nerve capable of transmitting a sense along
normal pathways?

Technological advances can also allow them the

experience of these missing sensations through a process known

as sensory substitution.
Sensory substitution relies on the principle of brain
plasticity, or neuroplasticity, which describes changes in the
structure of the brain in response to changes in the
environment, neural processes, and occasionally in response to
injury (Pascual-Leone, 2011).

These changes can be miniscule as

those found at the cellular level to the profound vicissitudes

involved in cortical remapping as a result of catastrophic
injuries.

In such instances, researchers have been able to

observe how such changes can fundamentally alter the pattern of

neural activity within the brain (Chaney, 2007).
The pioneer of sensory substitution was Paul Bach-y-Rita,
who constructed a prosthesis he called the Tactile Vision
Substitution System in 1969 which allowed him to transform a

6
PUSHING THE ENVELOPE
sensory input from one modality, vision, into another modality,
touch (Hardy, 2007).

Though counterintuitive, with practice

this tactile prosthesis granted his test subjects an experience

of sight.

Through this revolutionary technique, Bach-y-Rita

confirmed what was seen as a radical theory, that our brain

could decipher sensory signals even if they arrived via
unconventional pathways.
Currently, there are many projects exploring the idea of
sensory substitution.

A surprising amount focus on taking

sensory input from one modality, be it visual or auditory, and

transforming it into a tactile sensation.

Many of these systems

utilize the tongue as the interface between these disparate

modalities due to the interface being protected as well as the
ease at which electrical current can be transferred due to the
presence of saliva (Bach-y-Rita, 1998).
Another interesting study is that of Neil Harbisson, a
British contemporary artist and self-described cyborg activist
who currently lives in New York, NY.

Born with an extreme form

of color blindness, Harbisson experienced the world in greyscale

until 2004 when he underwent an operation to implant an antenna
into his skull which would allow him to perceive light as sound.
Through a process known as osseointegration, this eyeborg is
connected to his occipital bone and allows him to hear light

7
PUSHING THE ENVELOPE
frequencies including invisible spectrums like ultra violet and
infrared (Harbisson, 2012).
So far, we have contented ourselves to restoring the five
senses, but what happens when we allow our imaginations to
transcend the typical human experience?
As human beings, we are subject to the biological
limitations of our sensory organs.

Our eyesight, for example,

which allows us to experience a vivid spectrum of color, is

incapable of discerning the vast majority of frequencies present
in our objective reality.

Our sense of smell pales in

comparison to such olfactory titans as the loyal bloodhound that

sleeps at the foot of our bed.

Through technology can we expand

our very umwelt?

Recent advances in neuroscience have demonstrated that
though human beings are limited by our evolutionary biology, our
brains have the capacity to analyze the pattern and make sense
of data from non-traditional sensory input.
David Eagleman describes the potential for sensory addition
in his recent TED talk, Can we create new senses for humans

heres the big secret:

seeing any of this.

Your brain is not hearing or

Your brain is locked in a vault of

silence and darkness inside your skull.

All it ever sees

are electrochemical signals that come in along different

8
PUSHING THE ENVELOPE
data cables, and this is all it has to work with, and
nothing more.

Now, amazingly, the brain is really good at

taking in these signals and extracting patterns and

assigning meaning, so that it makes this inner cosmos and
puts together a story of this, your subjective world
(Eagleman, 2015).
He goes on to discuss an experiment currently being
conducted at the Baylor College of Medicine, where a student is
experiencing a live data stream from the internet via a vest
that provides a tactile sensation.

This data is transmitted for

five seconds, and the subject is presented with two buttons.

After choosing one, he receives feedback on the choice.
Unbeknownst to the subject, the sensation he is
experiencing is real-time data from the stock market and the
choices hes making are to buy or sell stocks.

The purpose of

this experiment is to see if the subject can develop a direct

perceptual experience of the economic movements of the planet
(Eagleman, 2015).
Beyond the capabilities for financial insights, this
technology offers a new way to experience the staggering amounts
of data we are currently inundated with.

Eagleman imagines

similar vests being worn by astronauts to allow them to

experience the relative condition of their spacecraft, by
factory managers to experience the state of their facilities,

9
PUSHING THE ENVELOPE
and even used by everyday individuals to monitor their health in
real time.
Although the quest to restore missing senses to those with
perception handicaps would once be viewed as groundbreaking,
advances in technology already allow us to repair senses damaged
by injury and disease that were unthinkable only a few years
ago.

As recently as fifteen years ago, researchers doubted we

would be able to develop technology that would allow digital

signals to be transmitted to and translated by the brain to
restore lost sensations.

Even more exciting, through sensory

substitution, we have technology that can allow an individual

who has been blind since birth to see through an implant on
their tongue!
Not only are we currently capable of restoring lost and
damaged senses, researchers such as Eagleman continue to push
the envelope by developing the means to perceive signals that
lay outside of the typical human umwelt.

From the ability to

experience the ultra violet and infrared spectrum though an

auditory implant, to a vest that gives us the ability to
experience the tactile sensations of the worlds stock markets,
these technologies are no longer the stuff of science fiction
and fantasy, they are the science of now.

10
PUSHING THE ENVELOPE

References
Chaney, Warren, Dynamic Mind, 2007, Las Vegas, Houghton-Brace
Publishing, pp 33-35, ISBN 0-9793392-0-0 [1]
D'Amico, A., & Di Natale, C. (2014). Beyond Human Senses:
Technologies, Strategies, Opportunities, and New
Responsibilities.
Darrah, M. (2012). The Use of Touch Technology in Science to Provide
Access for Students who are Visually Impaired. Journal Of
Technology Integration In The Classroom, 4(1), 45-50.
Desch, L. W. (1986). High Technology for Handicapped Children: A
Pediatrician's Viewpoint. Pediatrics, 77(1), 71
Eagleman, D. (2015, March). David Eagleman: Can we create new senses
for humans? [Video file]. Retrieved from
http://www.ted.com/talks/david_eagleman_can_we_create_new_senses
_for_humans
Foley, H., & Matlin, M. (2010). Sensation & Perception (5th ed.).
Boston, MA: Pearson Education Inc., publishing as Allyn & Bacon.
Gross, R. (2014, August 31). A Bionic Eye That Restores Sight.
Retrieved May 8, 2015, from
http://www.theatlantic.com/health/archive/2014/08/a-bionic-eyethat-restores-sight/378628/
Hakobyan, L., Lumsden, J., OSullivan, D., & Bartlett, H. (2013).
Mobile assistive technologies for the visually impaired. Survey
Of Ophthalmology, 58(6), 513-528.
doi:10.1016/j.survophthal.2012.10.004
Harbisson, N. (2012, June). Neil Harbisson: I listen to color
[Video File]. Retrieved from
http://www.ted.com/talks/neil_harbisson_i_listen_to_color
Hardy, B., Ramanantsoa, M., Hanneton, S., Lenay, C., Gapenne, O., &
Marque, C. K. (2007). Sensory Substitution: Perception Dedicated
to Action.
Harrington, T. (2014, February 1). Deaf Statistics. Retrieved May 11,
2015, from

11
PUSHING THE ENVELOPE
http://libguides.gallaudet.edu/content.php?pid=119476&sid=102919
0
Lobo, L., Travieso, D., Barrientos, A., & Jacobs, D. M. (2014).
Stepping on Obstacles with a Sensory Substitution Device on the
Lower Leg: Practice without Vision Is More Beneficial than
Practice with Vision. Plos ONE, 9(6), 1-12.
doi:10.1371/journal.pone.0098801
Nicolelis, M. (2014, October). Miguel Nicolelis: Brain-to-brain
communication has arrived. How we did it [Video File].
Retrieved from
https://www.ted.com/talks/miguel_nicolelis_brain_to_brain_commun
ication_has_arrived_how_we_did_it
Novich, S. D., & Eagleman, D. M. (2014). [D79] A vibrotactile sensory
substitution device for the deaf and profoundly hearing
impaired. 2014 IEEE Haptics Symposium (HAPTICS), 1.
doi:10.1109/HAPTICS.2014.6775558
Pascual-Leone, A., Freitas, C., Oberman, L., Horvath, J. C., Halko,
M., Eldaief, M. et al. (2011). Characterizing brain cortical
plasticity and network dynamics across the age-span in health
and disease with TMS-EEG and TMS-fMRI. Brain Topography, 24,
302-315. doi:10.1007/s10548-011-0196-8
Retinitis pigmentosa - Prevalence. (2012, February 9). Retrieved May
10, 2015, from http://www.institutvision.org/index.php?option=com_content&view=article&id=204&Item
id=66&lang=en&limitstart=1
Vitaliev, D. (2009). The eyeborg man. Engineering & Technology
(17509637), 4(8), 26-28. doi:10.1049/et.2009.0807
(2014). G2: 'I wanted to be a different kind of human': Thanks to the
antenna implanted in his skull, Neil Harbisson can hear images
and paint sounds. The world's first cyborg artist tells Stuart
Jeffries why ringtones are green and Amy Winehouse is red and
pink. The Guardian (London, England).