Sixth Sense Technology

Electronics & Communication Submitted by:-

Department Ms Anshita Mishra

HOD- Dr P. K. Jain EC-1 SEM III

Faculty-Mr. Puneet Namdeo

ACKNOWLEDGEMENT
1|Page
I am heartily thankful to my supervisor,
Mr. Puneet Namdeo, whose
encouragement, guidance and support
from the initial to the final level enabled
me to develop an understanding of the
subject.

Lastly, I offer my regards and blessings

to all of those who supported me in any
respect during the completion of the
project.

Anshita Mishra

2|Page
 INDEX

ABSTRACT.............................................................................4
INTRODUCTION...................................................................5
HISTORY................................................................................6
WHY THE NAME..................................................................7
‘SIXTH SENSE’......................................................................7
WORKING PRINCIPLE.......................................................7
Components...........................................................................8
APPLICATIONS...................................................................11
SOCIETAL ISSUES.............................................................12
CONCLUSION......................................................................13
ADVANTAGES AND ENHANCEMENTS........................13
Advantages...........................................................................14
Future Enhancements.........................................................15
BIBLIOGRAPHY.................................................................16

ABSTRACT

3|Page
'Sixth Sense' is a wearable gestural interface that augments the physical
world around us with digital information and lets us use natural hand
gestures to interact with that information. It is a sense that would give
seamless and easy access to Meta information or information that may
exist somewhere. The Sixth Sense prototype is comprised of a pocket
projector, a mirror and a camera. The hardware components are
coupled in a pendant like mobile wearable device. Both The projector
and the camera are connected to the mobile computing device in the
User’s pocket. The projector projects visual information enabling
surfaces, walls and physical objects around us to be used as interfaces.
‘Sixth Sense’ frees information from its confines by seamlessly
integrating it with reality, and thus making the entire world Your
Computer. Integrating information to our everyday objects will not only
help us to get rid of the gap between the physical world and digital
World, but it will also help us in some way to stay human, to be more
connected to our physical world. The goal is to bring a part of the
physical world to digital world to make computing devices more
computing devices more intuitive. The reverse process is also made
possible. In this paper, we explained the sixth sense device, its working
and the various applications that demonstrate the usefulness, viability
and flexibility of the system also providing information about its
availability and adoptability.

INTRODUCTION
We've evolved over millions of years to sense the world around
us. When we encounter something, someone or someplace, we
use our five natural senses to perceive information about it;

4|Page
that information helps us make decisions and chose the right
actions to take. But arguably the most useful information that
can help us make the right decision is not naturally perceivable
with our five senses, namely the data, information and
knowledge that mankind has accumulated about everything
and which is increasingly all available online. Although the
miniaturization of computing devices allows us to carry
computers in our pockets, keeping us continually connected to
the digital world, there is no link between our digital devices
and our interactions with the physical world. Information is
confined traditionally on paper or digitally on a screen. Most of
the people have some portable technology like smart phones,
laptops, e-book and iPods. But you need to carry all the stuff
with you. We grew up interacting with the physical objects
around us. And there are enormous numbers of them that we
use every day. Unlike our most computing devices these objects
are much more fun to use. When you talk about objects one
another thing that automatically comes to that thing is gestures
how we manipulate the objects. We use gestures not only to
interact with each other but also to interact with each other.

5|Page
 HISTORY
The exploration of this sixth sense technology started with a
question why can’t we use a computer to interact with physical
world. This literally started with a mouse that comes with a ball
in it and TWO ROLLERS that actually guides the movement. So
taking out rollers from another mouse totally we have four
rollers. Using these, a system interface device was formed.
With this, whatever you do in physical world will be reflected in
the digital world. With this initially a pen to draw 3-dimensions
in digital world was made that can help designers and
architects not only Think but they can actually draw so that it is
more intuitive to use that way. The goal is to bring part of
physical world to digital world to make computing devices more
intuitive. Not only this, the reverse process is made possible.
Initially the device started with a helmet consisting of the
camera, projector and color markers. Later it was converted
into a form so that it is easy to use it by
wearing around the neck. Sixth Sense bridges this gap, bringing
intangible, digital information out into the tangible world, and
allowing us to interact with this information via natural hand
gestures. ‘Sixth Sense’ frees information from its confines by
seamlessly integrating it with reality, and thus making the
entire world your computer.

6|Page
 WHY THE NAME
‘SIXTH SENSE’
From thousands of years mankind used the
five senses to gain information. But now as this device is completely
portable and it can provide information about anything, it
is named as ‘SIXTH SENSE DEVICE’.

WORKING PRINCIPLE
The Sixth Sense prototype is comprised of a pocket projector, a mirror
and a camera. The hardware components are coupled in a pendant like
mobile wearable device. Both the projector and the camera are
connected to the mobile
computing device in the user’s
pocket.

7|Page
The projector projects visual information enabling surfaces, walls and
physical objects around us to be used as interfaces; while the camera
recognizes and tracks user's hand gestures and physical objects using
computer-vision based techniques. The software program processes
the video stream data captured by the camera and tracks the locations
of the colored markers (visual tracking fiducially) at the tip of the user’s
fingers using simple computer-vision techniques. The movements and
arrangements of these fiducially are interpreted into gestures that act
as interaction instructions for the projected application interfaces. The
maximum number of tracked fingers is only constrained by the number
of unique fiducially, thus Sixth Sense also supports multi-touch and
multi-user interaction.

Components
The hardware components are coupled in a pendant like mobile
wearable device.
 Camera
 Projector
 Mirror
 Mobile Component
 Color Markers

The entire hardware apparatus is encompassed in a pendant-shaped

mobile wearable device. Basically the camera recognizes individuals,
images, pictures, gestures one makes with their hands and the
projector assists in projecting any information on whatever type of
surface is present in front of the person. The usage of the mirror is

8|Page
significant as the projector dangles pointing downwards from the neck.
To bring out variations on a much higher plane, in the demo video
which was broadcasted to showcase the prototype to the world, Mistry
uses colored caps on his fingers so that it becomes simpler for the
software to differentiate between the fingers, demanding various
applications. The software program analyses the video data caught by
the camera and also tracks down the locations of the colored markers
by utilizing single computer vision techniques. One can have any
number of hand gestures and movements as long as they are all
reasonably identified and differentiated for the system to interpret it,
preferably through unique and varied fiducially. This is possible only
because the Sixth Sense device supports multi-touch and multi-user
interaction. MIT basically plans to augment reality with a pendant
picoprojector: hold up an object at the store and the device blasts
relevant information onto it (like environmental stats, for instance),
which can be browsed and manipulated with hand gestures. The "sixth
sense" in question is the internet, which naturally supplies the data,
and that can be just about anything -- MIT has shown off the device
projecting information about a person you meet at a party on that
actual person (pictured), projecting flight status on a boarding pass,
along with an entire non-contextual interface for reading email or
making calls. It's pretty interesting technology that, like many MIT
Media Lab projects, makes the wearer look like a complete dork -- if the
projector doesn't give it away, the colored finger bands the device uses
to detect finger motion certainly might. The idea is that Sixth Sense
tries to determine not only what someone is interacting with, but also
how he or she is interacting with it. The software searches the internet
for information that is potentially relevant to that situation, and then

9|Page
the projector takes over. All the work is in the software," says Dr Maes.
"The system is constantly trying to figure out what's around you, and
what you're trying to do. It has to recognize the images you see, track
your gestures, and then relate it all to relevant information at the same
time."

10 | P a g e
APPLICATIONS
The Sixth Sense prototype implements several applications that
demonstrate the usefulness, viability and flexibility of the
system. The Sixth Sense device has a huge number of
applications. The following are few of the applications of Sixth
Sense Technology.

 Call up a map
 Check the time
 Create multimedia reading experience
 Drawing application
 Zooming features
 Get product information
 Get book information
 Get flight updates
 Feed information on people
 Take pictures
 Check the email
 Make a call


11 | P a g e
SOCIETAL ISSUES

This technology in essence seems fantastic; however, there are

a number of societal issues that come with it. The two major
ones are: the accuracy of the information and the privacy of
individuals. The first major issue which this new technology
arises is how accurate the information displayed is. For
example, if you wish to see the environmental statistics of
certain product, what guarantees that the information
displayed will be, at least, relatively accurate? If the company
which produces the product were to make sure to not let
certain information be displayed the product might seem
harmless to the environment when it really isn’t. The second
major issue that arises from "Sixth Sense" is the issue of
personal privacy. The project is being promised to be able to
get relevant information of people you meet. For example, if
you meet someone at a party you would get some relevant
facts about that person. The problem here is how much
"relevant facts” there are. Even if only the name and the
birthday of the person is displayed, this would facilitate identity
theft immensely not only because the information is readily
available but also because it would be much harder to track the
culprit since this technology would be used by a number of
people. The possible solutions to this, however, are
unfortunately not much different from the solutions to internet
privacy that is to not provide any personally identifiable
12 | P a g e
information. Of course, not even this guarantees personal
privacy since other people can upload information about
yourself without your knowledge.

CONCLUSION

Imagination is the only limit for technologies like this one. It

augments the physical world around us with digital information
and lets us use natural hand gestures to interact with that
information. It is a sense that would give seamless and easy
access to Meta information or information. So, with this
technology we can have our digital world with us wherever we
go. And we can start using any surface around us as an
interface. It will change the way we interact with people not
only with physical world. Actually the hardware is not that hard
to manufacture. The open source software is going to be made
available .It costs around$350.So the technology can reach
masses than staying to laboratories or environments.

13 | P a g e
 ADVANTAGES AND
ENHANCEMENTS

Advantages

 Sixth Sense is a user friendly interface which integrates digital

information into the physical world and its objects, making the entire
world your computer.
 Sixth Sense does not change human habits but causes computer and
other machines to adapt to human needs.  It uses hand gestures to
interact with digital information.
 Supports multi-touch and multi-user interaction

Data access directly from machine in real time

It is an open source and cost effective and we can mind map the idea
anywhere

 It is gesture-controlled wearable computing device that feeds our

relevant information and turns any surface into an interactive display.
 It is portable and easy to carry as we can wear it in our neck.

 The device could be used by anyone without even a basic knowledge

of a keyboard or mouse.

There is no need to carry a camera anymore. If we are going for a

holiday, then from now on wards it will be easy to capture photos by
using mere fingers

14 | P a g e
Future Enhancements
 To get rid of color markers

To incorporate camera and projector inside mobile computing device.

 Whenever we place pendant- style wearable device on table, it

should allow us to use the table as multi touch user interface. 
Applying this technology in various interest like gaming, education
systems etc.

To have 3D gesture tracking.

To make sixth sense work as fifth sense for disabled person.

15 | P a g e
 BIBLIOGRAPHY

 J· P. Maes, P. Mistry. Unveiling the “Sixth Sense," game

changing wearable tech. TED 2009. Long Beach, CA,
USA 2009

 GIZMODO - Sixth Sense Technology May Change How

We Look at the World Forever

 SOFTPEDIA - Next-Gen 'Sixth Sense' Device Created at

MIT

 PCWORLD - MIT Wearable Gadget Gives You Sixth

Sense

 http://www.pranavmistry.com/projects/sixthsense/

 BBC - Sixth Sense blurs digital and the real.

 http://lucasrichter.wordpress.com/2009/03/13/patti
e-maes-sixth-sense-technology-whats-stopping-this/

16 | P a g e