2017 JISOM ASurveyonAugmentedReality PDF
2017 JISOM ASurveyonAugmentedReality PDF
2017 JISOM ASurveyonAugmentedReality PDF
net/publication/333969219
CITATION READS
1 748
3 authors, including:
Costin-Anton Boiangiu
Polytechnic University of Bucharest
157 PUBLICATIONS 442 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Costin-Anton Boiangiu on 23 June 2019.
Abstract: The aim of this paper is to present the concept of Augmented Reality (AR) and
a summary of the approaches used for this technique. Augmented Reality is a technique
that superimposes 3D virtual objects into the user's environment in real time. We analyze
the technical requirements that are addressed in order to provide the user with the best AR
experience of his surrounding context. We also take into account the specificity of certain
domains and how AR systems interact with them. The purpose of this survey is to present
the current state-of-the-art in augmented reality.
Keywords: Survey, Augmented Reality
1. Introduction
Augmented reality is a technique that overlays some form of spatially registered
augmentation onto the physical world. The user can see in real time the world around him,
composited with virtual objects. These virtual objects are embedded into the user's world
with the help of additional wearable devices. The difference between augmented reality
(AR) and virtual reality (VR) is that the former is taking use of the real environment and
overlays virtual objects onto it, whereas VR creates a totally artificial environment. In
other words, AR adds virtual information to the real world, whereas VR completely
replaces the real world with a virtual one.
The motivation for this technology varies from application to application, but mostly it
provides the user with additional information that he cannot obtain using only his senses.
Because AR has the potential to address different problems, reputed corporations such as
Google, IBM, Sony, HP and many universities have put their efforts to develop it.
Augmented Reality is suitable for applications in almost every subject, especially physics,
chemistry, biology, mathematics, history, astronomy, medicine, and even music. These
big companies are working to develop suitable technologic devices that can accommodate
to any of these subjects and that can ultimately impact the user's life.
AR also has a big impact in education and it is probable to change the way students will
learn in the future. A study performed by [1] concluded that not only the students'
understanding of a lesson increased when using augmented reality, but they were also
more motivated and engaged into learning more.
The additional 3D virtual information will represent a powerful tool in the user's life,
because it has the ability to support and improve their senses and their efficacy. It will
Romania
impact the way the user learns, travels, talks, plays, treats some diseases and even the way
he feels the food's smell or flavors.
The first part of this paper presents what Augmented Reality is and the motivation behind
this technology. The second part focuses on what hardware and software has been
developed for this technology and how it is best put to use and the last part focuses on
some of the applications of this technology.
2. Technology
Hardware
The hardware components used for AR are the wearable devices that allow the user to see
and interact with the system. These components are: displays, sensors, processors and
input devices. The display offers the user an instant access to the AR environment and it
is usually a form of lightweight see-through optical device. The sensors are usually
MEMS (Micro-Electro-Mechanical Sensors) and they are useful in the tracking process.
The processor is the one that analyzes the visual field and responds to it according to the
AR's application. The input devices are consistent to the application's needs and represent
the way the user interacts with the AR environment. Modern mobile devices like
smartphones and tablet computers include these elements which makes them suitable for
integrated Augmented Reality.
Display Devices
The user can see the virtual world through various display devices, such as: head-mounted
display (HMD), hand held devices, monitors or any optical projection systems.
3. Applications
AR is a technology suitable for innovative and creative solutions for many problems. The
user's perception of life can be enhanced by bringing virtual information to his immediate
or indirect real surrounding. Although some research, such as [23] considers that AR is
limited to the display technology, AR systems can be developed to apply not only to the
sight sense but also to touch, smell, hearing or any combination of them. Because of this
reason, AR has a wide range of applicability.
Navigation
One of the first AR applications and probably the most used one is in navigation. With the
help of GPS data, an AR system can overlay the best route to get from point A to point B.
Wikitude Drive, shown in Figure 7, is an application that uses GPS data and with the help
of the user's mobile phone, the selected route is displayed over the image in front of him.
Figure 11. Example of a biology textbook using Augmented Reality (Photo: [32])
A survey completed by [33] presents the way students can interact with their lessons and
also how the teaching methods can improve by using AR. The teacher can use AR to
display different annotation that can help him with his course as presented in [34].
Entertainment
There are a lot of AR applications that exist for entertainment. From cultural apps, with
sightseeing and museum guidance, to gaming and many smart-phone apps, AR can
enhance the user's experience. While visiting a museum, the user can use a mobile phone
to project a multimedia presentation about what he is seeing. Or as presented by [35] the
user can virtually see the reconstruction of ancient ruins and have an intuitive feeling
about how the ruins looked back in the time, as shown in Figure 12. Wikitude World
Browser is an app that overlays information about stores, hotels, scenery and touristic
locations in real time.
Figure 12. Example of a view with the reconstruction of the Dashuifa's ruins (Photo: [7])
For gaming, the AR can offer more than the other physical board game by introducing
animation and other multimedia objects. Pokemon Go is an example of AR app, in which
the player needs to walk as much as possible and look for pokemons. Other games are
marker-based and with the help of some cards the user can see 3D objects. Piclings is an
iOS game in which the iPhone's user takes a picture, redefines it digitally and incorporates
it into the actual game. Junio Browser is a famous German app in which the user needed
to point the smart-phone to the TV and answer a quiz. This games was nationwide spread
and a lot of users started to compute against each other for the big prize. Zoombie
ShootAR from Metaio is an AR game where the players need to shoot the zombies that
are superimposed into the real world through their mobile device. Lego's offers the
possibility to see Lego products by simply scanning some cards from their website. Once
a webcam is put in front of the computer screen, a 3D Lego object will appear.
Another application for AR is in advertising and commercial, as shown in Figure 13.
Most techniques are marker-based in which the user needs to point to the advertising card
that will trigger an animation or a presentation of a product.
Figure 13. Example of a 3D virtual model of MINI car. This 3D object appeared as the
user pointed to the marker trigger (Photo: [7])
Military
From navigation to combat and simulation, AR has applicability also in military field. The
first head-mounted display gave pilots information about velocity, positioning and other
navigation information. Afterwards, AR was used to offer a better visualization of targets
and point of interest in combats. AR can also offer extra information to the soldiers by
using IR (infrared) cameras for night vision or cameras sensible to heat that can show if
someone is hidden nearby.
AR can be used also for battle planning, where more soldiers are connected to the same
interface and they can see in 3D the plan of the battle and decide the best way to take
action, as shown in Figure 14.
Figure 14. Example of AR application with the help of Hololens. In the picture, the two
soldiers share the same environment, a map on which they can make battle presumptions
(Photo: [36])
Assembly and manufacturing
In order for a product to come to life, there are many steps through which it needs to go:
planning, design, ergonomics assessment, etc. A survey done by [37] on the AR
application in assembly. Boeing used the first AR assembly system for guiding
technicians in building the airplane's electrical system. Another comprehensive survey
was performed by [38] and [39] about the use of augmented reality in manufacturing
industry, in which graphical assembly instruction and animation can be shown and [40]
wrote about the use of AR in design and manufacturing. State-of-the-art methods for
developing CAD (computer-aided designs) models from digital data acquisition, motion
capture, assembly modeling and human-computer interaction were presented by [41].
Figure 15 shows an example of a CAD assembly in AR.
Figure 15. Example of virtual door lock assembly (Photo: [28])
Robot path planning
Teleoperation is the process in which an assembly is controlled from distance. A robot
can be controlled from long distance and it can execute tasks already programmed. But,
because there might exist long distance communication problems, it may be better to
control a virtual version of the robot. AR allows for this to happen and the user can see in
real time the results of his manipulations, as shown in Figure 16. These virtual
manipulations can predict some errors that might appear in reality and improve their
performance. Robot Programming using AR (RPAR) is a form of offline programming
that uses a video-tracking method from ARToolKit to eliminate a lot of calibration issues.
Figure 16. Virtual lines that show the planned motion of a robot arm (Photo: [34])
Pervasive Augmented Reality
A survey was concluded by [42] about the future goal of augmented reality, pervasive
augmented reality (PAR). PAR aims to offer a continuous AR experience to the user, with
as little interaction as possible. If standard AR was a context-aware experience, PAR's
purpose would be to sense the current context of the user and adapt accordingly. So far,
most of AR applications are developed to address one problem, with a specific solution,
for a specific domain. PAR systems aims for an AR technology that can learn from the
user's experience and context and adapt to it, without the user's interaction. But PAR,
being a continuous AR experience, has some hardware challenges and also of ethics.
From the hardware point of view, the system needs to be able to collect and process a lot
of data in real time, in order to offer a reliable solution to whatever the current context and
situation might be. Also, the collected data needs to be safe to use and respect the privacy
of others.
4. Conclusion
Throughout this survey, the AR technology was presented, taking into account both the
technology behind it and its applicability. A lot of work was already developed for this
method, but taking into account its evolution and its possibilities, a lot more will be
developed in the future years. Just as personal computers and smartphones changed the
life of all the users, it is expected that all the wearable devices with AR technology will
also have a huge impact. The future expectancy of this technology is PAR, the continuous
AR experience and an easy-to-use technology.
Bibliography
[1] Y. S. Hsu, Y. H. Lin and B. Yang, "Impact of augmented reality lessons on students’ STEM
interest," Research and Practice in Technology Enhanced Learning, vol. 12, no. 1, pp. 1-14,
2017.
[3] W. Barfield, Fundamentals of wearable computers and augmented reality, CRC Press, 2015.
[4] J. P. Rolland, R. L. Holloway and H. Fuchs, "Comparison of optical and video see-through,
head-mounted displays," in Photonics for Industrial Applications, 1995, pp. 293-307.
[5] C. Christiansen, "Flip the Media," [Online]. Available: http:// flipthemedia.com/ 2012/ 01/
video-glasses-augmented-reality/ . [Accessed 11 May 2017].
[6] K. Kiyokawa, "A wide field-of-view head mounted projective display using hyperbolic half-
silvered mirrors," in 2007 6th IEEE and ACM International Symposium on Mixed and
Augmented Reality, Nara, Japan, 2007.
[8] R. Hamdani and Z. Liu, "Portable augmented-reality head-up display device," Google Patents,
2015.
[10] Google, "Google Glass," [Online]. Available: https:// support.google.com/ glass/ answer/
3064128?hl=en. [Accessed 17 May 2017].
[11] Google, "Quick Start," Google Glass, [Online]. Available: https:// developers.google.com/
glass/ develop/ mirror/ quickstart/ . [Accessed 17 May 2017].
[14] S. Mann, Intelligent image processing, John Wiley & Sons, Inc., 2001.
[15] T. Mike, "Foxtail Marketing: Virtual and Augmented Reality Marketing Strategies for the
coming decade," [Online]. Available: https:// foxtailmarketing.com/ virtual-augmented-
reality-marketing-strategies-coming-decade/ . [Accessed 2017 May 17].
[16] L. Yi-bo, K. Shao-peng, Q. Zhi-hua and Z. Qiong, "Development actuality and application of
registration technology in augmented reality," in Computational Intelligence and Design,
2008. ISCID'08. International Symposium on, vol. 2, IEEE, 2008, pp. 69-74.
[18] D. Van Krevelen and R. Poelman, "Augmented Reality: Technologies, Applications and
Limitations," 2007.
[19] I. Barakonyi, T. Fahmy and D. Schmalstieg, "Remote collaboration using augmented reality
videoconferencing," in Proceedings of Graphics interface 2004, Canadian Human-Computer
Communications Society, 2004, pp. 89-96.
[20] P. Mistry, P. Maes and L. Chang, "WUW-wear Ur world: a wearable gestural interface," in
CHI'09 extended abstracts on Human factors in computing systems, ACM, 2009, pp. 4111-
4116.
[21] J.-Y. Lee, S.-H. Lee, H.-M. Park, S.-K. Lee, J.-S. Choi and J.-S. Kwon, "Design and
implementation of a wearable AR annotation system using gaze interaction," in International
Conference on Consumer Electronics (ICCE), Digest of Technical Papers, 2010 , pp. 185-186.
[23] R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier and B. MacIntyre, "Recent advances
in augmented reality," IEEE computer graphics and applications, vol. 21, no. 6, pp. 34-47,
2001.
[24] J. E. Swan and J. L. Gabbard, "Quantitative and qualitative methods for human-subject
experiments in Virtual and Augmented Reality," in Virtual Reality (VR), IEEE, 2014, pp. 1-6.
[25] C. Bichlmeier, S. M. Heining, M. Feuerstein and N. Navab, "The virtual mirror: a new
interaction paradigm for augmented reality environments," IEEE Transactions on Medical
Imaging, vol. 28, no. 9, pp. 1498-1510, 2009.
[26] N. Navab, C. Bichlmeier and T. Sielhorst, Virtual penetrating mirror device for visualizing of
virtual objects within an augmented reality environment, Google Patents, 2012.
[28] D. Reiners, D. Stricker, G. Klinker and S. Muller, "Augmented reality for construction tasks:
Doorlock assembly," Proc. IEEE and ACM IWAR, vol. 98, no. 1, pp. 31-46, 1998.
[30] K. Lee, "Augmented reality in education and training," TechTrends, vol. 56, no. 2, pp. 13--21,
2012.
[31] M. C. J. Lizandra, "Augumented reality used for treating phobias," [Online]. Available: http://
users.dsic.upv.es/ ~mcarmen/ ar-phobia-small-animals.html. [Accessed 16 May 2017].
[34] R. T. Azuma, "A survey of augmented reality," Presence: Teleoperators and virtual
environments, vol. 6, no. 4, pp. 355-385, 1997.
[35] Y. Liu and Y. Wang, "AR-View: An augmented reality device for digital reconstruction of
Yuangmingyuan," in IEEE International Symposium on Mixed and Augmented Reality-Arts,
Media and Humanities, 2009. ISMAR-AMH 2009., IEEE, 2009, pp. 3-7.
[36] NextReality, "Royal Australian Air Force Using HoloLens to Experiment with Augmented
Reality," [Online]. Available: https:// hololens.reality.news/ news/ royal-australian-air-force-
using-hololens-experiment-with-augmented-reality-0175955/ . [Accessed 10 May 2017].
[37] X. Wang, S. Ong and A. Nee, "A comprehensive survey of augmented reality assembly
research," Advances in Manufacturing, vol. 4, no. 1, pp. 1-22, 2016.
[38] S. K. Ong and A. Y. C. Nee, Virtual and augmented reality applications in manufacturing,
Springer Science & Business Media, 2013.
[43] Y.-S. Hsu, Y.-H. Lin and B. Yang, "Impact of augmented reality lessons on students’ STEM
interest," Research and Practice in Technology Enhanced Learning, vol. 12, no. 1, pp. 1-14,
2017.