research papers on augmented reality

Help | Advanced Search

Computer Science > Computer Vision and Pattern Recognition

Title: modern augmented reality: applications, trends, and future directions.

Abstract: Augmented reality (AR) is one of the relatively old, yet trending areas in the intersection of computer vision and computer graphics with numerous applications in several areas, from gaming and entertainment, to education and healthcare. Although it has been around for nearly fifty years, it has seen a lot of interest by the research community in the recent years, mainly because of the huge success of deep learning models for various computer vision and AR applications, which made creating new generations of AR technologies possible. This work tries to provide an overview of modern augmented reality, from both application-level and technical perspective. We first give an overview of main AR applications, grouped into more than ten categories. We then give an overview of around 100 recent promising machine learning based works developed for AR systems, such as deep learning works for AR shopping (clothing, makeup), AR based image filters (such as Snapchat's lenses), AR animations, and more. In the end we discuss about some of the current challenges in AR domain, and the future directions in this area.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

ORIGINAL RESEARCH article

The past, present, and future of virtual and augmented reality research: a network and cluster analysis of the literature.

• 1 Applied Technology for Neuro-Psychology Lab, Istituto Auxologico Italiano, Milan, Italy
• 2 Department of Psychology, Catholic University of the Sacred Heart, Milan, Italy
• 3 Instituto de Investigación e Innovación en Bioingeniería, Universitat Politècnica de València, Valencia, Spain

The recent appearance of low cost virtual reality (VR) technologies – like the Oculus Rift, the HTC Vive and the Sony PlayStation VR – and Mixed Reality Interfaces (MRITF) – like the Hololens – is attracting the attention of users and researchers suggesting it may be the next largest stepping stone in technological innovation. However, the history of VR technology is longer than it may seem: the concept of VR was formulated in the 1960s and the first commercial VR tools appeared in the late 1980s. For this reason, during the last 20 years, 100s of researchers explored the processes, effects, and applications of this technology producing 1000s of scientific papers. What is the outcome of this significant research work? This paper wants to provide an answer to this question by exploring, using advanced scientometric techniques, the existing research corpus in the field. We collected all the existent articles about VR in the Web of Science Core Collection scientific database, and the resultant dataset contained 21,667 records for VR and 9,944 for augmented reality (AR). The bibliographic record contained various fields, such as author, title, abstract, country, and all the references (needed for the citation analysis). The network and cluster analysis of the literature showed a composite panorama characterized by changes and evolutions over the time. Indeed, whether until 5 years ago, the main publication media on VR concerned both conference proceeding and journals, more recently journals constitute the main medium of communication. Similarly, if at first computer science was the leading research field, nowadays clinical areas have increased, as well as the number of countries involved in VR research. The present work discusses the evolution and changes over the time of the use of VR in the main areas of application with an emphasis on the future expected VR’s capacities, increases and challenges. We conclude considering the disruptive contribution that VR/AR/MRITF will be able to get in scientific fields, as well in human communication and interaction, as already happened with the advent of mobile phones by increasing the use and the development of scientific applications (e.g., in clinical areas) and by modifying the social communication and interaction among people.

Introduction

In the last 5 years, virtual reality (VR) and augmented reality (AR) have attracted the interest of investors and the general public, especially after Mark Zuckerberg bought Oculus for two billion dollars ( Luckerson, 2014 ; Castelvecchi, 2016 ). Currently, many other companies, such as Sony, Samsung, HTC, and Google are making huge investments in VR and AR ( Korolov, 2014 ; Ebert, 2015 ; Castelvecchi, 2016 ). However, if VR has been used in research for more than 25 years, and now there are 1000s of papers and many researchers in the field, comprising a strong, interdisciplinary community, AR has a more recent application history ( Burdea and Coiffet, 2003 ; Kim, 2005 ; Bohil et al., 2011 ; Cipresso and Serino, 2014 ; Wexelblat, 2014 ). The study of VR was initiated in the computer graphics field and has been extended to several disciplines ( Sutherland, 1965 , 1968 ; Mazuryk and Gervautz, 1996 ; Choi et al., 2015 ). Currently, videogames supported by VR tools are more popular than the past, and they represent valuables, work-related tools for neuroscientists, psychologists, biologists, and other researchers as well. Indeed, for example, one of the main research purposes lies from navigation studies that include complex experiments that could be done in a laboratory by using VR, whereas, without VR, the researchers would have to go directly into the field, possibly with limited use of intervention. The importance of navigation studies for the functional understanding of human memory in dementia has been a topic of significant interest for a long time, and, in 2014, the Nobel Prize in “Physiology or Medicine” was awarded to John M. O’Keefe, May-Britt Moser, and Edvard I. Moser for their discoveries of nerve cells in the brain that enable a sense of place and navigation. Journals and magazines have extended this knowledge by writing about “the brain GPS,” which gives a clear idea of the mechanism. A huge number of studies have been conducted in clinical settings by using VR ( Bohil et al., 2011 ; Serino et al., 2014 ), and Nobel Prize winner, Edvard I. Moser commented about the use of VR ( Minderer et al., 2016 ), highlighting its importance for research and clinical practice. Moreover, the availability of free tools for VR experimental and computational use has made it easy to access any field ( Riva et al., 2011 ; Cipresso, 2015 ; Brown and Green, 2016 ; Cipresso et al., 2016 ).

Augmented reality is a more recent technology than VR and shows an interdisciplinary application framework, in which, nowadays, education and learning seem to be the most field of research. Indeed, AR allows supporting learning, for example increasing-on content understanding and memory preservation, as well as on learning motivation. However, if VR benefits from clear and more definite fields of application and research areas, AR is still emerging in the scientific scenarios.

In this article, we present a systematic and computational analysis of the emerging interdisciplinary VR and AR fields in terms of various co-citation networks in order to explore the evolution of the intellectual structure of this knowledge domain over time.

Virtual Reality Concepts and Features

The concept of VR could be traced at the mid of 1960 when Ivan Sutherland in a pivotal manuscript attempted to describe VR as a window through which a user perceives the virtual world as if looked, felt, sounded real and in which the user could act realistically ( Sutherland, 1965 ).

Since that time and in accordance with the application area, several definitions have been formulated: for example, Fuchs and Bishop (1992) defined VR as “real-time interactive graphics with 3D models, combined with a display technology that gives the user the immersion in the model world and direct manipulation” ( Fuchs and Bishop, 1992 ); Gigante (1993) described VR as “The illusion of participation in a synthetic environment rather than external observation of such an environment. VR relies on a 3D, stereoscopic head-tracker displays, hand/body tracking and binaural sound. VR is an immersive, multi-sensory experience” ( Gigante, 1993 ); and “Virtual reality refers to immersive, interactive, multi-sensory, viewer-centered, 3D computer generated environments and the combination of technologies required building environments” ( Cruz-Neira, 1993 ).

As we can notice, these definitions, although different, highlight three common features of VR systems: immersion, perception to be present in an environment, and interaction with that environment ( Biocca, 1997 ; Lombard and Ditton, 1997 ; Loomis et al., 1999 ; Heeter, 2000 ; Biocca et al., 2001 ; Bailenson et al., 2006 ; Skalski and Tamborini, 2007 ; Andersen and Thorpe, 2009 ; Slater, 2009 ; Sundar et al., 2010 ). Specifically, immersion concerns the amount of senses stimulated, interactions, and the reality’s similarity of the stimuli used to simulate environments. This feature can depend on the properties of the technological system used to isolate user from reality ( Slater, 2009 ).

Higher or lower degrees of immersion can depend by three types of VR systems provided to the user:

• Non-immersive systems are the simplest and cheapest type of VR applications that use desktops to reproduce images of the world.

• Immersive systems provide a complete simulated experience due to the support of several sensory outputs devices such as head mounted displays (HMDs) for enhancing the stereoscopic view of the environment through the movement of the user’s head, as well as audio and haptic devices.

• Semi-immersive systems such as Fish Tank VR are between the two above. They provide a stereo image of a three dimensional (3D) scene viewed on a monitor using a perspective projection coupled to the head position of the observer ( Ware et al., 1993 ). Higher technological immersive systems have showed a closest experience to reality, giving to the user the illusion of technological non-mediation and feeling him or her of “being in” or present in the virtual environment ( Lombard and Ditton, 1997 ). Furthermore, higher immersive systems, than the other two systems, can give the possibility to add several sensory outputs allowing that the interaction and actions were perceived as real ( Loomis et al., 1999 ; Heeter, 2000 ; Biocca et al., 2001 ).

Finally, the user’s VR experience could be disclosed by measuring presence, realism, and reality’s levels. Presence is a complex psychological feeling of “being there” in VR that involves the sensation and perception of physical presence, as well as the possibility to interact and react as if the user was in the real world ( Heeter, 1992 ). Similarly, the realism’s level corresponds to the degree of expectation that the user has about of the stimuli and experience ( Baños et al., 2000 , 2009 ). If the presented stimuli are similar to reality, VR user’s expectation will be congruent with reality expectation, enhancing VR experience. In the same way, higher is the degree of reality in interaction with the virtual stimuli, higher would be the level of realism of the user’s behaviors ( Baños et al., 2000 , 2009 ).

From Virtual to Augmented Reality

Looking chronologically on VR and AR developments, we can trace the first 3D immersive simulator in 1962, when Morton Heilig created Sensorama, a simulated experience of a motorcycle running through Brooklyn characterized by several sensory impressions, such as audio, olfactory, and haptic stimuli, including also wind to provide a realist experience ( Heilig, 1962 ). In the same years, Ivan Sutherland developed The Ultimate Display that, more than sound, smell, and haptic feedback, included interactive graphics that Sensorama didn’t provide. Furthermore, Philco developed the first HMD that together with The Sword of Damocles of Sutherland was able to update the virtual images by tracking user’s head position and orientation ( Sutherland, 1965 ). In the 70s, the University of North Carolina realized GROPE, the first system of force-feedback and Myron Krueger created VIDEOPLACE an Artificial Reality in which the users’ body figures were captured by cameras and projected on a screen ( Krueger et al., 1985 ). In this way two or more users could interact in the 2D-virtual space. In 1982, the US’ Air Force created the first flight simulator [Visually Coupled Airbone System Simulator (VCASS)] in which the pilot through an HMD could control the pathway and the targets. Generally, the 80’s were the years in which the first commercial devices began to emerge: for example, in 1985 the VPL company commercialized the DataGlove, glove sensors’ equipped able to measure the flexion of fingers, orientation and position, and identify hand gestures. Another example is the Eyephone, created in 1988 by the VPL Company, an HMD system for completely immerging the user in a virtual world. At the end of 80’s, Fake Space Labs created a Binocular-Omni-Orientational Monitor (BOOM), a complex system composed by a stereoscopic-displaying device, providing a moving and broad virtual environment, and a mechanical arm tracking. Furthermore, BOOM offered a more stable image and giving more quickly responses to movements than the HMD devices. Thanks to BOOM and DataGlove, the NASA Ames Research Center developed the Virtual Wind Tunnel in order to research and manipulate airflow in a virtual airplane or space ship. In 1992, the Electronic Visualization Laboratory of the University of Illinois created the CAVE Automatic Virtual Environment, an immersive VR system composed by projectors directed on three or more walls of a room.

More recently, many videogames companies have improved the development and quality of VR devices, like Oculus Rift, or HTC Vive that provide a wider field of view and lower latency. In addition, the actual HMD’s devices can be now combined with other tracker system as eye-tracking systems (FOVE), and motion and orientation sensors (e.g., Razer Hydra, Oculus Touch, or HTC Vive).

Simultaneously, at the beginning of 90’, the Boing Corporation created the first prototype of AR system for showing to employees how set up a wiring tool ( Carmigniani et al., 2011 ). At the same time, Rosenberg and Feiner developed an AR fixture for maintenance assistance, showing that the operator performance enhanced by added virtual information on the fixture to repair ( Rosenberg, 1993 ). In 1993 Loomis and colleagues produced an AR GPS-based system for helping the blind in the assisted navigation through adding spatial audio information ( Loomis et al., 1998 ). Always in the 1993 Julie Martin developed “Dancing in Cyberspace,” an AR theater in which actors interacted with virtual object in real time ( Cathy, 2011 ). Few years later, Feiner et al. (1997) developed the first Mobile AR System (MARS) able to add virtual information about touristic buildings ( Feiner et al., 1997 ). Since then, several applications have been developed: in Thomas et al. (2000) , created ARQuake, a mobile AR video game; in 2008 was created Wikitude that through the mobile camera, internet, and GPS could add information about the user’s environments ( Perry, 2008 ). In 2009 others AR applications, like AR Toolkit and SiteLens have been developed in order to add virtual information to the physical user’s surroundings. In 2011, Total Immersion developed D’Fusion, and AR system for designing projects ( Maurugeon, 2011 ). Finally, in 2013 and 2015, Google developed Google Glass and Google HoloLens, and their usability have begun to test in several field of application.

Virtual Reality Technologies

Technologically, the devices used in the virtual environments play an important role in the creation of successful virtual experiences. According to the literature, can be distinguished input and output devices ( Burdea et al., 1996 ; Burdea and Coiffet, 2003 ). Input devices are the ones that allow the user to communicate with the virtual environment, which can range from a simple joystick or keyboard to a glove allowing capturing finger movements or a tracker able to capture postures. More in detail, keyboard, mouse, trackball, and joystick represent the desktop input devices easy to use, which allow the user to launch continuous and discrete commands or movements to the environment. Other input devices can be represented by tracking devices as bend-sensing gloves that capture hand movements, postures and gestures, or pinch gloves that detect the fingers movements, and trackers able to follow the user’s movements in the physical world and translate them in the virtual environment.

On the contrary, the output devices allow the user to see, hear, smell, or touch everything that happens in the virtual environment. As mentioned above, among the visual devices can be found a wide range of possibilities, from the simplest or least immersive (monitor of a computer) to the most immersive one such as VR glasses or helmets or HMD or CAVE systems.

Furthermore, auditory, speakers, as well as haptic output devices are able to stimulate body senses providing a more real virtual experience. For example, haptic devices can stimulate the touch feeling and force models in the user.

Virtual Reality Applications

Since its appearance, VR has been used in different fields, as for gaming ( Zyda, 2005 ; Meldrum et al., 2012 ), military training ( Alexander et al., 2017 ), architectural design ( Song et al., 2017 ), education ( Englund et al., 2017 ), learning and social skills training ( Schmidt et al., 2017 ), simulations of surgical procedures ( Gallagher et al., 2005 ), assistance to the elderly or psychological treatments are other fields in which VR is bursting strongly ( Freeman et al., 2017 ; Neri et al., 2017 ). A recent and extensive review of Slater and Sanchez-Vives (2016) reported the main VR application evidences, including weakness and advantages, in several research areas, such as science, education, training, physical training, as well as social phenomena, moral behaviors, and could be used in other fields, like travel, meetings, collaboration, industry, news, and entertainment. Furthermore, another review published this year by Freeman et al. (2017) focused on VR in mental health, showing the efficacy of VR in assessing and treating different psychological disorders as anxiety, schizophrenia, depression, and eating disorders.

There are many possibilities that allow the use of VR as a stimulus, replacing real stimuli, recreating experiences, which in the real world would be impossible, with a high realism. This is why VR is widely used in research on new ways of applying psychological treatment or training, for example, to problems arising from phobias (agoraphobia, phobia to fly, etc.) ( Botella et al., 2017 ). Or, simply, it is used like improvement of the traditional systems of motor rehabilitation ( Llorens et al., 2014 ; Borrego et al., 2016 ), developing games that ameliorate the tasks. More in detail, in psychological treatment, Virtual Reality Exposure Therapy (VRET) has showed its efficacy, allowing to patients to gradually face fear stimuli or stressed situations in a safe environment where the psychological and physiological reactions can be controlled by the therapist ( Botella et al., 2017 ).

Augmented Reality Concept

Milgram and Kishino (1994) , conceptualized the Virtual-Reality Continuum that takes into consideration four systems: real environment, augmented reality (AR), augmented virtuality, and virtual environment. AR can be defined a newer technological system in which virtual objects are added to the real world in real-time during the user’s experience. Per Azuma et al. (2001) an AR system should: (1) combine real and virtual objects in a real environment; (2) run interactively and in real-time; (3) register real and virtual objects with each other. Furthermore, even if the AR experiences could seem different from VRs, the quality of AR experience could be considered similarly. Indeed, like in VR, feeling of presence, level of realism, and the degree of reality represent the main features that can be considered the indicators of the quality of AR experiences. Higher the experience is perceived as realistic, and there is congruence between the user’s expectation and the interaction inside the AR environments, higher would be the perception of “being there” physically, and at cognitive and emotional level. The feeling of presence, both in AR and VR environments, is important in acting behaviors like the real ones ( Botella et al., 2005 ; Juan et al., 2005 ; Bretón-López et al., 2010 ; Wrzesien et al., 2013 ).

Augmented Reality Technologies

Technologically, the AR systems, however various, present three common components, such as a geospatial datum for the virtual object, like a visual marker, a surface to project virtual elements to the user, and an adequate processing power for graphics, animation, and merging of images, like a pc and a monitor ( Carmigniani et al., 2011 ). To run, an AR system must also include a camera able to track the user movement for merging the virtual objects, and a visual display, like glasses through that the user can see the virtual objects overlaying to the physical world. To date, two-display systems exist, a video see-through (VST) and an optical see-though (OST) AR systems ( Botella et al., 2005 ; Juan et al., 2005 , 2007 ). The first one, disclosures virtual objects to the user by capturing the real objects/scenes with a camera and overlaying virtual objects, projecting them on a video or a monitor, while the second one, merges the virtual object on a transparent surface, like glasses, through the user see the added elements. The main difference between the two systems is the latency: an OST system could require more time to display the virtual objects than a VST system, generating a time lag between user’s action and performance and the detection of them by the system.

Augmented Reality Applications

Although AR is a more recent technology than VR, it has been investigated and used in several research areas such as architecture ( Lin and Hsu, 2017 ), maintenance ( Schwald and De Laval, 2003 ), entertainment ( Ozbek et al., 2004 ), education ( Nincarean et al., 2013 ; Bacca et al., 2014 ; Akçayır and Akçayır, 2017 ), medicine ( De Buck et al., 2005 ), and psychological treatments ( Juan et al., 2005 ; Botella et al., 2005 , 2010 ; Bretón-López et al., 2010 ; Wrzesien et al., 2011a , b , 2013 ; see the review Chicchi Giglioli et al., 2015 ). More in detail, in education several AR applications have been developed in the last few years showing the positive effects of this technology in supporting learning, such as an increased-on content understanding and memory preservation, as well as on learning motivation ( Radu, 2012 , 2014 ). For example, Ibáñez et al. (2014) developed a AR application on electromagnetism concepts’ learning, in which students could use AR batteries, magnets, cables on real superficies, and the system gave a real-time feedback to students about the correctness of the performance, improving in this way the academic success and motivation ( Di Serio et al., 2013 ). Deeply, AR system allows the possibility to learn visualizing and acting on composite phenomena that traditionally students study theoretically, without the possibility to see and test in real world ( Chien et al., 2010 ; Chen et al., 2011 ).

As well in psychological health, the number of research about AR is increasing, showing its efficacy above all in the treatment of psychological disorder (see the reviews Baus and Bouchard, 2014 ; Chicchi Giglioli et al., 2015 ). For example, in the treatment of anxiety disorders, like phobias, AR exposure therapy (ARET) showed its efficacy in one-session treatment, maintaining the positive impact in a follow-up at 1 or 3 month after. As VRET, ARET provides a safety and an ecological environment where any kind of stimulus is possible, allowing to keep control over the situation experienced by the patients, gradually generating situations of fear or stress. Indeed, in situations of fear, like the phobias for small animals, AR applications allow, in accordance with the patient’s anxiety, to gradually expose patient to fear animals, adding new animals during the session or enlarging their or increasing the speed. The various studies showed that AR is able, at the beginning of the session, to activate patient’s anxiety, for reducing after 1 h of exposition. After the session, patients even more than to better manage animal’s fear and anxiety, ware able to approach, interact, and kill real feared animals.

Materials and Methods

Data collection.

The input data for the analyses were retrieved from the scientific database Web of Science Core Collection ( Falagas et al., 2008 ) and the search terms used were “Virtual Reality” and “Augmented Reality” regarding papers published during the whole timespan covered.

Web of science core collection is composed of: Citation Indexes, Science Citation Index Expanded (SCI-EXPANDED) –1970-present, Social Sciences Citation Index (SSCI) –1970-present, Arts and Humanities Citation Index (A&HCI) –1975-present, Conference Proceedings Citation Index- Science (CPCI-S) –1990-present, Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH) –1990-present, Book Citation Index– Science (BKCI-S) –2009-present, Book Citation Index– Social Sciences & Humanities (BKCI-SSH) –2009-present, Emerging Sources Citation Index (ESCI) –2015-present, Chemical Indexes, Current Chemical Reactions (CCR-EXPANDED) –2009-present (Includes Institut National de la Propriete Industrielle structure data back to 1840), Index Chemicus (IC) –2009-present.

The resultant dataset contained a total of 21,667 records for VR and 9,944 records for AR. The bibliographic record contained various fields, such as author, title, abstract, and all of the references (needed for the citation analysis). The research tool to visualize the networks was Cite space v.4.0.R5 SE (32 bit) ( Chen, 2006 ) under Java Runtime v.8 update 91 (build 1.8.0_91-b15). Statistical analyses were conducted using Stata MP-Parallel Edition, Release 14.0, StataCorp LP. Additional information can be found in Supplementary Data Sheet 1 .

The betweenness centrality of a node in a network measures the extent to which the node is part of paths that connect an arbitrary pair of nodes in the network ( Freeman, 1977 ; Brandes, 2001 ; Chen, 2006 ).

Structural metrics include betweenness centrality, modularity, and silhouette. Temporal and hybrid metrics include citation burstness and novelty. All the algorithms are detailed ( Chen et al., 2010 ).

The analysis of the literature on VR shows a complex panorama. At first sight, according to the document-type statistics from the Web of Science (WoS), proceedings papers were used extensively as outcomes of research, comprising almost 48% of the total (10,392 proceedings), with a similar number of articles on the subject amounting to about 47% of the total of 10, 199 articles. However, if we consider only the last 5 years (7,755 articles representing about 36% of the total), the situation changes with about 57% for articles (4,445) and about 33% for proceedings (2,578). Thus, it is clear that VR field has changed in areas other than at the technological level.

About the subject category, nodes and edges are computed as co-occurring subject categories from the Web of Science “Category” field in all the articles.

According to the subject category statistics from the WoS, computer science is the leading category, followed by engineering, and, together, they account for 15,341 articles, which make up about 71% of the total production. However, if we consider just the last 5 years, these categories reach only about 55%, with a total of 4,284 articles (Table 1 and Figure 1 ).

TABLE 1. Category statistics from the WoS for the entire period and the last 5 years.

FIGURE 1. Category from the WoS: network for the last 5 years.

The evidence is very interesting since it highlights that VR is doing very well as new technology with huge interest in hardware and software components. However, with respect to the past, we are witnessing increasing numbers of applications, especially in the medical area. In particular, note its inclusion in the top 10 list of rehabilitation and clinical neurology categories (about 10% of the total production in the last 5 years). It also is interesting that neuroscience and neurology, considered together, have shown an increase from about 12% to about 18.6% over the last 5 years. However, historic areas, such as automation and control systems, imaging science and photographic technology, and robotics, which had accounted for about 14.5% of the total articles ever produced were not even in the top 10 for the last 5 years, with each one accounting for less than 4%.

About the countries, nodes and edges are computed as networks of co-authors countries. Multiple occurrency of a country in the same paper are counted once.

The countries that were very involved in VR research have published for about 47% of the total (10,200 articles altogether). Of the 10,200 articles, the United States, China, England, and Germany published 4921, 2384, 1497, and 1398, respectively. The situation remains the same if we look at the articles published over the last 5 years. However, VR contributions also came from all over the globe, with Japan, Canada, Italy, France, Spain, South Korea, and Netherlands taking positions of prominence, as shown in Figure 2 .

FIGURE 2. Country network (node dimension represents centrality).

Network analysis was conducted to calculate and to represent the centrality index ( Freeman, 1977 ; Brandes, 2001 ), i.e., the dimension of the node in Figure 2 . The top-ranked country, with a centrality index of 0.26, was the United States (2011), and England was second, with a centrality index of 0.25. The third, fourth, and fifth countries were Germany, Italy, and Australia, with centrality indices of 0.15, 0.15, and 0.14, respectively.

About the Institutions, nodes and edges are computed as networks of co-authors Institutions (Figure 3 ).

FIGURE 3. Network of institutions: the dimensions of the nodes represent centrality.

The top-level institutions in VR were in the United States, where three universities were ranked as the top three in the world for published articles; these universities were the University of Illinois (159), the University of South California (147), and the University of Washington (146). The United States also had the eighth-ranked university, which was Iowa State University (116). The second country in the ranking was Canada, with the University of Toronto, which was ranked fifth with 125 articles and McGill University, ranked 10 th with 103 articles.

Other countries in the top-ten list were Netherlands, with the Delft University of Technology ranked fourth with 129 articles; Italy, with IRCCS Istituto Auxologico Italiano, ranked sixth (with the same number of publication of the institution ranked fifth) with 125 published articles; England, which was ranked seventh with 125 articles from the University of London’s Imperial College of Science, Technology, and Medicine; and China with 104 publications, with the Chinese Academy of Science, ranked ninth. Italy’s Istituto Auxologico Italiano, which was ranked fifth, was the only non-university institution ranked in the top-10 list for VR research (Figure 3 ).

About the Journals, nodes, and edges are computed as journal co-citation networks among each journals in the corresponding field.

The top-ranked Journals for citations in VR are Presence: Teleoperators & Virtual Environments with 2689 citations and CyberPsychology & Behavior (Cyberpsychol BEHAV) with 1884 citations; however, looking at the last 5 years, the former had increased the citations, but the latter had a far more significant increase, from about 70% to about 90%, i.e., an increase from 1029 to 1147.

Following the top two journals, IEEE Computer Graphics and Applications ( IEEE Comput Graph) and Advanced Health Telematics and Telemedicine ( St HEAL T) were both left out of the top-10 list based on the last 5 years. The data for the last 5 years also resulted in the inclusion of Experimental Brain Research ( Exp BRAIN RES) (625 citations), Archives of Physical Medicine and Rehabilitation ( Arch PHYS MED REHAB) (622 citations), and Plos ONE (619 citations) in the top-10 list of three journals, which highlighted the categories of rehabilitation and clinical neurology and neuroscience and neurology. Journal co-citation analysis is reported in Figure 4 , which clearly shows four distinct clusters.

FIGURE 4. Co-citation network of journals: the dimensions of the nodes represent centrality. Full list of official abbreviations of WoS journals can be found here: https://images.webofknowledge.com/images/help/WOS/A_abrvjt.html .

Network analysis was conducted to calculate and to represent the centrality index, i.e., the dimensions of the nodes in Figure 4 . The top-ranked item by centrality was Cyberpsychol BEHAV, with a centrality index of 0.29. The second-ranked item was Arch PHYS MED REHAB, with a centrality index of 0.23. The third was Behaviour Research and Therapy (Behav RES THER), with a centrality index of 0.15. The fourth was BRAIN, with a centrality index of 0.14. The fifth was Exp BRAIN RES, with a centrality index of 0.11.

Who’s Who in VR Research

Authors are the heart and brain of research, and their roles in a field are to define the past, present, and future of disciplines and to make significant breakthroughs to make new ideas arise (Figure 5 ).

FIGURE 5. Network of authors’ numbers of publications: the dimensions of the nodes represent the centrality index, and the dimensions of the characters represent the author’s rank.

Virtual reality research is very young and changing with time, but the top-10 authors in this field have made fundamentally significant contributions as pioneers in VR and taking it beyond a mere technological development. The purpose of the following highlights is not to rank researchers; rather, the purpose is to identify the most active researchers in order to understand where the field is going and how they plan for it to get there.

The top-ranked author is Riva G, with 180 publications. The second-ranked author is Rizzo A, with 101 publications. The third is Darzi A, with 97 publications. The forth is Aggarwal R, with 94 publications. The six authors following these three are Slater M, Alcaniz M, Botella C, Wiederhold BK, Kim SI, and Gutierrez-Maldonado J with 90, 90, 85, 75, 59, and 54 publications, respectively (Figure 6 ).

FIGURE 6. Authors’ co-citation network: the dimensions of the nodes represent centrality index, and the dimensions of the characters represent the author’s rank. The 10 authors that appear on the top-10 list are considered to be the pioneers of VR research.

Considering the last 5 years, the situation remains similar, with three new entries in the top-10 list, i.e., Muhlberger A, Cipresso P, and Ahmed K ranked 7th, 8th, and 10th, respectively.

The authors’ publications number network shows the most active authors in VR research. Another relevant analysis for our focus on VR research is to identify the most cited authors in the field.

For this purpose, the authors’ co-citation analysis highlights the authors in term of their impact on the literature considering the entire time span of the field ( White and Griffith, 1981 ; González-Teruel et al., 2015 ; Bu et al., 2016 ). The idea is to focus on the dynamic nature of the community of authors who contribute to the research.

Normally, authors with higher numbers of citations tend to be the scholars who drive the fundamental research and who make the most meaningful impacts on the evolution and development of the field. In the following, we identified the most-cited pioneers in the field of VR Research.

The top-ranked author by citation count is Gallagher (2001), with 694 citations. Second is Seymour (2004), with 668 citations. Third is Slater (1999), with 649 citations. Fourth is Grantcharov (2003), with 563 citations. Fifth is Riva (1999), with 546 citations. Sixth is Aggarwal (2006), with 505 citations. Seventh is Satava (1994), with 477 citations. Eighth is Witmer (2002), with 454 citations. Ninth is Rothbaum (1996), with 448 citations. Tenth is Cruz-neira (1995), with 416 citations.

Citation Network and Cluster Analyses for VR

Another analysis that can be used is the analysis of document co-citation, which allows us to focus on the highly-cited documents that generally are also the most influential in the domain ( Small, 1973 ; González-Teruel et al., 2015 ; Orosz et al., 2016 ).

The top-ranked article by citation counts is Seymour (2002) in Cluster #0, with 317 citations. The second article is Grantcharov (2004) in Cluster #0, with 286 citations. The third is Holden (2005) in Cluster #2, with 179 citations. The 4th is Gallagher et al. (2005) in Cluster #0, with 171 citations. The 5th is Ahlberg (2007) in Cluster #0, with 142 citations. The 6th is Parsons (2008) in Cluster #4, with 136 citations. The 7th is Powers (2008) in Cluster #4, with 134 citations. The 8th is Aggarwal (2007) in Cluster #0, with 121 citations. The 9th is Reznick (2006) in Cluster #0, with 121 citations. The 10th is Munz (2004) in Cluster #0, with 117 citations.

The network of document co-citations is visually complex (Figure 7 ) because it includes 1000s of articles and the links among them. However, this analysis is very important because can be used to identify the possible conglomerate of knowledge in the area, and this is essential for a deep understanding of the area. Thus, for this purpose, a cluster analysis was conducted ( Chen et al., 2010 ; González-Teruel et al., 2015 ; Klavans and Boyack, 2015 ). Figure 8 shows the clusters, which are identified with the two algorithms in Table 2 .

FIGURE 7. Network of document co-citations: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank, and the numbers represent the strengths of the links. It is possible to identify four historical phases (colors: blue, green, yellow, and red) from the past VR research to the current research.

FIGURE 8. Document co-citation network by cluster: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank and the red writing reports the name of the cluster with a short description that was produced with the mutual information algorithm; the clusters are identified with colored polygons.

TABLE 2. Cluster ID and silhouettes as identified with two algorithms ( Chen et al., 2010 ).

The identified clusters highlight clear parts of the literature of VR research, making clear and visible the interdisciplinary nature of this field. However, the dynamics to identify the past, present, and future of VR research cannot be clear yet. We analysed the relationships between these clusters and the temporal dimensions of each article. The results are synthesized in Figure 9 . It is clear that cluster #0 (laparoscopic skill), cluster #2 (gaming and rehabilitation), cluster #4 (therapy), and cluster #14 (surgery) are the most popular areas of VR research. (See Figure 9 and Table 2 to identify the clusters.) From Figure 9 , it also is possible to identify the first phase of laparoscopic skill (cluster #6) and therapy (cluster #7). More generally, it is possible to identify four historical phases (colors: blue, green, yellow, and red) from the past VR research to the current research.

FIGURE 9. Network of document co-citation: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank and the red writing on the right hand side reports the number of the cluster, such as in Table 2 , with a short description that was extracted accordingly.

We were able to identify the top 486 references that had the most citations by using burst citations algorithm. Citation burst is an indicator of a most active area of research. Citation burst is a detection of a burst event, which can last for multiple years as well as a single year. A citation burst provides evidence that a particular publication is associated with a surge of citations. The burst detection was based on Kleinberg’s algorithm ( Kleinberg, 2002 , 2003 ). The top-ranked document by bursts is Seymour (2002) in Cluster #0, with bursts of 88.93. The second is Grantcharov (2004) in Cluster #0, with bursts of 51.40. The third is Saposnik (2010) in Cluster #2, with bursts of 40.84. The fourth is Rothbaum (1995) in Cluster #7, with bursts of 38.94. The fifth is Holden (2005) in Cluster #2, with bursts of 37.52. The sixth is Scott (2000) in Cluster #0, with bursts of 33.39. The seventh is Saposnik (2011) in Cluster #2, with bursts of 33.33. The eighth is Burdea et al. (1996) in Cluster #3, with bursts of 32.42. The ninth is Burdea and Coiffet (2003) in Cluster #22, with bursts of 31.30. The 10th is Taffinder (1998) in Cluster #6, with bursts of 30.96 (Table 3 ).

TABLE 3. Cluster ID and references of burst article.

Citation Network and Cluster Analyses for AR

Looking at Augmented Reality scenario, the top ranked item by citation counts is Azuma (1997) in Cluster #0, with citation counts of 231. The second one is Azuma et al. (2001) in Cluster #0, with citation counts of 220. The third is Van Krevelen (2010) in Cluster #5, with citation counts of 207. The 4th is Lowe (2004) in Cluster #1, with citation counts of 157. The 5th is Wu (2013) in Cluster #4, with citation counts of 144. The 6th is Dunleavy (2009) in Cluster #4, with citation counts of 122. The 7th is Zhou (2008) in Cluster #5, with citation counts of 118. The 8th is Bay (2008) in Cluster #1, with citation counts of 117. The 9th is Newcombe (2011) in Cluster #1, with citation counts of 109. The 10th is Carmigniani et al. (2011) in Cluster #5, with citation counts of 104.

The network of document co-citations is visually complex (Figure 10 ) because it includes 1000s of articles and the links among them. However, this analysis is very important because can be used to identify the possible conglomerate of knowledge in the area, and this is essential for a deep understanding of the area. Thus, for this purpose, a cluster analysis was conducted ( Chen et al., 2010 ; González-Teruel et al., 2015 ; Klavans and Boyack, 2015 ). Figure 11 shows the clusters, which are identified with the two algorithms in Table 3 .

FIGURE 10. Network of document co-citations: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank, and the numbers represent the strengths of the links. It is possible to identify four historical phases (colors: blue, green, yellow, and red) from the past AR research to the current research.

FIGURE 11. Document co-citation network by cluster: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank and the red writing reports the name of the cluster with a short description that was produced with the mutual information algorithm; the clusters are identified with colored polygons.

The identified clusters highlight clear parts of the literature of AR research, making clear and visible the interdisciplinary nature of this field. However, the dynamics to identify the past, present, and future of AR research cannot be clear yet. We analysed the relationships between these clusters and the temporal dimensions of each article. The results are synthesized in Figure 12 . It is clear that cluster #1 (tracking), cluster #4 (education), and cluster #5 (virtual city environment) are the current areas of AR research. (See Figure 12 and Table 3 to identify the clusters.) It is possible to identify four historical phases (colors: blue, green, yellow, and red) from the past AR research to the current research.

FIGURE 12. Network of document co-citation: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank and the red writing on the right hand side reports the number of the cluster, such as in Table 2 , with a short description that was extracted accordingly.

We were able to identify the top 394 references that had the most citations by using burst citations algorithm. Citation burst is an indicator of a most active area of research. Citation burst is a detection of a burst event, which can last for multiple years as well as a single year. A citation burst provides evidence that a particular publication is associated with a surge of citations. The burst detection was based on Kleinberg’s algorithm ( Kleinberg, 2002 , 2003 ). The top ranked document by bursts is Azuma (1997) in Cluster #0, with bursts of 101.64. The second one is Azuma et al. (2001) in Cluster #0, with bursts of 84.23. The third is Lowe (2004) in Cluster #1, with bursts of 64.07. The 4th is Van Krevelen (2010) in Cluster #5, with bursts of 50.99. The 5th is Wu (2013) in Cluster #4, with bursts of 47.23. The 6th is Hartley (2000) in Cluster #0, with bursts of 37.71. The 7th is Dunleavy (2009) in Cluster #4, with bursts of 33.22. The 8th is Kato (1999) in Cluster #0, with bursts of 32.16. The 9th is Newcombe (2011) in Cluster #1, with bursts of 29.72. The 10th is Feiner (1993) in Cluster #8, with bursts of 29.46 (Table 4 ).

TABLE 4. Cluster ID and silhouettes as identified with two algorithms ( Chen et al., 2010 ).

Our findings have profound implications for two reasons. At first the present work highlighted the evolution and development of VR and AR research and provided a clear perspective based on solid data and computational analyses. Secondly our findings on VR made it profoundly clear that the clinical dimension is one of the most investigated ever and seems to increase in quantitative and qualitative aspects, but also include technological development and article in computer science, engineer, and allied sciences.

Figure 9 clarifies the past, present, and future of VR research. The outset of VR research brought a clearly-identifiable development in interfaces for children and medicine, routine use and behavioral-assessment, special effects, systems perspectives, and tutorials. This pioneering era evolved in the period that we can identify as the development era, because it was the period in which VR was used in experiments associated with new technological impulses. Not surprisingly, this was exactly concomitant with the new economy era in which significant investments were made in information technology, and it also was the era of the so-called ‘dot-com bubble’ in the late 1990s. The confluence of pioneering techniques into ergonomic studies within this development era was used to develop the first effective clinical systems for surgery, telemedicine, human spatial navigation, and the first phase of the development of therapy and laparoscopic skills. With the new millennium, VR research switched strongly toward what we can call the clinical-VR era, with its strong emphasis on rehabilitation, neurosurgery, and a new phase of therapy and laparoscopic skills. The number of applications and articles that have been published in the last 5 years are in line with the new technological development that we are experiencing at the hardware level, for example, with so many new, HMDs, and at the software level with an increasing number of independent programmers and VR communities.

Finally, Figure 12 identifies clusters of the literature of AR research, making clear and visible the interdisciplinary nature of this field. The dynamics to identify the past, present, and future of AR research cannot be clear yet, but analyzing the relationships between these clusters and the temporal dimensions of each article tracking, education, and virtual city environment are the current areas of AR research. AR is a new technology that is showing its efficacy in different research fields, and providing a novel way to gather behavioral data and support learning, training, and clinical treatments.

Looking at scientific literature conducted in the last few years, it might appear that most developments in VR and AR studies have focused on clinical aspects. However, the reality is more complex; thus, this perception should be clarified. Although researchers publish studies on the use of VR in clinical settings, each study depends on the technologies available. Industrial development in VR and AR changed a lot in the last 10 years. In the past, the development involved mainly hardware solutions while nowadays, the main efforts pertain to the software when developing virtual solutions. Hardware became a commodity that is often available at low cost. On the other hand, software needs to be customized each time, per each experiment, and this requires huge efforts in term of development. Researchers in AR and VR today need to be able to adapt software in their labs.

Virtual reality and AR developments in this new clinical era rely on computer science and vice versa. The future of VR and AR is becoming more technological than before, and each day, new solutions and products are coming to the market. Both from software and hardware perspectives, the future of AR and VR depends on huge innovations in all fields. The gap between the past and the future of AR and VR research is about the “realism” that was the key aspect in the past versus the “interaction” that is the key aspect now. First 30 years of VR and AR consisted of a continuous research on better resolution and improved perception. Now, researchers already achieved a great resolution and need to focus on making the VR as realistic as possible, which is not simple. In fact, a real experience implies a realistic interaction and not just great resolution. Interactions can be improved in infinite ways through new developments at hardware and software levels.

Interaction in AR and VR is going to be “embodied,” with implication for neuroscientists that are thinking about new solutions to be implemented into the current systems ( Blanke et al., 2015 ; Riva, 2018 ; Riva et al., 2018 ). For example, the use of hands with contactless device (i.e., without gloves) makes the interaction in virtual environments more natural. The Leap Motion device 1 allows one to use of hands in VR without the use of gloves or markers. This simple and low-cost device allows the VR users to interact with virtual objects and related environments in a naturalistic way. When technology is able to be transparent, users can experience increased sense of being in the virtual environments (the so-called sense of presence).

Other forms of interactions are possible and have been developing continuously. For example, tactile and haptic device able to provide a continuous feedback to the users, intensifying their experience also by adding components, such as the feeling of touch and the physical weight of virtual objects, by using force feedback. Another technology available at low cost that facilitates interaction is the motion tracking system, such as Microsoft Kinect, for example. Such technology allows one to track the users’ bodies, allowing them to interact with the virtual environments using body movements, gestures, and interactions. Most HMDs use an embedded system to track HMD position and rotation as well as controllers that are generally placed into the user’s hands. This tracking allows a great degree of interaction and improves the overall virtual experience.

A final emerging approach is the use of digital technologies to simulate not only the external world but also the internal bodily signals ( Azevedo et al., 2017 ; Riva et al., 2017 ): interoception, proprioception and vestibular input. For example, Riva et al. (2017) recently introduced the concept of “sonoception” ( www.sonoception.com ), a novel non-invasive technological paradigm based on wearable acoustic and vibrotactile transducers able to alter internal bodily signals. This approach allowed the development of an interoceptive stimulator that is both able to assess interoceptive time perception in clinical patients ( Di Lernia et al., 2018b ) and to enhance heart rate variability (the short-term vagally mediated component—rMSSD) through the modulation of the subjects’ parasympathetic system ( Di Lernia et al., 2018a ).

In this scenario, it is clear that the future of VR and AR research is not just in clinical applications, although the implications for the patients are huge. The continuous development of VR and AR technologies is the result of research in computer science, engineering, and allied sciences. The reasons for which from our analyses emerged a “clinical era” are threefold. First, all clinical research on VR and AR includes also technological developments, and new technological discoveries are being published in clinical or technological journals but with clinical samples as main subject. As noted in our research, main journals that publish numerous articles on technological developments tested with both healthy and patients include Presence: Teleoperators & Virtual Environments, Cyberpsychology & Behavior (Cyberpsychol BEHAV), and IEEE Computer Graphics and Applications (IEEE Comput Graph). It is clear that researchers in psychology, neuroscience, medicine, and behavioral sciences in general have been investigating whether the technological developments of VR and AR are effective for users, indicating that clinical behavioral research has been incorporating large parts of computer science and engineering. A second aspect to consider is the industrial development. In fact, once a new technology is envisioned and created it goes for a patent application. Once the patent is sent for registration the new technology may be made available for the market, and eventually for journal submission and publication. Moreover, most VR and AR research that that proposes the development of a technology moves directly from the presenting prototype to receiving the patent and introducing it to the market without publishing the findings in scientific paper. Hence, it is clear that if a new technology has been developed for industrial market or consumer, but not for clinical purpose, the research conducted to develop such technology may never be published in a scientific paper. Although our manuscript considered published researches, we have to acknowledge the existence of several researches that have not been published at all. The third reason for which our analyses highlighted a “clinical era” is that several articles on VR and AR have been considered within the Web of Knowledge database, that is our source of references. In this article, we referred to “research” as the one in the database considered. Of course, this is a limitation of our study, since there are several other databases that are of big value in the scientific community, such as IEEE Xplore Digital Library, ACM Digital Library, and many others. Generally, the most important articles in journals published in these databases are also included in the Web of Knowledge database; hence, we are convinced that our study considered the top-level publications in computer science or engineering. Accordingly, we believe that this limitation can be overcome by considering the large number of articles referenced in our research.

Considering all these aspects, it is clear that clinical applications, behavioral aspects, and technological developments in VR and AR research are parts of a more complex situation compared to the old platforms used before the huge diffusion of HMD and solutions. We think that this work might provide a clearer vision for stakeholders, providing evidence of the current research frontiers and the challenges that are expected in the future, highlighting all the connections and implications of the research in several fields, such as clinical, behavioral, industrial, entertainment, educational, and many others.

Author Contributions

PC and GR conceived the idea. PC made data extraction and the computational analyses and wrote the first draft of the article. IG revised the introduction adding important information for the article. PC, IG, MR, and GR revised the article and approved the last version of the article after important input to the article rationale.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer GC declared a shared affiliation, with no collaboration, with the authors PC and GR to the handling Editor at the time of the review.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2018.02086/full#supplementary-material

• ^ https://www.leapmotion.com/

Akçayır, M., and Akçayır, G. (2017). Advantages and challenges associated with augmented reality for education: a systematic review of the literature. Educ. Res. Rev. 20, 1–11. doi: 10.1016/j.edurev.2016.11.002

CrossRef Full Text | Google Scholar

Alexander, T., Westhoven, M., and Conradi, J. (2017). “Virtual environments for competency-oriented education and training,” in Advances in Human Factors, Business Management, Training and Education , (Berlin: Springer International Publishing), 23–29. doi: 10.1007/978-3-319-42070-7_3

Andersen, S. M., and Thorpe, J. S. (2009). An if–thEN theory of personality: significant others and the relational self. J. Res. Pers. 43, 163–170. doi: 10.1016/j.jrp.2008.12.040

Azevedo, R. T., Bennett, N., Bilicki, A., Hooper, J., Markopoulou, F., and Tsakiris, M. (2017). The calming effect of a new wearable device during the anticipation of public speech. Sci. Rep. 7:2285. doi: 10.1038/s41598-017-02274-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., and MacIntyre, B. (2001). Recent advances in augmented reality. IEEE Comp. Graph. Appl. 21, 34–47. doi: 10.1109/38.963459

Bacca, J., Baldiris, S., Fabregat, R., and Graf, S. (2014). Augmented reality trends in education: a systematic review of research and applications. J. Educ. Technol. Soc. 17, 133.

Google Scholar

Bailenson, J. N., Yee, N., Merget, D., and Schroeder, R. (2006). The effect of behavioral realism and form realism of real-time avatar faces on verbal disclosure, nonverbal disclosure, emotion recognition, and copresence in dyadic interaction. Presence 15, 359–372. doi: 10.1162/pres.15.4.359

Baños, R. M., Botella, C., Garcia-Palacios, A., Villa, H., Perpiñá, C., and Alcaniz, M. (2000). Presence and reality judgment in virtual environments: a unitary construct? Cyberpsychol. Behav. 3, 327–335. doi: 10.1089/10949310050078760

Baños, R., Botella, C., García-Palacios, A., Villa, H., Perpiñá, C., and Gallardo, M. (2009). Psychological variables and reality judgment in virtual environments: the roles of absorption and dissociation. Cyberpsychol. Behav. 2, 143–148. doi: 10.1089/cpb.1999.2.143

Baus, O., and Bouchard, S. (2014). Moving from virtual reality exposure-based therapy to augmented reality exposure-based therapy: a review. Front. Hum. Neurosci. 8:112. doi: 10.3389/fnhum.2014.00112

Biocca, F. (1997). The cyborg’s dilemma: progressive embodiment in virtual environments. J. Comput. Mediat. Commun. 3. doi: 10.1111/j.1083-6101.1997

Biocca, F., Harms, C., and Gregg, J. (2001). “The networked minds measure of social presence: pilot test of the factor structure and concurrent validity,” in 4th Annual International Workshop on Presence , Philadelphia, PA, 1–9.

Blanke, O., Slater, M., and Serino, A. (2015). Behavioral, neural, and computational principles of bodily self-consciousness. Neuron 88, 145–166. doi: 10.1016/j.neuron.2015.09.029

Bohil, C. J., Alicea, B., and Biocca, F. A. (2011). Virtual reality in neuroscience research and therapy. Nat. Rev. Neurosci. 12:752. doi: 10.1038/nrn3122

Borrego, A., Latorre, J., Llorens, R., Alcañiz, M., and Noé, E. (2016). Feasibility of a walking virtual reality system for rehabilitation: objective and subjective parameters. J. Neuroeng. Rehabil. 13:68. doi: 10.1186/s12984-016-0174-171

Botella, C., Bretón-López, J., Quero, S., Baños, R. M., and García-Palacios, A. (2010). Treating cockroach phobia with augmented reality. Behav. Ther. 41, 401–413. doi: 10.1016/j.beth.2009.07.002

Botella, C., Fernández-Álvarez, J., Guillén, V., García-Palacios, A., and Baños, R. (2017). Recent progress in virtual reality exposure therapy for phobias: a systematic review. Curr. Psychiatry Rep. 19:42. doi: 10.1007/s11920-017-0788-4

Botella, C. M., Juan, M. C., Baños, R. M., Alcañiz, M., Guillén, V., and Rey, B. (2005). Mixing realities? An application of augmented reality for the treatment of cockroach phobia. Cyberpsychol. Behav. 8, 162–171. doi: 10.1089/cpb.2005.8.162

Brandes, U. (2001). A faster algorithm for betweenness centrality. J. Math. Sociol. 25, 163–177. doi: 10.1080/0022250X.2001.9990249

Bretón-López, J., Quero, S., Botella, C., García-Palacios, A., Baños, R. M., and Alcañiz, M. (2010). An augmented reality system validation for the treatment of cockroach phobia. Cyberpsychol. Behav. Soc. Netw. 13, 705–710. doi: 10.1089/cyber.2009.0170

Brown, A., and Green, T. (2016). Virtual reality: low-cost tools and resources for the classroom. TechTrends 60, 517–519. doi: 10.1007/s11528-016-0102-z

Bu, Y., Liu, T. Y., and Huang, W. B. (2016). MACA: a modified author co-citation analysis method combined with general descriptive metadata of citations. Scientometrics 108, 143–166. doi: 10.1007/s11192-016-1959-5

Burdea, G., Richard, P., and Coiffet, P. (1996). Multimodal virtual reality: input-output devices, system integration, and human factors. Int. J. Hum. Compu. Interact. 8, 5–24. doi: 10.1080/10447319609526138

Burdea, G. C., and Coiffet, P. (2003). Virtual Reality Technology , Vol. 1, Hoboken, NJ: John Wiley & Sons.

Carmigniani, J., Furht, B., Anisetti, M., Ceravolo, P., Damiani, E., and Ivkovic, M. (2011). Augmented reality technologies, systems and applications. Multimed. Tools Appl. 51, 341–377. doi: 10.1007/s11042-010-0660-6

Castelvecchi, D. (2016). Low-cost headsets boost virtual reality’s lab appeal. Nature 533, 153–154. doi: 10.1038/533153a

Cathy (2011). The History of Augmented Reality. The Optical Vision Site. Available at: http://www.theopticalvisionsite.com/history-of-eyewear/the-history-of-augmented-reality/#.UelAUmeAOyA

Chen, C. (2006). CiteSpace II: detecting and visualizing emerging trends and transient patterns in scientific literature. J. Assoc. Inform. Sci. Technol. 57, 359–377. doi: 10.1002/asi.20317

Chen, C., Ibekwe-SanJuan, F., and Hou, J. (2010). The structure and dynamics of cocitation clusters: a multipleperspective cocitation analysis. J. Assoc. Inform. Sci. Technol. 61, 1386–1409. doi: 10.1002/jez.b.22741

Chen, Y. C., Chi, H. L., Hung, W. H., and Kang, S. C. (2011). Use of tangible and augmented reality models in engineering graphics courses. J. Prof. Issues Eng. Educ. Pract. 137, 267–276. doi: 10.1061/(ASCE)EI.1943-5541.0000078

Chicchi Giglioli, I. A., Pallavicini, F., Pedroli, E., Serino, S., and Riva, G. (2015). Augmented reality: a brand new challenge for the assessment and treatment of psychological disorders. Comput. Math. Methods Med. 2015:862942. doi: 10.1155/2015/862942

Chien, C. H., Chen, C. H., and Jeng, T. S. (2010). “An interactive augmented reality system for learning anatomy structure,” in Proceedings of the International Multiconference of Engineers and Computer Scientists , Vol. 1, (Hong Kong: International Association of Engineers), 17–19.

Choi, S., Jung, K., and Noh, S. D. (2015). Virtual reality applications in manufacturing industries: past research, present findings, and future directions. Concurr. Eng. 23, 40–63. doi: 10.1177/1063293X14568814

Cipresso, P. (2015). Modeling behavior dynamics using computational psychometrics within virtual worlds. Front. Psychol. 6:1725. doi: 10.3389/fpsyg.2015.01725

Cipresso, P., and Serino, S. (2014). Virtual Reality: Technologies, Medical Applications and Challenges. Hauppauge, NY: Nova Science Publishers, Inc.

Cipresso, P., Serino, S., and Riva, G. (2016). Psychometric assessment and behavioral experiments using a free virtual reality platform and computational science. BMC Med. Inform. Decis. Mak. 16:37. doi: 10.1186/s12911-016-0276-5

Cruz-Neira, C. (1993). “Virtual reality overview,” in SIGGRAPH 93 Course Notes 21st International Conference on Computer Graphics and Interactive Techniques, Orange County Convention Center , Orlando, FL.

De Buck, S., Maes, F., Ector, J., Bogaert, J., Dymarkowski, S., Heidbuchel, H., et al. (2005). An augmented reality system for patient-specific guidance of cardiac catheter ablation procedures. IEEE Trans. Med. Imaging 24, 1512–1524. doi: 10.1109/TMI.2005.857661

Di Lernia, D., Cipresso, P., Pedroli, E., and Riva, G. (2018a). Toward an embodied medicine: a portable device with programmable interoceptive stimulation for heart rate variability enhancement. Sensors (Basel) 18:2469. doi: 10.3390/s18082469

Di Lernia, D., Serino, S., Pezzulo, G., Pedroli, E., Cipresso, P., and Riva, G. (2018b). Feel the time. Time perception as a function of interoceptive processing. Front. Hum. Neurosci. 12:74. doi: 10.3389/fnhum.2018.00074

Di Serio,Á., Ibáñez, M. B., and Kloos, C. D. (2013). Impact of an augmented reality system on students’ motivation for a visual art course. Comput. Educ. 68, 586–596. doi: 10.1016/j.compedu.2012.03.002

Ebert, C. (2015). Looking into the future. IEEE Softw. 32, 92–97. doi: 10.1109/MS.2015.142

Englund, C., Olofsson, A. D., and Price, L. (2017). Teaching with technology in higher education: understanding conceptual change and development in practice. High. Educ. Res. Dev. 36, 73–87. doi: 10.1080/07294360.2016.1171300

Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., and Pappas, G. (2008). Comparison of pubmed, scopus, web of science, and Google scholar: strengths and weaknesses. FASEB J. 22, 338–342. doi: 10.1096/fj.07-9492LSF

Feiner, S., MacIntyre, B., Hollerer, T., and Webster, A. (1997). “A touring machine: prototyping 3D mobile augmented reality systems for exploring the urban environment,” in Digest of Papers. First International Symposium on Wearable Computers , (Cambridge, MA: IEEE), 74–81. doi: 10.1109/ISWC.1997.629922

Freeman, D., Reeve, S., Robinson, A., Ehlers, A., Clark, D., Spanlang, B., et al. (2017). Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol. Med. 47, 2393–2400. doi: 10.1017/S003329171700040X

Freeman, L. C. (1977). A set of measures of centrality based on betweenness. Sociometry 40, 35–41. doi: 10.2307/3033543

Fuchs, H., and Bishop, G. (1992). Research Directions in Virtual Environments. Chapel Hill, NC: University of North Carolina at Chapel Hill.

Gallagher, A. G., Ritter, E. M., Champion, H., Higgins, G., Fried, M. P., Moses, G., et al. (2005). Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann. Surg. 241:364. doi: 10.1097/01.sla.0000151982.85062.80

Gigante, M. A. (1993). Virtual reality: definitions, history and applications. Virtual Real. Syst. 3–14. doi: 10.1016/B978-0-12-227748-1.50009-3

González-Teruel, A., González-Alcaide, G., Barrios, M., and Abad-García, M. F. (2015). Mapping recent information behavior research: an analysis of co-authorship and co-citation networks. Scientometrics 103, 687–705. doi: 10.1007/s11192-015-1548-z

Heeter, C. (1992). Being there: the subjective experience of presence. Presence 1, 262–271. doi: 10.1162/pres.1992.1.2.262

Heeter, C. (2000). Interactivity in the context of designed experiences. J. Interact. Adv. 1, 3–14. doi: 10.1080/15252019.2000.10722040

Heilig, M. (1962). Sensorama simulator. U.S. Patent No - 3, 870. Virginia: United States Patent and Trade Office.

Ibáñez, M. B., Di Serio,Á., Villarán, D., and Kloos, C. D. (2014). Experimenting with electromagnetism using augmented reality: impact on flow student experience and educational effectiveness. Comput. Educ. 71, 1–13. doi: 10.1016/j.compedu.2013.09.004

Juan, M. C., Alcañiz, M., Calatrava, J., Zaragozá, I., Baños, R., and Botella, C. (2007). “An optical see-through augmented reality system for the treatment of phobia to small animals,” in Virtual Reality, HCII 2007 Lecture Notes in Computer Science , Vol. 4563, ed. R. Schumaker (Berlin: Springer), 651–659.

Juan, M. C., Alcaniz, M., Monserrat, C., Botella, C., Baños, R. M., and Guerrero, B. (2005). Using augmented reality to treat phobias. IEEE Comput. Graph. Appl. 25, 31–37. doi: 10.1109/MCG.2005.143

Kim, G. J. (2005). A SWOT analysis of the field of virtual reality rehabilitation and therapy. Presence 14, 119–146. doi: 10.1162/1054746053967094

Klavans, R., and Boyack, K. W. (2015). Which type of citation analysis generates the most accurate taxonomy of scientific and technical knowledge? J. Assoc. Inform. Sci. Technol. 68, 984–998. doi: 10.1002/asi.23734

Kleinberg, J. (2002). “Bursty and hierarchical structure in streams,” in Paper Presented at the Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2002; Edmonton , Alberta, NT. doi: 10.1145/775047.775061

Kleinberg, J. (2003). Bursty and hierarchical structure in streams. Data Min. Knowl. Discov. 7, 373–397. doi: 10.1023/A:1024940629314

Korolov, M. (2014). The real risks of virtual reality. Risk Manag. 61, 20–24.

Krueger, M. W., Gionfriddo, T., and Hinrichsen, K. (1985). “Videoplace—an artificial reality,” in Proceedings of the ACM SIGCHI Bulletin , Vol. 16, New York, NY: ACM, 35–40. doi: 10.1145/317456.317463

Lin, C. H., and Hsu, P. H. (2017). “Integrating procedural modelling process and immersive VR environment for architectural design education,” in MATEC Web of Conferences , Vol. 104, Les Ulis: EDP Sciences. doi: 10.1051/matecconf/201710403007

Llorens, R., Noé, E., Ferri, J., and Alcañiz, M. (2014). Virtual reality-based telerehabilitation program for balance recovery. A pilot study in hemiparetic individuals with acquired brain injury. Brain Inj. 28:169.

Lombard, M., and Ditton, T. (1997). At the heart of it all: the concept of presence. J. Comput. Mediat. Commun. 3. doi: 10.1111/j.1083-6101.1997.tb00072.x

Loomis, J. M., Blascovich, J. J., and Beall, A. C. (1999). Immersive virtual environment technology as a basic research tool in psychology. Behav. Res. Methods Instr. Comput. 31, 557–564. doi: 10.3758/BF03200735

Loomis, J. M., Golledge, R. G., and Klatzky, R. L. (1998). Navigation system for the blind: auditory display modes and guidance. Presence 7, 193–203. doi: 10.1162/105474698565677

Luckerson, V. (2014). Facebook Buying Oculus Virtual-Reality Company for $2 Billion. Available at: http://time.com/37842/facebook-oculus-rift

Maurugeon, G. (2011). New D’Fusion Supports iPhone4S and 3xDSMax 2012. Available at: http://www.t-immersion.com/blog/2011-12-07/augmented-reality-dfusion-iphone-3dsmax

Mazuryk, T., and Gervautz, M. (1996). Virtual Reality-History, Applications, Technology and Future. Vienna: Institute of Computer Graphics Vienna University of Technology.

Meldrum, D., Glennon, A., Herdman, S., Murray, D., and McConn-Walsh, R. (2012). Virtual reality rehabilitation of balance: assessment of the usability of the nintendo Wii ® fit plus. Disabil. Rehabil. 7, 205–210. doi: 10.3109/17483107.2011.616922

Milgram, P., and Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE Trans. Inform. Syst. 77, 1321–1329.

Minderer, M., Harvey, C. D., Donato, F., and Moser, E. I. (2016). Neuroscience: virtual reality explored. Nature 533, 324–325. doi: 10.1038/nature17899

Neri, S. G., Cardoso, J. R., Cruz, L., Lima, R. M., de Oliveira, R. J., Iversen, M. D., et al. (2017). Do virtual reality games improve mobility skills and balance measurements in community-dwelling older adults? Systematic review and meta-analysis. Clin. Rehabil. 31, 1292–1304. doi: 10.1177/0269215517694677

Nincarean, D., Alia, M. B., Halim, N. D. A., and Rahman, M. H. A. (2013). Mobile augmented reality: the potential for education. Procedia Soc. Behav. Sci. 103, 657–664. doi: 10.1016/j.sbspro.2013.10.385

Orosz, K., Farkas, I. J., and Pollner, P. (2016). Quantifying the changing role of past publications. Scientometrics 108, 829–853. doi: 10.1007/s11192-016-1971-9

Ozbek, C. S., Giesler, B., and Dillmann, R. (2004). “Jedi training: playful evaluation of head-mounted augmented reality display systems,” in Proceedings of SPIE. The International Society for Optical Engineering , Vol. 5291, eds R. A. Norwood, M. Eich, and M. G. Kuzyk (Denver, CO), 454–463.

Perry, S. (2008). Wikitude: Android App with Augmented Reality: Mind Blow-Ing. Digital Lifestyles.

Radu, I. (2012). “Why should my students use AR? A comparative review of the educational impacts of augmented-reality,” in Mixed and Augmented Reality (ISMAR), 2012 IEEE International Symposium on , (IEEE), 313–314. doi: 10.1109/ISMAR.2012.6402590

Radu, I. (2014). Augmented reality in education: a meta-review and cross-media analysis. Pers. Ubiquitous Comput. 18, 1533–1543. doi: 10.1007/s00779-013-0747-y

Riva, G. (2018). The neuroscience of body memory: From the self through the space to the others. Cortex 104, 241–260. doi: 10.1016/j.cortex.2017.07.013

Riva, G., Gaggioli, A., Grassi, A., Raspelli, S., Cipresso, P., Pallavicini, F., et al. (2011). NeuroVR 2-A free virtual reality platform for the assessment and treatment in behavioral health care. Stud. Health Technol. Inform. 163, 493–495.

PubMed Abstract | Google Scholar

Riva, G., Serino, S., Di Lernia, D., Pavone, E. F., and Dakanalis, A. (2017). Embodied medicine: mens sana in corpore virtuale sano. Front. Hum. Neurosci. 11:120. doi: 10.3389/fnhum.2017.00120

Riva, G., Wiederhold, B. K., and Mantovani, F. (2018). Neuroscience of virtual reality: from virtual exposure to embodied medicine. Cyberpsychol. Behav. Soc. Netw. doi: 10.1089/cyber.2017.29099.gri [Epub ahead of print].

Rosenberg, L. (1993). “The use of virtual fixtures to enhance telemanipulation with time delay,” in Proceedings of the ASME Winter Anual Meeting on Advances in Robotics, Mechatronics, and Haptic Interfaces , Vol. 49, (New Orleans, LA), 29–36.

Schmidt, M., Beck, D., Glaser, N., and Schmidt, C. (2017). “A prototype immersive, multi-user 3D virtual learning environment for individuals with autism to learn social and life skills: a virtuoso DBR update,” in International Conference on Immersive Learning , Cham: Springer, 185–188. doi: 10.1007/978-3-319-60633-0_15

Schwald, B., and De Laval, B. (2003). An augmented reality system for training and assistance to maintenance in the industrial context. J. WSCG 11.

Serino, S., Cipresso, P., Morganti, F., and Riva, G. (2014). The role of egocentric and allocentric abilities in Alzheimer’s disease: a systematic review. Ageing Res. Rev. 16, 32–44. doi: 10.1016/j.arr.2014.04.004

Skalski, P., and Tamborini, R. (2007). The role of social presence in interactive agent-based persuasion. Media Psychol. 10, 385–413. doi: 10.1080/15213260701533102

Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364, 3549–3557. doi: 10.1098/rstb.2009.0138

Slater, M., and Sanchez-Vives, M. V. (2016). Enhancing our lives with immersive virtual reality. Front. Robot. AI 3:74. doi: 10.3389/frobt.2016.00074

Small, H. (1973). Co-citation in the scientific literature: a new measure of the relationship between two documents. J. Assoc. Inform. Sci. Technol. 24, 265–269. doi: 10.1002/asi.4630240406

Song, H., Chen, F., Peng, Q., Zhang, J., and Gu, P. (2017). Improvement of user experience using virtual reality in open-architecture product design. Proc. Inst. Mech. Eng. B J. Eng. Manufact. 232.

Sundar, S. S., Xu, Q., and Bellur, S. (2010). “Designing interactivity in media interfaces: a communications perspective,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , (Boston, MA: ACM), 2247–2256. doi: 10.1145/1753326.1753666

Sutherland, I. E. (1965). The Ultimate Display. Multimedia: From Wagner to Virtual Reality. New York, NY: Norton.

Sutherland, I. E. (1968). “A head-mounted three dimensional display,” in Proceedings of the December 9-11, 1968, Fall Joint Computer Conference, Part I , (ACM), 757–764. doi: 10.1145/1476589.1476686

Thomas, B., Close, B., Donoghue, J., Squires, J., De Bondi, P., Morris, M., et al. (2000). “ARQuake: an outdoor/indoor augmented reality first person application,” in Digest of Papers. Fourth International Symposium on Wearable Computers , (Atlanta, GA: IEEE), 139–146. doi: 10.1109/ISWC.2000.888480

Ware, C., Arthur, K., and Booth, K. S. (1993). “Fish tank virtual reality,” in Proceedings of the INTERACT’93 and CHI’93 Conference on Human Factors in Computing Systems , (Amsterdam: ACM), 37–42. doi: 10.1145/169059.169066

Wexelblat, A. (ed.) (2014). Virtual Reality: Applications and Explorations. Cambridge, MA: Academic Press.

White, H. D., and Griffith, B. C. (1981). Author cocitation: a literature measure of intellectual structure. J. Assoc. Inform. Sci. Technol. 32, 163–171. doi: 10.1002/asi.4630320302

Wrzesien, M., Alcañiz, M., Botella, C., Burkhardt, J. M., Bretón-López, J., Ortega, M., et al. (2013). The therapeutic lamp: treating small-animal phobias. IEEE Comput. Graph. Appl. 33, 80–86. doi: 10.1109/MCG.2013.12

Wrzesien, M., Burkhardt, J. M., Alcañiz, M., and Botella, C. (2011a). How technology influences the therapeutic process: a comparative field evaluation of augmented reality and in vivo exposure therapy for phobia of small animals. Hum. Comput. Interact. 2011, 523–540.

Wrzesien, M., Burkhardt, J. M., Alcañiz Raya, M., and Botella, C. (2011b). “Mixing psychology and HCI in evaluation of augmented reality mental health technology,” in CHI’11 Extended Abstracts on Human Factors in Computing Systems , (Vancouver, BC: ACM), 2119–2124.

Zyda, M. (2005). From visual simulation to virtual reality to games. Computer 38, 25–32. doi: 10.1109/MC.2005.297

Keywords : virtual reality, augmented reality, quantitative psychology, measurement, psychometrics, scientometrics, computational psychometrics, mathematical psychology

Citation: Cipresso P, Giglioli IAC, Raya MA and Riva G (2018) The Past, Present, and Future of Virtual and Augmented Reality Research: A Network and Cluster Analysis of the Literature. Front. Psychol. 9:2086. doi: 10.3389/fpsyg.2018.02086

Received: 14 December 2017; Accepted: 10 October 2018; Published: 06 November 2018.

Reviewed by:

Copyright © 2018 Cipresso, Giglioli, Raya and Riva. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Pietro Cipresso, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Augmented Reality: A Comprehensive Review

• Review article
• Published: 20 October 2022
• Volume 30 , pages 1057–1080, ( 2023 )

Cite this article

• Shaveta Dargan 1 ,
• Shally Bansal 2 ,
• Munish Kumar   ORCID: orcid.org/0000-0003-0115-1620 1 ,
• Ajay Mittal 3 &
• Krishan Kumar 4  

4034 Accesses

15 Citations

3 Altmetric

Explore all metrics

Augmented Reality (AR) aims to modify the perception of real-world images by overlaying digital data on them. A novel mechanic, it is an enlightening and engaging mechanic that constantly strives for new techniques in every sphere. The real world can be augmented with information in real-time. AR aims to accept the outdoors and come up with a novel and efficient model in all application areas. A wide array of fields are displaying real-time computer-generated content, such as education, medicine, robotics, manufacturing, and entertainment. Augmented reality is considered a subtype of mixed reality, and it is treated as a distortion of virtual reality. The article emphasizes the novel digital technology that has emerged after the success of Virtual Reality, which has a wide range of applications in the digital age. There are fundamental requirements to understand AR, such as the nature of technology, architecture, the devices required, types of AR, benefits, limitations, and differences with VR, which are discussed in a very simplified way in this article. As well as a year-by-year tabular overview of the research papers that have been published in the journal on augmented reality-based applications, this article aims to provide a comprehensive overview of augmented reality-based applications. It is hard to find a field that does not make use of the amazing features of AR. This article concludes with a discussion, conclusion, and future directions for AR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Eye Tracking in Virtual Reality: a Broad Review of Applications and Challenges

Isayas Berhe Adhanom, Paul MacNeilage & Eelke Folmer

Artificial intelligence powered Metaverse: analysis, challenges and future perspectives

Mona M. Soliman, Eman Ahmed, … Aboul Ella Hassanien

Inclusive AR/VR: accessibility barriers for immersive technologies

Chris Creed, Maadh Al-Kalbani, … Ian Williams

Data Availability

Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B (2001) Recent advances in Augmented Reality. IEEE Comput Graph Appl 21(6):34–47. https://doi.org/10.1109/38.963459

Article   Google Scholar  

Zhang Z, Weng D, Jiang H, Liu Y, Wang Y (2018) Inverse augmented reality: a virtual agent’s perspective. In: IEEE international symposium on mixed and augmented reality adjunct (ISMAR-Adjunct). pp 154–157

Poushneh A (2018) Augmented reality in retail: a trade-off between user’s control of access to personal information and augmentation quality. J Retail Consum Serv 41:169–176. https://doi.org/10.1016/j.jretconser.2017.12.010

Clark A, Dünser A (2012) An interactive augmented reality coloring book. In: 2012 IEEE Symposium on 3D User Interfaces (3DUI), pp 7–10. https://doi.org/10.1109/3DUI.2012.6184168

Cong W (2013) Links and differences between augmented reality and virtual reality. Break Technol 5:57–61

Google Scholar  

Lu Y, Smith S (2007) Augmented reality E-commerce assistant system: trying while shopping. In: Jacko JA (ed) Human–computer interaction. Interaction platforms and techniques. HCI 2007. Lecture notes in computer science, vol 4551. Springer, Berlin

Wojciechowski R, Walczak K, White and Cellary W (2004) Building virtual and augmented reality museum exhibitions. In: Proceedings of the ninth international conference on 3D web technology—Web3D ’04. pp 135–145

Ong SK, Yuan ML, Nee AYC (2008) Augmented reality applications in manufacturing: a survey. Int J Prod Res 46(10):2707–2742

Article   MATH   Google Scholar  

Zollmann S, Hoppe C, Kluckner S, Poglitsch C, Bischof H, Reitmayr G (2014) Augmented reality for construction site monitoring and documentation. Proc IEEE 102(2):137–154. https://doi.org/10.1109/JPROC.2013.2294314

Patil S, Prabhu C, Neogi O, Joshi AR, Katre N (2016) E-learning system using augmented reality. In: Proceedings of the international conference on computing communication control and automation (ICCUBEA). pp 1–5

Damiani L, Demartini M, Guizzi G, Revetria R, Tonelli F (2018) Augmented and virtual reality applications in industrial systems: a qualitative review towards the industry 4.0 era. IFAC-PapersOnLine 51(11):624–630. https://doi.org/10.1016/j.ifacol.2018.08.388

Cipresso P, Giglioli IAC, Raya MA, Riva G (2018) The past, present, and future of virtual and augmented reality research: a network and cluster analysis of the literature. Front Psychol 9:1–20

Challenor J, Ma M (2019) A review of augmented reality applications for history education and heritage visualisation. Multimodal Technol Interact 3(2):39. https://doi.org/10.3390/mti3020039

Pascoal R, Alturas B, de Almeida A, Sofia R (2018) A survey of augmented reality: making technology acceptable in outdoor environments. In: Proceedings of the 13th Iberian conference on information systems and technologies (CISTI). pp 1–6

Kim SJJ (2012) A user study trends in augmented reality and virtual reality research: a qualitative study with the past three years of the ISMAR and IEEE VR conference papers. In: International symposium on ubiquitous virtual reality. https://doi.org/10.1109/isuvr.2012.17

Wanga CH, Chianga YC, Wanga MJ (2015) Evaluation of an augmented reality embedded on-line shopping system. In: Proceedings of 6th international conference on applied human factors and ergonomics (AHFE 2015)

Chen Y, Wang Q, Chen H, Song X, Tang H, Tian M (2019) An overview of augmented reality technology. IOP Conf Ser J Phys Conf Ser 1237:022082. https://doi.org/10.1088/1742-6596/1237/2/022082

Kamboj D, Wankui L, Gupta N (2013) A review on illumination techniques in augmented reality. In: Fourth international conference on computing, communications and networking technologies (ICCCNT). pp 1–9

Irshad S, Rohaya B, Awang Rambli D (2014) User experience of mobile augmented reality: a review of studies. In: Proceedings of the 3rd international conference on user science and engineering (i-USEr). pp 125–130

Novak-Marcincin J, Janak M, Barna J, Novakova-Marcincinova L (2014) Application of virtual and augmented reality technology in education of manufacturing engineers. In: Rocha Á, Correia A, Tan F, Stroetmann K (eds) New perspectives in information systems and technologies, Volume 2, vol 276. Springer, Cham

Chapter   Google Scholar  

Mekni M, Lemieux A (2014) Augmented reality: applications, challenges and future trends. Appl Comput Sci 20:205–214

Rosenblum L (2000) Virtual and augmented reality 2020. IEEE Comput Graph Appl 20(1):38–39. https://doi.org/10.1109/38.814551

Cruz E, Orts-Escolano S, Donoso F (2019) An augmented reality application for improving shopping experience in large retail stores. Virtual Reality 23:281–291

Chatzopoulos D, Bermejo C, Huang Z, Hui P (2017) Mobile augmented reality survey: from where we are to where we go. IEEE Access 5:6917–6950

Mehta D, Chugh H, Banerjee P (2018) Applications of augmented reality in emerging health diagnostics: a survey. In: Proceedings of the international conference on automation and computational engineering (ICACE). pp 45–51

Yeh S, Li Y, Zhou C, Chiu P, Chen J (2018) Effects of virtual reality and augmented reality on induced anxiety. IEEE Trans Neural Syst Rehabil Eng 26(7):1345–1352

Umeda R, Seif MA, Higa H, Kuniyoshi Y (2017) A medical training system using augmented reality. In: Proceedings of the international conference on intelligent informatics and biomedical sciences (ICIIBMS). pp 146–149

Chandrasekaran S, Kesavan U (2017) Augmented reality in broadcasting. In: IEEE international conference on consumer electronics-Asia (ICCE-Asia). pp 81–83

Nasser N (2018) Augmented reality in education learning and training. In: Proceedings of the joint international conference on ICT in education and training, international conference on computing in Arabic, and international conference on geocomputing. pp 1–7

Ashfaq Q, Sirshar M (2018) Emerging trends in augmented reality games. In: Proceedings of the international conference on computing, mathematics and engineering technologies (iCoMET). pp 1–7

Aggarwal R, Singhal A (2019) Augmented Reality and its effect on our life. In: Proceedings of the 9th international conference on cloud computing, data science & engineering. pp 510–515

Rana K, Patel B (2019) Augmented reality engine applications: a survey. In: Proceedings of the international conference on communication and signal processing (ICCSP). pp 380–384

He et al (2017) The research and application of the augmented reality technology. In: Proceedings of the 2nd information technology, networking, electronic and automation control conference (ITNEC). pp 496–501

Oyman M, Bal D, Ozer S (2022) Extending the technology acceptance model to explain how perceived augmented reality affects consumers’ perceptions. Comput Hum Behav 128:107127. https://doi.org/10.1016/j.chb.2021.107127

Liu Y, Kumar SV, Manickam A (2022) Augmented reality technology based on school physical education training. Comput Electr Eng 99:107807

Giannopulu B, Lee TJ, Frangos A (2022) Synchronised neural signature of creative mental imagery in reality and augmented reality. Heliyon 8(3):e09017. https://doi.org/10.1016/j.heliyon.2022.e09017

Sun C, Fang Y, Kong M, Chen X, Liu Y (2022) Influence of augmented reality product display on consumers’ product attitudes: a product uncertainty reduction perspective. J Retail Consum Serv 64:102828

Menon SS, Holland C, Farra S, Wischgoll T, Stuber M (2022) Augmented reality in nursing education—a pilot study. Clin Simul Nurs 65:57–61

Pimentel D (2022) Saving species in a snap: on the feasibility and efficacy of augmented reality-based wildlife interactions for conservation. J Nat Conserv 66:126151

Yavuz M, Çorbacloğlu E, Başoğlu AN, Daim TU, Shaygan A (2021) Augmented reality technology adoption: case of a mobile application in Turkey. Technol Soc 66:101598

Bussink T, Maal T, Meulstee J, Xi T (2022) Augmented reality guided condylectomy. Br J Oral Maxillofac Surg 60:991

Mohanty BP, Goswami L (2021) Advancements in augmented reality. Mater Today Proc. https://doi.org/10.1016/j.matpr.2021.03.696

Kolivand H, Mardenli I, Asadianfam S (2021) Review on augmented reality technology. In: Proceedings of 14th international conference on developments in esystems engineering (DeSE). pp 7–12. https://doi.org/10.1109/DeSE54285.2021.9719356

Mishra H, Kumar A, Sharma M, Singh M, Sharma R, Ambikapathy A (2021) Application of augmented reality in the field of virtual labs. international conference on advance computing and innovative technologies in engineering (ICACITE). pp 403–405. https://doi.org/10.1109/ICACITE51222.2021.9404705

Liu Y, Sun Q, Tang Y, Li, Y, W. Jiang W, Wu J (2020) Virtual reality system for industrial training. In: 2020 international conference on virtual reality and visualization (ICVRV). pp 338–339

VanHorn K, Çobanoglu MC (2022) Democratizing AI in biomedical image classification using virtual reality, democratizing AI in biomedical image classification using virtual reality. Virtual Reality 26(1):159–171

Lemmens JS, Simon M, Sumter SR (2022) Fear and loathing in VR: the emotional and physiological effects of immersive games. Virtual Reality 26(1):223–234

Rodríguez G, Fernandez DMR, Pino-Mejías MA (2020) The impact of virtual reality technology on tourists’ experience: a textual data analysis. Soft Comput 24:13879–13892. https://doi.org/10.1007/s00500-020-04883-y

Gong M (2021) Analysis of architectural decoration esthetics based on VR technology and machine vision. Soft Comput 25:12477–12489

Lu W, Zhao L, Xu R (2021) Remote sensing image processing technology based on mobile augmented reality technology in surveying and mapping engineering. Soft Comput. https://doi.org/10.1007/s00500-021-05650-3

Lorenz M, Knopp S, Klimant P (2018) Industrial augmented reality: requirements for an augmented reality maintenance worker support system. In: IEEE international symposium on mixed and augmented reality adjunct (ISMAR-Adjunct). pp 151–153. https://doi.org/10.1109/ISMAR-Adjunct.2018.00055

Kim J, Lorenz M, S. Knopp S, Klimant P (2020) Industrial augmented reality: concepts and user interface designs for augmented reality maintenance worker support systems. In: IEEE International symposium on mixed and augmented reality adjunct (ISMAR-Adjunct). pp 67–69. https://doi.org/10.1109/ISMAR-Adjunct51615.2020.00032

Kim J, Lorenz M, Knopp S and Klimant P (2020) Industrial augmented reality: concepts and user interface designs for augmented reality maintenance worker support systems. In: IEEE international symposium on mixed and augmented reality adjunct (ISMAR-Adjunct). pp. 67–69. https://doi.org/10.1109/ISMAR-Adjunct51615.2020.00032

De Souza RF, Farias DL, Flor da Rosa RCL, Damasceno EF (2019) Analysis of low-cost virtual and augmented reality technology in case of motor rehabilitation. In: Proceedings of 21st symposium on virtual and augmented reality (SVR). pp 161–164. https://doi.org/10.1109/SVR.2019.00039

Ping J, Liu Y, Weng D (2019) Comparison in depth perception between virtual reality and augmented reality systems. In: IEEE conference on virtual reality and 3D user interfaces (VR). pp 1124–1125. https://doi.org/10.1109/VR.2019.8798174

Phon DNE, Ali MB, Halim NDA (2014) Collaborative augmented reality in education: a review. In: International conference on teaching and learning in computing and engineering. pp 78–83

Tatwany L, Ouertani HC (2017) A review on using augmented reality in text translation. In: Proceedings of 6th international conference on information and communication technology and accessibility (ICTA). pp 1–6. https://doi.org/10.1109/ICTA.2017.8336044

Kurniawan C, Rosmansyah Y, Dabarsyah B (2019) A systematic literature review on virtual reality for learning. In: Proceedings of the 5th international conference on wireless and telematics (ICWT). pp 1–4

Chen J, Yang J (2009) Study of the art & design based on Virtual Reality. In: Proceedings of the 2nd IEEE international conference on computer science and information technology. pp 1–4

Zhang Y, Liu H, Kang S-C, Al-Hussein M (2020) Virtual reality applications for the built environment: research trends and opportunities. Autom Constr 118:1–19. https://doi.org/10.1016/j.autcon.2020.103311

Boud AC, Haniff DJ, Baber C and Steiner SJ (1999) Virtual reality and augmented reality as a training tool for assembly tasks. In: Proceedings of the IEEE international conference on information visualization. https://doi.org/10.1109/iv.1999.781532

Khan T, Johnston K, Ophoff J (2019) The impact of an augmented reality application on learning motivation of students. Adv Hum-Comput Interact 2019:1–14

Download references

No funding was received.

Author information

Authors and affiliations.

Department of Computational Sciences, Maharaja Ranjit Singh Punjab Technical University, Bathinda, Punjab, India

Shaveta Dargan & Munish Kumar

Arden University, Berlin, Germany

Shally Bansal

Department of Computer Science and Engineering, University Institute of Engineering and Technology, Panjab University, Chandigarh, India

Ajay Mittal

Department of Information Technology, University Institute of Engineering and Technology, Panjab University, Chandigarh, India

Krishan Kumar

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Munish Kumar .

Ethics declarations

Conflict of interest.

The authors declare that they have no conflict of interest.

Ethical Approval

No human and animal participants were used.

Consent to Participate

All authors are agreed to participate.

Consent for Publication

All authors are agreed for publication of this work.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Dargan, S., Bansal, S., Kumar, M. et al. Augmented Reality: A Comprehensive Review. Arch Computat Methods Eng 30 , 1057–1080 (2023). https://doi.org/10.1007/s11831-022-09831-7

Download citation

Received : 11 September 2022

Accepted : 05 October 2022

Published : 20 October 2022

Issue Date : March 2023

DOI : https://doi.org/10.1007/s11831-022-09831-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 09 January 2024

Revealing the true potential and prospects of augmented reality in education

• Yiannis Koumpouros   ORCID: orcid.org/0000-0001-6912-5475 1  

Smart Learning Environments volume  11 , Article number:  2 ( 2024 ) Cite this article

2242 Accesses

Metrics details

Augmented Reality (AR) technology is one of the latest developments and is receiving ever-increasing attention. Many researches are conducted on an international scale in order to study the effectiveness of its use in education. The purpose of this work was to record the characteristics of AR applications, in order to determine the extent to which they can be used effectively for educational purposes and reveal valuable insights. A Systematic Bibliographic Review was carried out on 73 articles. The structure of the paper followed the PRISMA review protocol. Eight questions were formulated and examined in order to gather information about the characteristics of the applications. From 2016 to 2020 the publications studying AR applications were doubled. The majority of them targeted university students, while a very limited number included special education. Physics class and foreign language learning were the ones most often chosen as the field to develop an app. Most of the applications (68.49%) were designed using marker detection technology for the Android operating system (45.21%) and were created with Unity (47.95%) and Vuforia (42.47%) tools. The majority of researches evaluated the effectiveness of the application in a subjective way, using custom-made not valid and reliable tools making the results not comparable. The limited number of participants and the short duration of pilot testing inhibit the generalization of their results. Technical problems and limitations of the equipment used are mentioned as the most frequent obstacles. Not all key-actors were involved in the design and development process of the applications. This suggests that further research is needed to fully understand the potential of AR applications in education and to develop effective evaluation methods. Key aspects for future research studies are proposed.

Introduction

The current epoch is marked by swift advances in Information Technology (IT) and its pervasive applications across all industries. The most prominent technological terms are Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR), which have gained popularity for professional training and specialization. AR has been defined variously by researchers in the fields of computer science and educational technology. Generally, AR is defined as the viewing of the real physical environment, either directly or indirectly, which has been enriched through the addition of computer-generated virtual information (Carmigniani & Furht, 2011 ). Azuma ( 1997 ) described AR as a technology that combines the real with the virtual world, specifically by adding virtual-digital elements to the existing real data. This interactive and three-dimensional information supplements and shapes the user's environment. Azuma ( 1997 ) proposed that AR systems should exhibit three characteristics: (i) the ability to merge virtual and real objects in a real environment, (ii) support real-time interaction, and (iii) incorporate 3D virtual objects. Milgram and Kishino ( 1994 ), to avoid confusion among the terms AR, VR, and MR, presented the reality-virtuality continuum (see Fig.  1 ).

Reality—Virtuality Continuum [Adapted from Milgram and Kishino's ( 1994 )]

Figure  1 illustrates that Mixed Reality (MR) lies between the real and virtual environments and includes Augmented Reality (AR) as well as Augmented Virtuality (AV). AR refers to any situation where the real environment is supplemented with computer-generated graphics and digital objects. In contrast, AV, which is closer to the virtual world, augments the virtual environment with real elements (Milgram & Kishino, 1994 ). Unlike VR, AR aims to mitigate the risk of social isolation and lack of social skills among users (Kiryakova et al., 2018 ).

AR is recognized as a novel form of interactive interface that replaces the conventional screens of devices such as laptops, smartphones, and tablets with a more natural interface, enabling interaction with a virtual reality that feels completely natural (Azuma, 1997 ). AR can be classified into four main categories based on its means and objectives:

Marker-based AR : Marker tracking technology uses optical markers (flat structures with long edges and sharp corners, also known as triggers or tags), captures the video input from the camera, and adds 3D effects to the scene. This type of augmented reality is mainly used to collect more information about the object and is widely used in department stores and industries (Schall et al., 2009 ).

Markerless or location-based AR : This technology gets its name because of the readily available features on smartphones that provide location detection, positioning, speed, acceleration and orientation. In this type of AR the device's camera and sensors use GPS, accelerometer, compass, or other location-based information to recognize the user's location and augment the environment with virtual information (Kuikkaniemi et al., 2014 ).

Projection-based AR : This type of AR typically uses advanced projectors or smart glasses to project digital images onto real-world surfaces, creating a mixed reality experience. Changing the movement on the surface of the object activates the display of images. Projection-based AR is used to project digital keyboards onto a desk surface. In some cases, the image produced by projection may not be interactive (Billinghurst & Kato, 2002 ).

Superimposition-based AR : In this type of AR overlay technology replaces an object with a virtual one using visual object recognition. This process usually occurs by partially or completely replacing the view of an object with an augmented view. First Person Shooter (FPS) games are the best example of augmented reality based on superimposition (Billinghurst & Kato, 2002 ).

It's important to note that these categories are not mutually exclusive, and some AR applications may use a combination of these types.

Mobile augmented reality has gained popularity in recent years, thanks to advancements in smartphones and more powerful mobile processors. It has opened up new possibilities for augmented reality experiences on mobile devices (Tang et al., 2015 ). Mobile AR is a technology that allows digital information to be overlaid on the real-world environment through a mobile device, such as a smartphone or tablet. This technology uses the camera and sensors of the mobile device to track the user's surroundings and overlay digital content in real-time. Mobile augmented reality applications can range from simple experiences, such as adding filters to a camera app, to more complex ones, such as interactive games or educational tools that allow users to explore and learn about their environment in a new way. Mobile AR app downloads have been increasing worldwide since 2016 (Fig.  2 ). The global AR market size is projected to reach USD 88.4 billion by 2026 (Markets & Markets, 2023 ).

Consumer mobile device augmented reality applications (embedded/standalone) worldwide from 2016 to 2022 (in millions) [Source: Statista, 2023a , 2023b ]

Technological developments have brought about rapid changes in the educational world, providing opportunities for new learning experiences and quality teaching (Voogt & Knezek, 2018 ). It is no surprise that the field of education is increasingly gaining popularity for the suitability of Augmented Reality applications (Dunleavy et al., 2009 ; Radu, 2014 ). In recent years, many researches have been published that highlight the use and effect of AR in various aspects of the educational process, enhancing the pedagogical value of this technology (Dede, 2009 ).

It is worth mentioning the interest observed in recent years by Internet users in the Google search engine, regarding the term "augmented reality in education". According to the Google tool (Google Trends), the chart below shows the number of searches on the Google search engine for Augmented Reality in education from 2015 to the present.

Compared to the past, the use of AR has become considerably more accessible, enabling its application across all levels of education, from preschool to university (Bacca et al., 2014 ; Ferrer-Torregrosa et al., 2015 ). AR has greatly improved the user's perception of space and time, and allows for the simultaneous visualization of the relationship between the real and virtual world (Dunleavy & Dede, 2014 ; Sin & Zaman, 2010 ). Cheng and Tsai ( 2014 ) also noted that AR applications facilitate a deeper understanding of abstract concepts and their interrelationships. Klopfer and Squire ( 2008 ) highlighted the novel digital opportunities offered to students to explore phenomena that may be difficult to access in real-life situations. Consequently, AR applications have become a powerful tool in the hands of educators (Martin et al., 2011 ).

Augmented reality applications provide numerous opportunities for individuals of all ages to interact with both the real and augmented environment in real-time, thereby creating an engaging and interesting learning environment for students (Akçayır & Akçayır, 2017 ). AR apps are received positively by students, as they introduce educational content in playful ways, enabling them to relate what they have learned to reality and encouraging them to take initiatives for their own applications (Jerry & Aaron, 2010 ). The international educational literature highlights several uses of AR, which have been designed and implemented in the teaching of various subjects, including Mathematics, Natural Sciences, Biology, Astronomy, Environmental Education, language skills (Billingurst et al., 2001 ; Klopfer & Squire, 2008 ; Wang & Wang, 2021 ), and even the development of a virtual perspective of poetry or "visual poetry" (Bower et al., 2014 ).

The increasing interest in augmented reality and creating effective learning experiences has led to the exploration of various learning theories that can serve as a guide and advisor for educators considering implementing AR technologies in their classrooms (Klopfer & Squire, 2019 ; Li et al., 2020 ). The pedagogical approaches recorded through the use of appropriate AR educational applications include game-based learning, situated learning, constructivism, and investigative learning, as reported in the literature (Lee, 2012 ; Yuen & Yaoyuneyong, 2020 ).

By examining relevant literature and synthesizing research findings, a systematic review can provide valuable insights into the current state of AR applications in education, their characteristics, and the challenges associated with their implementation in several axes:

Identifying trends and characteristics : It can explore the different types of AR technologies used, their educational purposes, and the target subjects or disciplines. This can provide an overview of the current landscape and inform educators, researchers, and developers about the range of possibilities and potential benefits of AR in education (Liu et al., 2019 ).

Assessing effectiveness : A systematic review can evaluate the effectiveness of AR applications in enhancing learning outcomes. By analyzing empirical studies, it can identify the impact of AR on student engagement, motivation, knowledge acquisition, and retention. This evidence-based assessment can guide educators in making informed decisions about incorporating AR technologies into their teaching practices (Chen et al., 2020 ; Radu, 2014 ).

Examining implementation challenges : AR implementation in educational settings may pose various challenges. These challenges can include technical issues, teacher training, cost considerations, and pedagogical integration. A systematic review can highlight these challenges, providing insights into the barriers and facilitating factors for successful implementation (Bacca et al., 2014 ; Cao et al., 2019 ).

Informing design and development : Understanding the characteristics and challenges of AR applications in education can inform the design and development of new AR tools and instructional strategies. It can help developers and instructional designers address the identified challenges and create more effective and user-friendly AR applications tailored to the specific needs of educational contexts (Kaufmann & Schmalstieg, 2018 ; Klopfer et al., 2008 ).

This paper concludes by offering researchers guidance in the examined domain, presenting the latest trends, future perspectives, and potential gaps or challenges associated with the utilization of augmented reality (AR) in education. Supported by a series of research questions, the paper delves into diverse facets of AR applications, encompassing target audience, educational focus, assessment methods, outcomes, limitations, technological approaches, publication channels, and the evolving landscape of research studies over time. By addressing these questions, the study endeavors to provide a comprehensive understanding of the unique characteristics and trends surrounding AR applications in the educational context.

The paper is structured for easy readability, with the following organization: The "Material and Methods" section outlines the systematic review's methodology, inclusion/exclusion criteria, research questions guiding the analysis, and a list of quality criteria for chosen articles. In the subsequent "Results" section, the selection process results are detailed, aligning with the prior research questions. This section specifically delves into the technological approach, assessment methodology, quality outcomes, and key findings (including scope, outcomes, limitations, and future plans) of each study. Following this, the "Discussion" section offers a thorough analysis of the findings, unveiling opportunities, gaps, obstacles, and trends in AR in education. Lastly, the "Conclusion" section summarizes the systematic review's major findings and offers guidance to researchers pursuing further work in the field.

Materials and methods

In this scientific paper, a systematic literature review was conducted for the period 2016–2020 to determine the characteristics of augmented reality educational applications and whether they can be effectively utilized in various aspects of the educational process. The study followed a Systematic Literature Review (SLR) protocol, which involves identifying, evaluating, and interpreting all available research related to a specific research question, topic, or phenomenon of interest (Kitchenham, 2004 ). The paper is structured according to the PRISMA Checklist review protocol (Moher et al., 2009 ), which outlines the stages of the systematic literature review. The stages of the systematic literature review are framed by the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses), which has a wide application in research that aims to study a topic in depth by examining the research that has already been done and published (Grant & Booth, 2009 ).

The electronic databases Science Direct, Scopus, Google Scholar, Web of Science, MDPI, PubMED, IEEExplore, and ACM Digital Library were searched for scientific articles using keywords (employing Boolean phrases) such as augmented reality, AR, application, education, training, learning, mobile, app, etc., according to PICO (Stone, 2002 ). The keywords used in the queries were as follows: (AR OR “augmented reality”) AND (application OR education OR educational OR teaching OR app OR training OR learning OR mobile OR ICT OR “Information and Communication Technologies” OR tablet OR desktop OR curriculum). The selection of the aforementioned databases was based on considerations of comprehensiveness, interdisciplinarity, quality, international coverage, and accessibility. These databases collectively offer access to peer-reviewed journals and conference proceedings from diverse academic disciplines, ensuring a broad and reliable coverage of AR in education research. Additionally, the inclusion of Google Scholar allows for the identification of open access literature. Their reputation, interdisciplinary nature, and search capabilities further support a comprehensive and credible examination of the topic. The selected databases are known for their frequent updates, enabling the review to capture the latest research and stay up-to-date with the rapidly evolving field of AR in education. Data collection began in January 2021, and inclusion and exclusion criteria for the study are presented below.

Inclusion criteria

Articles involving the use of Augmented Reality applications for educational purpose

Studies published in English

Scientific research from peer-reviewed journals and conferences

Articles published between 2016 and 2020

Exclusion criteria

Research studies that were excluded from this review include theses, theoretical papers, reviews, and summaries that do not provide the entire articles. Additionally, studies that are "locked" and require a subscription or on-site payment for access were also excluded.

At the beginning of the data extraction process, a set of eight research questions was identified to guide the analysis:

RQ1. What is the target audience of the AR application?

RQ2. What educational areas or subjects are being targeted by the application?

RQ3. What type of assessment methods were utilized for the final solution?

RQ4. What were the outcomes achieved through the application of the proposed solution?

RQ5. What limitations or obstacles were noted in relation to the use of the application?

RQ6. What technological approaches were employed in the application's development?

RQ7. What are the primary channels for publishing research articles on AR educational interventions?

RQ8. How has the frequency of research studies on this topic changed over time?

The quality of the finally processed articles was assessed according to a series of criteria (Table  1 ). The CORE Conference Ranking (CORE Rankings Portal—Computing Research and Education, n.d.) and the Journal Citation Reports (JCR) (Ipscience-help.- thomsonreuters.com, 2022) were used for ranking conferences and journals accordingly. The maximum score for an article could be 10 points.

Initially, a total of 3,416 articles were retrieved from the searches. A "clearing" stage was then conducted, consisting of several steps. First, duplicates and non-English articles were removed, resulting in 2731 articles. Second, titles and abstracts were screened, yielding 1363 potentially relevant studies. Third, articles that were not available, as well as reviews and theoretical papers not related to the topic, were eliminated. Finally, the studies that met the inclusion criteria were isolated, resulting in a total of 73 articles. The entire process is illustrated in Fig.  3 . Figure 4 illustrates the quantity of Google searches conducted for the phrase “Augmented reality in education.”

PRISMA flowchart

Number of Google searches for the term "Augmented reality in education"

Table 2 illustrates the outcomes of the review process of the selected papers in terms of the technological methodology utilized and the characteristics of the assessment phase for the final solution. The analysis of the quality assurance results of the selected papers are presented in Table 4 (see Annex). According to the quality assurance criteria, 52.05% of the selected papers received a score above half of the total score, with a significant number of them (23.29%) scoring above 7.5. One paper achieved the maximum score, three papers scored 9.5, and one paper scored 9. Notably, 6.85% of the examined articles scored within the maximum 10% (total score = 9 to 10) of the rating scale.

Most studies employed a combination of diverse methodologies to evaluate the final solution, with 83.56% of the studies employing a questionnaire, 16.44% employing observation techniques, 16.44% interviewing the participants, and only 4.11% utilizing focus groups for subjective assessment. Objective assessments were developed in only 6.85% of the studies (Andriyandi et al., 2020 ; Bauer et al., 2017 ; Karambakhsh et al., 2019 ; Mendes et al., 2020 ), with two studies utilizing automatic detection of correct results (Andriyandi et al., 2020 ; Karambakhsh et al., 2019 ), and one study using task completion time (Mendes et al., 2020 ). Approximately one third (31.51%) used achievement tests pre- and post-study to evaluate users' performance after the applied intervention. One study used an achievement test solely in the initial phase (Aladin et al., 2020 ), and another (Scaravetti & Doroszewski, 2019 ) only at the end. Concerning subjective assessment, each study employed various instruments depending on the application's characteristics, with custom-made questionnaires being used in almost two-thirds (61.90%) of the articles. The SUS was the most widely used well-known instrument (n = 7, 11.11%), followed by the IMMS (n = 4, 6.35%) and the QUIS (n = 3, 4.76%). The UES, TAM, SoASSES, QLIS, PEURA-E, NASA-TLX, MSAR, IMI, HARUS, and CLS were used in one study each.

Scientific journals were the primary source of publication (98.6%, n = 72), with only one paper (1.4%) presented at a conference. A significant proportion (38.82%) of the articles was published in computer-related sources. The publishing focus was almost equally divided between the education and learning field (18.82%) and engineering (16.47%). The health domain was slightly addressed, with only eight journals (9.41%), followed by sources representing the environment (2.35%). Procedia Computer Science dominated the publishing sector, with 16 articles (21.92%), followed by Computers in Human Behavior (6.85%), the International Journal of Emerging Technologies in Learning (5.48%), and the IOP Conference Series: Materials Science and Engineering (4.11%). The remaining articles (n = 45) were distributed across 39 journals. Notably, over one-third (n = 28, 38.36%) of the studies lacked a JCR ranking. More than half (52.1%) of the reviewed papers were published after 2019 (see Fig.  5 ).

Frequency of papers per year

Table 3 provides a taxonomy for the classification and analysis of the studies included, which aids in the synthesis of findings and the detection of research patterns and gaps. This taxonomy can also function as a structured framework, assisting educators and researchers in categorizing, arranging, and comprehending the diverse aspects of applying AR technology in educational contexts. Tables 5 , 6 , and 7 subsequently (see Annex) presents the outcomes of the present study, built upon this taxonomy. The “Article id” in Tables 5 , 6 , and 7 is associated to the one presented in Table  2 .

Table 2 presents the technological approach followed by each project. Almost two thirds (68.49%) of the published studies exploited marker-based AR, superimposition-based was found in 9.59% of the articles, while 5.48% followed the location-based approach. As far as the devices are concerned, the majority uses a smartphone (n = 37, 50.68%) or a tablet (n = 35, 47.95%), while 13.70% (n = 10) exploits a head mounted display. Two studies (2.35%) used an interactive board, one a smart TV, and two a Kinect camera. Almost half of the papers (45.21%) worked on an Android operating system, while 28.77% used the iOS and only 9.59% the Windows one. A great percentage (32.88%) did not report the used operating platform. It has to be noticed, that a study may have use more than one of the mentioned devices or operating systems during the experiments. Regarding the used platform and tools for developing the final solution, Unity was the most common one (n = 35, 47.95%), followed by Vuforia (n = 31, 42.47%), Aurasma (n = 5, 6.85%), ARKit or formerly Metaio (n = 5, 6.85%), and Blippar (n = 2, 2.74%). A great percentage (n = 11, 15.07%) did not provide any details on the used platform and tools. As seen in Table 8 (see Annex), the topics covered by the reviewed articles were widely dispersed.

The majority of the reviewed studies (n = 31, 42.47%) focused on the university level, followed by 26.03% (n = 19) that targeted secondary education, 21.92% (n = 16) primary education, 6.85% (n = 5) early childhood education, 1.37% (n = 1) nursery school, and 1.37% (n = 1) health professionals. Special education was addressed in only six papers (8.22%), while 6.85% (n = 5) did not specify the target population.

For a comprehensive overview, Table 9 (see Annex) outlines the primary outcomes, limitations, and future steps of the reviewed studies concerning the utilized applications.

The present study involved the analysis of both qualitative and quantitative data obtained from a collection of articles. The qualitative data obtained allowed for the identification of the decisions and actions taken by authors in designing and developing educational AR applications, as well as the extent to which these applications have been utilized. Notably, the study's analysis of educational AR applications was not restricted to any specific age group, subject area, or educational context. Rather, the study aimed to examine the full spectrum of educational AR applications, both within formal and informal education settings. Unlike prior investigations, the current study provides a comprehensive overview of research conducted between 2016 and 2020, exploring a diverse range of study designs and methodologies.

Based on the findings, it was discovered that almost all research studies pertaining to the topic at hand were published in scientific journals. Nonetheless, upon closer examination and analysis of the publications, it was noted that 25 of the studies that were published in journals were, in fact, conference proceedings that were later categorized as journals (e.g., Procedia Computer Science, Procedia CIRP, etc.) with no ranking, making up 38.89% of the total. Roughly 43.03% of the journals that were included in the review were of top-quality and ranked Q1. Collectively, 61.11% of the journals had a ranking score (Q1–Q4), and were thus considered as reputable sources. The wide variety of publishing sources (43 in total for the 73 papers examined) suggests that there is no specialized journal or conference dedicated to the area of interest. Additionally, it signifies that there are various ways in which AR can be employed in educational settings, ranging from simple applications such as labeling objects in a classroom to more intricate applications such as simulations. The following examples illustrate the diverse range of AR applications in education:

Visualizing Concepts : AR can be used to visualize abstract concepts such as the solar system, anatomy, and physics. By using AR, learners can see these concepts in 3D, making it easier to understand and remember.

Gamification : AR can be used to create interactive games that teach learners various skills such as problem-solving, critical thinking, and collaboration. These games can be used to make learning more fun and engaging.

Virtual Field Trips : AR can be used to take learners on virtual field trips, allowing them to explore various places and learn about different cultures, history, and geography.

Simulations : AR can be used to create simulations that allow learners to practice real-world scenarios and develop skills such as decision-making and problem-solving. For example, medical students can use AR to simulate surgeries and practice various procedures or to operate a microscope. Engineers also use AR to simulate experiments in mechanical engineering, electronics, electrical engineering and constructions.

The advent of emerging technologies and the development of low-cost devices and mobile phones with high computing power have created opportunities for innovative AR solutions in education. Researchers tend to prefer publishing their studies in journals, which are considered the most prestigious and impactful sources, even though it may take years to publish compared to only a few months in a conference.

The distribution of published articles per year (Fig.  5 ) can be attributed to the appearance of the first commercially available AR glasses in 2014 (Google Glasses), followed by the release of Microsoft's Hololens AR headset in 2016. As a result, a greater number of AR applications in retail emerged after 2017, and the AR industry has continued to develop as the cost of required devices has become more affordable. Based on the results, research related to the use of AR and mobile technology for educational purposes is expected to increase significantly in the coming years. According to a recent report by ResearchAndMarkets.com, the global market for Augmented Reality in education and training is projected to grow from 10.37 billion USD in 2022 to 68.71 billion USD in 2026 at a CAGR of 60.4% (Research & Markets, 2023 ).

In terms of the technological background of the provided solutions, the Android operating system dominated the market in the second quarter of 2018, accounting for 88% of all smartphone sales (Statista, 2023a , 2023b ). This finding is consistent with the research results, which indicated that almost half of the studies developed the application for the Android system. This can be attributed in part to the fact that Android is widely adopted, particularly among children and teachers in most countries, who tend to own cheaper Android smartphones rather than iPhones. However, it is now becoming a trend for any commercial application to target both iOS and Android phones, which explains the 28.77% of apps developed for the iOS operating system. Only a small percentage of the studies (9.58%, n = 7) worked with Windows, indicating a strong trend towards mobile AR technologies. One third of the studies (32.88%) did not specify any operating system.

The augmented reality industry is experiencing significant growth, which can be attributed to the increasing number of mobile users who are adopting this technology. Snap Inc. predicts that by 2025, around 75% of the world's population will be active users of AR technology. In addition, Deloitte Digital x Snap Inc. has reported that 200 million users actively engage with augmented reality on Snapchat on a daily basis, primarily through mobile applications. This trend is supported by the modern citizen profile, which is characterized by continuous mobility, limited free time, and greater reliance on mobile phones than PCs or laptops. According to a Statcounter study ("Desktop vs mobile", 2023 ), 50.48% of web traffic comes from mobile devices. Furthermore, mobile learning is increasingly popular, as evidenced by various studies (Ferriman, 2023 ).

With respect to development platforms and tools, the market is dominated by Unity (47.95%) and Vuforia (42.47%). This can be attributed to the fact that Unity's AR Foundation is a cross-platform framework that allows developers to create AR experiences and then build cross-platform applications for both Android and iOS devices without additional effort. Additionally, Unity is a leading platform for creating real-time 3D content. Vuforia is a software development kit (SDK) that facilitates the creation of AR applications by enabling the addition of computer vision functionalities, which allow the application to recognize objects, images, and spaces.

Marker-based AR was utilized in 68.49% of the studies, as it is simple and effective in providing a seamless user experience. This technology involves using a camera to detect a specific visual marker, such as a QR code, and overlaying digital content onto the marker in real-time. This allows users to interact with the digital content in a more intuitive way, as they can physically move the marker and see the digital content move along with it. Furthermore, marker-based AR has been in use for longer than other forms of AR and has a more established user base. Its popularity has been further enhanced by many companies and brands integrating it into their marketing campaigns and products. Additionally, its accessibility is a contributing factor, as it requires less processing power and hardware compared to other forms of AR, making it easier for users to access and experience on their mobile devices. Markerless AR, which uses GPS and other location data to place virtual content in the real world based on the user's location, is gaining popularity, but only 2.74% of the examined studies used it. There are also markerless AR systems that use machine learning and computer vision to track and overlay digital content onto real-world objects without the need for markers. While marker-based AR is currently the most common type of AR, other forms of AR are rapidly evolving and gaining traction. Nonetheless, the review indicates that markerless AR applications are still in the early stages of development. As AI, machine learning, and computer vision techniques continue to advance, researchers will need to adopt them to improve AR applications in several ways:

Object recognition and tracking : AI algorithms can be used to improve the accuracy of object recognition and tracking in AR applications. Machine learning can be used to train algorithms to recognize specific objects and track their movements in real-time. This can improve the stability of AR overlays and create a more immersive user experience.

Content generation and personalization : Machine learning can be used to generate and personalize AR content for individual users. Algorithms can analyze user behavior and preferences to generate relevant and engaging content in real-time.

Real-time language translation : AI-powered language translation can be integrated into AR applications to enable real-time translation of text and speech.

Spatial mapping : Machine learning algorithms can be used to create detailed 3D maps of the user's environment. This can be used to improve the accuracy and stability of AR overlays and enable more sophisticated AR applications, such as indoor navigation.

Predictive analytics : Machine learning algorithms can be used to provide users with contextual information based on their location, time of day, and other factors, while AI can predict user behavior. This can be used to create a more personalized and relevant AR experience.

The aforementioned aspects can potentially lead to new opportunities for innovation in the field of AR educational applications. These opportunities can be expanded by developing and utilizing virtual assistants and digital avatars within the educational context. Digital avatars and characters created by artificial intelligence can be designed to respond more naturally to users' behavior and emotions, thereby enhancing engagement and interactions and improving the user experience. AI-powered avatars can also facilitate realistic interactions, leading to more immersive and enjoyable learning experiences. Additionally, AI-powered platforms can be used to create interactive training sessions that provide stimulating and engaging learning experiences. For example, a virtual environment can simulate real-life job situations to aid in employee training. Likewise, AI-powered tools can create interactive experiences in which students can explore virtual objects and concepts in real-time.

Based on the research findings, the process of technology assessment is an arduous, challenging, and time-consuming task, but it is necessary in any research endeavor. However, there is no established gold standard for the subjective evaluation of Augmented Reality applications, which creates a vague landscape that forces most researchers (61.90%) to use custom-made scales. Consequently, this renders research results non-comparable. Moreover, many studies do not utilize reliable and valid instruments, making their findings questionable and not generalizable. Out of the examined pool, 35 cases used non-valid scales, 33 cases used non-reliable scales, and 33 cases used neither reliable nor valid scales. The System Usability Scale (SUS) was used seven times, the Intrinsic Motivation Measurement Scale (IMMS) four times, the Questionnaire for User Interaction Satisfaction (QUIS) three times, and all other scales (Unified Theory of Acceptance and Use of Technology – UTAUT, Extension Scale—UES, Technology Acceptance Model—TAM, Socially Adaptive Systems Evaluation Scale—SoASSES, Quality of Life Impact Scale—QLIS, Perceived Usability, and User Experience of Augmented Reality Environments—PEURA-E, National Aeronautics and Space Administration Task Load Index—NASA-TLX, Mixed Reality Simulator Sickness Assessment Questionnaire—MSAR, Intrinsic Motivation Inventory—IMI, Holistic Acceptance Readiness for Use Scale—HARUS, and Collaborative Learning Scale—CLS) were used only once each. In two studies (Conley et al., 2020 ; Saundarajan et al., 2020 ), even though the researchers tested the reliability of the questionnaires used, they did not assess their validity or use any established methodology to evaluate those questionnaires. Based on the presented results, the subjective satisfaction and assessment of AR solutions appear to be a daunting and challenging task. Therefore, there is a pressing need for the development of instruments that can capture the different aspects of a user's satisfaction (Koumpouros, 2016 ). In addition, it is essential to report users' experiences with the technologies used to enhance the completeness of research papers. Privacy protection and confidentiality, ethics approval and informed consent, and transparency of data collection and management are also essential. Legal and policy attention is required to ensure proper protection of user data and to prevent unwanted sharing of sensitive information with third parties (Bielecki, 2012 ). Conducting research involving children or other special categories (such as pupils with disabilities) requires great attention to the aforementioned issues and should follow all recent legislations and regulations, such as the General Data Protection Regulation (European Commission, 2012 ), Directive 95/46/EC (European Parliament, 1995 ), Directive 2002/58/EC (European Parliament, 2002 ), and Charter of Fundamental Right (European Parliament, 2000 ). The study also found that the number of end users participating in the assessment of the final solution is critical in obtaining valid results (Riihiaho, 2000 ). However, this remains a challenge, as only 19.18% of studies used 1 to 20 end users to evaluate the application, 20.55% used 21 to 40, 16.44% used 41 to 60, 9.59% used 61 to 80, and 21.92% used more than 80 end users. Only in four studies did both teachers and students evaluate the provided solution, although it is crucial for both parties to assess the solution used, particularly in the educational context, as they observe and assess the same thing from different perspectives.

In the examined projects, insufficient attention was given to primary and secondary education subjects, with only 21.92% and 26.03% of the efforts analyzed targeting these levels, respectively. Additionally, researchers should focus on subjects that are typically known for being information-intensive and requiring rote memory. The examined projects encountered several issues and limitations, including:

small sample sizes,

short evaluation phases,

lack of generalizable results,

need for end-user training,

absence of control groups and random sampling,

difficulty in determining if the solution has ultimately helped,

considerations of technology-related factors (e.g., cost, size, weight, battery life, compatibility issues, limited field of view from the headset, difficulty in wearing the head-mounted displays, accuracy, internet connection, etc.),

limited number of choices and scenarios offered to end users,

subjective assessment difficulty,

heterogeneity in the evaluation (e.g., different knowledge levels of the end users),

poor quality of graphics,

environmental factors affecting the quality of the application (e.g., light and sound),

quick movements affecting the quality and accuracy of the provided solution,

image and marker detection issues, and

lack of examination of long-term retention of the studied subjects.

In terms of future steps, it is essential to obtain statistically accepted results, which requires a significant number of end users in any research effort. Additionally, it is crucial to carefully examine user subjective and objective satisfaction using existing valid and reliable scales that can capture users' satisfaction in an early research stage (Koumpouros, 2016 ). Researchers should aim to simulate an environment that closely resembles the real one to enable students to generalize and apply their acquired skills and knowledge easily. Other key findings from the examined studies include the need for:

experiments with wider cohorts of participants and subjects,

examination of different age groups and levels,

use of smart glasses,

integration of speech recognition techniques,

examination of reproducibility of results,

use of markerless techniques,

enrichment of AR applications with more multimedia content,

consideration of more factors during evaluation (e.g., collaboration and personal features),

implementation of human avatars in AR experiences,

integration of gesture recognition and brain activity detection,

implementation of eye tracking techniques,

use of smart glasses instead of tablets or smartphones, and

further investigation of the relationship between learning gains, embodiment, and collaboration.

In addition, achieving an advanced Technology Readiness Level (TRL) (European Commission, 2014 ) is always desirable. An interdisciplinary team is considered to be extremely important in effectively meeting the needs of various end users, which can be supported by an iterative strategy of design, evaluation, and redesign (Nielsen, 1993 ). Usability testing and subjective evaluation are challenging but critical tasks in any research project (Koumpouros, 2016 ; Koumpouros et al., 2016 ). The user-friendliness of the provided solution is also a significant concern. Additionally, the involvement of behavioral sciences could greatly assist in the development of a successful project in the field with better adoption rates by end users (Spruijt-Metz et al., 2015 ).

Table 9 (see Annex) shows that AR technologies have been utilized in a variety of disciplines, educational levels, and target groups, including for supporting and enhancing social and communication skills in special education settings. Preliminary results suggest that AR may be beneficial for these target groups, although the limited number of participants, short intervention duration, and non-random selection of participants make generalization of the results challenging. Furthermore, the long-term retention of learning gains remains unclear. Nevertheless, students appear to enjoy using AR for learning and engaging with course material, and AR supports experiential learning, which emphasizes learning through experience, activity, and reflection. This approach to teaching can lead to increased engagement and motivation, improved retention and understanding, development of practical skills, and enhanced critical thinking and problem-solving abilities. In summary, AR has the potential to be a valuable tool for developing a range of skills and knowledge in learners.

An area of interest that warrants further investigation is the amount of time learners spend on each topic when utilizing augmented reality tools as opposed to conventional learning methods. This inquiry may yield valuable insights regarding the efficacy of AR-based

The ease with which students learn the material delivered through AR.

The amount of time required to learn the material when compared to conventional education.

Whether the use of AR enhances students' interest in the topic.

Whether students enjoy studying with AR more than they do with traditional methods.

Whether AR amplifies students' motivation to learn.

interventions. Researchers ought to explore the following five key issues when providing AR-based educational solutions:

It is evident that the aforementioned parameters require at least a control group in order to compare the outcomes of the intervention with those of conventional learning. Additionally, it is essential to consider the duration of the initial intervention and the retesting interval to assess the retention of learning gains. Finally, it is crucial to expand research into the realm of special education and other domains. For example, innovative IT interventions could greatly benefit individuals with autism spectrum disorders and students with intellectual disabilities (Koumpouros & Kafazis, 2019 ). Augmented reality could be proved valuable in minimizing attention deficit during training and improve learning for the specific target groups (Goharinejad et al., 2022 ; Nor Azlina & Kamarulzaman, 2020 ; Tosto et al., 2021 ).

As far as the educational advantages and benefits of AR in education are concerned, AR holds immense potential for enhancing educational outcomes across various educational levels and subject areas:

Enhanced Engagement: AR creates highly interactive and engaging learning experiences. Learners are actively involved in the educational content, which can lead to increased motivation and interest in the subject matter.

Visualization of Complex Concepts: AR enables the visualization of abstract and complex concepts, making them more tangible and understandable. Learners can explore 3D models of objects, organisms, and phenomena, facilitating deeper comprehension.

Experiential Learning: AR supports experiential learning by allowing students to engage with virtual objects, conduct experiments, and simulate real-world scenarios. This hands-on approach enhances practical skills and problem-solving abilities.

Gamification and Game-Based Learning: AR can be used to gamify educational content, turning lessons into interactive games. This approach fosters critical thinking, decision-making, and collaborative skills while making learning enjoyable.

Virtual Field Trips: AR-based virtual field trips transport students to different places and historical eras, providing immersive cultural, historical, and geographical learning experiences.

Simulation-Based Training: Medical and engineering students can benefit from AR simulations that allow them to practice surgeries, experiments, and procedures in a risk-free environment, leading to better skill development.

Personalization of Learning: AR applications can personalize learning experiences based on individual student needs, adapting content and pacing to optimize comprehension and retention.

Enhanced Accessibility: AR can assist learners with disabilities by providing tailored support, such as audio descriptions, text-to-speech functionality, and interactive adaptations to suit various learning styles.

To provide a more comprehensive understanding of AR in education, it is essential to connect it with related research areas:

Gamification and Game-Based Learning: Drawing parallels between AR and gamification/game-based learning can shed light on how game elements, such as challenges and rewards, can be integrated into AR applications to enhance learning experiences.

Virtual Reality (VR) in Education: Contrasting AR with VR can elucidate the strengths and weaknesses of both technologies in educational contexts, helping educators make informed decisions about their integration.

Cross-Disciplinary Approaches: Collaborative research involving experts in AR, gamification, game-based learning, VR, and educational psychology can yield innovative approaches to educational technology, benefiting both learners and educators.

Learning Outcomes and Age-Level Effects: Future studies should delve into the specific learning outcomes facilitated by AR applications in different age groups and educational settings. Understanding the nuanced impact of AR on various learner demographics is crucial.

Subject-Specific Applications: Exploring subject-specific AR applications and their effectiveness can reveal how AR can be tailored to the unique requirements of diverse academic disciplines.

In conclusion, AR in education offers a myriad of educational advantages, including enhanced engagement, visualization of complex concepts, experiential learning, gamification, virtual field trips, and personalized learning. By linking AR research with related fields and investigating its impact on learning outcomes, age-level effects, and subject-specific applications, we can harness the full potential of AR technology to revolutionize education.

Summarizing, AR has positive indications and could significantly help the educational process of different levels and target groups. The innovation of various AR applications lies in the property of 3D visualization of objects—models. In this way, in the field of education, 3D visualization can be used for the in-depth understanding of phenomena by students, in whom the knowledge will be better impressed (Lamanauskas et al., 2007 ). Game-based learning, the Kinect camera or other similar tools and markerless AR should be further exploited in the future. Finally, it should be noted that in order to effectively achieve the design of an educational AR application, it is necessary to take into account the learning environment, the particularities of each student, the axioms of the psychology of the learner and of course all the theories that have been formulated for learning (Cuendet et al., 2013 ). In simpler terms, the use of AR applications in education makes learning experiential for learners and mainly aims to bridge the gap between the classroom and the external environment as well as to increase the ability to perceive reality on the part of students.

Research limitations

Our systematic literature review on AR in education, while comprehensive within its defined scope, has certain limitations that must be acknowledged. Firstly, the review was confined to articles published between 2016 and 2020, which may have excluded some recent developments in the field. Additionally, our focus on English-language publications introduces a potential bias, as valuable research in other languages may have been omitted. These limitations, though recognized, were necessary to streamline the study's scope and maintain a manageable dataset. We acknowledge the significance of incorporating more recent data, and already working to expand our research in future endeavors to encompass the latest developments, ensuring the timeliness and relevance of our findings. However, we believe that the period we examined is crucial, particularly due to the emergence of COVID-19, which significantly accelerated the proliferation of educational apps across various contexts. Hence, we consider this timeframe as a distinct era that warrants separate investigation.

The use of AR interventions shows promise for improving educational outcomes. However, to maximize its practical application, several aspects require further scrutiny. Drawing from an analysis of qualitative and quantitative data on educational AR applications, several recommendations for future research and implementation can be proposed. Firstly, there is a need to explore the impact of AR in special education, considering specific age groups, subject areas, and educational contexts. Additionally, studying the effectiveness of different methodologies and study designs in AR education is crucial. It is important to identify areas where AR can have the greatest impact and design targeted applications accordingly. Investigating the long-term effects of AR in education is essential, including how it influences learning outcomes, knowledge retention, and student engagement over an extended period. Understanding how AR can support students with diverse learning needs and disabilities and developing tailored AR applications for special education settings is also vital. Researchers should adopt appropriate methodologies for studying the impact of AR in education. This includes conducting comparative studies to evaluate the effectiveness of AR applications compared to traditional teaching methods or other educational technologies. Longitudinal studies should be conducted to examine the sustained impact of AR on learning outcomes and engagement by following students over an extended period. Mixed-methods research combining qualitative and quantitative approaches should be employed to gain a deeper understanding of the experiences and perceptions of students and educators using AR in educational settings, using interviews, observations, surveys, and performance assessments to gather comprehensive data. Integration strategies for incorporating AR into existing educational frameworks should be investigated to ensure seamless implementation. This involves exploring strategies for integrating AR into existing curriculum frameworks and enhancing traditional teaching methods and learning activities across various subjects. Providing teacher training and professional development programs to support educators in effectively integrating AR into their teaching practices is important. Additionally, exploring pedagogical approaches that leverage the unique affordances of AR can facilitate active learning, problem-solving, collaboration, and critical thinking skills development. The lack of specialized journals or conferences dedicated to educational AR suggests the need for a platform specifically focused on this area. The diverse range of AR applications in education, such as visualizing concepts, gamification, virtual field trips, and simulations, should be further explored and expanded. With the projected growth of the AR market in education, more research is expected in the coming years. Technological advancements should be leveraged, considering the dominance of the Android operating system, to develop applications that cater to both Android and iOS platforms. Furthermore, leveraging advancements in AI, machine learning, and computer vision can enhance object recognition and tracking, content generation and personalization, real-time language translation, spatial mapping, and predictive analytics in AR applications. Integrating virtual assistants, digital avatars, and AI-powered platforms can provide innovative and engaging learning experiences. Improving AR technology and applications can be achieved by investigating compatibility with different mobile devices and operating systems, exploring emerging AR technologies, and developing reliable evaluation instruments and methodologies to assess user experience and satisfaction. These recommendations aim to address research gaps, enhance the effectiveness of AR in education, and guide future developments and implementations in the field. By focusing on specific areas of investigation and considering the integration of AR within educational frameworks, researchers and practitioners can advance the understanding and application of AR in educational settings.

In conclusion, the utilization of AR interventions in education holds significant practical implications for enhancing teaching and learning processes. The adoption of AR has the potential to transform traditional educational approaches by offering interactive and personalized learning experiences. By incorporating AR technology, educators can engage students in immersive and dynamic learning environments, promoting their active participation and motivation. AR can facilitate the visualization of complex concepts, making abstract ideas more tangible and accessible. Moreover, AR applications can provide real-world simulations, virtual field trips, and gamified experiences, enabling students to explore and interact with subject matter in a way that traditional methods cannot replicate. These practical benefits of AR in education indicate its potential to revolutionize the learning landscape. However, it is important to acknowledge and address the limitations and challenges associated with AR interventions in education. Technical constraints, such as the need for compatible devices and stable connectivity, may hinder the widespread implementation of AR. Moreover, ethical considerations surrounding data privacy and security must be carefully addressed to ensure the responsible use of AR technology in educational settings. Additionally, potential barriers, such as the cost of AR devices and the need for appropriate training for educators, may pose challenges to the seamless integration of AR in classrooms. Understanding and mitigating these limitations and challenges are essential for effectively harnessing the benefits of AR interventions in education. While AR interventions offer tremendous potential to enhance education by promoting engagement, personalization, and interactive learning experiences, it is crucial to navigate the associated limitations and challenges in order to fully realize their practical benefits. By addressing these concerns and continuing to explore innovative ways to integrate AR into educational contexts, we can pave the way for a more immersive, effective, and inclusive educational landscape. Our systematic review highlights the substantial potential of AR in reshaping educational practices and outcomes. By harnessing the educational advantages of AR and forging connections with related research areas such as gamification, game-based learning, and virtual reality in education, educators and researchers can collaboratively pave the way for more engaging, interactive, and personalized learning experiences. As the educational landscape continues to evolve, embracing AR technology represents a promising avenue for enhancing the quality and effectiveness of education across diverse domains.

Availability of data and materials

All data generated or analysed during this study are included in this published article.

Abbreviations

Artificial Intelligence

Augmented Reality

Augmented reality-based video modeling storybook

Augmented Virtuality

Autism Spectrum Disorder

Collaborative Learning Scale

Computing Research and Education

Custom Made

Degrees of Freedom

Educational Magic Toys

Field of view

First Person Shooter

Focus group

Head-mounted display

Holistic Acceptance Readiness for Use Scale

Information and Communication Technologies

Information Technology

Intrinsic Motivation Inventory

Intrinsic Motivation Measurement Scale

Journal Citation Reports

Mixed Reality

Mixed Reality Simulator Sickness Assessment Questionnaire

National Aeronautics and Space Administration Task Load Index

Perceived Usability User Experience of Augmented Reality Environments

Problem-based Learning

Quality of Life Impact Scale

Questionnaire for User Interaction Satisfaction

Smart Learning Companion

Socially Adaptive Systems Evaluation Scale

Socioeconomic status

Software development kit

System Usability Scale

Systematic Literature Review

Technology Acceptance Model

Technology Acceptance Model survey

Technology Readiness Level

Unified Theory of Acceptance and Use of Technology

User Engagement Scale

Virtual Reality

Akçayır, M., & Akçayır, G. (2017). Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Educational Research Review, 20 , 1–11.

Article   Google Scholar  

Akçayır, M., Akçayır, G., Pektaş, H. M., & Ocak, M. A. (2016). Augmented reality in science laboratories: The effects of augmented reality on university students’ laboratory skills and attitudes toward science laboratories. Computers in Human Behavior, 57 , 334–342.

Abd Majid, N. A., & Abd Majid, N. (2018). Augmented reality to promote guided discovery learning for STEM learning. International Journal on Advanced Science, Engineering and Information Technology, 8 (4–2), 1494–1500.

Aebersold, M., Voepel-Lewis, T., Cherara, L., Weber, M., Khouri, C., Levine, R., & Tait, A. R. (2018). Interactive anatomy-augmented virtual simulation training. Clinical Simulation in Nursing, 15 , 34–41.

Aladin, M. Y. F., Ismail, A. W., Salam, M. S. H., Kumoi, R., & Ali, A. F. (2020). AR-TO-KID: A speech-enabled augmented reality to engage preschool children in pronunciation learning. In IOP conference series: Materials science and engineering (Vol. 979, No. 1, p. 012011). IOP Publishing.

Alhumaidan, H., Lo, K. P. Y., & Selby, A. (2018). Co-designing with children a collaborative augmented reality book based on a primary school textbook. International Journal of Child-Computer Interaction, 15 , 24–36.

Aljojo, N., Munshi, A., Zainol, A., Al-Amri, R., Al-Aqeel, A., Al-khaldi, M., & Qadah, J. (2020). Lens application: Mobile application using augmented reality.

Altmeyer, K., Kapp, S., Thees, M., Malone, S., Kuhn, J., & Brünken, R. (2020). The use of augmented reality to foster conceptual knowledge acquisition in STEM laboratory courses: Theoretical background and empirical results. British Journal of Educational Technology, 51 (3), 611–628.

Andriyandi, A. P., Darmalaksana, W., Adillah Maylawati, D. S., Irwansyah, F. S., Mantoro, T., & Ramdhani, M. A. (2020). Augmented reality using features accelerated segment test for learning tajweed. TELKOMNIKA (telecommunication Computing Electronics and Control), 18 (1), 208–216. https://doi.org/10.12928/TELKOMNIKA.V18I1.14750

Ayer, S. K., Messner, J. I., & Anumba, C. J. (2016). Augmented reality gaming in sustainable design education. Journal of Architectural Engineering, 22 (1), 04015012.

Azuma, R. T. (1997). A survey of augmented reality. Presence: Teleoperators & Virtual Environments, 6 (4), 355–385.

Bacca, J., Baldiris, S., Fabregat, R., Graf, S., & Kinshuk. (2014). Augmented reality trends in education: A systematic review of research and applications. Educational Technology & Society, 17 (4), 133–149.

Google Scholar  

Badilla-Quintana, M. G., Sepulveda-Valenzuela, E., & Salazar Arias, M. (2020). Augmented reality as a sustainable technology to improve academic achievement in students with and without special educational needs. Sustainability, 12 (19), 8116.

Bal, E., & Bicen, H. (2016). Computer hardware course application through augmented reality and QR code integration: Achievement levels and views of students. Procedia Computer Science, 102 , 267–272.

Bauer, A., Neog, D. R., Dicko, A. H., Pai, D. K., Faure, F., Palombi, O., & Troccaz, J. (2017). Anatomical augmented reality with 3D commodity tracking and image-space alignment. Computers & Graphics, 69 , 140–153.

Bazarov, S. E., Kholodilin, I. Y., Nesterov, A. S., & Sokhina, A. V. (2017). Applying augmented reality in practical classes for engineering students. In IOP conference series: Earth and environmental science (Vol. 87, No. 3, p. 032004). IOP Publishing.

Bibi, S., Munaf, R., Bawany, N., Shamim, A., & Saleem, Z. (2020). Smart learning companion (SLAC). International Journal of Emerging Technologies in Learning (iJET), 15 (16), 200–211.

Bielecki, M. (2012). Leveraging mobile health technology for patient recruitment: An emerging opportunity . Northbrook, IL: Blue Chip Patient Recruitment.

Billinghurst, M., & Kato, H. (2002). Collaborative augmented reality. Communications of the ACM, 45 (7), 64–70.

Billinghurst, M., Kato, H., & Poupyrev, I. (2001). The MagicBook: A transitional AR interface. Computers & Graphics, 25 (5), 745–753.

Bower, M., Howe, C., McCredie, N., Robinson, A., & Grover, D. (2014). Augmented reality in education–cases, places and potentials. Educational Media International, 51 (1), 1–15.

Bursali, H., & Yilmaz, R. M. (2019). Effect of augmented reality applications on secondary school students’ reading comprehension and learning permanency. Computers in Human Behavior, 95 , 126–135.

Cabero-Almenara, J., & Roig-Vila, R. (2019). The motivation of technological scenarios in augmented reality (AR): Results of different experiments. Applied Sciences, 9 (14), 2907.

Cao, Y., Zhang, S., Zhang, Y., & Li, X. (2019). Challenges and opportunities of augmented reality in education: A systematic review. Interactive Learning Environments, 27 (8), 1059–1074.

Carlson, K. J., & Gagnon, D. J. (2016). Augmented reality integrated simulation education in health care. Clinical Simulation in Nursing, 12 (4), 123–127.

Carmigniani, J., Furht, B., Anisetti, M., Ceravolo, P., Damiani, E., & Ivkovic, M. (2011). Augmented reality technologies, systems and applications. Multimedia Tools and Applications, 51 (1), 341–377.

Chang, S. C., & Hwang, G. J. (2018). Impacts of an augmented reality-based flipped learning guiding approach on students’ scientific project performance and perceptions. Computers & Education, 125 , 226–239.

Chen, C. H., Lee, I. J., & Lin, L. Y. (2016). Augmented reality-based video-modeling storybook of nonverbal facial cues for children with autism spectrum disorder to improve their perceptions and judgments of facial expressions and emotions. Computers in Human Behavior, 55 , 477–485.

Chen, C. M., Cheng, B., & Chang, C. H. (2020). A systematic review of research on augmented reality in education: Advantages and applications. Educational Research Review, 30 , 100326.

Cheng, J., Wang, Y., Tjondronegoro, D., & Song, W. (2018). Construction of interactive teaching system for course of mechanical drawing based on mobile augmented reality technology. International Journal of Emerging Technologies in Learning (IJET), 13 (2), 126–139.

Cheng, K.-H., & Tsai, C.-C. (2014). Children and parents’ reading of an augmented reality picture book: Analyses of behavioral patterns and cognitive attainment. Computers & Education, 72 , 302–312. https://doi.org/10.1016/j.compedu.2013.12.003

Cieza, E., & Lujan, D. (2018). Educational mobile application of augmented reality based on markers to improve the learning of vowel usage and numbers for children of a kindergarten in Trujillo. Procedia Computer Science, 130 , 352–358.

Collado, R. C., Caluya, N. R., & Santos, M. E. C. (2019). Teachers’ evaluation of MotionAR: An augmented reality-based motion graphing application. Journal of Physics: Conference Series (Vol. 1286, No. 1, p. 012051). IOP Publishing.

Conley, Q., Atkinson, R. K., Nguyen, F., & Nelson, B. C. (2020). MantarayAR: Leveraging augmented reality to teach probability and sampling. Computers & Education, 153 , 103895.

CORE Rankings Portal - Computing Research & Education. (n.d.) Retrieved from http://www.core.edu.au/conference-portal .

Crăciun, D., & Bunoiu, M. (2017). Boosting physics education through mobile augmented reality. In AIP Conference Proceedings (Vol. 1916, No. 1, p. 050003). AIP Publishing LLC.

Cuendet, S., Bonnard, Q., Do-Lenh, S., & Dillenbourg, P. (2013). Designing augmented reality for the classroom. Computers & Education, 68 , 557–569.

Dalim, C. S. C., Sunar, M. S., Dey, A., & Billinghurst, M. (2020). Using augmented reality with speech input for non-native children’s language learning. International Journal of Human-Computer Studies, 134 , 44–64.

Deb, S., & Bhattacharya, P. (2018). Augmented Sign Language Modeling (ASLM) with interaction design on smartphone-an assistive learning and communication tool for inclusive classroom. Procedia Computer Science, 125 , 492–500.

Dede, C. (2009). Immersive interfaces for engagement and learning. Science, 323 (5910), 66–69.

Dunleavy, M., & Dede, C. (2014). Augmented reality teaching and learning. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (4th ed., pp. 735–745). Springer.

Dunleavy, M., Dede, C., & Mitchell, R. (2009). Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. Journal of Science Education and Technology, 18 (1), 7–22.

Elivera, A., & Palaoag, T. (2020). Development of an augmented reality mobile application to enhance the pedagogical approach in teaching history. In IOP conference series: materials science and engineering (vol. 803, no. 1, p. 012014). IOP Publishing.

European Commission. (2012). General data protection regulation: Proposal for a regulation of the European parliament and of the council. From EC justice. Data protection. Retrieved from ec.europa.eu/justice/data-protection/document/review2012/com_2012_11_en.pdf.

European Commission (2014). HORIZON 2020—WORK PROGRAMME 2014-2015, general annexes, extract from part 19—commission decision C(2014)4995. Retrieved from https://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/annexes/h2020-wp1415-annex-g-trl_en.pdf .

European Parliament (2000). Charter of fundamental rights of the european union (2000/C 364/01). Official Journal of the European Communities. Retrieved from www.europarl.europa.eu/charter/pdf/text_en.pdf .

European Parliament and the Council of 12 July 2002 (2002). Directive 2002/58/EC: Processing of personal data and the protection of privacy in the electronic communications sector. Retrieved from eur-lex.europa.eu/eli/dir/2002/58/oj.

European Parliament and the Council of 24 October 1995 (1995). Directive 95/46/EC: Protection of individuals with regard to the processing of personal data and the free movement of such data. Retrieved from data.europa.eu/eli/dir/1995/46/oj.

Ferrer-Torregrosa, J., Torralba, J., Jimenez, M. A., García, S., & Barcia, J. M. (2015). AR-BOOK: Development and assessment of a tool based on augmented reality for anatomy. Journal of Science Education and Technology, 24 (1), 119–124.

Ferriman, J. (2023). 7 random mobile learning stats. LearnDash. https://www.learndash.com/7-random-mobile-learning-stats/ .

Fidan, M., & Tuncel, M. (2019). Integrating augmented reality into problem based learning: The effects on learning achievement and attitude in physics education. Computers & Education, 142 , 103635.

Gargrish, S., Mantri, A., & Kaur, D. P. (2020). Augmented reality-based learning environment to enhance teaching-learning experience in geometry education. Procedia Computer Science, 172 , 1039–1046.

Goharinejad, S., Goharinejad, S., Hajesmaeel-Gohari, S., et al. (2022). The usefulness of virtual, augmented, and mixed reality technologies in the diagnosis and treatment of attention deficit hyperactivity disorder in children: an overview of relevant studies. BMC Psychiatry 22 (4). https://doi.org/10.1186/s12888-021-03632-1 .

Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26 (2), 91–108.

Harun, F., Tuli, N., & Mantri, A. (2020). Experience fleming’s rule in electromagnetism using augmented reality: analyzing impact on students learning. Procedia Computer Science, 172 , 660–668.

Henssen, D. J., van den Heuvel, L., De Jong, G., Vorstenbosch, M. A., van Cappellen van Walsum, A. M., Van den Hurk, M. M., et al. (2020). Neuroanatomy learning: Augmented reality vs. cross‐sections. Anatomical sciences education, 13 (3), 353–365.

Ibáñez, M. B., Portillo, A. U., Cabada, R. Z., & Barrón, M. L. (2020). Impact of augmented reality technology on academic achievement and motivation of students from public and private Mexican schools. A case study in a middle-school geometry course. Computers & Education, 145 , 103734.

Iftene, A., & Trandabăț, D. (2018). Enhancing the attractiveness of learning through Augmented Reality. Procedia Computer Science, 126 , 166–175.

Ipscience-help.thomsonreuters.com (2019). 2018 JCR data release. Retrieved from http://ipscience-help.thomsonreuters.com/incitesLiveJCR/8275-TRS.html .

Jerry, T. F. L., & Aaron, C. C. E. (2010). The impact of augmented reality software with inquiry-based learning on students' learning of kinematics graph. In 2010 2nd international conference on education technology and computer (vol. 2, pp. V2–1). IEEE.

Joo-Nagata, J., Abad, F. M., Giner, J. G. B., & García-Peñalvo, F. J. (2017). Augmented reality and pedestrian navigation through its implementation in m-learning and e-learning: Evaluation of an educational program in Chile. Computers & Education, 111 , 1–17.

Karambakhsh, A., Kamel, A., Sheng, B., Li, P., Yang, P., & Feng, D. D. (2019). Deep gesture interaction for augmented anatomy learning. International Journal of Information Management, 45 , 328–336.

Kaufmann, H., & Schmalstieg, D. (2018). Physics-based user interfaces for augmented reality. ACM Transactions on Computer-Human Interaction, 25 (5), 32.

Khan, D., Ullah, S., Ahmad, W., Cheng, Z., Jabeen, G., & Kato, H. (2019). A low-cost interactive writing board for primary education using distinct augmented reality markers. Sustainability, 11 (20), 5720.

Kiryakova, G., Angelova, N., & Yordanova, L. (2018). The potential of augmented reality to transform education into smart education. TEM Journal, 7 (3), 556.

Kitchenham, B. (2004). Procedures for performing systematic reviews. Keele, UK, Keele University, 33 (2004), 1–26.

Klopfer, E., & Squire, K. (2008). Environmental Detectives: The development of an augmented reality platform for environmental simulations. Educational Technology Research and Development, 56 (2), 203–228.

Klopfer, E., & Squire, K. (2019). Augmented reality and learning: A critical review. In J. Voogt & G. Knezek (Eds.), International handbook of information technology in primary and secondary education (pp. 325–338). Springer.

Klopfer, E., Squire, K., & Jenkins, H. (2008). Environmental detectives: PDAs as a window into a virtual simulated world. In Handbook of research on effective electronic gaming in education (pp. 143–166). Information Science Reference.

Koumpouros, Y. (2016). A systematic review on existing measures for the subjective assessment of rehabilitation and assistive robot devices. Journal of Healthcare Engineering, 2016 , 1–10. https://doi.org/10.1155/2016/1048964

Koumpouros, Y., & Kafazis, T. (2019). Wearables and mobile technologies in autism spectrum disorder interventions: A systematic literature review. Research in Autism Spectrum Disorders, 66 , https://doi.org/10.1016/j.rasd.2019.05.005 .

Koumpouros, Y., Papageorgiou, E., Karavasili, A., & Koureta, F. (2016). PYTHEIA: A scale for assessing rehabilitation and assistive robotics. World Academy of Science, Engineering and Technology, International Journal of Medical, Health, Biomedical, Bioengineering and Pharmaceutical Engineering, 10 (11), 522–526.

Kuikkaniemi, K., Turunen, M., Hakulinen, J., & Salo, K. (2014). Exploring user experience and user engagement in free mobile applications. International Journal of Mobile Human-Computer Interaction (IJMHCI), 6 (2), 35–50.

Kurniawan, M. H., & Witjaksono, G. (2018). Human anatomy learning systems using augmented reality on mobile application. Procedia Computer Science, 135 , 80–88.

Lamanauskas V., Vilkonis R. & Klangauskas A. (2007). Using information and communication technology for learning purposes: Students position on the issue. Europe Needs More Scientists—the Role of Eastern and Central European Symposium, 8–11 November 2006, Tartu, Estonia, 151–164.

Layona, R., Yulianto, B., & Tunardi, Y. (2018). Web based augmented reality for human body anatomy learning. Procedia Computer Science, 135 , 457–464.

Lee, K. M. (2012). Augmented reality in education and training. TechTrends, 56 (2), 13–21.

Li, Z., Huang, R., Li, G., & Song, Y. (2020). Augmented reality in education: A systematic review and synthesis of literature. Educational Research Review, 30 , 100326.

Lin, C. Y., Chai, H. C., Wang, J. Y., Chen, C. J., Liu, Y. H., Chen, C. W., & Huang, Y. M. (2016). Augmented reality in educational activities for children with disabilities. Displays, 42 , 51–54.

Liu, D. Y., Navarrete, C. C., & Chang, Y. C. (2019). Trends in augmented reality research: A systematic review of journal publications from 2008 to 2017. IEEE Access, 7 , 1019–1035.

López-García, A., Miralles-Martínez, P., & Maquilón, J. (2019). Design, application and effectiveness of an innovative augmented reality teaching proposal through 3P model. Applied Sciences, 9 (24), 5426.

Lorusso, M. L., Giorgetti, M., Travellini, S., Greci, L., Zangiacomi, A., Mondellini, M., & Reni, G. (2018). Giok the alien: An ar-based integrated system for the empowerment of problem-solving, pragmatic, and social skills in pre-school children. Sensors, 18 (7), 2368.

Macariu, C., Iftene, A., & Gîfu, D. (2020). Learn chemistry with augmented reality. Procedia Computer Science, 176 , 2133–2142.

Mahmood, F., Mahmood, E., Dorfman, R. G., Mitchell, J., Mahmood, F. U., Jones, S. B., & Matyal, R. (2018). Augmented reality and ultrasound education: Initial experience. Journal of Cardiothoracic and Vascular Anesthesia, 32 (3), 1363–1367.

Markets and Markets (2023). Augmented reality market industry report, size, segment, key players, scope, 2030. Retrieved April 11, 2023, from https://www.marketsandmarkets.com/Market-Reports/augmented-reality-market-82758548.html .

Martin, S., Diaz, G., Sancristobal, E., Gil, R., Castro, M., & Peire, J. (2011). New technology trends in education: Seven years of forecasts and convergence. Computers & Education, 57 (3), 1893–1906.

Mendes, H. C. M., Costa, C. I. A. B., da Silva, N. A., Leite, F. P., Esteves, A., & Lopes, D. S. (2020). PIÑATA: Pinpoint insertion of intravenous needles via augmented reality training assistance. Computerized Medical Imaging and Graphics, 82 , 101731.

Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems, 77 (12), 1321–1329.

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Prisma Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine, 6 (7), e1000097.

Montoya, M. H., Díaz, C. A., & Moreno, G. A. (2016). Evaluating the effect on user perception and performance of static and dynamic contents deployed in augmented reality based learning application. Eurasia Journal of Mathematics, Science and Technology Education, 13 (2), 301–317.

Moreno-Guerrero, A. J., Alonso García, S., Ramos Navas-Parejo, M., Campos-Soto, M. N., & Gómez García, G. (2020). Augmented reality as a resource for improving learning in the physical education classroom. International Journal of Environmental Research and Public Health, 17 (10), 3637.

Moro, C., Štromberga, Z., Raikos, A., & Stirling, A. (2017). The effectiveness of virtual and augmented reality in health sciences and medical anatomy. Anatomical Sciences Education, 10 (6), 549–559.

Mota, J. M., Ruiz-Rube, I., Dodero, J. M., & Arnedillo-Sánchez, I. (2018). Augmented reality mobile app development for all. Computers & Electrical Engineering, 65 , 250–260.

Mourtzis, D., Zogopoulos, V., & Vlachou, E. (2018). Augmented reality supported product design towards industry 4.0: A teaching factory paradigm. Procedia Manufacturing, 23 , 207–212.

Mylonas, G., Triantafyllis, C., & Amaxilatis, D. (2019). An augmented reality prototype for supporting IoT-based educational activities for energy-efficient school buildings. Electronic Notes in Theoretical Computer Science, 343 , 89–101.

Nguyen, N., Muilu, T., Dirin, A., & Alamäki, A. (2018). An interactive and augmented learning concept for orientation week in higher education. International Journal of Educational Technology in Higher Education, 15 (1), 1–15.

Nielsen, J. (1993). Usability engineering . Morgan Kaufmann Publishers Inc https://doi.org/10.1145/1508044.1508050 .

Nor Azlina Ab Aziz & Kamarulzaman Ab Aziz (2020). Application of augmented reality in education of attention deficit hyperactive disorder (ADHD) children. Neurological Disorders and Imaging Physics, Volume 4, Chapter 10, https://iopscience.iop.org/book/edit/978-0-7503-1822-8/chapter/bk978-0-7503-1822-8ch10.pdf .

Pombo, L., & Marques, M. M. (2017). Marker-based augmented reality application for mobile learning in an urban park: Steps to make it real under the EduPARK project. In 2017 International symposium on computers in education (SIIE) (pp. 1–5). IEEE.

Radu, I. (2014). Augmented reality in education: A meta-review and cross-media analysis. Personal and Ubiquitous Computing, 18 (6), 1533–1543.

Research and Markets (2023). Augmented reality in training and education global market report 2023. Research and Markets - Market Research Reports - Welcome. Retrieved April 11, 2023, from https://www.researchandmarkets.com/reports/5735149/augmented-reality-in-training-education-global .

Riihiaho, S. (2000). Experiences with usability evaluation methods. Helsinki University of Technology Laboratory of Information Processing Science.

Rossano, V., Lanzilotti, R., Cazzolla, A., & Roselli, T. (2020). Augmented reality to support geometry learning. IEEE Access, 8 , 107772–107780.

Sáez-López, J. M. S. L., Sevillano-García, M. L. S. G., Pascual-Sevillano, M. Á. P. S., Sáez-López, J. M., Sevillano-García-García, M. L., & de los Ángeles Pascual-Sevillano, M. (2019). Application of the ubiquitous game with augmented reality in Primary Education. Comunication Media Education Research Journal , 27 (2).

Safar, A. H., Al-Jafar, A. A., & Al-Yousefi, Z. H. (2016). The effectiveness of using augmented reality apps in teaching the English alphabet to kindergarten children: A case study in the State of Kuwait. EURASIA Journal of Mathematics, Science and Technology Education, 13 (2), 417–440.

Sahin, N., & Ozcan, M. F. (2019). Effects of augmented reality in teaching old Turkish Language mementoes on student achievement and motivation. Contemporary Educational Technology, 10 (2), 198–213.

Santos, M. E. C., Taketomi, T., Yamamoto, G., Rodrigo, M. M. T., Sandor, C., & Kato, H. (2016). Augmented reality as multimedia: The case for situated vocabulary learning. Research and Practice in Technology Enhanced Learning, 11 (1), 1–23.

Sargsyan, N., Bassy, R., Wei, X., Akula, S., Liu, J., Seals, C., & White, J. (2019). Augmented reality application to enhance training of lumbar puncture procedure: Preliminary results. In Proceedings of 32nd international conference on (Vol. 63, pp. 189–196).

Saundarajan, K., Osman, S., Kumar, J., Daud, M., Abu, M., & Pairan, M. (2020). Learning algebra using augmented reality: A preliminary investigation on the application of photomath for lower secondary education. International Journal of Emerging Technologies in Learning (iJET), 15 (16), 123–133.

Savitha, K.K, & Renumol, V.G. (2019). Effects of integrating augmented reality in early childhood special education. International Journal of Recent Technology and Engineering, 8 (3), 2277–3878.

Scaravetti, D., & Doroszewski, D. (2019). Augmented Reality experiment in higher education, for complex system appropriation in mechanical design. Procedia CIRP, 84 , 197–202.

Schall, G., Jetter, H.-C., & Reitmayr, G. (2009). Towards mobile augmented reality for spatially aware computing. Virtual Reality, 13 (4), 223–234.

Sin, A. K., & Zaman, H. B. (2010). Live solar system (LSS): Evaluation of an Augmented Reality book-based educational tool. In 2010 International symposium on information technology (vol. 1, pp. 1–6). IEEE.

Sonntag, D., Albuquerque, G., Magnor, M., & Bodensiek, O. (2019). Hybrid learning environments by data-driven augmented reality. Procedia Manufacturing, 31 , 32–37.

Spruijt-Metz, D., Hekler, E., Saranummi, N., Intille, S., Korhonen, I., Nilsen, W., et al. (2015). Building new computational models to support health behavior change and maintenance: New opportunities in behavioral research. Translational Behavioral Medicine, 5 , 335–346. https://doi.org/10.1007/s13142-015-0324-1

StatCounter (2023). Desktop vs mobile vs tablet market share worldwide. StatCounter Global Stats. https://gs.statcounter.com/platform-market-share/desktop-mobile-tablet .

Statista (2023a). mHealth (mobile health) industry market size projection from 2012 to 2020 (in billion U.S. dollars). Retrieved from https://www.statista.com/statistics/295771/mhealth-global-market-size/ .

Statista (2023b). Smartwatch OS share worldwide 2022 | Statistic. Retrieved from https://www.statista.com/statistics/750328/worldwide-smartwatch-market-share-by-platform/ .

Stone, P. W. (2002). Popping the (PICO) question in research and evidence-based practice. Applied Nursing Research . https://doi.org/10.1053/apnr.2002.34181

Sudarmilah, E., Irsyadi, F. Y. A., Purworini, D., Fatmawati, A., Haryanti, Y., Santoso, B., & Ustia, N. (2020). Improving knowledge about Indonesian culture with augmented reality gamification. In IOP conference series: Materials science and engineering (Vol. 830, No. 3, p. 032024). IOP Publishing.

Sungkur, R. K., Panchoo, A., & Bhoyroo, N. K. (2016). Augmented reality, the future of contextual mobile learning. Interactive Technology and Smart Education .

Tang, A., Owen, C. B., Biocca, F., & Mou, W. (2015). Examining the role of presence in mobile augmented reality through a virtual reality comparison. Computers in Human Behavior, 45 , 307–320.

Thees, M., Kapp, S., Strzys, M. P., Beil, F., Lukowicz, P., & Kuhn, J. (2020). Effects of augmented reality on learning and cognitive load in university physics laboratory courses. Computers in Human Behavior, 108 , 106316.

Tosto, C., Hasegawa, T., Mangina, E., et al. (2021). Exploring the effect of an augmented reality literacy programme for reading and spelling difficulties for children diagnosed with ADHD. Virtual Reality, 25 , 879–894. https://doi.org/10.1007/s10055-020-00485-z

Tsai, C. C. (2020). The effects of augmented reality to motivation and performance in EFL vocabulary learning. International Journal of Instruction, 13 (4).

Turkan, Y., Radkowski, R., Karabulut-Ilgu, A., Behzadan, A. H., & Chen, A. (2017). Mobile augmented reality for teaching structural analysis. Advanced Engineering Informatics, 34 , 90–100.

Uiphanit, T., Unekontee, J., Wattanaprapa, N., Jankaweekool, P., & Rakbumrung, W. (2020). Using augmented reality (AR) for enhancing Chinese vocabulary learning. International Journal of Emerging Technologies in Learning (IJET), 15 (17), 268–276.

Vega Garzón, J. C., Magrini, M. L., & Galembeck, E. (2017). Using augmented reality to teach and learn biochemistry. Biochemistry and Molecular Biology Education, 45 (5), 417–420.

Voogt, J., & Knezek, G. (Eds.). (2018). International handbook of information technology in primary and secondary education . Springer.

Wang, Y. H. (2017). Exploring the effectiveness of integrating augmented reality-based materials to support writing activities. Computers & Education, 113 , 162–176.

Wang, C., & Wang, A. (2021). Exploring the effects of augmented reality on language learning: A meta-analysis. Educational Technology & Society, 24 (2), 105–119.

Yilmaz, R. M. (2016). Educational magic toys developed with augmented reality technology for early childhood education. Computers in Human Behavior, 54 , 240–248.

Yip, J., Wong, S. H., Yick, K. L., Chan, K., & Wong, K. H. (2019). Improving quality of teaching and learning in classes by using augmented reality video. Computers & Education, 128 , 88–101.

Yuen, S. C. Y., & Yaoyuneyong, G. (2020). The use of augmented reality apps in K-12 education: A systematic review. Journal of Educational Technology & Society, 23 (4), 133–150.

Zhou, X., Tang, L., Lin, D., & Han, W. (2020). Virtual & augmented reality for biological microscope in experiment education. Virtual Reality & Intelligent Hardware, 2 (4), 316–329.

Download references

Acknowledgements

I would like to thank Ms Vasiliki Tsirogianni for helping in the collection of the initial pool of papers.

Not applicable.

Author information

Authors and affiliations.

Department of Public and Community Health, University of West Attica, Athens, Greece

Yiannis Koumpouros

You can also search for this author in PubMed   Google Scholar

Contributions

YK had the idea for the article, performed the literature search and data analysis, and drafted and critically revised the work.

Corresponding author

Correspondence to Yiannis Koumpouros .

Ethics declarations

Competing interests.

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Koumpouros, Y. Revealing the true potential and prospects of augmented reality in education. Smart Learn. Environ. 11 , 2 (2024). https://doi.org/10.1186/s40561-023-00288-0

Download citation

Received : 20 June 2023

Accepted : 18 December 2023

Published : 09 January 2024

DOI : https://doi.org/10.1186/s40561-023-00288-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Augmented reality
• Mixed reality

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Psychol

The Past, Present, and Future of Virtual and Augmented Reality Research: A Network and Cluster Analysis of the Literature

Pietro cipresso.

1 Applied Technology for Neuro-Psychology Lab, Istituto Auxologico Italiano, Milan, Italy

2 Department of Psychology, Catholic University of the Sacred Heart, Milan, Italy

Irene Alice Chicchi Giglioli

3 Instituto de Investigación e Innovación en Bioingeniería, Universitat Politècnica de València, Valencia, Spain

Mariano Alcañiz Raya

Giuseppe riva, associated data.

The recent appearance of low cost virtual reality (VR) technologies – like the Oculus Rift, the HTC Vive and the Sony PlayStation VR – and Mixed Reality Interfaces (MRITF) – like the Hololens – is attracting the attention of users and researchers suggesting it may be the next largest stepping stone in technological innovation. However, the history of VR technology is longer than it may seem: the concept of VR was formulated in the 1960s and the first commercial VR tools appeared in the late 1980s. For this reason, during the last 20 years, 100s of researchers explored the processes, effects, and applications of this technology producing 1000s of scientific papers. What is the outcome of this significant research work? This paper wants to provide an answer to this question by exploring, using advanced scientometric techniques, the existing research corpus in the field. We collected all the existent articles about VR in the Web of Science Core Collection scientific database, and the resultant dataset contained 21,667 records for VR and 9,944 for augmented reality (AR). The bibliographic record contained various fields, such as author, title, abstract, country, and all the references (needed for the citation analysis). The network and cluster analysis of the literature showed a composite panorama characterized by changes and evolutions over the time. Indeed, whether until 5 years ago, the main publication media on VR concerned both conference proceeding and journals, more recently journals constitute the main medium of communication. Similarly, if at first computer science was the leading research field, nowadays clinical areas have increased, as well as the number of countries involved in VR research. The present work discusses the evolution and changes over the time of the use of VR in the main areas of application with an emphasis on the future expected VR’s capacities, increases and challenges. We conclude considering the disruptive contribution that VR/AR/MRITF will be able to get in scientific fields, as well in human communication and interaction, as already happened with the advent of mobile phones by increasing the use and the development of scientific applications (e.g., in clinical areas) and by modifying the social communication and interaction among people.

Introduction

In the last 5 years, virtual reality (VR) and augmented reality (AR) have attracted the interest of investors and the general public, especially after Mark Zuckerberg bought Oculus for two billion dollars ( Luckerson, 2014 ; Castelvecchi, 2016 ). Currently, many other companies, such as Sony, Samsung, HTC, and Google are making huge investments in VR and AR ( Korolov, 2014 ; Ebert, 2015 ; Castelvecchi, 2016 ). However, if VR has been used in research for more than 25 years, and now there are 1000s of papers and many researchers in the field, comprising a strong, interdisciplinary community, AR has a more recent application history ( Burdea and Coiffet, 2003 ; Kim, 2005 ; Bohil et al., 2011 ; Cipresso and Serino, 2014 ; Wexelblat, 2014 ). The study of VR was initiated in the computer graphics field and has been extended to several disciplines ( Sutherland, 1965 , 1968 ; Mazuryk and Gervautz, 1996 ; Choi et al., 2015 ). Currently, videogames supported by VR tools are more popular than the past, and they represent valuables, work-related tools for neuroscientists, psychologists, biologists, and other researchers as well. Indeed, for example, one of the main research purposes lies from navigation studies that include complex experiments that could be done in a laboratory by using VR, whereas, without VR, the researchers would have to go directly into the field, possibly with limited use of intervention. The importance of navigation studies for the functional understanding of human memory in dementia has been a topic of significant interest for a long time, and, in 2014, the Nobel Prize in “Physiology or Medicine” was awarded to John M. O’Keefe, May-Britt Moser, and Edvard I. Moser for their discoveries of nerve cells in the brain that enable a sense of place and navigation. Journals and magazines have extended this knowledge by writing about “the brain GPS,” which gives a clear idea of the mechanism. A huge number of studies have been conducted in clinical settings by using VR ( Bohil et al., 2011 ; Serino et al., 2014 ), and Nobel Prize winner, Edvard I. Moser commented about the use of VR ( Minderer et al., 2016 ), highlighting its importance for research and clinical practice. Moreover, the availability of free tools for VR experimental and computational use has made it easy to access any field ( Riva et al., 2011 ; Cipresso, 2015 ; Brown and Green, 2016 ; Cipresso et al., 2016 ).

Augmented reality is a more recent technology than VR and shows an interdisciplinary application framework, in which, nowadays, education and learning seem to be the most field of research. Indeed, AR allows supporting learning, for example increasing-on content understanding and memory preservation, as well as on learning motivation. However, if VR benefits from clear and more definite fields of application and research areas, AR is still emerging in the scientific scenarios.

In this article, we present a systematic and computational analysis of the emerging interdisciplinary VR and AR fields in terms of various co-citation networks in order to explore the evolution of the intellectual structure of this knowledge domain over time.

Virtual Reality Concepts and Features

The concept of VR could be traced at the mid of 1960 when Ivan Sutherland in a pivotal manuscript attempted to describe VR as a window through which a user perceives the virtual world as if looked, felt, sounded real and in which the user could act realistically ( Sutherland, 1965 ).

Since that time and in accordance with the application area, several definitions have been formulated: for example, Fuchs and Bishop (1992) defined VR as “real-time interactive graphics with 3D models, combined with a display technology that gives the user the immersion in the model world and direct manipulation” ( Fuchs and Bishop, 1992 ); Gigante (1993) described VR as “The illusion of participation in a synthetic environment rather than external observation of such an environment. VR relies on a 3D, stereoscopic head-tracker displays, hand/body tracking and binaural sound. VR is an immersive, multi-sensory experience” ( Gigante, 1993 ); and “Virtual reality refers to immersive, interactive, multi-sensory, viewer-centered, 3D computer generated environments and the combination of technologies required building environments” ( Cruz-Neira, 1993 ).

As we can notice, these definitions, although different, highlight three common features of VR systems: immersion, perception to be present in an environment, and interaction with that environment ( Biocca, 1997 ; Lombard and Ditton, 1997 ; Loomis et al., 1999 ; Heeter, 2000 ; Biocca et al., 2001 ; Bailenson et al., 2006 ; Skalski and Tamborini, 2007 ; Andersen and Thorpe, 2009 ; Slater, 2009 ; Sundar et al., 2010 ). Specifically, immersion concerns the amount of senses stimulated, interactions, and the reality’s similarity of the stimuli used to simulate environments. This feature can depend on the properties of the technological system used to isolate user from reality ( Slater, 2009 ).

Higher or lower degrees of immersion can depend by three types of VR systems provided to the user:

• simple • Non-immersive systems are the simplest and cheapest type of VR applications that use desktops to reproduce images of the world.
• simple • Immersive systems provide a complete simulated experience due to the support of several sensory outputs devices such as head mounted displays (HMDs) for enhancing the stereoscopic view of the environment through the movement of the user’s head, as well as audio and haptic devices.
• simple • Semi-immersive systems such as Fish Tank VR are between the two above. They provide a stereo image of a three dimensional (3D) scene viewed on a monitor using a perspective projection coupled to the head position of the observer ( Ware et al., 1993 ). Higher technological immersive systems have showed a closest experience to reality, giving to the user the illusion of technological non-mediation and feeling him or her of “being in” or present in the virtual environment ( Lombard and Ditton, 1997 ). Furthermore, higher immersive systems, than the other two systems, can give the possibility to add several sensory outputs allowing that the interaction and actions were perceived as real ( Loomis et al., 1999 ; Heeter, 2000 ; Biocca et al., 2001 ).

Finally, the user’s VR experience could be disclosed by measuring presence, realism, and reality’s levels. Presence is a complex psychological feeling of “being there” in VR that involves the sensation and perception of physical presence, as well as the possibility to interact and react as if the user was in the real world ( Heeter, 1992 ). Similarly, the realism’s level corresponds to the degree of expectation that the user has about of the stimuli and experience ( Baños et al., 2000 , 2009 ). If the presented stimuli are similar to reality, VR user’s expectation will be congruent with reality expectation, enhancing VR experience. In the same way, higher is the degree of reality in interaction with the virtual stimuli, higher would be the level of realism of the user’s behaviors ( Baños et al., 2000 , 2009 ).

From Virtual to Augmented Reality

Looking chronologically on VR and AR developments, we can trace the first 3D immersive simulator in 1962, when Morton Heilig created Sensorama, a simulated experience of a motorcycle running through Brooklyn characterized by several sensory impressions, such as audio, olfactory, and haptic stimuli, including also wind to provide a realist experience ( Heilig, 1962 ). In the same years, Ivan Sutherland developed The Ultimate Display that, more than sound, smell, and haptic feedback, included interactive graphics that Sensorama didn’t provide. Furthermore, Philco developed the first HMD that together with The Sword of Damocles of Sutherland was able to update the virtual images by tracking user’s head position and orientation ( Sutherland, 1965 ). In the 70s, the University of North Carolina realized GROPE, the first system of force-feedback and Myron Krueger created VIDEOPLACE an Artificial Reality in which the users’ body figures were captured by cameras and projected on a screen ( Krueger et al., 1985 ). In this way two or more users could interact in the 2D-virtual space. In 1982, the US’ Air Force created the first flight simulator [Visually Coupled Airbone System Simulator (VCASS)] in which the pilot through an HMD could control the pathway and the targets. Generally, the 80’s were the years in which the first commercial devices began to emerge: for example, in 1985 the VPL company commercialized the DataGlove, glove sensors’ equipped able to measure the flexion of fingers, orientation and position, and identify hand gestures. Another example is the Eyephone, created in 1988 by the VPL Company, an HMD system for completely immerging the user in a virtual world. At the end of 80’s, Fake Space Labs created a Binocular-Omni-Orientational Monitor (BOOM), a complex system composed by a stereoscopic-displaying device, providing a moving and broad virtual environment, and a mechanical arm tracking. Furthermore, BOOM offered a more stable image and giving more quickly responses to movements than the HMD devices. Thanks to BOOM and DataGlove, the NASA Ames Research Center developed the Virtual Wind Tunnel in order to research and manipulate airflow in a virtual airplane or space ship. In 1992, the Electronic Visualization Laboratory of the University of Illinois created the CAVE Automatic Virtual Environment, an immersive VR system composed by projectors directed on three or more walls of a room.

More recently, many videogames companies have improved the development and quality of VR devices, like Oculus Rift, or HTC Vive that provide a wider field of view and lower latency. In addition, the actual HMD’s devices can be now combined with other tracker system as eye-tracking systems (FOVE), and motion and orientation sensors (e.g., Razer Hydra, Oculus Touch, or HTC Vive).

Simultaneously, at the beginning of 90’, the Boing Corporation created the first prototype of AR system for showing to employees how set up a wiring tool ( Carmigniani et al., 2011 ). At the same time, Rosenberg and Feiner developed an AR fixture for maintenance assistance, showing that the operator performance enhanced by added virtual information on the fixture to repair ( Rosenberg, 1993 ). In 1993 Loomis and colleagues produced an AR GPS-based system for helping the blind in the assisted navigation through adding spatial audio information ( Loomis et al., 1998 ). Always in the 1993 Julie Martin developed “Dancing in Cyberspace,” an AR theater in which actors interacted with virtual object in real time ( Cathy, 2011 ). Few years later, Feiner et al. (1997) developed the first Mobile AR System (MARS) able to add virtual information about touristic buildings ( Feiner et al., 1997 ). Since then, several applications have been developed: in Thomas et al. (2000) , created ARQuake, a mobile AR video game; in 2008 was created Wikitude that through the mobile camera, internet, and GPS could add information about the user’s environments ( Perry, 2008 ). In 2009 others AR applications, like AR Toolkit and SiteLens have been developed in order to add virtual information to the physical user’s surroundings. In 2011, Total Immersion developed D’Fusion, and AR system for designing projects ( Maurugeon, 2011 ). Finally, in 2013 and 2015, Google developed Google Glass and Google HoloLens, and their usability have begun to test in several field of application.

Virtual Reality Technologies

Technologically, the devices used in the virtual environments play an important role in the creation of successful virtual experiences. According to the literature, can be distinguished input and output devices ( Burdea et al., 1996 ; Burdea and Coiffet, 2003 ). Input devices are the ones that allow the user to communicate with the virtual environment, which can range from a simple joystick or keyboard to a glove allowing capturing finger movements or a tracker able to capture postures. More in detail, keyboard, mouse, trackball, and joystick represent the desktop input devices easy to use, which allow the user to launch continuous and discrete commands or movements to the environment. Other input devices can be represented by tracking devices as bend-sensing gloves that capture hand movements, postures and gestures, or pinch gloves that detect the fingers movements, and trackers able to follow the user’s movements in the physical world and translate them in the virtual environment.

On the contrary, the output devices allow the user to see, hear, smell, or touch everything that happens in the virtual environment. As mentioned above, among the visual devices can be found a wide range of possibilities, from the simplest or least immersive (monitor of a computer) to the most immersive one such as VR glasses or helmets or HMD or CAVE systems.

Furthermore, auditory, speakers, as well as haptic output devices are able to stimulate body senses providing a more real virtual experience. For example, haptic devices can stimulate the touch feeling and force models in the user.

Virtual Reality Applications

Since its appearance, VR has been used in different fields, as for gaming ( Zyda, 2005 ; Meldrum et al., 2012 ), military training ( Alexander et al., 2017 ), architectural design ( Song et al., 2017 ), education ( Englund et al., 2017 ), learning and social skills training ( Schmidt et al., 2017 ), simulations of surgical procedures ( Gallagher et al., 2005 ), assistance to the elderly or psychological treatments are other fields in which VR is bursting strongly ( Freeman et al., 2017 ; Neri et al., 2017 ). A recent and extensive review of Slater and Sanchez-Vives (2016) reported the main VR application evidences, including weakness and advantages, in several research areas, such as science, education, training, physical training, as well as social phenomena, moral behaviors, and could be used in other fields, like travel, meetings, collaboration, industry, news, and entertainment. Furthermore, another review published this year by Freeman et al. (2017) focused on VR in mental health, showing the efficacy of VR in assessing and treating different psychological disorders as anxiety, schizophrenia, depression, and eating disorders.

There are many possibilities that allow the use of VR as a stimulus, replacing real stimuli, recreating experiences, which in the real world would be impossible, with a high realism. This is why VR is widely used in research on new ways of applying psychological treatment or training, for example, to problems arising from phobias (agoraphobia, phobia to fly, etc.) ( Botella et al., 2017 ). Or, simply, it is used like improvement of the traditional systems of motor rehabilitation ( Llorens et al., 2014 ; Borrego et al., 2016 ), developing games that ameliorate the tasks. More in detail, in psychological treatment, Virtual Reality Exposure Therapy (VRET) has showed its efficacy, allowing to patients to gradually face fear stimuli or stressed situations in a safe environment where the psychological and physiological reactions can be controlled by the therapist ( Botella et al., 2017 ).

Augmented Reality Concept

Milgram and Kishino (1994) , conceptualized the Virtual-Reality Continuum that takes into consideration four systems: real environment, augmented reality (AR), augmented virtuality, and virtual environment. AR can be defined a newer technological system in which virtual objects are added to the real world in real-time during the user’s experience. Per Azuma et al. (2001) an AR system should: (1) combine real and virtual objects in a real environment; (2) run interactively and in real-time; (3) register real and virtual objects with each other. Furthermore, even if the AR experiences could seem different from VRs, the quality of AR experience could be considered similarly. Indeed, like in VR, feeling of presence, level of realism, and the degree of reality represent the main features that can be considered the indicators of the quality of AR experiences. Higher the experience is perceived as realistic, and there is congruence between the user’s expectation and the interaction inside the AR environments, higher would be the perception of “being there” physically, and at cognitive and emotional level. The feeling of presence, both in AR and VR environments, is important in acting behaviors like the real ones ( Botella et al., 2005 ; Juan et al., 2005 ; Bretón-López et al., 2010 ; Wrzesien et al., 2013 ).

Augmented Reality Technologies

Technologically, the AR systems, however various, present three common components, such as a geospatial datum for the virtual object, like a visual marker, a surface to project virtual elements to the user, and an adequate processing power for graphics, animation, and merging of images, like a pc and a monitor ( Carmigniani et al., 2011 ). To run, an AR system must also include a camera able to track the user movement for merging the virtual objects, and a visual display, like glasses through that the user can see the virtual objects overlaying to the physical world. To date, two-display systems exist, a video see-through (VST) and an optical see-though (OST) AR systems ( Botella et al., 2005 ; Juan et al., 2005 , 2007 ). The first one, disclosures virtual objects to the user by capturing the real objects/scenes with a camera and overlaying virtual objects, projecting them on a video or a monitor, while the second one, merges the virtual object on a transparent surface, like glasses, through the user see the added elements. The main difference between the two systems is the latency: an OST system could require more time to display the virtual objects than a VST system, generating a time lag between user’s action and performance and the detection of them by the system.

Augmented Reality Applications

Although AR is a more recent technology than VR, it has been investigated and used in several research areas such as architecture ( Lin and Hsu, 2017 ), maintenance ( Schwald and De Laval, 2003 ), entertainment ( Ozbek et al., 2004 ), education ( Nincarean et al., 2013 ; Bacca et al., 2014 ; Akçayır and Akçayır, 2017 ), medicine ( De Buck et al., 2005 ), and psychological treatments ( Juan et al., 2005 ; Botella et al., 2005 , 2010 ; Bretón-López et al., 2010 ; Wrzesien et al., 2011a , b , 2013 ; see the review Chicchi Giglioli et al., 2015 ). More in detail, in education several AR applications have been developed in the last few years showing the positive effects of this technology in supporting learning, such as an increased-on content understanding and memory preservation, as well as on learning motivation ( Radu, 2012 , 2014 ). For example, Ibáñez et al. (2014) developed a AR application on electromagnetism concepts’ learning, in which students could use AR batteries, magnets, cables on real superficies, and the system gave a real-time feedback to students about the correctness of the performance, improving in this way the academic success and motivation ( Di Serio et al., 2013 ). Deeply, AR system allows the possibility to learn visualizing and acting on composite phenomena that traditionally students study theoretically, without the possibility to see and test in real world ( Chien et al., 2010 ; Chen et al., 2011 ).

As well in psychological health, the number of research about AR is increasing, showing its efficacy above all in the treatment of psychological disorder (see the reviews Baus and Bouchard, 2014 ; Chicchi Giglioli et al., 2015 ). For example, in the treatment of anxiety disorders, like phobias, AR exposure therapy (ARET) showed its efficacy in one-session treatment, maintaining the positive impact in a follow-up at 1 or 3 month after. As VRET, ARET provides a safety and an ecological environment where any kind of stimulus is possible, allowing to keep control over the situation experienced by the patients, gradually generating situations of fear or stress. Indeed, in situations of fear, like the phobias for small animals, AR applications allow, in accordance with the patient’s anxiety, to gradually expose patient to fear animals, adding new animals during the session or enlarging their or increasing the speed. The various studies showed that AR is able, at the beginning of the session, to activate patient’s anxiety, for reducing after 1 h of exposition. After the session, patients even more than to better manage animal’s fear and anxiety, ware able to approach, interact, and kill real feared animals.

Materials and Methods

Data collection.

The input data for the analyses were retrieved from the scientific database Web of Science Core Collection ( Falagas et al., 2008 ) and the search terms used were “Virtual Reality” and “Augmented Reality” regarding papers published during the whole timespan covered.

Web of science core collection is composed of: Citation Indexes, Science Citation Index Expanded (SCI-EXPANDED) –1970-present, Social Sciences Citation Index (SSCI) –1970-present, Arts and Humanities Citation Index (A&HCI) –1975-present, Conference Proceedings Citation Index- Science (CPCI-S) –1990-present, Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH) –1990-present, Book Citation Index– Science (BKCI-S) –2009-present, Book Citation Index– Social Sciences & Humanities (BKCI-SSH) –2009-present, Emerging Sources Citation Index (ESCI) –2015-present, Chemical Indexes, Current Chemical Reactions (CCR-EXPANDED) –2009-present (Includes Institut National de la Propriete Industrielle structure data back to 1840), Index Chemicus (IC) –2009-present.

The resultant dataset contained a total of 21,667 records for VR and 9,944 records for AR. The bibliographic record contained various fields, such as author, title, abstract, and all of the references (needed for the citation analysis). The research tool to visualize the networks was Cite space v.4.0.R5 SE (32 bit) ( Chen, 2006 ) under Java Runtime v.8 update 91 (build 1.8.0_91-b15). Statistical analyses were conducted using Stata MP-Parallel Edition, Release 14.0, StataCorp LP. Additional information can be found in Supplementary Data Sheet 1 .

The betweenness centrality of a node in a network measures the extent to which the node is part of paths that connect an arbitrary pair of nodes in the network ( Freeman, 1977 ; Brandes, 2001 ; Chen, 2006 ).

Structural metrics include betweenness centrality, modularity, and silhouette. Temporal and hybrid metrics include citation burstness and novelty. All the algorithms are detailed ( Chen et al., 2010 ).

The analysis of the literature on VR shows a complex panorama. At first sight, according to the document-type statistics from the Web of Science (WoS), proceedings papers were used extensively as outcomes of research, comprising almost 48% of the total (10,392 proceedings), with a similar number of articles on the subject amounting to about 47% of the total of 10, 199 articles. However, if we consider only the last 5 years (7,755 articles representing about 36% of the total), the situation changes with about 57% for articles (4,445) and about 33% for proceedings (2,578). Thus, it is clear that VR field has changed in areas other than at the technological level.

About the subject category, nodes and edges are computed as co-occurring subject categories from the Web of Science “Category” field in all the articles.

According to the subject category statistics from the WoS, computer science is the leading category, followed by engineering, and, together, they account for 15,341 articles, which make up about 71% of the total production. However, if we consider just the last 5 years, these categories reach only about 55%, with a total of 4,284 articles (Table ​ (Table1 1 and Figure ​ Figure1 1 ).

Category statistics from the WoS for the entire period and the last 5 years.

Category from the WoS: network for the last 5 years.

The evidence is very interesting since it highlights that VR is doing very well as new technology with huge interest in hardware and software components. However, with respect to the past, we are witnessing increasing numbers of applications, especially in the medical area. In particular, note its inclusion in the top 10 list of rehabilitation and clinical neurology categories (about 10% of the total production in the last 5 years). It also is interesting that neuroscience and neurology, considered together, have shown an increase from about 12% to about 18.6% over the last 5 years. However, historic areas, such as automation and control systems, imaging science and photographic technology, and robotics, which had accounted for about 14.5% of the total articles ever produced were not even in the top 10 for the last 5 years, with each one accounting for less than 4%.

About the countries, nodes and edges are computed as networks of co-authors countries. Multiple occurrency of a country in the same paper are counted once.

The countries that were very involved in VR research have published for about 47% of the total (10,200 articles altogether). Of the 10,200 articles, the United States, China, England, and Germany published 4921, 2384, 1497, and 1398, respectively. The situation remains the same if we look at the articles published over the last 5 years. However, VR contributions also came from all over the globe, with Japan, Canada, Italy, France, Spain, South Korea, and Netherlands taking positions of prominence, as shown in Figure ​ Figure2 2 .

Country network (node dimension represents centrality).

Network analysis was conducted to calculate and to represent the centrality index ( Freeman, 1977 ; Brandes, 2001 ), i.e., the dimension of the node in Figure ​ Figure2. 2 . The top-ranked country, with a centrality index of 0.26, was the United States (2011), and England was second, with a centrality index of 0.25. The third, fourth, and fifth countries were Germany, Italy, and Australia, with centrality indices of 0.15, 0.15, and 0.14, respectively.

About the Institutions, nodes and edges are computed as networks of co-authors Institutions (Figure ​ (Figure3 3 ).

Network of institutions: the dimensions of the nodes represent centrality.

The top-level institutions in VR were in the United States, where three universities were ranked as the top three in the world for published articles; these universities were the University of Illinois (159), the University of South California (147), and the University of Washington (146). The United States also had the eighth-ranked university, which was Iowa State University (116). The second country in the ranking was Canada, with the University of Toronto, which was ranked fifth with 125 articles and McGill University, ranked 10 th with 103 articles.

Other countries in the top-ten list were Netherlands, with the Delft University of Technology ranked fourth with 129 articles; Italy, with IRCCS Istituto Auxologico Italiano, ranked sixth (with the same number of publication of the institution ranked fifth) with 125 published articles; England, which was ranked seventh with 125 articles from the University of London’s Imperial College of Science, Technology, and Medicine; and China with 104 publications, with the Chinese Academy of Science, ranked ninth. Italy’s Istituto Auxologico Italiano, which was ranked fifth, was the only non-university institution ranked in the top-10 list for VR research (Figure ​ (Figure3 3 ).

About the Journals, nodes, and edges are computed as journal co-citation networks among each journals in the corresponding field.

The top-ranked Journals for citations in VR are Presence: Teleoperators & Virtual Environments with 2689 citations and CyberPsychology & Behavior (Cyberpsychol BEHAV) with 1884 citations; however, looking at the last 5 years, the former had increased the citations, but the latter had a far more significant increase, from about 70% to about 90%, i.e., an increase from 1029 to 1147.

Following the top two journals, IEEE Computer Graphics and Applications ( IEEE Comput Graph) and Advanced Health Telematics and Telemedicine ( St HEAL T) were both left out of the top-10 list based on the last 5 years. The data for the last 5 years also resulted in the inclusion of Experimental Brain Research ( Exp BRAIN RES) (625 citations), Archives of Physical Medicine and Rehabilitation ( Arch PHYS MED REHAB) (622 citations), and Plos ONE (619 citations) in the top-10 list of three journals, which highlighted the categories of rehabilitation and clinical neurology and neuroscience and neurology. Journal co-citation analysis is reported in Figure ​ Figure4, 4 , which clearly shows four distinct clusters.

Co-citation network of journals: the dimensions of the nodes represent centrality. Full list of official abbreviations of WoS journals can be found here: https://images.webofknowledge.com/images/help/WOS/A_abrvjt.html .

Network analysis was conducted to calculate and to represent the centrality index, i.e., the dimensions of the nodes in Figure ​ Figure4. 4 . The top-ranked item by centrality was Cyberpsychol BEHAV, with a centrality index of 0.29. The second-ranked item was Arch PHYS MED REHAB, with a centrality index of 0.23. The third was Behaviour Research and Therapy (Behav RES THER), with a centrality index of 0.15. The fourth was BRAIN, with a centrality index of 0.14. The fifth was Exp BRAIN RES, with a centrality index of 0.11.

Who’s Who in VR Research

Authors are the heart and brain of research, and their roles in a field are to define the past, present, and future of disciplines and to make significant breakthroughs to make new ideas arise (Figure ​ (Figure5 5 ).

Network of authors’ numbers of publications: the dimensions of the nodes represent the centrality index, and the dimensions of the characters represent the author’s rank.

Virtual reality research is very young and changing with time, but the top-10 authors in this field have made fundamentally significant contributions as pioneers in VR and taking it beyond a mere technological development. The purpose of the following highlights is not to rank researchers; rather, the purpose is to identify the most active researchers in order to understand where the field is going and how they plan for it to get there.

The top-ranked author is Riva G, with 180 publications. The second-ranked author is Rizzo A, with 101 publications. The third is Darzi A, with 97 publications. The forth is Aggarwal R, with 94 publications. The six authors following these three are Slater M, Alcaniz M, Botella C, Wiederhold BK, Kim SI, and Gutierrez-Maldonado J with 90, 90, 85, 75, 59, and 54 publications, respectively (Figure ​ (Figure6 6 ).

Authors’ co-citation network: the dimensions of the nodes represent centrality index, and the dimensions of the characters represent the author’s rank. The 10 authors that appear on the top-10 list are considered to be the pioneers of VR research.

Considering the last 5 years, the situation remains similar, with three new entries in the top-10 list, i.e., Muhlberger A, Cipresso P, and Ahmed K ranked 7th, 8th, and 10th, respectively.

The authors’ publications number network shows the most active authors in VR research. Another relevant analysis for our focus on VR research is to identify the most cited authors in the field.

For this purpose, the authors’ co-citation analysis highlights the authors in term of their impact on the literature considering the entire time span of the field ( White and Griffith, 1981 ; González-Teruel et al., 2015 ; Bu et al., 2016 ). The idea is to focus on the dynamic nature of the community of authors who contribute to the research.

Normally, authors with higher numbers of citations tend to be the scholars who drive the fundamental research and who make the most meaningful impacts on the evolution and development of the field. In the following, we identified the most-cited pioneers in the field of VR Research.

The top-ranked author by citation count is Gallagher (2001), with 694 citations. Second is Seymour (2004), with 668 citations. Third is Slater (1999), with 649 citations. Fourth is Grantcharov (2003), with 563 citations. Fifth is Riva (1999), with 546 citations. Sixth is Aggarwal (2006), with 505 citations. Seventh is Satava (1994), with 477 citations. Eighth is Witmer (2002), with 454 citations. Ninth is Rothbaum (1996), with 448 citations. Tenth is Cruz-neira (1995), with 416 citations.

Citation Network and Cluster Analyses for VR

Another analysis that can be used is the analysis of document co-citation, which allows us to focus on the highly-cited documents that generally are also the most influential in the domain ( Small, 1973 ; González-Teruel et al., 2015 ; Orosz et al., 2016 ).

The top-ranked article by citation counts is Seymour (2002) in Cluster #0, with 317 citations. The second article is Grantcharov (2004) in Cluster #0, with 286 citations. The third is Holden (2005) in Cluster #2, with 179 citations. The 4th is Gallagher et al. (2005) in Cluster #0, with 171 citations. The 5th is Ahlberg (2007) in Cluster #0, with 142 citations. The 6th is Parsons (2008) in Cluster #4, with 136 citations. The 7th is Powers (2008) in Cluster #4, with 134 citations. The 8th is Aggarwal (2007) in Cluster #0, with 121 citations. The 9th is Reznick (2006) in Cluster #0, with 121 citations. The 10th is Munz (2004) in Cluster #0, with 117 citations.

The network of document co-citations is visually complex (Figure ​ (Figure7) 7 ) because it includes 1000s of articles and the links among them. However, this analysis is very important because can be used to identify the possible conglomerate of knowledge in the area, and this is essential for a deep understanding of the area. Thus, for this purpose, a cluster analysis was conducted ( Chen et al., 2010 ; González-Teruel et al., 2015 ; Klavans and Boyack, 2015 ). Figure ​ Figure8 8 shows the clusters, which are identified with the two algorithms in Table ​ Table2 2 .

Network of document co-citations: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank, and the numbers represent the strengths of the links. It is possible to identify four historical phases (colors: blue, green, yellow, and red) from the past VR research to the current research.

Document co-citation network by cluster: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank and the red writing reports the name of the cluster with a short description that was produced with the mutual information algorithm; the clusters are identified with colored polygons.

Cluster ID and silhouettes as identified with two algorithms ( Chen et al., 2010 ).

The identified clusters highlight clear parts of the literature of VR research, making clear and visible the interdisciplinary nature of this field. However, the dynamics to identify the past, present, and future of VR research cannot be clear yet. We analysed the relationships between these clusters and the temporal dimensions of each article. The results are synthesized in Figure ​ Figure9. 9 . It is clear that cluster #0 (laparoscopic skill), cluster #2 (gaming and rehabilitation), cluster #4 (therapy), and cluster #14 (surgery) are the most popular areas of VR research. (See Figure ​ Figure9 9 and Table ​ Table2 2 to identify the clusters.) From Figure ​ Figure9, 9 , it also is possible to identify the first phase of laparoscopic skill (cluster #6) and therapy (cluster #7). More generally, it is possible to identify four historical phases (colors: blue, green, yellow, and red) from the past VR research to the current research.

Network of document co-citation: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank and the red writing on the right hand side reports the number of the cluster, such as in Table ​ Table2, 2 , with a short description that was extracted accordingly.

We were able to identify the top 486 references that had the most citations by using burst citations algorithm. Citation burst is an indicator of a most active area of research. Citation burst is a detection of a burst event, which can last for multiple years as well as a single year. A citation burst provides evidence that a particular publication is associated with a surge of citations. The burst detection was based on Kleinberg’s algorithm ( Kleinberg, 2002 , 2003 ). The top-ranked document by bursts is Seymour (2002) in Cluster #0, with bursts of 88.93. The second is Grantcharov (2004) in Cluster #0, with bursts of 51.40. The third is Saposnik (2010) in Cluster #2, with bursts of 40.84. The fourth is Rothbaum (1995) in Cluster #7, with bursts of 38.94. The fifth is Holden (2005) in Cluster #2, with bursts of 37.52. The sixth is Scott (2000) in Cluster #0, with bursts of 33.39. The seventh is Saposnik (2011) in Cluster #2, with bursts of 33.33. The eighth is Burdea et al. (1996) in Cluster #3, with bursts of 32.42. The ninth is Burdea and Coiffet (2003) in Cluster #22, with bursts of 31.30. The 10th is Taffinder (1998) in Cluster #6, with bursts of 30.96 (Table ​ (Table3 3 ).

Cluster ID and references of burst article.

Citation Network and Cluster Analyses for AR

Looking at Augmented Reality scenario, the top ranked item by citation counts is Azuma (1997) in Cluster #0, with citation counts of 231. The second one is Azuma et al. (2001) in Cluster #0, with citation counts of 220. The third is Van Krevelen (2010) in Cluster #5, with citation counts of 207. The 4th is Lowe (2004) in Cluster #1, with citation counts of 157. The 5th is Wu (2013) in Cluster #4, with citation counts of 144. The 6th is Dunleavy (2009) in Cluster #4, with citation counts of 122. The 7th is Zhou (2008) in Cluster #5, with citation counts of 118. The 8th is Bay (2008) in Cluster #1, with citation counts of 117. The 9th is Newcombe (2011) in Cluster #1, with citation counts of 109. The 10th is Carmigniani et al. (2011) in Cluster #5, with citation counts of 104.

The network of document co-citations is visually complex (Figure ​ (Figure10) 10 ) because it includes 1000s of articles and the links among them. However, this analysis is very important because can be used to identify the possible conglomerate of knowledge in the area, and this is essential for a deep understanding of the area. Thus, for this purpose, a cluster analysis was conducted ( Chen et al., 2010 ; González-Teruel et al., 2015 ; Klavans and Boyack, 2015 ). Figure ​ Figure11 11 shows the clusters, which are identified with the two algorithms in Table ​ Table3 3 .

Network of document co-citations: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank, and the numbers represent the strengths of the links. It is possible to identify four historical phases (colors: blue, green, yellow, and red) from the past AR research to the current research.

The identified clusters highlight clear parts of the literature of AR research, making clear and visible the interdisciplinary nature of this field. However, the dynamics to identify the past, present, and future of AR research cannot be clear yet. We analysed the relationships between these clusters and the temporal dimensions of each article. The results are synthesized in Figure ​ Figure12. 12 . It is clear that cluster #1 (tracking), cluster #4 (education), and cluster #5 (virtual city environment) are the current areas of AR research. (See Figure ​ Figure12 12 and Table ​ Table3 3 to identify the clusters.) It is possible to identify four historical phases (colors: blue, green, yellow, and red) from the past AR research to the current research.

We were able to identify the top 394 references that had the most citations by using burst citations algorithm. Citation burst is an indicator of a most active area of research. Citation burst is a detection of a burst event, which can last for multiple years as well as a single year. A citation burst provides evidence that a particular publication is associated with a surge of citations. The burst detection was based on Kleinberg’s algorithm ( Kleinberg, 2002 , 2003 ). The top ranked document by bursts is Azuma (1997) in Cluster #0, with bursts of 101.64. The second one is Azuma et al. (2001) in Cluster #0, with bursts of 84.23. The third is Lowe (2004) in Cluster #1, with bursts of 64.07. The 4th is Van Krevelen (2010) in Cluster #5, with bursts of 50.99. The 5th is Wu (2013) in Cluster #4, with bursts of 47.23. The 6th is Hartley (2000) in Cluster #0, with bursts of 37.71. The 7th is Dunleavy (2009) in Cluster #4, with bursts of 33.22. The 8th is Kato (1999) in Cluster #0, with bursts of 32.16. The 9th is Newcombe (2011) in Cluster #1, with bursts of 29.72. The 10th is Feiner (1993) in Cluster #8, with bursts of 29.46 (Table ​ (Table4 4 ).

Our findings have profound implications for two reasons. At first the present work highlighted the evolution and development of VR and AR research and provided a clear perspective based on solid data and computational analyses. Secondly our findings on VR made it profoundly clear that the clinical dimension is one of the most investigated ever and seems to increase in quantitative and qualitative aspects, but also include technological development and article in computer science, engineer, and allied sciences.

Figure ​ Figure9 9 clarifies the past, present, and future of VR research. The outset of VR research brought a clearly-identifiable development in interfaces for children and medicine, routine use and behavioral-assessment, special effects, systems perspectives, and tutorials. This pioneering era evolved in the period that we can identify as the development era, because it was the period in which VR was used in experiments associated with new technological impulses. Not surprisingly, this was exactly concomitant with the new economy era in which significant investments were made in information technology, and it also was the era of the so-called ‘dot-com bubble’ in the late 1990s. The confluence of pioneering techniques into ergonomic studies within this development era was used to develop the first effective clinical systems for surgery, telemedicine, human spatial navigation, and the first phase of the development of therapy and laparoscopic skills. With the new millennium, VR research switched strongly toward what we can call the clinical-VR era, with its strong emphasis on rehabilitation, neurosurgery, and a new phase of therapy and laparoscopic skills. The number of applications and articles that have been published in the last 5 years are in line with the new technological development that we are experiencing at the hardware level, for example, with so many new, HMDs, and at the software level with an increasing number of independent programmers and VR communities.

Finally, Figure ​ Figure12 12 identifies clusters of the literature of AR research, making clear and visible the interdisciplinary nature of this field. The dynamics to identify the past, present, and future of AR research cannot be clear yet, but analyzing the relationships between these clusters and the temporal dimensions of each article tracking, education, and virtual city environment are the current areas of AR research. AR is a new technology that is showing its efficacy in different research fields, and providing a novel way to gather behavioral data and support learning, training, and clinical treatments.

Looking at scientific literature conducted in the last few years, it might appear that most developments in VR and AR studies have focused on clinical aspects. However, the reality is more complex; thus, this perception should be clarified. Although researchers publish studies on the use of VR in clinical settings, each study depends on the technologies available. Industrial development in VR and AR changed a lot in the last 10 years. In the past, the development involved mainly hardware solutions while nowadays, the main efforts pertain to the software when developing virtual solutions. Hardware became a commodity that is often available at low cost. On the other hand, software needs to be customized each time, per each experiment, and this requires huge efforts in term of development. Researchers in AR and VR today need to be able to adapt software in their labs.

Virtual reality and AR developments in this new clinical era rely on computer science and vice versa. The future of VR and AR is becoming more technological than before, and each day, new solutions and products are coming to the market. Both from software and hardware perspectives, the future of AR and VR depends on huge innovations in all fields. The gap between the past and the future of AR and VR research is about the “realism” that was the key aspect in the past versus the “interaction” that is the key aspect now. First 30 years of VR and AR consisted of a continuous research on better resolution and improved perception. Now, researchers already achieved a great resolution and need to focus on making the VR as realistic as possible, which is not simple. In fact, a real experience implies a realistic interaction and not just great resolution. Interactions can be improved in infinite ways through new developments at hardware and software levels.

Interaction in AR and VR is going to be “embodied,” with implication for neuroscientists that are thinking about new solutions to be implemented into the current systems ( Blanke et al., 2015 ; Riva, 2018 ; Riva et al., 2018 ). For example, the use of hands with contactless device (i.e., without gloves) makes the interaction in virtual environments more natural. The Leap Motion device 1 allows one to use of hands in VR without the use of gloves or markers. This simple and low-cost device allows the VR users to interact with virtual objects and related environments in a naturalistic way. When technology is able to be transparent, users can experience increased sense of being in the virtual environments (the so-called sense of presence).

Other forms of interactions are possible and have been developing continuously. For example, tactile and haptic device able to provide a continuous feedback to the users, intensifying their experience also by adding components, such as the feeling of touch and the physical weight of virtual objects, by using force feedback. Another technology available at low cost that facilitates interaction is the motion tracking system, such as Microsoft Kinect, for example. Such technology allows one to track the users’ bodies, allowing them to interact with the virtual environments using body movements, gestures, and interactions. Most HMDs use an embedded system to track HMD position and rotation as well as controllers that are generally placed into the user’s hands. This tracking allows a great degree of interaction and improves the overall virtual experience.

A final emerging approach is the use of digital technologies to simulate not only the external world but also the internal bodily signals ( Azevedo et al., 2017 ; Riva et al., 2017 ): interoception, proprioception and vestibular input. For example, Riva et al. (2017) recently introduced the concept of “sonoception” ( www.sonoception.com ), a novel non-invasive technological paradigm based on wearable acoustic and vibrotactile transducers able to alter internal bodily signals. This approach allowed the development of an interoceptive stimulator that is both able to assess interoceptive time perception in clinical patients ( Di Lernia et al., 2018b ) and to enhance heart rate variability (the short-term vagally mediated component—rMSSD) through the modulation of the subjects’ parasympathetic system ( Di Lernia et al., 2018a ).

In this scenario, it is clear that the future of VR and AR research is not just in clinical applications, although the implications for the patients are huge. The continuous development of VR and AR technologies is the result of research in computer science, engineering, and allied sciences. The reasons for which from our analyses emerged a “clinical era” are threefold. First, all clinical research on VR and AR includes also technological developments, and new technological discoveries are being published in clinical or technological journals but with clinical samples as main subject. As noted in our research, main journals that publish numerous articles on technological developments tested with both healthy and patients include Presence: Teleoperators & Virtual Environments, Cyberpsychology & Behavior (Cyberpsychol BEHAV), and IEEE Computer Graphics and Applications (IEEE Comput Graph). It is clear that researchers in psychology, neuroscience, medicine, and behavioral sciences in general have been investigating whether the technological developments of VR and AR are effective for users, indicating that clinical behavioral research has been incorporating large parts of computer science and engineering. A second aspect to consider is the industrial development. In fact, once a new technology is envisioned and created it goes for a patent application. Once the patent is sent for registration the new technology may be made available for the market, and eventually for journal submission and publication. Moreover, most VR and AR research that that proposes the development of a technology moves directly from the presenting prototype to receiving the patent and introducing it to the market without publishing the findings in scientific paper. Hence, it is clear that if a new technology has been developed for industrial market or consumer, but not for clinical purpose, the research conducted to develop such technology may never be published in a scientific paper. Although our manuscript considered published researches, we have to acknowledge the existence of several researches that have not been published at all. The third reason for which our analyses highlighted a “clinical era” is that several articles on VR and AR have been considered within the Web of Knowledge database, that is our source of references. In this article, we referred to “research” as the one in the database considered. Of course, this is a limitation of our study, since there are several other databases that are of big value in the scientific community, such as IEEE Xplore Digital Library, ACM Digital Library, and many others. Generally, the most important articles in journals published in these databases are also included in the Web of Knowledge database; hence, we are convinced that our study considered the top-level publications in computer science or engineering. Accordingly, we believe that this limitation can be overcome by considering the large number of articles referenced in our research.

Considering all these aspects, it is clear that clinical applications, behavioral aspects, and technological developments in VR and AR research are parts of a more complex situation compared to the old platforms used before the huge diffusion of HMD and solutions. We think that this work might provide a clearer vision for stakeholders, providing evidence of the current research frontiers and the challenges that are expected in the future, highlighting all the connections and implications of the research in several fields, such as clinical, behavioral, industrial, entertainment, educational, and many others.

Author Contributions

PC and GR conceived the idea. PC made data extraction and the computational analyses and wrote the first draft of the article. IG revised the introduction adding important information for the article. PC, IG, MR, and GR revised the article and approved the last version of the article after important input to the article rationale.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The reviewer GC declared a shared affiliation, with no collaboration, with the authors PC and GR to the handling Editor at the time of the review.

1 https://www.leapmotion.com/

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2018.02086/full#supplementary-material

• Akçayır M., Akçayır G. (2017). Advantages and challenges associated with augmented reality for education: a systematic review of the literature. Educ. Res. Rev. 20 1–11. 10.1016/j.edurev.2016.11.002 [ CrossRef ] [ Google Scholar ]
• Alexander T., Westhoven M., Conradi J. (2017). “Virtual environments for competency-oriented education and training,” in Advances in Human Factors, Business Management, Training and Education , (Berlin: Springer International Publishing; ), 23–29. 10.1007/978-3-319-42070-7_3 [ CrossRef ] [ Google Scholar ]
• Andersen S. M., Thorpe J. S. (2009). An if–thEN theory of personality: significant others and the relational self. J. Res. Pers. 43 163–170. 10.1016/j.jrp.2008.12.040 [ CrossRef ] [ Google Scholar ]
• Azevedo R. T., Bennett N., Bilicki A., Hooper J., Markopoulou F., Tsakiris M. (2017). The calming effect of a new wearable device during the anticipation of public speech. Sci. Rep. 7 : 2285 . 10.1038/s41598-017-02274-2 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Azuma R., Baillot Y., Behringer R., Feiner S., Julier S., MacIntyre B. (2001). Recent advances in augmented reality. IEEE Comp. Graph. Appl. 21 34–47. 10.1109/38.963459 [ CrossRef ] [ Google Scholar ]
• Bacca J., Baldiris S., Fabregat R., Graf S. (2014). Augmented reality trends in education: a systematic review of research and applications. J. Educ. Technol. Soc. 17 133 . [ Google Scholar ]
• Bailenson J. N., Yee N., Merget D., Schroeder R. (2006). The effect of behavioral realism and form realism of real-time avatar faces on verbal disclosure, nonverbal disclosure, emotion recognition, and copresence in dyadic interaction. Presence 15 359–372. 10.1162/pres.15.4.359 [ CrossRef ] [ Google Scholar ]
• Baños R. M., Botella C., Garcia-Palacios A., Villa H., Perpiñá C., Alcaniz M. (2000). Presence and reality judgment in virtual environments: a unitary construct? Cyberpsychol. Behav. 3 327–335. 10.1089/10949310050078760 [ CrossRef ] [ Google Scholar ]
• Baños R., Botella C., García-Palacios A., Villa H., Perpiñá C., Gallardo M. (2009). Psychological variables and reality judgment in virtual environments: the roles of absorption and dissociation. Cyberpsychol. Behav. 2 143–148. 10.1089/cpb.1999.2.143 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Baus O., Bouchard S. (2014). Moving from virtual reality exposure-based therapy to augmented reality exposure-based therapy: a review. Front. Hum. Neurosci. 8 : 112 . 10.3389/fnhum.2014.00112 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Biocca F. (1997). The cyborg’s dilemma: progressive embodiment in virtual environments. J. Comput. Mediat. Commun. 3 10.1111/j.1083-6101.1997 [ CrossRef ] [ Google Scholar ]
• Biocca F., Harms C., Gregg J. (2001). “The networked minds measure of social presence: pilot test of the factor structure and concurrent validity,” in 4th Annual International Workshop on Presence , Philadelphia, PA, 1–9. [ Google Scholar ]
• Blanke O., Slater M., Serino A. (2015). Behavioral, neural, and computational principles of bodily self-consciousness. Neuron 88 145–166. 10.1016/j.neuron.2015.09.029 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bohil C. J., Alicea B., Biocca F. A. (2011). Virtual reality in neuroscience research and therapy. Nat. Rev. Neurosci. 12 : 752 . 10.1038/nrn3122 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Borrego A., Latorre J., Llorens R., Alcañiz M., Noé E. (2016). Feasibility of a walking virtual reality system for rehabilitation: objective and subjective parameters. J. Neuroeng. Rehabil. 13 : 68 . 10.1186/s12984-016-0174-171 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Botella C., Bretón-López J., Quero S., Baños R. M., García-Palacios A. (2010). Treating cockroach phobia with augmented reality. Behav. Ther. 41 401–413. 10.1016/j.beth.2009.07.002 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Botella C., Fernández-Álvarez J., Guillén V., García-Palacios A., Baños R. (2017). Recent progress in virtual reality exposure therapy for phobias: a systematic review. Curr. Psychiatry Rep. 19 : 42 . 10.1007/s11920-017-0788-4 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Botella C. M., Juan M. C., Baños R. M., Alcañiz M., Guillén V., Rey B. (2005). Mixing realities? An application of augmented reality for the treatment of cockroach phobia. Cyberpsychol. Behav. 8 162–171. 10.1089/cpb.2005.8.162 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Brandes U. (2001). A faster algorithm for betweenness centrality. J. Math. Sociol. 25 163–177. 10.1080/0022250X.2001.9990249 [ CrossRef ] [ Google Scholar ]
• Bretón-López J., Quero S., Botella C., García-Palacios A., Baños R. M., Alcañiz M. (2010). An augmented reality system validation for the treatment of cockroach phobia. Cyberpsychol. Behav. Soc. Netw. 13 705–710. 10.1089/cyber.2009.0170 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Brown A., Green T. (2016). Virtual reality: low-cost tools and resources for the classroom. TechTrends 60 517–519. 10.1007/s11528-016-0102-z [ CrossRef ] [ Google Scholar ]
• Bu Y., Liu T. Y., Huang W. B. (2016). MACA: a modified author co-citation analysis method combined with general descriptive metadata of citations. Scientometrics 108 143–166. 10.1007/s11192-016-1959-5 [ CrossRef ] [ Google Scholar ]
• Burdea G., Richard P., Coiffet P. (1996). Multimodal virtual reality: input-output devices, system integration, and human factors. Int. J. Hum. Compu. Interact. 8 5–24. 10.1080/10447319609526138 [ CrossRef ] [ Google Scholar ]
• Burdea G. C., Coiffet P. (2003). Virtual Reality Technology Vol. 1 Hoboken, NJ: John Wiley & Sons. [ Google Scholar ]
• Carmigniani J., Furht B., Anisetti M., Ceravolo P., Damiani E., Ivkovic M. (2011). Augmented reality technologies, systems and applications. Multimed. Tools Appl. 51 341–377. 10.1007/s11042-010-0660-6 [ CrossRef ] [ Google Scholar ]
• Castelvecchi D. (2016). Low-cost headsets boost virtual reality’s lab appeal. Nature 533 153–154. 10.1038/533153a [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cathy (2011). The History of Augmented Reality. The Optical Vision Site. Available at: http://www.theopticalvisionsite.com/history-of-eyewear/the-history-of-augmented-reality/#.UelAUmeAOyA [ Google Scholar ]
• Chen C. (2006). CiteSpace II: detecting and visualizing emerging trends and transient patterns in scientific literature. J. Assoc. Inform. Sci. Technol. 57 359–377. 10.1002/asi.20317 [ CrossRef ] [ Google Scholar ]
• Chen C., Ibekwe-SanJuan F., Hou J. (2010). The structure and dynamics of cocitation clusters: a multipleperspective cocitation analysis. J. Assoc. Inform. Sci. Technol. 61 1386–1409. 10.1002/jez.b.22741 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Chen Y. C., Chi H. L., Hung W. H., Kang S. C. (2011). Use of tangible and augmented reality models in engineering graphics courses. J. Prof. Issues Eng. Educ. Pract. 137 267–276. 10.1061/(ASCE)EI.1943-5541.0000078 [ CrossRef ] [ Google Scholar ]
• Chicchi Giglioli I. A., Pallavicini F., Pedroli E., Serino S., Riva G. (2015). Augmented reality: a brand new challenge for the assessment and treatment of psychological disorders. Comput. Math. Methods Med. 2015 : 862942 . 10.1155/2015/862942 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Chien C. H., Chen C. H., Jeng T. S. (2010). “An interactive augmented reality system for learning anatomy structure,” in Proceedings of the International Multiconference of Engineers and Computer Scientists , Vol. 1 (Hong Kong: International Association of Engineers; ), 17–19. [ Google Scholar ]
• Choi S., Jung K., Noh S. D. (2015). Virtual reality applications in manufacturing industries: past research, present findings, and future directions. Concurr. Eng. 23 40–63. 10.1177/1063293X14568814 [ CrossRef ] [ Google Scholar ]
• Cipresso P. (2015). Modeling behavior dynamics using computational psychometrics within virtual worlds. Front. Psychol. 6 : 1725 . 10.3389/fpsyg.2015.01725 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cipresso P., Serino S. (2014). Virtual Reality: Technologies, Medical Applications and Challenges. Hauppauge, NY: Nova Science Publishers, Inc. [ Google Scholar ]
• Cipresso P., Serino S., Riva G. (2016). Psychometric assessment and behavioral experiments using a free virtual reality platform and computational science. BMC Med. Inform. Decis. Mak. 16 : 37 . 10.1186/s12911-016-0276-5 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cruz-Neira C. (1993). “Virtual reality overview,” in SIGGRAPH 93 Course Notes 21st International Conference on Computer Graphics and Interactive Techniques, Orange County Convention Center , Orlando, FL. [ Google Scholar ]
• De Buck S., Maes F., Ector J., Bogaert J., Dymarkowski S., Heidbuchel H., et al. (2005). An augmented reality system for patient-specific guidance of cardiac catheter ablation procedures. IEEE Trans. Med. Imaging 24 1512–1524. 10.1109/TMI.2005.857661 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Di Lernia D., Cipresso P., Pedroli E., Riva G. (2018a). Toward an embodied medicine: a portable device with programmable interoceptive stimulation for heart rate variability enhancement. Sensors (Basel) 18 : 2469 . 10.3390/s18082469 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Di Lernia D., Serino S., Pezzulo G., Pedroli E., Cipresso P., Riva G. (2018b). Feel the time. Time perception as a function of interoceptive processing. Front. Hum. Neurosci. 12 : 74 . 10.3389/fnhum.2018.00074 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Di Serio Á., Ibáñez M. B., Kloos C. D. (2013). Impact of an augmented reality system on students’ motivation for a visual art course. Comput. Educ. 68 586–596. 10.1016/j.compedu.2012.03.002 [ CrossRef ] [ Google Scholar ]
• Ebert C. (2015). Looking into the future. IEEE Softw. 32 92–97. 10.1109/MS.2015.142 [ CrossRef ] [ Google Scholar ]
• Englund C., Olofsson A. D., Price L. (2017). Teaching with technology in higher education: understanding conceptual change and development in practice. High. Educ. Res. Dev. 36 73–87. 10.1080/07294360.2016.1171300 [ CrossRef ] [ Google Scholar ]
• Falagas M. E., Pitsouni E. I., Malietzis G. A., Pappas G. (2008). Comparison of pubmed, scopus, web of science, and Google scholar: strengths and weaknesses. FASEB J. 22 338–342. 10.1096/fj.07-9492LSF [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Feiner S., MacIntyre B., Hollerer T., Webster A. (1997). “A touring machine: prototyping 3D mobile augmented reality systems for exploring the urban environment,” in Digest of Papers. First International Symposium on Wearable Computers , (Cambridge, MA: IEEE; ), 74–81. 10.1109/ISWC.1997.629922 [ CrossRef ] [ Google Scholar ]
• Freeman D., Reeve S., Robinson A., Ehlers A., Clark D., Spanlang B., et al. (2017). Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol. Med. 47 2393–2400. 10.1017/S003329171700040X [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Freeman L. C. (1977). A set of measures of centrality based on betweenness. Sociometry 40 35–41. 10.2307/3033543 [ CrossRef ] [ Google Scholar ]
• Fuchs H., Bishop G. (1992). Research Directions in Virtual Environments. Chapel Hill, NC: University of North Carolina at Chapel Hill. [ Google Scholar ]
• Gallagher A. G., Ritter E. M., Champion H., Higgins G., Fried M. P., Moses G., et al. (2005). Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann. Surg. 241 : 364 . 10.1097/01.sla.0000151982.85062.80 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gigante M. A. (1993). Virtual reality: definitions, history and applications. Virtual Real. Syst. 3–14. 10.1016/B978-0-12-227748-1.50009-3 [ CrossRef ] [ Google Scholar ]
• González-Teruel A., González-Alcaide G., Barrios M., Abad-García M. F. (2015). Mapping recent information behavior research: an analysis of co-authorship and co-citation networks. Scientometrics 103 687–705. 10.1007/s11192-015-1548-z [ CrossRef ] [ Google Scholar ]
• Heeter C. (1992). Being there: the subjective experience of presence. Presence 1 262–271. 10.1162/pres.1992.1.2.262 [ CrossRef ] [ Google Scholar ]
• Heeter C. (2000). Interactivity in the context of designed experiences. J. Interact. Adv. 1 3–14. 10.1080/15252019.2000.10722040 [ CrossRef ] [ Google Scholar ]
• Heilig M. (1962). Sensorama simulator . U.S. Patent No - 3, 870. Virginia: United States Patent and Trade Office. [ Google Scholar ]
• Ibáñez M. B., Di Serio Á., Villarán D., Kloos C. D. (2014). Experimenting with electromagnetism using augmented reality: impact on flow student experience and educational effectiveness. Comput. Educ. 71 1–13. 10.1016/j.compedu.2013.09.004 [ CrossRef ] [ Google Scholar ]
• Juan M. C., Alcañiz M., Calatrava J., Zaragozá I., Baños R., Botella C. (2007). “An optical see-through augmented reality system for the treatment of phobia to small animals,” in Virtual Reality, HCII 2007 Lecture Notes in Computer Science Vol. 4563 ed. Schumaker R. (Berlin: Springer; ), 651–659. [ Google Scholar ]
• Juan M. C., Alcaniz M., Monserrat C., Botella C., Baños R. M., Guerrero B. (2005). Using augmented reality to treat phobias. IEEE Comput. Graph. Appl. 25 31–37. 10.1109/MCG.2005.143 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kim G. J. (2005). A SWOT analysis of the field of virtual reality rehabilitation and therapy. Presence 14 119–146. 10.1162/1054746053967094 [ CrossRef ] [ Google Scholar ]
• Klavans R., Boyack K. W. (2015). Which type of citation analysis generates the most accurate taxonomy of scientific and technical knowledge? J. Assoc. Inform. Sci. Technol. 68 984–998. 10.1002/asi.23734 [ CrossRef ] [ Google Scholar ]
• Kleinberg J. (2002). “Bursty and hierarchical structure in streams,” in Paper Presented at the Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2002; Edmonton , Alberta, NT: 10.1145/775047.775061 [ CrossRef ] [ Google Scholar ]
• Kleinberg J. (2003). Bursty and hierarchical structure in streams. Data Min. Knowl. Discov. 7 373–397. 10.1023/A:1024940629314 [ CrossRef ] [ Google Scholar ]
• Korolov M. (2014). The real risks of virtual reality. Risk Manag. 61 20–24. [ Google Scholar ]
• Krueger M. W., Gionfriddo T., Hinrichsen K. (1985). “Videoplace—an artificial reality,” in Proceedings of the ACM SIGCHI Bulletin Vol. 16 New York, NY: ACM, 35–40. 10.1145/317456.317463 [ CrossRef ] [ Google Scholar ]
• Lin C. H., Hsu P. H. (2017). “Integrating procedural modelling process and immersive VR environment for architectural design education,” in MATEC Web of Conferences Vol. 104 Les Ulis: EDP Sciences; 10.1051/matecconf/201710403007 [ CrossRef ] [ Google Scholar ]
• Llorens R., Noé E., Ferri J., Alcañiz M. (2014). Virtual reality-based telerehabilitation program for balance recovery. A pilot study in hemiparetic individuals with acquired brain injury. Brain Inj. 28 : 169 . [ Google Scholar ]
• Lombard M., Ditton T. (1997). At the heart of it all: the concept of presence. J. Comput. Mediat. Commun. 3 10.1111/j.1083-6101.1997.tb00072.x [ CrossRef ] [ Google Scholar ]
• Loomis J. M., Blascovich J. J., Beall A. C. (1999). Immersive virtual environment technology as a basic research tool in psychology. Behav. Res. Methods Instr. Comput. 31 557–564. 10.3758/BF03200735 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Loomis J. M., Golledge R. G., Klatzky R. L. (1998). Navigation system for the blind: auditory display modes and guidance. Presence 7 193–203. 10.1162/105474698565677 [ CrossRef ] [ Google Scholar ]
• Luckerson V. (2014). Facebook Buying Oculus Virtual-Reality Company for $2 Billion. Available at: http://time.com/37842/facebook-oculus-rift [ Google Scholar ]
• Maurugeon G. (2011). New D’Fusion Supports iPhone4S and 3DSMax 2012. Available at: http://www.t-immersion.com/blog/2011-12-07/augmented-reality-dfusion-iphone-3dsmax [ Google Scholar ]
• Mazuryk T., Gervautz M. (1996). Virtual Reality-History, Applications, Technology and Future. Vienna: Institute of Computer Graphics Vienna University of Technology. [ Google Scholar ]
• Meldrum D., Glennon A., Herdman S., Murray D., McConn-Walsh R. (2012). Virtual reality rehabilitation of balance: assessment of the usability of the nintendo Wii ® fit plus. Disabil. Rehabil. 7 205–210. 10.3109/17483107.2011.616922 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Milgram P., Kishino F. (1994). A taxonomy of mixed reality visual displays. IEICE Trans. Inform. Syst. 77 1321–1329. [ Google Scholar ]
• Minderer M., Harvey C. D., Donato F., Moser E. I. (2016). Neuroscience: virtual reality explored. Nature 533 324–325. 10.1038/nature17899 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Neri S. G., Cardoso J. R., Cruz L., Lima R. M., de Oliveira R. J., Iversen M. D., et al. (2017). Do virtual reality games improve mobility skills and balance measurements in community-dwelling older adults? Systematic review and meta-analysis. Clin. Rehabil. 31 1292–1304. 10.1177/0269215517694677 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Nincarean D., Alia M. B., Halim N. D. A., Rahman M. H. A. (2013). Mobile augmented reality: the potential for education. Procedia Soc. Behav. Sci. 103 657–664. 10.1016/j.sbspro.2013.10.385 [ CrossRef ] [ Google Scholar ]
• Orosz K., Farkas I. J., Pollner P. (2016). Quantifying the changing role of past publications. Scientometrics 108 829–853. 10.1007/s11192-016-1971-9 [ CrossRef ] [ Google Scholar ]
• Ozbek C. S., Giesler B., Dillmann R. (2004). “Jedi training: playful evaluation of head-mounted augmented reality display systems,” in Proceedings of SPIE. The International Society for Optical Engineering Vol. 5291 eds Norwood R. A., Eich M., Kuzyk M. G. (Denver, CO: ), 454–463. [ Google Scholar ]
• Perry S. (2008). Wikitude: Android App with Augmented Reality: Mind Blow-Ing. Digital Lifestyles. [ Google Scholar ]
• Radu I. (2012). “Why should my students use AR? A comparative review of the educational impacts of augmented-reality,” in Mixed and Augmented Reality (ISMAR), 2012 IEEE International Symposium on , (IEEE) , 313–314. 10.1109/ISMAR.2012.6402590 [ CrossRef ] [ Google Scholar ]
• Radu I. (2014). Augmented reality in education: a meta-review and cross-media analysis. Pers. Ubiquitous Comput. 18 1533–1543. 10.1007/s00779-013-0747-y [ CrossRef ] [ Google Scholar ]
• Riva G. (2018). The neuroscience of body memory: From the self through the space to the others. Cortex 104 241–260. 10.1016/j.cortex.2017.07.013 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Riva G., Gaggioli A., Grassi A., Raspelli S., Cipresso P., Pallavicini F., et al. (2011). NeuroVR 2-A free virtual reality platform for the assessment and treatment in behavioral health care. Stud. Health Technol. Inform. 163 493–495. [ PubMed ] [ Google Scholar ]
• Riva G., Serino S., Di Lernia D., Pavone E. F., Dakanalis A. (2017). Embodied medicine: mens sana in corpore virtuale sano. Front. Hum. Neurosci. 11 : 120 . 10.3389/fnhum.2017.00120 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Riva G., Wiederhold B. K., Mantovani F. (2018). Neuroscience of virtual reality: from virtual exposure to embodied medicine. Cyberpsychol. Behav. Soc. Netw. 10.1089/cyber.2017.29099.gri [Epub ahead of print]. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rosenberg L. (1993). “The use of virtual fixtures to enhance telemanipulation with time delay,” in Proceedings of the ASME Winter Anual Meeting on Advances in Robotics, Mechatronics, and Haptic Interfaces Vol. 49 (New Orleans, LA: ), 29–36. [ Google Scholar ]
• Schmidt M., Beck D., Glaser N., Schmidt C. (2017). “A prototype immersive, multi-user 3D virtual learning environment for individuals with autism to learn social and life skills: a virtuoso DBR update,” in International Conference on Immersive Learning , Cham: Springer, 185–188. 10.1007/978-3-319-60633-0_15 [ CrossRef ] [ Google Scholar ]
• Schwald B., De Laval B. (2003). An augmented reality system for training and assistance to maintenance in the industrial context. J. WSCG 11 . [ Google Scholar ]
• Serino S., Cipresso P., Morganti F., Riva G. (2014). The role of egocentric and allocentric abilities in Alzheimer’s disease: a systematic review. Ageing Res. Rev. 16 32–44. 10.1016/j.arr.2014.04.004 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Skalski P., Tamborini R. (2007). The role of social presence in interactive agent-based persuasion. Media Psychol. 10 385–413. 10.1080/15213260701533102 [ CrossRef ] [ Google Scholar ]
• Slater M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364 3549–3557. 10.1098/rstb.2009.0138 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Slater M., Sanchez-Vives M. V. (2016). Enhancing our lives with immersive virtual reality. Front. Robot. AI 3 : 74 10.3389/frobt.2016.00074 [ CrossRef ] [ Google Scholar ]
• Small H. (1973). Co-citation in the scientific literature: a new measure of the relationship between two documents. J. Assoc. Inform. Sci. Technol. 24 265–269. 10.1002/asi.4630240406 [ CrossRef ] [ Google Scholar ]
• Song H., Chen F., Peng Q., Zhang J., Gu P. (2017). Improvement of user experience using virtual reality in open-architecture product design. Proc. Inst. Mech. Eng. B J. Eng. Manufact. 232 . [ Google Scholar ]
• Sundar S. S., Xu Q., Bellur S. (2010). “Designing interactivity in media interfaces: a communications perspective,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , (Boston, MA: ACM; ), 2247–2256. 10.1145/1753326.1753666 [ CrossRef ] [ Google Scholar ]
• Sutherland I. E. (1965). The Ultimate Display. Multimedia: From Wagner to Virtual Reality. New York, NY: Norton. [ Google Scholar ]
• Sutherland I. E. (1968). “A head-mounted three dimensional display,” in Proceedings of the December 9-11, 1968, Fall Joint Computer Conference, Part I , (ACM) , 757–764. 10.1145/1476589.1476686 [ CrossRef ] [ Google Scholar ]
• Thomas B., Close B., Donoghue J., Squires J., De Bondi P., Morris M., et al. (2000). “ARQuake: an outdoor/indoor augmented reality first person application,” in Digest of Papers. Fourth International Symposium on Wearable Computers , (Atlanta, GA: IEEE; ), 139–146. 10.1109/ISWC.2000.888480 [ CrossRef ] [ Google Scholar ]
• Ware C., Arthur K., Booth K. S. (1993). “Fish tank virtual reality,” in Proceedings of the INTERACT’93 and CHI’93 Conference on Human Factors in Computing Systems , (Amsterdam: ACM; ), 37–42. 10.1145/169059.169066 [ CrossRef ] [ Google Scholar ]
• Wexelblat A. (ed.) (2014). Virtual Reality: Applications and Explorations. Cambridge, MA: Academic Press. [ Google Scholar ]
• White H. D., Griffith B. C. (1981). Author cocitation: a literature measure of intellectual structure. J. Assoc. Inform. Sci. Technol. 32 163–171. 10.1002/asi.4630320302 [ CrossRef ] [ Google Scholar ]
• Wrzesien M., Alcañiz M., Botella C., Burkhardt J. M., Bretón-López J., Ortega M., et al. (2013). The therapeutic lamp: treating small-animal phobias. IEEE Comput. Graph. Appl. 33 80–86. 10.1109/MCG.2013.12 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wrzesien M., Burkhardt J. M., Alcañiz M., Botella C. (2011a). How technology influences the therapeutic process: a comparative field evaluation of augmented reality and in vivo exposure therapy for phobia of small animals. Hum. Comput. Interact. 2011 523–540. [ Google Scholar ]
• Wrzesien M., Burkhardt J. M., Alcañiz Raya M., Botella C. (2011b). “Mixing psychology and HCI in evaluation of augmented reality mental health technology,” in CHI’11 Extended Abstracts on Human Factors in Computing Systems , (Vancouver, BC: ACM; ), 2119–2124. [ Google Scholar ]
• Zyda M. (2005). From visual simulation to virtual reality to games. Computer 38 25–32. 10.1109/MC.2005.297 [ CrossRef ] [ Google Scholar ]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 04 April 2024

AI and augmented reality for 3D Indian dance pose reconstruction cultural revival

• J. Jayanthi 1 &
• P. Uma Maheswari 1  

Scientific Reports volume  14 , Article number:  7906 ( 2024 ) Cite this article

Metrics details

• Computational science
• Computer science

This paper delves into the specialized domain of human action recognition, focusing on the Identification of Indian classical dance poses, specifically Bharatanatyam. Within the dance context, a “Karana” embodies a synchronized and harmonious movement encompassing body, hands, and feet, as defined by the Natyashastra. The essence of Karana lies in the amalgamation of nritta hasta (hand movements), sthaana (body postures), and chaari (leg movements). Although numerous, Natyashastra codifies 108 karanas, showcased in the intricate stone carvings adorning the Nataraj temples of Chidambaram, where Lord Shiva’s association with these movements is depicted. Automating pose identification in Bharatanatyam poses challenges due to the vast array of variations, encompassing hand and body postures, mudras (hand gestures), facial expressions, and head gestures. To simplify this intricate task, this research employs image processing and automation techniques. The proposed methodology comprises four stages: acquisition and pre-processing of images involving skeletonization and Data Augmentation techniques, feature extraction from images, classification of dance poses using a deep learning network-based convolution neural network model (InceptionResNetV2), and visualization of 3D models through mesh creation from point clouds. The use of advanced technologies, such as the MediaPipe library for body key point detection and deep learning networks, streamlines the identification process. Data augmentation, a pivotal step, expands small datasets, enhancing the model’s accuracy. The convolution neural network model showcased its effectiveness in accurately recognizing intricate dance movements, paving the way for streamlined analysis and interpretation. This innovative approach not only simplifies the identification of Bharatanatyam poses but also sets a precedent for enhancing accessibility and efficiency for practitioners and researchers in the Indian classical dance.

Introduction

Temples ensconced in the historic towns of Thanjavur, Chidambaram, Kumbakonam, Satara, and Prambanan exhibit intricate panels adorned with inscriptions detailing the Karanas, presenting a captivating mosaic of diverse poses upon closer examination. At the heart of Indian classical dance lies the Natya Shastra 1 , revered as the foundational scripture akin to a sacred “bible” of this artistic discipline. Crafted by the venerable Sage Bharata, also known as Bharata Muni, this ancient text stands as a guiding light, meticulously outlining the principles and regulations governing the expansive realms of performing arts. Within its profound teachings, the Natya Shastra meticulously codifies 108 Karanas, each bearing unique appellations such as Talapuspaputam, Vartitam, Valitorukam, and numerous others, encapsulating the intricate lexicon of movements enshrined within this cultural treasure trove 2 .

Bharatanatyam, the quintessential embodiment of this classical heritage, derives its name from the fusion of fundamental elements. The term itself weaves a poetic narrative: “Bha” representing Bhava, the essence of emotion; “Ra” symbolizing Raaga, the soulful resonance of music; “Ta” standing for Taala, the rhythmic heartbeat; and “Natyam” encapsulates the art of dance. In this amalgamation of emotions, melody, and rhythm, Bharatanatyam emerges as a profound art form transcending temporal boundaries, captivating the hearts of connoisseurs and enthusiasts alike. In the vibrant tapestry of Indian classical dances, Bharatanatyam occupies a distinguished position, sharing the stage with other esteemed classical styles such as Odissi from Odisha, Kuchipudi from Andhra Pradesh, Kathakali from Kerala, Mohiniattam from Kerala, and Kathak from Northern India. Its influence extends far beyond the realms of performance, permeating the very stones of ancient Hindu temples. Within these sanctified precincts, timeless sculptures draw inspiration from the dynamic postures and fluid movements of Bharatanatyam, immortalizing the dance form’s elegance and grace for generations to come. Rooted in the venerable traditions of southern India, Bharatanatyam flourishes within the sanctified environs of temples and royal courts, echoing the cultural ethos of the land. It venerates not only the aesthetic beauty of the human body but also embraces the cosmic harmony of the universe itself 3 . However, the dance finds its truest expression when harmoniously synchronized with music. The soul-stirring strains of Carnatic music, a classical genre originating from the southern regions of India, provide the perfect accompaniment, creating a symphony of movement and melody that enchants the senses.

A distinguishing feature of Indian classical dance lies in the intricate language of hand gestures known as Mudras 4 . These Mudras, numbering approximately fifty-five, serve as a means of clear communication, conveying specific ideas, events, actions, or even creatures. Among them, thirty-two Mudras are ‘Asamyukta Hasta,’ requiring only one hand, while the remaining twenty-three are ‘Samyukta Hasta,’ necessitating the graceful interplay of both hands. These gestures, akin to an ancient sign language, infuse the dance with depth and nuance, allowing for a profound narrative to unfold through the dancer’s fingertips.Comprehending dance poses holds immense significance for aspiring dancers; the precise replication of these poses signifies the completion of a dance performance. Bharatanatyam, often regarded as the cosmic dance or the dance of the universe, embodies profound symbolism. However, there is a scarcity of documentation concerning the 3D augmentation of Bharatanatyam dance poses.

The principles governing movement in Indian Classical Dances (ICDs) are elucidated in the Natyashastra 5 . Studies showcase the fusion of deep descriptors and handcrafted pose signatures on the ICD dataset, enabling the classification of Indian classical dance sequences, regardless of specific poses. Moreover, Kishore et al. 6 advocate the use of CNN for classifying ICD images, achieving an impressive recognition rate of 93.33%, surpassing other classifier models reported in the ICD dataset. In their endeavours, Guo and Qian 7 have developed a dedicated system for recognizing and identifying 3D dance postures. Saha et al. 8 Introduce an algorithm for gesture recognition in ICD, utilizing joint coordinates captured by Kinect. This algorithm accurately identifies gestures associated with emotions such as happiness, fear, anger, relaxation, and sadness. Mallik et al. 9 Employ the Multimedia Web Ontology Language (MOWL) to effectively represent the domain knowledge of Indian Classical Dance (ICD). Furthermore, Kalpana et al. 10 delves into the application of classical Indian dance as a pedagogical tool, suggesting a categorical content analysis methodology. This framework enables Asian Indian students to learn mathematical shapes through Bharatanatyam. Additionally, Rodriguez 11 establishes a chronological relation between Kathak footwork and geometry, significantly contributing to the interdisciplinary understanding of dance and mathematics. In a pioneering effort, Kim et al. 12 introduce the Rectified Linear Unit (ReLU)-based Extreme Learning Machine Classifier (ELMC). This meticulously designed classifier can classify 800 dance movement data points across 200 different dance types. Moreover, Bisht et al. 13 focus on the recognition of classical dance mudras in India, leveraging images of hand mudras from diverse classical dances obtained through online sources. The Histogram of Oriented Gradients (HOG) features of these hand mudras serve as input for the classifier, which employs Support Vector Machine (SVM) for recognition purposes. The tradition of classical Chinese dance is meticulously preserved by the New York-based Shen Yun Performing Arts 14 . Their public performances serve as an instrumental method for conserving Chinese classical dance, enriching people’s understanding of this art form and sparking interest in it. Recent research efforts 15 aim to differentiate between movements in Bharatanatyam and Kathak. This analysis, primarily visual in nature, scrutinizes the positioning and tension of body limbs and hand postures. In an innovative approach, Kim et al. 16 propose a technique for estimating human poses. This method utilizes MediaPipe Pose and an optimization approach rooted in a humanoid model. The accurate estimation of human poses is a formidable challenge, critical for applications in virtual reality, robotics, and human–computer interaction. Lastly, recent research endeavours 17 introduce a generative model within a deep learning framework. Leveraging an extensive dataset of human motion capture data, this model has the ability to generate unprecedented movements, expanding the horizons of understanding in the realm of dance. These diverse research pursuits, spanning from intricate pose recognition to the preservation of traditional dance forms, collectively enrich the tapestry of knowledge and innovation in the field of Indian classical dance and its global counterparts.

In this research, our proposed methodology consists of four distinct stages: initial image acquisition and pre-processing incorporating skeletonization and Data Augmentation techniques, followed by feature extraction from the images. Subsequently, the dance poses are classified utilizing a convolution neural network model based on deep learning, specifically the InceptionResNetV2 architecture. Finally, the study involves the visualization of three-dimensional models through the creation of meshes derived from point clouds.

The adopted framework integrates image pre-processing, data augmentation; pose estimation, classification, and 3D model reconstruction to address challenges in dance pose identification.

Firstly, the process begins with image acquisition and pre-processing. This involves the initial collection of images followed by preparatory steps to ensure their suitability for analysis. Techniques such as skeletonization are employed to simplify the images, focusing on the essential structural elements and removing unnecessary details. Additionally, Data Augmentation techniques are applied to augment the dataset by generating new images from existing ones, thereby diversifying the training data and enhancing the model’s robustness. Next, the feature extraction stage involves extracting meaningful features from the pre-processed images. This step aims to capture the relevant characteristics of the dance poses that can be used for classification. Features may include aspects such as shape, texture, or spatial relationships within the image, which are crucial for distinguishing between different poses. The third stage focuses on classification using a deep learning convolutional neural network (CNN) model, specifically the InceptionResNetV2 architecture. CNNs are well-suited for image classification tasks due to their ability to automatically learn hierarchical features from the data. InceptionResNetV2, in particular, is known for its effectiveness in handling complex visual data and achieving high accuracy in classification tasks. Finally, the visualization of 3D models through mesh creation from point clouds adds an additional dimension to the analysis. This stage allows for the creation of three-dimensional representations of the dance poses, providing insights into their spatial structure and dynamics. By visualizing the poses in 3D, researchers gain a deeper understanding of their anatomical intricacies and movement patterns.

Throughout the methodology, advanced technologies such as the MediaPipe library for body key point detection are utilized to streamline the identification process. Data augmentation emerges as a pivotal step, expanding small datasets and improving the model’s accuracy. The effectiveness of the convolutional neural network model in accurately recognizing intricate dance movements demonstrates its potential for streamlined analysis and interpretation. Overall, this innovative approach not only simplifies the identification of Bharatanatyam poses but also sets a precedent for enhancing accessibility and efficiency for practitioners and researchers in the field of Indian classical dance.

The proposed method depicted in Fig.  1 is designed to classify input images into 108 distinct dance form categories: Talapuspaputam, Vartitam,,Valitorukam,Apaviddham,Samanakham, Linam,Swastikarechitam, Mandalaswastikam, Nikuttakam, Ardhanikuttakam, Katicchinnam, Ardharechitakam, Vaksahswastikam, etc. The approach involves generating a dataset that is evenly distributed among all 108 classes. Subsequently, the dataset undergoes several pre-processing steps such as resizing, thresholding, scaling and skeletonization utilizing the MediaPipe library for body key point detection. The resulting processed frames are then inputted into a deep convolution neural network based on the Inception-ResNet-v2 architecture, which performs the classification task by assigning the images to their respective dance form categories mentioned above and visualize 3D models reconstruction process through creating a mesh from point clouds.

proposed architecture.

Dataset and pre-processing Image

Dataset selection and significance.

The method proposed in this research involves utilizing camera-captured images along with publicly available sources 18 , 19 , as depicted in Figs. 2 and 3 . Specifically, the karanas poses were captured from the Chidambaram Nataraja Temple, which dates back to the period of Raja Raja Chola in the tenth century. These temple wall carvings depict all 108 karanas from the Natya Shastra by Bharata Muni, serving as the foundational postures of Bharatanatyam, an Indian classical dance form. To capture these karanas, a Canon EOS-600D DSLR Camera was utilized. The camera setup included a 3-inch LCD screen, allowing for clear view and enabling shots from various angles. The camera features an 18-megapixel sensor with high ISO 6400 for low-light capture, auto focus, and flash capability. A total of 1721 images were captured, comprising 15 samples from each of the 108 dance poses. The dataset for the study consists of these 1721 images, sourced from both publicly available sources and those captured by the camera. Care was taken to ensure an equal distribution of data across all 108 classes. Despite the small size of the dataset, the presence of varying dance poses within the same category, as well as diverse backgrounds, adds a challenge to classification tasks. The process of enhancing and preparing dance pose images encompasses several essential sub-processes aimed at improving the image quality. These include resizing and cropping, grayscale conversion, binarization, and noise removal. These techniques collectively contribute to the enhancement and preparation of dance pose images, ensuring that they are optimized for further analysis and readability.

camera captured Image of Dance poses in Chidambaram Temple.

Publically available data source of dance poses.

The dataset used in this research is significant due to its unique attributes that align with the study’s objectives. It comprises images captured both by a camera and from the Chidambaram Nataraja Temple, showcasing ancient dance poses dating back to the tenth century. These poses, outlined in the Natya Shastra, offer valuable insights into Indian classical dance. Despite its relatively small size, the dataset is meticulously balanced across all 108 pose categories, making it highly useful for training models. Moreover, it presents challenges akin to those encountered in real dance performances, thereby enhancing its realism. Furthermore, the images undergo thorough processing to enhance their quality, rendering them suitable for analysis. Overall, this dataset contributes significantly to fields such as computer vision, pattern recognition, and cultural heritage preservation by effectively bridging technology with cultural understanding.

Gray scale conversion

The luminosity method, chosen for grayscale conversion in Indian pose identification, amalgamates RGB channels using weighted averages, emphasizing green due to its significance in human visual perception. This sophisticated approach enhances image quality by considering human visual sensitivity, distinguishing it from the conventional average method. in the Eq. ( 1 ).

Binarization

The adaptive threshold T dynamically adjusts between minimum and maximum pixel intensity values in Indian dance pose images, enabling precise segmentation in diverse illumination conditions. This adaptive approach classifies pixels below T as background and those above it as foreground, facilitating effective analysis for Indian dance pose applications as shown in Eq. ( 2 ).

Noise removal

Noise removal is pivotal for image clarity and accurate analysis, with Median filters effectively reducing noise while preserving image integrity. In Indian dance pose analysis, our system employs Median filters to enhance image quality by replacing pixel values with median neighbourhood values, mitigating noise effectively. Mathematically, the Median filter can be represented as in Eq. ( 3 ):

where \(I_{{{\text{filtered}}}} \left( {x,y} \right)\) represents the filtered intensity value at pixel coordinates ( (x, y) , I(x + i, y + j) denotes the intensity value of neighboring pixels, and k determines the size of the neighborhood.

Morphological operations

Morphological operations, erosion and dilation, are essential for enhancing binary image quality by addressing noise and texture distortions. Erosion reduces noise by shrinking white regions, while dilation enhances image features, improving dance pose visibility as represented in Eqs. ( 4 ) and ( 5 ).

where A represents the input binary image, B denotes the structuring element, and the symbols  ⊖  and  ⊕  denote erosion and dilation operations, respectively.

Normalization

The Min–Max Normalization method is applied to normalize the image, scaling the data between 0 and 1 for simplified interpretation. This technique enhances comprehension of the image’s content for Indian dance pose analysis. Mathematically, normalization is represented as in the Eq. ( 6 ).

where MPPi, the pixel value of the image after applying a median filter between new minimum and maximum values based on the original minimum and maximum values.

Data augmentation

Data augmentation 20 , 21 is a crucial method in image pre-processing used to expand the size of small datasets. By generating extra training data from the original dataset, image data augmentation techniques significantly improve the learning process without the need for additional storage memory. Common approaches to generate new images involve horizontal or vertical flipping, inward or outward scaling, rotation at different angles, translation, random cropping, and the addition of Gaussian noise (Fig.  4 ) to prevent over fitting and enhance learning capabilities.

Data Augmentation of dance poses.

In the context of the Indian dance pose classification system, the equations define various transformations applied to the image data.

where the coordinates x′ and y′ represent positions in the resized image, while x and y denote coordinates in the original image. W stands for the width of the image, and H denotes its height. The rotation angle is denoted by R, with ΔR representing a random adjustment within a specified range for each color channel. Additionally, A is a matrix constructed based on specific random parameters for affine transformations, which combine translation, rotation, scaling, and shearing. These transformations are crucial for augmenting the dataset and improving the robustness of the classification system for Indian dance poses.

The research incorporates Google’s human posture detection library, such as MediaPipe, along with Inception-ResNet-V2 transfer learning architectures. These models were utilized to compare our proposed model with existing techniques.

Dance pose Enhance (Image Dataset).

Dance poses Data Augmentation (Image Dataset).

Dance poses—Skeletonization and estimate the pose (Image Dataset).

Dance poses Recognition (Image Dataset).

Dance poses—3D point cloud from mesh.

MediaPipe is an advanced Machine Learning solution designed for precise body pose tracking, enabling the inference of 33 major 3D landmarks (as shown in Fig.  5 ) across the entire body. With this information, it becomes possible to construct a skeletal orientation, accurately representing the positioning and orientation of the body’s skeletal structure (as shown in Fig.  6 ).The facial land marking procedure involves the use of landmarks ranging from 0 to 10 for facial features. Landmarks 11 to 22 are specifically used for detecting upper body parts such as the shoulders, wrists, elbows, and hands. Lastly, landmarks 23 to 32 are utilized to determine the position of lower body components including the hips, knees, legs, and feet. These landmarks provide precise spatial information in three-dimensional (3D) space about the respective body regions 22 , 23 . To represent a set of 33 points mathematically, you can use vectors in three-dimensional space. Each point consists of three coordinates (x, y, and z). Here’s how you can represent 33 points as mathematical vectors in the Eq. ( 13 ):

33- Landmarks detected on the human body using MediaPipe.

Posture detection using MediaPipe.

Let P be the set of 33 points: P  = { p 1, p 2, p 3,…, p 33}Each pi represents a 3D point:

where xi ​ is the x-coordinate of the i -th point, yi ​ is the y-coordinate of the i -th point, zi ​ is the z-coordinate of the i -th point.

Model- Inception- ResNet V2

A Convolutional Neural Network (CNN) 24 , 25 , 26 is a deep learning algorithm specifically designed for image recognition and processing tasks. It comprises various layers, including convolutional layers, pooling layers, and fully connected layers (Figs. 7 and 8 ). These layers work together to extract and learn relevant features from images, enabling the CNN to make accurate predictions and perform complex image-related tasks. The initial layer in the network is the Convolution Layer, responsible for extracting features from an input image. By utilizing a small set of input data, it learns image features while maintaining the interconnections between pixels. The pooling layer is an essential component of a CNN and performs a crucial role in image pre-processing. Its purpose is to reduce the number of parameters in cases where the image size is excessively large. Following the pooling layer, the subsequent layer is known as flattening. As the name implies, this layer takes the pooled results and flattens them. The pooling matrix, which is generated from the pooling layer, is transformed into a one-dimensional matrix, where all the values are arranged in columns sequentially. The pixel values of the input image are not directly linked to the output layer. Nevertheless, in the fully-connected layer, every neuron in the output layer establishes a direct connection with a node in the preceding layer. This layer is responsible for performing classification tasks by utilizing the features extracted from the previous layers and their diverse filters 27 , 28 .

convolution network architecture.

convolution, max pooling and flatten process.

In the context of image processing, where the input image is denoted as ‘f’ and the kernel as ‘h’, Eq. ( 14 ) assigns ‘m’ and ‘n’ as the row and column indices of the resulting matrix, respectively. Moving forward, Eq. ( 15 ) defines the width of the padding, ‘p’, in terms of the filter dimension ‘f’. Subsequently, Eq. ( 16 ) computes the dimensions of the output matrix, factoring in padding and stride effects. Further, Eq. ( 17 ) delineates the dimensions of the received tensor, accounting for image size ‘n’, filter size ‘f’, number of channels ‘nc’, padding ‘p’, stride ‘s’, and the number of filters ‘nf’. Finally, an activation function is introduced, with the widely-used Rectified Linear Unit (ReLU) applied in Eq. ( 18 ) to filter the output produced by the layer.

The Inception-ResNet-v2 is a convolutional neural network that has been trained using a dataset consisting of over a million images sourced from the ImageNet database.Inception-ResNet-V2 is a hybrid model that combines the strengths of both the Inception net and residual connection models 29 , 30 , 31 , 32 . Inception-ResNet-V2 consists of a remarkable 164 deep layers and approximately 55 million parameters. The Residual Inception Block integrates convolutional filters of various sizes along with residual connections. By employing residual connections, this architecture effectively mitigates the issue of performance degradation caused by deep networks and significantly reduces training time.

Visualize 3D models reconstruction

In the process involving a 3D dance pose model, the model begins as a gray; un-textured mesh that can be interactively rotated for viewing. To enhance its appearance as a 3D object 33 , 34 , normal for vertices and surfaces are computed, enabling realistic rendering. A coordinate frame is introduced with XYZ axes, originating at the model’s centre, facilitating an understanding of its spatial orientation. The mesh is converted into a point cloud by sampling points, and colors in the point cloud represent the Z-axis position. The point cloud can be rotated to achieve different viewpoints, offering a versatile way to view the dance pose model from various angles and orientations in 3D space.

Results and discussion

The paper’s models are created using Python libraries, including NumPy, Pandas, OpenCV (cv2), PIL, OS, Matplotlib, MediaPipe, etc., running on a Dell G15 Gaming Laptop equipped with an 8 GB RAM, an Intel Core i5 processor, and an NVIDIA GeForce GTX graphics card. The dataset utilized for the study comprises 1721 images, sourced from a combination of publicly available sources and images captured by a camera.

In our analysis of Dance pose dataset, pre-processing successfully enhanced data quality, reducing noise and ensuring data consistency, as evidenced by a higher signal-to-noise ratio and improved feature preservation in the Table 1 . Data augmentation significantly improved model performance, increasing accuracy by 10% compared to the non-augmented dataset, indicating its effectiveness in mitigating overfitting and handling real-world data variations in the Table 2 . Regarding the use of MediaPipe for pose estimation, results exhibited a keypoint localization error of 5 pixels on average, reflecting precise pose estimation, though occasional inaccuracies were observed during fast motion. Further fine-tuning of tracking parameters and post-processing steps were applied to enhance tracking stability, ultimately improving the reliability of the MediaPipe-based results for our specific application.

As previously mentioned, the dataset consists of 1721 images categorized into 108 classes. The mediapipe based skeletonized input image of size for our architecture is 50 × 50 × 3. The model’s output is first flattened before being passed to the dense layers. The final dense layer consists of 108 units with softmax activation function. Each unit represents the probability of a Dance pose belonging to one of the 108 categories in the dataset. Softmax is employed due to the multi-class nature of the dataset, as it produces a multinomial probability distribution as the desired output in Fig.  9 .

Experimental results of dance pose Identification system.

Table 3 and Fig.  11 , presents the accuracy, precision, recall, and f1-score achieved by the proposed Media Pipe version of the Inception-ResNet-v2 architectures 35 , 36 for the classification problem using the specified dataset. Furthermore, a comparison was made between the results obtained from the Inception-ResNet-v2 model using both the Media Pipe and non-Media Pipe versions of the dataset to evaluate the impact of skeletonization on model accuracy. The Table 3 shows that our proposed models achieved significantly better performance on the skeletonized dataset compared to the original dataset. This improvement underscores the effectiveness of using skeletonized Dance pose images for the classification task. Notably, our proposed model exhibited the most substantial enhancement, with its performance rising from 86.46 to 92.75 when utilizing preprocessed images on the testing set. Additionally, the proposed model outperformed existing models in terms of precision, recall, and f1-score, showcasing its superiority in accurately classifying Dance poses. The observed increase in model performance can be attributed to the skeletonization process, which successfully removes background disturbances from the Dance posture images. This allows the CNN layers to focus solely on the required Dance poses, leading to more precise feature extraction and more accurate classification results. Consequently, the positive impact of skeletonization, carried out using a posture recognition library, on the performance of various deep learning models is evident. This pre-processing step is deemed critical in enhancing the effectiveness and accuracy of models for Dance pose classification tasks.

In our evaluation of the 3D Dance pose data processing pipeline (Fig.  10 ), we observed variations in execution time, with filtering being the most time-consuming step. While point density was consistent in the resulting point clouds, there were discrepancies in point-to-surface distances, suggesting room for improvement in capturing fine surface details. Data loss was minimal, with a 90%-point retention rate, indicating the pipeline’s ability to preserve most of the original data. In registration experiments, the pipeline demonstrated good accuracy with an average registration error of 0.10 units. These findings underscore the need for optimizing the filtering algorithm and addressing point-to-surface distance variations to enhance overall pipeline performance, while further validation and real-world testing are essential to ensure robustness.

Visualize 3D models reconstruction.

Limitations

A potential drawback of the model is its dependence on the accuracy of pose estimation provided by the MediaPipe library, which might encounter occasional inaccuracies, particularly during fast motion or complex poses. These inaccuracies have the potential to impact the quality of input data for classification tasks, leading to potential misclassifications or reduced model performance. Additionally, the effectiveness of the model may be influenced by the diversity and representativeness of the training dataset, as well as potential biases inherent in the data. Furthermore, the computational resources required for training and inference with deep learning models, such as the Inception-ResNet-V2 architecture, could pose constraints in terms of processing power and time, especially for large-scale datasets or real-time applications. Addressing these challenges may involve refining pose estimation techniques, enhancing dataset diversity, and optimizing model architecture and training procedures to enhance overall robustness and performance (Fig.  11 ).

Accuracy and Loss rate.

Proposed approach advantages and future directions

The proposed method for Bharatanatyam pose identification excels due to its tailored approach, leveraging traditional knowledge from the Natyashastra, advanced image processing techniques, deep learning with CNNs 37 , and 3D visualization. Specialized for Bharatanatyam, it captures the nuances of hand gestures, body postures, and leg movements. By integrating traditional wisdom with modern technology, it ensures authenticity and accuracy. Advanced image processing enhances dataset quality, while deep learning enables effective feature extraction and classification. 3D visualization provides deeper insights into pose dynamics. Integration of technologies like MediaPipe streamlines the process. Ultimately, this method preserves cultural heritage and sets a new standard for Bharatanatyam pose identification.

In terms of future directions, potential areas for improvement include exploring more sophisticated data augmentation techniques, investigating alternative model architectures, and incorporating domain-specific knowledge to enhance the model’s understanding of dance poses. Furthermore, conducting experiments on larger and more diverse datasets, as well as deploying the model in real-world settings for user feedback, could provide valuable insights for further refinement and optimization. Overall, addressing these future directions will contribute to advancing dance pose recognition and furthering the field of computer vision and human motion analysis.

Integrating 3D reconstruction in dance pose identification

The paper aims to identify dance poses, treating it as a classification problem. However, it incorporates 3D reconstruction to provide a more comprehensive understanding of the poses. This decision offers benefits such as enhanced understanding of spatial structure and dynamics, improved visualization for analysis, validation and verification of classification models, and practical applications like virtual reality simulations. The inclusion of 3D reconstruction enriches the study beyond mere classification, offering deeper insights and facilitating various applications in dance analysis and education.

Computational complexity analysis of models

Computational complexity analysis assesses the efficiency and resource requirements of models. For image processing tasks like pose recognition, complexities vary. Skeletonization algorithms, used for thinning images, exhibit complexity relative to pixel or edge count. Feature extraction in deep learning, involving convolutions and pooling layers, depends on input size, layer count, and filter dimensions. Classification complexity, determined by parameters in fully connected layers, influences computational demand. 3D reconstruction complexity, based on point cloud size and mesh generation algorithms, varies. Integration of advanced technologies like MediaPipe for key point detection streamlines processing, while preserving cultural heritage with automated pose recognition. Optimization for real-time applications necessitates managing complexity to ensure efficient performance.

Ablation study to the paper

The ablation study aimed to assess the individual contributions of key components in the proposed Indian dance pose identification system. Firstly, we evaluated the impact of image preprocessing techniques, such as noise reduction and data consistency enhancement 38 . By comparing classification metrics, we observed a significant improvement in model performance with preprocessing, increasing accuracy by 15%, precision by 12%, recall by 10%, and F1-score by 13%. Secondly, data augmentation experiments showed a notable increase in accuracy from 86.4 to 91.2% when augmenting the dataset, indicating a 5.8% improvement. Lastly, the use of MediaPipe for pose estimation led to precise results with a low keypoint localization error of 5 pixels on average. Fine-tuning and post-processing further enhanced stability, resulting in a 3% increase in accuracy. Overall, image preprocessing, data augmentation, and MediaPipe pose estimation contributed significantly to the model’s performance, with improvements of 15, 5.8, and 3%, respectively, highlighting their critical roles in enhancing classification accuracy and reliability.

The task of human pose detection has posed significant challenges in the field of computer vision due to its wide-ranging and diverse applications in everyday life. Consequently, the identification of poses in the context of Indian classical dance, specifically Bharatanatyam, holds immense importance for its potential impact on human well-being. In our study, we have put forth a novel deep-learning-network-based convolutional neural network model, InceptionResNetV2. This model is designed to work on key points identified using MediaPipe and has proven to be highly effective in accurately classifying 108 distinct dance poses. Our approach was developed following a comprehensive review of existing related research. The core idea behind our architecture is to separately extract spatial and depth features from the images and then leverage both sets of features for pose recognition. This unique approach provides our architecture with an advantage, enabling it to distinguish among poses more effectively, as initially hypothesized in our methodology and subsequently validated through result analysis and comparisons conducted in our research. Furthermore, our proposed architecture holds the potential to accommodate a greater number of poses, thanks to its feature extraction strategy. Future research endeavors will also focus on enhancing performance through hyperparameter tuning. In conclusion, our contribution has added significant value to ongoing efforts in the identification of Indian classical dance poses, particularly within the domain of Bharatanatyam. By employing advanced techniques in human pose detection and 3D model reconstruction, our work has not only improved the accuracy and robustness of pose recognition in this intricate dance form but has also opened avenues for broader applications in the field of human pose detection. Our research has not only enriched the understanding and preservation of the rich cultural heritage of Bharatanatyam but has also contributed to the advancement of computer vision and 3D modeling techniques with implications in diverse domains such as healthcare, sports analysis, and animation. We anticipate that our work will guide researchers in this area toward achieving near-perfect performance metrics, benefiting all stakeholders involved in this endeavour. Evaluation highlights the effectiveness of augmentation, preprocessing, and skeletonization, while future work focuses on optimization and validation for enhanced pipeline performance and robustness.

Data availability

The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

Bose, M., & Bose, M. The literature of dance: Movement and mimesis: The idea of dance in the Sanskritic tradition, pp. 13–107. (1991).

Bennink, L. P., Deekshithar, K. R., Deekshithar, J. R., &Deekshithar, S. R. Shiva’s Karanas in the temples of Tamil Nadu: The Natya Shastra in stone (2013).

O’shea, J. At home in the world? The Bharatanatyam dancer as transnational interpreter. Drama Rev. 47 (1), 176–186 (2003).

Article   Google Scholar  

Malavath, P. & Devarakonda, N. Natya Shastra: Deep learning for automatic classification of hand mudra in Indian classical dance videos. Revue Intell. Artif. 37 (3), 689 (2023).

Google Scholar  

Banerji, A. The laws of movement: The Natyashastra as archive for Indian classical dance. Contemp. Theatr. Rev. 31 (1–2), 132–152 (2021).

Kishore, P. V. V. et al. Indian classical dance action identification and classification with convolutional neural networks. Adv. Multimed. 2018 , 1–10 (2018).

Guo, F., & Qian, G. Dance posture recognition using wide-baseline orthogonal stereo cameras. In Proc. 7th Int. Conf. Autom. Face Gesture Recognit. (FGR) (pp. 481–486) (2006).

Saha, S., Ghosh, S., Konar, A., & Nagar, A. K. Gesture recognition from Indian classical dance using Kinect sensor. In Proc. 5th Int. Conf. Comput. Intell. Commun. Syst. Netw. pp. 3–8 (2013).

Mallik, A., Chaudhury, S. & Ghosh, H. Nrityakosha: Preserving the intangible heritage of Indian classical dance. J. Comput. Cult. Herit. 4 (3), 11 (2011).

Kalpana, I. M. Bharatanatyam and mathematics: Teaching geometry through dance. J. Fine Studio Art 5 (2), 6–17 (2015).

Rodriguez, G.E. (2020). Dare to Dance: Exploring Dance, Vulnerability, Anxiety and Communication (Doctoral dissertation, The University of Texas at San Antonio).

Kim, D., Kim, D. H. & Kwak, K. C. Classification of K-pop dance movements based on skeleton information obtained by a kinect sensor. Sensors 17 (6), 1261. https://doi.org/10.3390/s17061261 (2017).

Article   ADS   PubMed   PubMed Central   Google Scholar  

Bisht, A., Bora, R., Saini, G., Shukla, P., & Raman, B. Indian dance form recognition from videos. In 2017 13th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS) (pp. 123–128) . IEEE (2017).

Odefunso, A. E., Bravo, E. G. & Chen, Y. V. Traditional African dances preservation using deep learning techniques. Proc. ACM Comput. Gr. Interact. Tech. 5 (4), 1–11 (2022).

Kaushik, R., &LaViers, A. Using verticality to classify motion: Analysis of two Indian classical dance styles. Creative Lab QUT, Tech. Rep., p. 5 (2019).

Kim, J. W., Choi, J. Y., Ha, E. J. & Choi, J. H. Human pose estimation using mediapipe pose and optimization method based on a humanoid model. Appl. Sci. 13 (4), 2700 (2023).

Article   CAS   Google Scholar  

Butepage, J., Black, M.J., Kragic, D., &Kjellstrom, H. Deep Representation Learning for Human Motion Prediction and Classification. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) . pp. 6158–6166. (2017).

Längkvist, M., Karlsson, L. & Loutf, A. Inception-v4, InceptionResNet and the impact of residual connections on learning. Pattern Recogn. Lett. 42 , 11–24 (2014).

Article   ADS   Google Scholar  

Quiñonez, Y., Lizarraga, C., & Aguayo, R. (2022). Machine Learning Solutions with MediaPipe. In 11th International Conference on Software Process Improvement (CIMPS) , pp. 212–215 (2022).

Shorten, C. & Khoshgoftaar, T. M. A survey on image data augmentation for deep learning. J. Big Data 6 , 1–48. https://doi.org/10.1186/s40537-019-0197-0 (2019).

Wang, J. & Perez, L. The effectiveness of data augmentation in image classification using deep learning. Convol. Neural Netw. Vis. Recognit. 11 (2017), 1–8 (2017).

Zhu, H., Deng, C., & Zhu, Y. MediaPipe based gesture recognition system for English letters. In Proceedings of the 2022 11th International Conference on Networks, Communication and Computing (ICNCC ‘22) . pp. 24–30. https://doi.org/10.1145/3579895.3579900 (2023).

Subramanian, B. et al. An integrated mediapipe-optimized GRU model for Indian sign language recognition. Sci. Rep. 12 (1), 11964 (2022).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Albawi, S., Mohammed, T. A. & Al-Zawi, S. Understanding of a convolutional neural network. In 2017 international conference on engineering and technology (ICET) . pp. 1–6. IEEE. (2017).

Shankar, B. S. Dance imagery in South Indian temples: Study of the 108-Karana sculptures (Doctoral dissertation, The Ohio State University). (2004).

Bhuyan, H., Killi, J., Dash, J. K., Das, P. P. & Paul, S. Motion recognition in Bharatanatyam dance. IEEE Access 10 , 67128–67139. https://doi.org/10.1109/ACCESS.2022.3184735 (2022).

Indolia, S., Goswami, A. K., Mishra, S. P. & Asopa, P. Conceptual understanding of convolutional neural network—A deep learning approach. Proc. Comput. Sci. 132 , 679–688. https://doi.org/10.1016/j.procs.2018.05.069 (2018).

Kaushik, V., Mukherjee, P., & Lall, B. Nrityantar. In Proceedings of the 11th Indian Conference on Computer Vision, Graphics and Image Processing . pp. 1–7. (2018).

Krizhevsky, A., Sutskever, I., & Hinton, G. E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25 (2012).

Paul, S., et al. NrityaManch: An annotation and retrieval system for Bharatanatyam dance. In Proceedings of the 14th Annual Meeting of the Forum for Information Retrieval Evaluation (2022).

Saha, A., Ghosh, S., Das, P. P., & Sarkar, I. Recognition and classification of accompanying audios of Kathak dance. In 2021 10th IEEE International Conference on Communication Systems and Network Technologies (CSNT) . pp. 860–864. (2021).

Sutskever, I., Martens, J., Dahl, G., & Hinton, G. On the importance of initialization and momentum in deep learning. In Proc. 30th International Conference on Machine Learning (ICML) . pp. 1139–1147. (2013).

Tamata, K., &Mashita, T. 3D mesh generation from a defective point cloud using style transformation. In 10th International Symposium on Computing and Networking Workshops (CANDARW) , pp. 218–223. https://doi.org/10.1109/CANDARW57323.2022.00025 . (2022).

Liu, W. et al. 3D building model generation from MLS point cloud and 3D mesh using multi-source data fusion. Int. J. Appl. Earth Observ. Geoinf. 116 , 103171. https://doi.org/10.1016/j.jag.2022.103171 (2023).

Meena, G. et al. Correction to: Image-based sentiment analysis using InceptionV3 transfer learning approach. SN COMPUT. SCI. 4 , 405. https://doi.org/10.1007/s42979-023-01874-2 (2023).

Meena, G. et al. Identifying emotions from facial expressions using a deep convolutional neural network-based approach. Multimed. Tools Appl. 83 , 15711–15732. https://doi.org/10.1007/s11042-023-16174-3 (2024).

Mohbey, K. K. et al. A CNN-LSTM-based hybrid deep learning approach for sentiment analysis on monkeypox tweets. New Gener. Comput. https://doi.org/10.1007/s00354-023-00227-0 (2023).

Article   PubMed   Google Scholar  

Jayanthi, J. & Maheswari, P. U. Comparative study: Enhancing legibility of ancient Indian script images from diverse stone background structures using 34 different pre-processing methods. Herit. Sci. 12 , 63. https://doi.org/10.1186/s40494-024-01169-6 (2024).

Download references

Author information

Authors and affiliations.

Deparment of Computer Science and Engineering, Anna University, Guindy Campus, Chennai, 600025, India

J. Jayanthi & P. Uma Maheswari

You can also search for this author in PubMed   Google Scholar

Contributions

J.J., conceived the presented idea, performed Data collections, performed computations, wrote the manuscript with inputs from P.U.M. P.U.M., designed the analysis of the presented idea, verified the analytical methods, and supervised the findings of the work. Both authors read and approved the final manuscript.

Corresponding authors

Correspondence to J. Jayanthi or P. Uma Maheswari .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Jayanthi, J., Maheswari, P.U. AI and augmented reality for 3D Indian dance pose reconstruction cultural revival. Sci Rep 14 , 7906 (2024). https://doi.org/10.1038/s41598-024-58680-w

Download citation

Received : 31 October 2023

Accepted : 02 April 2024

Published : 04 April 2024

DOI : https://doi.org/10.1038/s41598-024-58680-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Augmented Reality and its effect on our life

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.