Extended Reality: Bridging the Gap Between Reality and Fantasy
The other day, I picked up a book entitled Otherworld. It claims that if you’re a fan of the novel or movie “Ready Player One”, then you could get hooked.
I’m not really a fan of sci-fi, but I read it anyway. It took me just one day to finish reading it, cover to cover. True, it does have a gripping storyline. I was mainly drawn to it because of the vivid picture being painted had the virtual reality technology fallen in the wrong hands. Occasionally, a paranoid voice inside of me silently screamed, “Could all this happen to us?” as I read through the chapters.
As I snapped back to reality, I was weighed down with more questions. What’s the deal with this extended, alternate, digital, <insert confusing tech-sounding adjective here> universe sprung to us from the latest tech offerings?
The Hype Around Extended Reality
Extended reality (XR) technology is the umbrella term for technology that emulates reality. It encompasses 3D views, virtual reality, augmented reality, and other immersive technologies that allow individuals to experience a close-to-reality, yet still-artificial environment through computer-generated content.
Today, we’re able to experience reality technology to a certain extent using apps on our smartphones. Just think of Snapchat and Pokémon Go. For a more immersive experience, wearables like spectacles, head-mounted displays (HMDs), and headsets equipped with consoles have been designed and manufactured to serve this purpose.
Yet, so many of these terms are being carelessly referenced that the real meaning and usage behind them often get mixed up. It’s easy to confuse which is which.
First, let’s demystify the XR terms that keep making the headlines — virtual reality, augmented reality, and mixed reality.
Virtual Reality (VR)
VR refers to the experience of a digital, made-up world with no link whatsoever to the real world.
In a more simplistic setting, VR can be experienced through smartphone screens by inserting them into standalone headsets. Google Cardboard has famously served this purpose.
Of course, the most popular application would be VR games where you use either PC-connected or standalone headsets to achieve total immersion. From the generic flying and landing an airplane in a simulated flight cockpit to donning an Iron Man’s suit and taking down the Chitauri in one fell swoop, the choice of VR games varies.
The headsets have to be specially designed to support a fluid, lucid, and convincing experience. In this space, Oculus Quest, HTC Vive, and Microsoft HoloLens are the frontrunners.
Augmented Reality (AR)
On the other hand, AR allows you to view the physical world from a digital perspective. Essentially, the digital details act as an additional layer to the real world. A typical AR device needs to first identify surrounding objects and replicate a 3D model of the environment. Sensors like cameras, infrared detectors, gaze trackers, accelerometers, and microphones help to achieve this. Then, the information is further contextualized using software depending on the use case.
For example, an e-commerce software would be able to label the object in front of you and show online stores that sell them along with price tags.
Several AR wearables are in the market, ranging from the conventional ones like Microsoft HoloLens and Magic Leap headsets to lightweight glasses like Vuzix Blade and Google Glass. However, most of them are geared towards industrial uses.
You can experience AR using your smartphone too. Remember the much-hyped Pokémon Go? Players launch the game from their smartphones and start exploring the places around them, checking out PokéStops and catching Pokémon, among others. They’re able to see the stops and pocket monsters through the digital overlays provided by the app.
And there are also Snapchat and Instagram filters. The apps need to create a complete 3D map of your face first through facial and depth recognition before applying photo filters of your choosing.
Mixed Reality (MR)
This is where it gets more interesting. MR is the enhancement of AR in that the digital content is able to interact with the surrounding environment, using advanced sensing and imaging technologies. Put simply, MR is the interactive form of AR. More superior holographic devices are needed to experience this.
To avoid confusion, let’s consider Pokémon Go again. If it’s indeed an MR app, you can take a real-life ball and hit a Pokémon that you see on your smartphone. Or, you can take a virtual ball from the game itself and flick it to the innocent bystander in front of you. Can you do all that? I didn’t think so. That’s why it’s an AR game!
Notable Use Cases Using Reality Technology
All the while, we’ve been associating XR with gaming and entertainment. But now, people are beginning to recognize its disruptive capabilities in revolutionizing how we approach daily tasks, be it in industrial, commercial, or personal space.
Retail
Online shopping has become a big part of our lives. But one would argue that the experience is still different from shopping at physical stores. Take buying apparel as an example. Sure, an online shopping platform would have photos from different angles and size charts to help you visualize, but nothing beats going to the brick-and-mortar store and trying out the clothing yourself. It’s hard to end up with the wrong item once you’ve tried out different combinations. The same can’t be said for online shopping, though.
An AR app calls the “virtual shop” might just be the answer. You only have to find a wide enough space, download the app from your smartphone, scan the space to set up the virtual store, and voila, the store is ready for browsing! GAP has been doing this successfully, garnering media attention in CES 2017. Users can set up a virtual dressing room and try GAP clothes of different types and sizes from the comfort of their own home.
Virtual shops are not confined to the fashion industry. IKEA has also employed this concept by inviting customers to try out different furniture from its online catalog. All they need to do is just scan the furniture from the IKEA Place app which will create a true-to-scale 3D model, then they can place it anywhere in their living space to determine its fit with the rest of their furniture.
Check out the IKEA Place app demo here:
Manufacturing
With so much hype around Industry 4.0, it’s unthinkable that some forms of XR are not in the mix. From design stage right up to repairs and inspection, there are many use cases that manufacturing industry can benefit from. It’s high time for them, owing to the resource-consuming elements and inherent safety risks in this line of work.
Let’s take a shipbuilding company as an example. A typical design stage would involve poring over stacks of blueprint and computer images and flying in experts the world over to review ship design before the build takes place. Sounds cumbersome, right?
All this can be simplified and expedited by employing VR technology. The team can step into the virtual design platform and view the design components from different angles using their respective VR headsets or HMDs. They can shift equipment, machinery, and even doors around and remodel in real-time. Virtual collaboration and peer review sessions for design proposals can be made possible too without having physical meet-ups.
Imagine what VR can do for designs of smaller and simpler machinery and equipment!
Having said that, maintenance and repairs are essential in manufacturing and form a huge part of day-to-day operations. Safety procedures need to be adhered to as personnel would need to deal with broken parts and volatile surrounding environment that can induce harm. This is where MR can help.
The factory personnel can utilize MR headsets or glasses that can project holographic instructions right in front of them, overlaid on the equipment to be repaired/inspected. Sensors are used to detect the conditions of equipment and prescribe the next steps on tackling the repair works. For example, a heat sensor can warn the personnel that the equipment is hot to the touch, so the next step is to don protective gear before doing the repair work.
Healthcare
It’s of utmost importance to produce more competent and certified doctors for a better healthcare system. This is where VR can make a difference. It allows healthcare training to be done in virtually immersive mode, allowing an in-depth look into human anatomy, medical gears, and equipment, thus enabling doctors in training to sharpen their knowledge and skills.
By taking part in surgery simulations using VR headsets or glasses, it would complement the routine observations held at operating theatres where procedures can only be seen from a distance. Trainees would be able to make more sense of what they’re seeing with these virtual simulations.
For emergency first-responders, an AR platform that simulates emergency situations from locating the necessary medical equipment right up to administering AED to patients would ensure they have immersive, repeatable training. This would increase their level of preparedness in handling emergency situations, thus improving the chances of saving more lives.
Education
21st-century education is not confined to the classroom anymore. There may be a silver lining behind COVID-19 where it serves to accelerate boundaryless learning.
New learning platforms have been created where students can ‘experience’ lessons in a holographic, immersive environment. For example, instead of learning about Roman civilization from textbooks, they can be transported to the Colosseum and witness showdowns between gladiators. Instead of reading about the different animal species and their habitats, they can view them in 3D form and inspect the body parts to determine whether they’re mammals, reptiles, insects, and so on.
Learning process can still continue without having to go to school. This can solve current issues like the pandemic where it’s unsafe for children to be exposed in a crowded environment as well as other pre-existing issues like logistics and transportation. An immersive learning platform can complement online classes provided by teachers.
Knowing children’s inclination towards online games, incorporating gamification into AR platforms will increase their interest to learn. They can be given quizzes at the end of every module and their scores will be published on leaderboards as added incentives.
Check out this demo of VR learning about dinosaurs developed by Unimersiv:
Navigation
Yes, we already have Waze and Google Maps installed on our smartphones to show us the way. Yet, switching from looking at the phone to the road in front of you can be a hassle, even downright dangerous. Sometimes, a fraction of a second of not looking on the road can cost lives!
That’s why AR navigation has been seriously explored in road driving. Most car manufacturers like Mercedes and Cadillac are installing AR-enabled dashboard navigation systems in their cars. With GPS and map data along with camera at the front of the car, the driver can have real-time navigation with arrows showing where to go and chevrons showing the destination. Although the driver will still have to refer to the dashboard, this reduces distraction of having to translate maps on the smartphone screen into real-world understanding.
This technology can be extended to aerial navigation too. In fact, F-35 aircrafts come with AR helmets that provide 360° view of their surroundings, merge them into feeds, and label objects in terms of distance, bearing, and altitude. This helps pilots to gather intel and decide next courses of action.
Recognizing the Risks
Extended reality technology seems to be gaining ground in terms of the advancement of new designs and use cases. Yet, investigation on issues (and the drive behind it) like governance, code of ethics, and social and psychological impact to the larger community isn’t getting much traction.
Authorities should adopt a more hands-on approach in defining the governance and upholding the rules and regulations. Ethical issues in terms of their psycho-social, psychological, and mental health impacts on the younger generation as well as the general public shouldn’t be taken lightly too.
Safety and privacy
Leaks of privacy data have been viewed as a serious issue. The public is now more aware of data privacy with the enforcement of EU’s GDPR (General Data Protection Regulation) and other local data privacy acts. XR technology devices have to function within the regulations.
However, some finer details have yet to be addressed and tech companies need to come clean about this. For example, if Snapchat records my facial features to do the photo filters, is the information stored locally or somewhere in a cloud? If it’s sent to a cloud, is the information encrypted? Can this data be accessed by third parties, possibly used for future targeted ads? Are there ways for us to express consent/non-consent of sharing this kind of data?
It’s one thing to have adult acne cream ad popping up on my social feeds — I might just be mildly annoyed. But it’s quite another to have my personal data misused by some shady third party for their own unlawful gains.
It wouldn’t augur well to find out that XR can lead to major data breaches like the one that happened with Facebook, as an example.
Mental health
We’re already aware of Internet addiction. And with XR fast gaining popularity, it could get worse. It’s not uncommon to hear about children (and adults) who become engrossed in XR gaming until they become completely unaware of what’s happening around them.
When we take away their gaming equipment, they start exhibiting restlessness, mood swings, and violent behavior, not unlike drug addicts who are deprived of drugs.
With the convenience of having an avatar in an AR gaming or app, it’s a breeding ground for virtual harassment. Hiding behind different virtual personas, some individuals tend to show their irresponsible and darker side in front of other players.
The abuser can conveniently hide their actions while those at the receiving end would feel the full brunt of it due to virtual embodiment. Evidence has shown that psychological trauma can still be felt after witnessing one’s own avatar being harassed (e.g., being groped, grabbed, fondled). Make no mistake that online harassment could inflict the same negative effects, similar to those that happen in real life.
This could lead to mental health decline. If left untreated, it can manifest to something even worse like stress, depression, and suicidal tendencies. It’s bad enough that mental health issues are on the rise. We don’t want XR to contribute to the stats too.
Discerning what’s real and what’s not
Childhood and adolescence are important stages where brains continue to develop and mature. As we need to distinguish between the reality and the artificial in an XR activity, its effects on children and teens should be holistically analyzed.
Adults may be able to seamlessly transition between reality and imagery with their more mature cognitive functions. But what about the younger generation in this day and age who spend most of their time on the Internet, and now rapidly shifting to the XR world?
In fact, a research had documented a case where teens asked whether aliens had indeed landed on their school campus after playing an outdoor AR game, Alien Contact.
Studying and isolating XR effects on segmented groups of adults, teens, and children should be prioritized, in tandem with the technological advancements of XR itself.
Keeping it Real with XR Technology
In the book I read, the mixed reality project was initiated by a tech giant with the noble intention of letting people in a coma exercise their brain functions by being immersed in an alternate, artificial reality. They theorized that it can be a stimulus to awaken their body parts, eventually bringing them out of the coma. But it turned awry when some patient results didn’t turn out as expected, causing uproar and mistrust among their loved ones. These people started to question the ulterior motives of the corporation that started this initiative.
Although this story is entirely fictional, there are lessons to learn. As mentioned earlier, coordinated research should be done to study XR impact from the most granular to the highest level in different facets of society. Any recommendations or precautions should be brought up for public awareness.
There are also legal boundaries that should be defined clearly in regulating the usage of XR devices and apps. Ugly consequences can happen if authorities simply let tech companies run the show without any check-and-balance, just like what happened in the book.
After all, it’s the people and community that should benefit from all these innovations and advancements — we can’t lose sight of that.
