Hallucinating XR

There is a distant possibility of an Extended reality experience through hallucination. Let’s see the current state of tech, advancements & feasibility.

Before we get there,

Do you know Extended Reality (XR) is one of the art forms practiced in Ancient India?

It’s called Indrajaal (இந்திர ஜாலம்: Tamil) — The art of Magic, illusion and deception. Listed in the 64 art forms that were practiced in ancient India. Wondering how Magic, illusion and deception is related to Extended Reality? Read on…

In fact, the earliest form of Augmented Reality was put to use by stage magicians. The first ever stage illusion of a ghost was produced in 1862 where a glass panel placed at an angle in front of the stage reflecting the ghost artist acting below the stage is made to appear on stage due to Refraction of light by the glass. This effect was called “Pepper’s Ghost” and produced the desired augmentation for the audience. Augmented reality is a century old technique

Most of the modern day Smart glass technology (Hololens, Lenovo ThinkReality, nReal) uses this Pepper’s Ghost effect at a very small scale in the wearing glass to produce the Mixed reality effect. In a nutshell, most smart glasses hold the glass that does the refraction of light mounted in the front and the glass is black tinted to make sure that the refraction is visible for various lighting conditions. A small projector beams the content on the glass surface, the refraction of light gives an illusion that the content appears at a distance from the viewer wearing the glass. That’s the art of illusion elevated by current scientific improvements in electronics.

The technology has gone through several transformations and it has reached our hands almost in the form factor of a wearing glass today. There are however advancements in this domain which lead me to project where the future is headed.

Wearable form factor is reduced to the form factor of contact lens. The early prototypes released by Mojo vision promises just that.

This prototype wearable contact lens produces monochromatic augmented reality for the wearer.

This is just the beginning of the huge potential that XR has to offer for humans. The interaction with this form of XR system is currently limited to the retina movements. This may change seeing the advancements that are made to the BCI or BMI (Brain computer interface, Brain Machine Interface) and AI.

While this may sound more science fiction at this point, projecting further on this space and trends I cannot exclude the possibility of a “Hallucinating XR”.

Neuralink — Most of us know that this Elon Musk backed startup is working in the space of brain implants to stimulate cortex and sensory parts of the brain to attain desirable effects.

While using Neuralink as an enabler to record and induce neural stimulations, there are advancements in decoding these signals into visual imagery

Decoding models trained on stimulus-induced brain activity in visual cortical areas showed accurate classification, detection, and identification of contents.

Source: https://www.science.org/doi/10.1126/science.1234330

Combining Neuralink with AI models trained for visual decoding may open up possibilities for recording visual experiences from the visual cortex of the brain. If we reverse the process we get the desired stimulations to induce visual imagery in a brain signal encoded form. An experience in this form the end user will start to hallucinate an extended reality experience. The possibility of this happening is still far enough with low probability, But we cannot hide the fact that the science is inching towards it.

Ethical & Responsible Tech:

While this possibility may look scary as with any other technological innovation. While we are working on these technologies, we must also work in parallel on regulatory and ethical use of this technology so that we retain our human values and respect our biological evolution process.

About the Author:

Mr. Raju Kandaswamy
Lead Consultant
ThoughtWorks