Apple’s truly revolutionary headset will usher in the age of augmented reality
- Up until now, augmented reality headsets largely have been limited to professional applications. One reason, among many, is that headsets hide the user’s eyes from everyone else, making the technology isolating and a bit creepy.
- Apple’s innovative Vision Pro headset solves this through its “EyeSight” technology, which simulates the user’s eyes behind a glass surface, creating the illusion for others that they can see the user’s face.
- This innovation will greatly increase adoption, as it eliminates the cold and dehumanizing vibe associated with headset use.
After years of speculation, Apple made its most significant announcement since the launch of the iPhone in 2007, unveiling its new mixed reality headset called the Vision Pro. And like the iPhone before it, which kicked off the age of mobile computing, the Vision Pro aims to transform our lives for years to come, blurring the boundaries between our physical and digital worlds.
In this augmented future, virtual content will be seamlessly merged with our surroundings, creating experiences throughout our daily lives that are wildly artistic, fancifully creative, deeply informative, or just plain useful. As explained in the keynote, Apple expects Vision Pro to be used for everything from exercise and entertainment to productivity, communication, education, gaming, content development, and of course, shopping — and all of it will be fully immersive.
How Apple revolutionized the headset
This is not a new vision for computing. The first experiments with immersive media go back to the 1960s and a device called the Ultimate Display developed at Harvard. It enabled “head tracking,” so users could look around and explore 3D models from different perspectives. This inspired the field of virtual reality in the late 1980s when computers were finally able to create immersive worlds in real-time. And by 1992, the first mixed reality system was developed at Air Force Research Laboratory (the Virtual Fixtures platform), enabling users to reach out and interact with real and virtual objects.
That was three decades ago, but thus far, augmented reality headsets have been limited to professional applications. This is why the launch of the Vision Pro is such a milestone. Apple has taken the field to an entirely new level of performance and usability. The Vision Pro has 12 cameras onboard, some to track hand motions, others to track eye motions, and yet others to capture the real world and display it on internal screens. This is a technique called “camera passthrough,” and it allows the device to accurately merge the real and virtual worlds.
Passthrough has long been a painful tradeoff for headset developers because it enables high-fidelity visuals but hides the user’s eyes from everyone else. This has made the technology feel creepy and inaccessible for mainstream applications. But Apple solved this by simulating the user’s eyes behind a glass surface on the front of the Vision Pro, creating the illusion for bystanders that they can see the user’s face. Apple calls this technology “EyeSight,” and I am confident it will greatly increase adoption, as it eliminates the cold and dehumanizing vibe associated with headset use.
Our augmented future
Of course, Apple is not the only major corporation driving our augmented future. Earlier this year, Google, Samsung, and Qualcomm announced a partnership to create an augmented reality platform based on Android. Last month, Google announced Geospatial Creator, a powerful tool for AR developers. Two weeks ago, it was reported that Meta is in licensing discussions with Magic Leap to bring next-gen AR devices to market. And just a few days ago, Meta announced its Quest 3 headset, its smallest and lightest product equipped with passthrough AR capabilities.
For me this is both exciting and scary. As a longtime researcher in the field, I am excited because I firmly believe that augmenting our world will make it seem as if we all have superpowers. But it is also scary because the technology is so powerful and has significant potential for abuse. For instance, AR devices could track our behaviors and monitor our emotions throughout our daily lives, and when combined with AI technology, it could be used as a powerful tool of manipulation.* Therefore, we must remain vigilant to ensure that our augmented future will be truly magical.
*With support from Minderoo Pictures, the Responsible Metaverse Alliance, and the XR Guild, I helped develop a short film called “Privacy Lost.” Unveiled at the Augmented World Expo (AWE) in Silicon Valley last week, the film depicts a young family eating in a restaurant while augmented reality transports them to a beachfront café. Using AI technology, the family’s emotions are processed in real-time, enabling an AI-generated waitress to optimize her sales tactics, persuading each user through conversational influence.
