Fixing the metaverse: Augmented reality pioneer shares ideas for avoiding dystopia

In a Big Think article last month, I warned that a corporate-controlled metaverse is rapidly coming our way, and it won’t be all sim-rainbows and sim-roses. Yes, there will be magical applications, but the real metaverse also will be intrusive and overwhelming, adding a pay-for-play layer to our lives that will make today’s problems with social media seem quaint.

In response to that piece, many have reached out, as the warnings I outlined resonated with them. They wanted to know what we can do to avoid the dangers. I have thought a lot about this over the last decade and have several ideas to share. To present this methodically, it is helpful to address three key questions: (1) What will the metaverse really be like when widely deployed? (2) What are the most significant risks? and (3) What are the most helpful solutions?    

What will the metaverse really be like?

To answer this, we must predict what will replace today’s ecosystem of desktops and mobile phones as our primary means of accessing digital content. There are two likely scenarios: either we will live in a virtual world, wearing headsets and gloves like Facebook is currently pitching, or we will live in an augmented world, wearing see-through glasses with content splashed all around us. Both futures are possible, but I firmly believe augmented reality (AR) will be the platform of our lives by the end of this decade. Virtual reality (VR) will be popular but for limited duration uses like gaming, entertainment, and targeted business applications.

Why does AR win? Having been involved in both technologies from the beginning, I see no way to escape a very simple fact: people do not want shoeboxes strapped to their faces. It is not just the size and weight that are off-putting, but the sense of being cut-off from your surroundings. In fact, it was this feeling of being enclosed and isolated while working with early vision systems at NASA that inspired me to pitch augmenting our reality to the Air Force 30 years ago. And as I studied people using those early prototypes, I was convinced AR would eventually become the medium of our lives.   

People do not want shoeboxes strapped to their faces.

Dr. Louis Rosenberg on why VR will lose out to AR

I know many believe VR will be the dominant platform, but I sincerely hope it does not go this way. I say that because the more we distance ourselves from in-person interaction, the less empathy we will feel for each other, reducing fellow humans to sim-characters in a sim-world. I have been concerned about this for a long time, even writing a dystopian graphic novel in 2008 (Upgrade) that describes the callous and oppressive society that can result if we all retreat into a virtual world. Coincidentally, the key plot point of that book was a perpetual global pandemic that forced everyone indoors, giving us no choice but to embrace purely virtual lives.

Still, barring such a disaster, I remain confident that augmented reality will inherit the world from phones and desktops, shaping our lives for the next few decades. I also believe AR will make our world magical, allowing us to embellish our surroundings with virtual content that seems real but is playfully untethered from the laws of physics. This will open amazing opportunities for artists and designers, entertainers and educators, and of course, marketeers. AR will also give us superpowers, enabling us to alter our surroundings with the flick of a finger or the blink of an eye. Rather than give pages of examples here, I point you to Metaverse 2030, a piece I wrote to convey what our augmented lives will be like 10 years out.

What are the primary risks we face?

It’s not the technology of the metaverse that is so dangerous, but the fact that powerful corporations will be able to mediate all aspects of our lives, selling access to our eyeballs to the highest bidder. I know this sounds a lot like today’s social media, but in the metaverse, the intrusion will be far more intimate than any media technology ever created. This will enable the big platform providers to monitor, manipulate, and monetize us like never before. I call these risks The Three M’s of the Metaverse:    

(1) Monitoring. Over the last two decades, tech companies have made a science of tracking our behaviors, analyzing how we browse and where we click so they can peddle our profiles to advertisers. Many consider this an egregious invasion of privacy, but little has been done to solve the problem. In addition, this obsession with tracking and profiling has made social media a destructively polarizing force, allowing platform providers to target us with custom messaging that amplifies our existing biases and preconceptions, radicalizing populations.

In the metaverse, this gets much worse. The technology will not simply track what you click on, but where you go, what you do, and what you look at — even how long your gaze lingers. The platforms will also track your facial expressions, vocal inflections, and vital signs (as captured by your trusty smartwatch), while intelligent algorithms will use this data to predict your emotional states. This means the companies controlling the metaverse will not just know how you act, but how you react, profiling your responses at the deepest level. Of course, the danger here is not that they track these things, but that they can use this data to manipulate our wants and needs, influencing not just what we buy but what we believe.

(2) Manipulation. From the early days of radio and TV, advertisers have targeted us by demographic, skillfully influencing our views. With the advent of social media, segmenting the public has gotten far more precise, enabling hyper-targeted messaging. In the metaverse, this targeting will get far more personal, the content much harder to resist. After all, in today’s world, we usually know when we are being advertised to and can muster a healthy dose of skepticism. In the metaverse, we will not be hit with overt pop-up ads or promo-videos but simulated people, products, and activities that seem just as real as everything else around us.

For example, in the metaverse you will meet people who look and act like any other user, but they will be computer generated personas (SimGens, I call them) that are programmed to engage you in conversation, reading your facial expressions and vocal inflections so they can pitch you more skillfully than any used car salesman. And they will be crafty, armed with a database of your interests and inclinations, plus a history of your previous interactions with similar ads. Even the manner in which these SimGens appear to you — their gender, hair color, eye color, clothing style — will be custom crafted by algorithms that predict which features are most likely to influence you personally. I know this sounds creepy, but it will happen unless we demand regulation to prevent it. (More on regulation below.)

(3) Monetization. As a longtime entrepreneur, I appreciate that platform providers are not charities: They need business models that generate real revenue. And because the public has resisted paid subscriptions, the model they have adopted is free access in exchange for advertising. This is why so much effort has gone into tracking us and profiling us and targeting us. We the public have chosen to be the product that is bought and sold rather than the customer paying the bills. I point this out because a great way to solve these problems is for us users to change our mindset, being willing to pay for access to these environments rather than selling access to ourselves.

What are the most helpful solutions?

Credit: Kaspars Grinvalds / Adobe Stock

As described above, shifting from ad-based to subscription-based models could be a powerful fix, eliminating the motivation that platform providers have to monitor and manipulate their users. Unfortunately, this only works if consumers are willing to pay for access. I suspect some users will be willing to pay for a safer metaverse, which will inspire entrepreneurs to create subscription-based platforms, but we cannot assume that this will become the norm anytime soon. We also cannot expect people to simply opt-out of the metaverse, as it will be our primary interface with digital content. Opting out will mean missing out on critical information in our world.

So what can we do to protect the public? At the risk of sounding cliché, the best solution is to enact significant and meaningful regulation. Of course, the nuanced question is: What specifically needs to be regulated?  

First and foremost, we need to limit the level of monitoring allowed in the metaverse. The providers will have access to everything we do and say and touch and see. In my opinion, they should not be allowed to store this data for more than the short periods of time required to mediate whatever real-time experience is being generated. That will reduce the degree to which they can profile our behaviors over time. In addition, they should be required to inform the public as to what is being tracked and how long it will be retained. For example, if they are monitoring your gaze, you need to be overtly notified.

At the same time, there should be strict limits on what type of tracking is allowed and for what purposes. For example, the public could demand restrictions on advertising algorithms that monitor your facial expressions, vocal inflections, posture, and vital signs (including your heart rate, respiration rate, pupil dilation, and even galvanic skin response). I know this type of tracking sounds extreme, but it is the direction we are headed, and it is not far off. Unless we strictly regulate the metaverse, these very personal physiological reactions will be used to fine-tune marketing messages, adapting their strategy to influence us in real-time.

In addition, we need to assume the metaverse will move away from traditional marketing methods like pop-up ads and promo-videos, instead targeting us in far more natural ways, injecting promotional objects and activities into our world that look and feel real. If a third-party pays for a virtual product placementin your augmented surroundings, perhaps they should be required to inform you that it is a targeted placement, not a serendipitous interaction that you just happened to stumble upon.

The same is true when advertisers target us with simulated personas that engage us in what feels like natural conversations. They could be required to inform us clearly and overtly whenever we interact with conversational agents controlled by intelligent algorithms, especially when the algorithms have a hidden promotional agenda. This becomes even more important when those algorithms are also monitoring our reactions, for example assessing our posture and breathing so it can skillfully adapt its approach in real-time. This type of “interactive manipulation,” optimized by AI, will happen soon, and it will be profoundly coercive unless highly regulated.

Is the metaverse worth it?

Clearly there are dangers to avoid as we transition from phones and desktops to immersive worlds. This begs the question: Is the metaverse even worth it? Personally, I believe the technology has the potential to make our lives magical, expanding what it means to be human. But to avoid the hidden perils, we should consider proactively regulating this space. And we need to do it now, before the problems become so ingrained in the infrastructure and business models that they are impossible to unwind.

I know, meaningful regulation is never easy and rarely a popular pursuit. But without restrictions on platform providers, we may find ourselves in a fully mediated world that looks and feels natural, while behind the scenes, powerful corporations are manipulating our lives for the highest bidder, altering our experiences without us even realizing it. This is not the future I want for myself or my kids, so I would encourage regulation now.

Like it or not, the metaverse is coming soon. Let’s help make sure it is a wonderful experience.