New 3D hologram technology projects soccer games on your kitchen table

Millions of people will watch the 2018 FIFA World Cup this week on TVs and computers. But new technology suggests the next World Cup might be watched through 3D holograms.

You might watch the next FIFA World Cup in hologram mode. (Credit: University of Washington)
You might watch the next FIFA World Cup in hologram mode. (Credit: University of Washington)


As the 2018 FIFA World Cup kicks off today in Russia, millions of people around the world will watch the matches on their TVs, computers, and smartphones. But what if you could watch it on your kitchen table in the form of a 3D hologram?

That was the goal of a recent project between researchers from the University of Washington, Facebook, and Google. The team developed a first-of-its-kind deep learning-based system that was able to transform videos of a soccer game into a “moving 3D reconstruction, in which the players and field can be rendered interactively with a 3D viewer or through an Augmented Reality device.”

Past experiments have shown it’s possible to reconstruct, say, a soccer match in a 3D hologram by outfitting the field with many high-tech cameras. But that’s expensive and difficult. That’s why the researchers set out to accomplish the same thing by using only a single YouTube video to reconstruct the soccer game in 3D.


Stills from video game converted into 3D imagery

 

First, the team had to train something called a convolutional neural network, a type of A.I. algorithm commonly used to analyze and produce visual imagery. The researchers decided to use soccer video games: They showed the network 12,000 2D images from FIFA games and fed it data sent between the GPU and game engine. This helped the neural network create a depth map from which a hologram could be created.

“FIFA, similar to most games, uses deferred shading during gameplay,” the team wrote. “Having access to the GPU calls enables capture of the depth and color buffers per frame. Once depth and color are captured for a given frame, we process it to extract the players.”

The next step was to try rendering 3D holograms from real-life soccer matches on YouTube. The results were impressive.


Stills from real-life soccer games converted into 3D imagery

But there are still kinks to work out: the hologram didn’t reproduce a moving ball, its image quality was somewhat low compared to the 4k-resolution source videos, it wasn’t produced in real time, and it could only be viewed from one side of the field.

Still, it's possible researchers will be able to develop the technology in time for the 2022 World Cup, which will be held in Qatar.

Weird science shows unseemly way beetles escape after being eaten

Certain water beetles can escape from frogs after being consumed.

R. attenuata escaping from a black-spotted pond frog.

Surprising Science
  • A Japanese scientist shows that some beetles can wiggle out of frog's butts after being eaten whole.
  • The research suggests the beetle can get out in as little as 7 minutes.
  • Most of the beetles swallowed in the experiment survived with no complications after being excreted.
Keep reading Show less

We're creating pigs with human immune systems to study illness

Are "humanized" pigs the future of medical research?

Surprising Science

The U.S. Food and Drug Administration requires all new medicines to be tested in animals before use in people. Pigs make better medical research subjects than mice, because they are closer to humans in size, physiology and genetic makeup.

Keep reading Show less

A new warning to sign to predict volcanic eruptions?

Satellite imagery can help better predict volcanic eruptions by monitoring changes in surface temperature near volcanoes.

Volcano erupting lava, volcanic sky active rock night Ecuador landscape

Credit: Ammit via Adobe Stock
Surprising Science
  • A recent study used data collected by NASA satellites to conduct a statistical analysis of surface temperatures near volcanoes that erupted from 2002 to 2019.
  • The results showed that surface temperatures near volcanoes gradually increased in the months and years prior to eruptions.
  • The method was able to detect potential eruptions that were not anticipated by other volcano monitoring methods, such as eruptions in Japan in 2014 and Chile in 2015.
Keep reading Show less
Politics & Current Affairs

Moral and economic lessons from Mario Kart

The design of a classic video game yields insights on how to address global poverty.

Quantcast