As the technology of virtual reality improves, we are going to start spending more time and getting more emotional inside it, says VR filmmaker Danfung Dennis. We are nearing the age when VR will cause profoundly intense and transformative experiences. But first, we need to create a new visual language that can really take advantage of the new possibilities of this amazing tech.

Danfung Dennis: Yes, it’s actually too early to really have any hard and fast rules of VR.

I think it’s maybe like the internet in 1998 where you could see that it was coming but it’s pretty hacked together. Everything from the cameras, the stitching pipeline, to the headsets themselves—they’ve been at this prototype phase, literally taking GoPros and sticking them together to make a 360 degree camera—to manually aligning images so they stitch correctly, and popping phones into plastic holders.

I mean all of that is going away and is being replaced with the real tools. We’re getting real VR cameras, we’ve built an automated stitching pipeline, and we’re going to have headsets that are affordable and easily accessible. And so I think we’re really learning from just the difficulty of the technical challenges of creating VR, and some of those challenges are melting away.

The “language” of it is actually much harder. How do we use this effectively? We’re finding things that do work. We’re finding a lot of things that don’t work. But we know that these experiences need to be longer. They’ve been pretty short, under ten minutes. We’re finding that the longer you spend in VR – and it has to be comfortable – the more immersion and feeling that you are in that world, the more your mind starts to accept it. And so we’re creating potentially 40 minutes of content, up to 60 minutes where you can go in, have these long durations and come out and have these profound experiences within that timeframe.

We know that the visual fidelity needs to get much better. Right now we’re at 4K by 2K, and when stretched out 360 degrees it looks low-res. We need resolutions of 8K by 4K and frame rates of 60 frames per second. We need perfect synchronization between all of these cameras, and ambisonic audio where an audio source sounds like it’s actually coming from this position.

When all of these factors start combining we’re going to have these high-fidelity experiences where the fluidity of emotion that can transfer in these worlds is going to be unparalleled. And so I think we’re just beginning on this curve of VR where the technology, the storytelling are starting to come together where we’re passing the prototype phase and we could actually use it to create these profound experiences — people come out after even ten minutes, come out of a headset and they will say, “I was so moved by that.” And a year later will come back and say, “that experience changed my life.” And I haven’t heard that before in films and images, that a short experience can have such a profound and lasting impact.

And so I think there is this real potential that we’re just starting to crack. Right now it’s a little bit of radio on the television; It’s an entirely new medium, so a lot of the cinematic language from traditional documentary and cinema, it has to be rethought in this new medium of VR. From the basics of composition – there is no frame.You’re working in a 360 degree environment. Cuts can be very abrupt and kind of take time for the viewer to reorient themselves in a new scene.

So even some of these real basics change. So we’re still learning what this language looks like, but we know it’s very spatial. You are feeling like you’re actually there, this sense of presence, it can be very strong. You are trying to interpret, your brain is interpreting these scenes as real and your body is reacting to them as they’re real as well. So you can have these very intense or meditative experiences in VR depending on how you use it.

We first thought that the whole crew had to disappear from a shot, and so we would set it up on a tripod and everybody would leave, and we’d get what we’d get. But then that was very limiting to shoot everything on tripod. We really wanted to move the camera and move with our subjects.

So we decided to leave our camera operator in the shot carrying our camera which we really streamlined in weight so that we could put it on a stabilized gimbal. This allowed us to move the camera through space which hadn’t really been done before in VR, but in a really comfortable way. It’s moving in a straightforward, in a level manner. It’s comfortable for the viewer and the headset. But it also really heightens this sense of space when you’ve got this much more information passing by you, you really start to feel the environment.

But it also meant that the camera operator was in the shot. If you look behind you, there he is, holding the camera. And that broke the standard rule of “no crew in the shot”. But it really didn’t bother anybody. It was just sort of, “Yes, that’s the process of making VR. It’s 360. You can’t hide everybody.” And he was just part of the scene, and some people really enjoyed it. They wanted to know sort of what he was feeling in response to some of the things that he was filming. And there he was taking them through.

So that’s just one example of where we sort of discarded a rule that had been established, made a new one, and I’m sure new ones will replace that. So we’re still constantly iterating on what works and what’s effective.

Filming anything that has the potential to be very emotional, that will transfer very well in VR. Filming things that are hard to access, that you can’t either travel to or they’re behind closed doors or they’re hidden on purpose, and there’s just no visual record of it. Those are very effective in VR. You’re experiencing something completely new, completely novel that you otherwise never would have.

So good access, an emotional story, and then strong characters always really take you into a world. And those few things are some of the building blocks that we’re working with. But we’re still experimenting. We’re still trying to understand this language. But we know it’s very spatial. It’s very emotional, and has the potential for these profound experiences.