NCTA — The Internet & Television Association

Virtual Reality, Virtual Sets, Real Entertainment

Virtual Reality, Virtual Sets, Real Entertainment

Mandalorian Header

While 2020 has forced many industries to quickly adapt to new ways of working, TV and film have always sought creative solutions to create amazing programming. These challenges, more recently, have been made easier with new technologies that require high-capacity broadband connections. How would different parts of a TV or film crew, be it the set design department or post-production team or visual effects team, learn to work together, remotely? That conundrum was at the center of a panel discussion at this year’s Comic-Con, which was held remotely this year due to COVID-19.

“The main thing I’ve noticed is that organization has to be so much better than it was before. It may have even been a false sense of organization, with having your whole team actually physically there, you just assume they’re all working on the things you want them to be working on,” Ryan McCoy, Senior Visualization Supervisor for Halon Entertainment remarked on the panel. Halon is a virtual visual production company that recently worked on Disney+’s new Emmy-nominated series The Mandalorian. “And now that you’ve pulled the rug out under that, you actually have to make sure things are accountable and that there are ways of quickly checking in that in some cases are more effective than they were before.”

“Part of what’s been fun about collaborating on The Mandalorian with LucasFilm and Disney, is that we have been able see through a few technical innovations and a few firsts that I think are going to have a lot of impact on the way television and movies are made moving forward,” Jon Favreau, Creator and Executive Producer of The Mandalorian said in an interview. The show, billed as a space western, is set in the Star Wars cinematic universe and therefore requires a lot of believable out-of-this-world effects, characters, and sets. A challenge that, in the past, was solved by scouting remote locations and building detailed prosthetics.

But new innovations in technology are starting to change that. One example is Unreal Engine, a platform normally used to develop cinema-quality video games including The Outer Worlds and the Bioshock franchise. The software allows multiple designers to create open worlds complete with textures and near-perfect lighting which are also able to change at the drop of a hat. It also means that team members can collaborate on these virtual sets no matter where they are. That’s how The Mandalorian went about production, even before the COVID-19 outbreak. The show uses a state-of-the-art digital-screen stage in lieu of traditional sets or on-location filming, which gives it a sense of reality rarely achieved in Sci-Fi. The stage combined with the virtual-world map means directors can completely manipulate their surroundings, almost like a holodeck. “Just being able to shoot a 12 hour sunrise or a 12 hour sunset…those opportunities are endless,” said Kenny DiGiordano, Halon’s Virtual Art Department Supervisor.

Mandalorian Virtual Stage
Image via StarWars

Another benefit of this virtual production method means, “rather than everyone going to the location, the location comes to you,” Halon founder and director Daniel Gregoire pointed out. Thanks to crowd sourced images and satellite pictures, “We can go to a [location] and fully take…a couple thousand pictures and turn around and rebuild that,” DiGiordano explained. For The Mandalorian, crews used a set of drones to capture detailed images of Monument Valley in Navajo Nation, AZ. That data was used to build a hyper-realistic digital set of the rock formations and desert, which could be used at any time, meaning no long treks for the cast and crew to get a few shots.

Remote editing is being upended as well. Directors can review scenes directly in a platform like Unreal Engine, which allows them to adjust things like lighting and camera position without having to wait for a file to review and pass back to the team to make edits, regardless of their physical location. That type of flexibility is what allowed director Wes Ball to work mostly remotely on his project Mouse Guard, showing that even before COVID-19 remote solutions were on the rise.

Digital world building also allows location scouts to create a virtual landscape that the crew can not only drop itself into, but also manipulate and move how they see fit. And depending on the type of project, “…we could go for a pre-vis quality where it’s a scout and it’s just the initial feeling of how this will be taken in, how you’re going to experience this as an actor, as a cameraman, where should we even stage these things? Then we can take it a step further and actually film this stuff with the quality and fidelity to bring it higher than that into a more final realm,” McCoy noted.

It’s not hard to imagine the implications this kind of technology has for all sorts of films and shows, but only time will tell how standard this practice becomes. For now, shows like The Mandalorian are using next-generation technology platforms and giving us a sneak peek at what the future of TV and film production might look like.