Virtual film production provides real-world skills for School of Informatics and Computing students

Hollywood could be a future stop for some IUPUI students who learned skills and technology that have become increasingly important in the filmmaking industry after they teamed up on a unique School of Informatics and Computing project.

Zebulun Wood reacts to the presence of a virtual dragon on screen during filming of a commercialView print quality image
Virtual production, a popular filmmaking technique, captures actors' performances while allowing filmmakers to control and create the physical settings. Photo by Liz Kaye, Indiana University

A team of five students in Zebulun Wood's N420 Multimedia Production Management class used virtual production, a technique popularized by "The Mandalorian," to create a commercial for the Media Arts and Science program.

"These skills gave the students an early sneak peek at trends in the filmmaking industry," Wood said.

Virtual production uses on-set cameras and actors but then layers digital backgrounds that are created from video-game engines, Wood said. It's a way to capture the performances of actors while controlling the physical location.

"This method of filmmaking has been popularized during the COVID-19 pandemic because there is less need for the entire team of filmmakers to travel to multiple locations around the world," Wood said.

The students -- Kayleigh Jones, Joel Holland, Isaac Johnston, Brandon Winkler and Christian Cochran -- pooled their skill sets from multiple specializations in film, 3D and game design to combine cameras and camera movement from the real world with the same movements happening in the virtual world, Wood said.

This type of project was a first for any team in his class, Wood added, and the challenge for the students was to use the school's existing technology for the new method of filmmaking.

Holland is a senior in the Media Arts and Science program who is focusing on game design and development. He said he first learned about film and visual effects in high school, and he knew that game engines had been used for real-time animation and rendering. He said he'd been curious about the technology but never thought he'd have the opportunity to see it up close.

"So, when I saw that the school believed it had the equipment to make it happen, I couldn't sign up fast enough," Holland said.

Winkler, a senior in Media Arts and Science, said that part of the team spent several weeks researching the hardware and software needed. The team used Unreal Engine for the game engine. Some students produced a text script, and one created a visual script using the game engine.

A student stands in front of a virtual world during filming of a commercialView print quality image
The students used game-engine technology to create the virtual worlds that appeared in their commercial. Photo by Liz Kaye, Indiana University

For camera expertise in form of directors of cinematography, professor Thomas Lewis recommended undergraduate students Cory Shearer and Alex Gao, who volunteered for the project, Wood said. They worked with the team to ensure a smooth shoot, tested camera light in real and digital worlds, and checked the final image quality.

Unreal Engine allowed the team to build its virtual worlds. Production also involved removing the greenscreen and adding a special C-shaped screen to wrap around the actors in the environment they were acting in. With the help of virtual reality, the team could move a real-life camera and have the in-game camera move synchronously, Winkler said.

For the final product, the students placed an actor on stage to give the illusion of being in both a futuristic city and in a medieval town with a fire-breathing dragon.

"The students met and exceeded all expectations, completed the project earlier than anticipated and have left our program in a very good position to continue the development of these skills within our coursework and curriculum offerings," Wood said.

Winkler said that the technical and communication skills he learned on the project are crucial to his career as a game developer and project manager.

Cochran, a senior who is specializing in game development, said the project taught him how to work with virtual cameras and improve his video editing. It also took him out of his comfort zone and has given him a look at what he can do in the future.

Jones, a junior who is focusing on game design and development and digital storytelling, said she learned a lot from the project and in her role as project manager and writer because the class provided an experience close to one in the real-life workforce.

"The project taught me a lot about filmmaking and innovative technology entering the workforce," Jones said. "It also taught me a lot about how to work with the client and adjust your plan to the new specifications they can spring on you. I can easily use these skills in my future career."