Chapman University Looks to the Future with Virtual Production Technology

One of Chapman's first forays into virtual production.

Chapman reached out to AbelCine to design and install the LED setup, comprised of AOTO LED displays, as well as to provide training for their own instructors. Daniel Leonard, Associate Dean, and Madeline Warren, Associate Professor, Media Arts Division Chair, were kind enough to answer a few questions about the project.

What attracted you to virtual production and creating an LED volume in the first place?

It’s clear that this is where the industry is headed, and we wanted to get ahead of the curve. Having an LED wall on campus, on one of our soundstages, provides an opportunity for students to learn new industry-standard skills that would otherwise not be possible. All credit to The Mandalorian for inspiring us.

How was your experience working with AbelCine and The Virtual Company?

Great! They put it together on a tight timeline, were flexible with our financial arrangements, and have been very responsive to questions from faculty, staff and students.

Noah Kadner (The Virtual Company), with the support of AbelCine Tech Specialists, conducts a training session.

Have you seen a positive reaction from students towards the option of virtual production and do you find it is a draw to new students?

Yes to both! One group of students has already shot a music video on the LED wall, and we are reviewing many requests from other students. It is definitely a draw to new and prospective students.

What are some of the challenges and new workflows you’re exploring?

One challenge is that the cutting-edge technology of virtual production is in a rapid phase of development, and the tools are evolving rapidly. Another challenge is creating worlds in 3D, and making the virtual production design and lighting match the physical set. As for new workflows, we’re exploring working with greenscreen within the wall.

Student's filming at Chapman's LED wall.

What's your approach to camera tracking?

Right now we’re using the VIVE Tracker 1.0 and having some issues with it, so we’ll soon be moving to the 2.0. We’re also exploring advanced tracking systems like the Mo-Sys system, which will allow us to expand the boundaries of the wall beyond its present configuration and allow for extremely wide shots blending the LED wall and the environment generated in Unreal beyond the edges of the LED wall.

What's your approach to camera setup, config, and lens choice working in the environment?

We are careful to match the real world lens on the physical camera with settings of a virtual camera within the Unreal environment to simulate photorealistic imagery. We are experimenting with nDisplay for output to the wall and creating a config file for this purpose, and creating a frustrum to reduce the load on Unreal for rendering parts of the world that are not visible to the camera.

Can you talk about interaction of real light and shadow? For instance, specific changes in the approach to lighting with an LED wall as an environmental/practical and potentially dynamic source.

We’re very interested in experimenting with UE’s [Unreal Engine's] ability to create lighting from the wall, as well as experimenting with UE’s ability to dynamically change lighting within a scene. We are also interested in connecting to Unreal’s ability to control additional lights via a DMX interface.

Students working at the LED wall.

What new curriculum are you building around this technology?

We started to develop new curriculum in January, just as the wall was being installed, by holding a four-week intensive Virtual Production Workshop. Students in the course learned Unreal Engine (many for the first time) and built environments for a proposed music video. Then the music video was shot last week, using those environments.

Looking ahead to Fall semester, we plan to offer a new course called Unreal Engine for Filmmakers and to create additional courses to develop specific skills needed for Virtual Production. We envision building out a Virtual Production curriculum that will bring together filmmaking students, including cinematographers, production designers, producers, directors, and other disciplines, to collaborate with Animation and Visual Effects students, as well as Computer Science and Engineering students. As educators, we’re excited that Virtual Production offers a unique opportunity to develop a new interdisciplinary area of study.

What are the challenges in pursuing cutting-edge workflows in a higher education institution?

One challenge is a lack of funds. Another challenge is working with new software whose documentation is often insufficient or outdated, so it is challenging to successfully use the software. And, new updates occur so rapidly that it can be difficult to keep up with them.

Students learning Unreal Engine at Chapman University.

What can Chapman achieve through virtual production techniques that it couldn’t before?

By creating virtual environments, we can shoot on almost any location that in reality would have been unavailable, too expensive, or too distant. We can also supplement location shooting by shooting inserts or closeups on the stage. Virtual production facilitates easier integration of animation with live action. We can provide quicker feedback on how a scene will look. Less set building, less company moves, and faster set changes are all advantages. Virtual production will fuse together post-production with pre-production and production. And it will foster more interdisciplinary collaboration between students and faculty in the departments of Film Production, Animation and VFX, and Computer Science and Engineering.

How will this workflow evolve as the next generation of filmmakers become familiar and skilled with the technology?

It will become faster, easier, and will allow greater production value for modestly budgeted productions. Together with greenscreen it may democratize filmmaking with tools that were previously available only on a big budget.

Responses have been lightly edited for space and clarity. Visit AbelCine's Virtual Production page to learn more about the technology and our solutions, and subscribe to Noah Kadner's Virtual Production Podcast.