So a major proportion of the Mandalorian was shot on a virtual set. Whats different is the technological leap forward thats come from throwing together a stage surrounded by high resolution screens, a game engine and motion sensing attached to the movie camera.
Whats been achieved is essentially the holodeck. The secret sauce is the Unreal game engine. Virtually any place indoor or outdoor can be displayed. At any time of the day. That scene can also be dynamically adjusted or changed to suit whatever’s required.
The displayed background reacts to camera movement in real time based on the position sensing rig connected to it. This simulates the relative view as if the camera were actually on site.
This is being called “real pixels”. The actors can see and react to the final set rather than having to imagine it as they would with a green screen set.
Perhaps Terence Stamp could be persuaded to do a block buster now that actors’ don’t need to work with tennis balls on sticks.
Theres less work in getting the final takes because most all of it is shot in camera. The LCD screens are bright enough to light the set and the details reflect realistically. Something that needs heavy post production with green screens.
Whats more remarkable is that the screen resolution is enough to register bokeh and depth of field. Although at times, the technology wasn’t trusted enough to deliver true fidelity and they leaned on shallow depth of field to mask any perceived imperfections.
In essence, within the size of the set. Any scene can be shot in a controlled environment that can be reproduced. Crews no longer have to wait for golden hour. Golden hour is now twenty four by seven.
Back projected scenes are perfectly reproducible. The same on the first take and the last.
The trade off for this flexibility is that the scene designs need to be close to final. In order that they can be projected and shot with.
Production technology has slowly been building to this. Gravity used an early small scale version of this idea. Mainly focused on getting the required footage of actors responding to scenes.
This was dubbed the “light box”. A large enough LCD screen filled box that encompassed an actor and camera crew. Strong enough to light the scene and to simulate what was happening for the actors to respond to.
Gravity was heavily computer generated and composited to achieve the deep space visuals that Alfonso Cuaron wanted. It was so good it took the Oscar for best visual FX.
Large scale projection has been used in many movies to give scenes that extra visual impact. With new projectors they’re reaching a scale where they can fill huge sound stages.
In Oblivion, the Sky Tower is a floating base situated above the clouds.
Andrea Riseborough’s Victoria and Tom Cruise’s Jack have many discussions at all times of day on this base. Whats striking is the views from the floor to ceiling windows.
These wrap around views were achieved by projecting pre-shot visuals from a Hawaiian mountain top. This could have easily been achieved with green or blue screen but the colour spill would have required huge amounts of work to remove.
Similarly the recent Star Wars films made use of the same technique for the Millennium Falcon cockpit scenes. The Falcon cockpit set had a wrap around projected screen. This lit the set and allowed the actors to see what was actually happening.
Even Parasite, a modern day drama set in South Korean society has significant amounts of VFX.
Bong Joon Ho leans heavily on VFX to create the visual language in many of his movies. Where Snow Piercer relies on the back to front of a train to signify social status, in Parasite he uses height. Both in terms of camera movement and character perspective / points of view.
I was amazed to learn that both houses don’t actually exist. They’re sets designed to facilitate the visual metaphors the director wanted.
Parasite could be filmed without the effects and be just as affecting. Which is a tribute to the script.
The effects are over all more subtle. But effective nonetheless. Allowing the director more scope to express himself.
Even for John Wick which bills itself on Keanu Reeve’s extraordinary stunt work and physicality. Parabellum makes use of VFX to give us extraordinary action scenes.
Arguably theres more VFX in Parabellum than in the Matrix. But I can’t argue that it doesn’t have a place in making a great movie.
We get to have outlandish knife fight scenes where no one actually gets stabbed.
Visual effects are at this point an indispensable tool. It lets film makers achieve shots that otherwise wouldn’t be available. I don’t think we’d see anything as epic as Avengers Endgame if we didn’t.
Whats new is that visual effects are now reaching a point where it can be shot in camera. There’ll be much less need for “We’ll fix it in post”.