When a director glances at the on-set monitor during a virtual production shoot, the foreground actor and the background environment have to feel like one image. Not “close enough” — one image. Skin tones, shadows, highlights, the gentle tint of a window-lit wall in a digital set: they all need to sit together as if a single lens captured them.
That’s a colour problem. A real-time, multi-stage colour problem.
We get asked about this a lot. People assume the hard part of green screen VP is the key — pulling a clean edge against the green. The key matters, of course, but it’s only one step. The harder problem, and the one that quietly makes or breaks a final image, is matching the colour of the foreground plate to the colour of the rendered background. This post walks through how we approach that at Vedri North Wales, from the sensor on the Blackmagic Pyxis 6K through Unreal Engine and into our on-set composite.
Start at the Sensor
A colour pipeline begins with the camera. The Pyxis 6K records Blackmagic RAW with full sensor metadata, including the native gamut and the gamma curve chosen at acquisition. We almost always shoot in Blackmagic Wide Gamut Gen 5 with Blackmagic Film Gen 5 gamma — it gives us the widest container to work with and clean roll-off in highlights, which matters when the background has bright virtual sky elements that need to feel diffused, not clipped.
Before we even shoot, we run a colour chart pass at the lighting level the production will use. We hold an X-Rite ColorChecker Video into the key light and record it for ten seconds. That gives us a reference for grading later, and just as importantly, it gives us a reference patch we can use to confirm the live composite isn’t lying to us during the day.
The On-Set Composite Path
This is where most colour problems happen, because the live composite isn’t just a preview — it’s what the director and DP are basing performance and lighting decisions on. If the comp lies, the day goes wrong.
Our pipeline sends an SDI feed from the Pyxis into our compositor and a separate SDI feed back from the compositor to a director’s monitor. The Mo-Sys StarTracker Max feeds positional data into our render PC over a 10Gb link to Unreal Engine, which renders the virtual environment with M-Path driving the virtual camera. Unreal outputs the background frame, the compositor pulls the key, and out goes a single frame to monitor — all in under two frames of latency.
The colour decisions sit inside that loop. We use OpenColorIO (OCIO) on both the Unreal side and the compositor side so that every transformation is logged, reversible, and matched to a shared config. The Pyxis footage comes in as Blackmagic Wide Gamut, gets transformed to ACEScg for the comp, and Unreal renders directly into ACEScg via its OCIO integration. Final output goes through a Rec.709 view transform for the on-set monitor.
The benefit of doing this in ACEScg rather than sRGB or some bespoke working space is that lighting in Unreal behaves physically. A 100,000-lux virtual sun gives the same exposure offsets you’d expect on a real exterior. The comp doesn’t have to fight against arbitrary white points.
Why Linear Maths Saves Hours
Most colour mistakes we’ve seen on smaller VP shoots come from doing comp maths in display-referred space. If your key, your fill, and your screen correction are all happening on gamma-corrected pixels, edges go grey, motion blur smears, and additive elements like virtual practicals look painted-on.
We do everything composite-side in linear floating point. The key is generated in linear, the screen correction subtracts a linear value, and the final blend happens linear before the view transform converts everything for the monitor. It sounds pedantic, but it’s the difference between a comp that holds up in a wide and one that falls apart the moment the camera moves.
The other small thing that matters: our Unreal scenes are lit assuming the same ACES output transform that the comp uses. If the environment artist has been grading their level under a different view transform, the comp will never match — even though the maths is technically correct. We sync this at the start of every project so the lookdev and the on-set pipeline are speaking the same language.
Grading Twice — On Set and in Post
We deliver two grades to clients. The on-set grade is a fast pass: a CDL applied through the compositor’s view transform, tuned during prep so the director can see something cinematic on the monitor without us slowing the day down. It’s not the final look, but it’s emotionally honest — the contrast, warmth, and saturation feel right.
The post grade happens later in DaVinci Resolve with the recorded plates, the matching Unreal renders at higher quality, and the CDL travelling through as a starting point. Because we’ve kept everything in ACEScg and logged every OCIO transformation, the colourist gets clean access to the original sensor data and a background plate already in a matched colour space. There’s nothing baked in that can’t be undone.
This split — fast on set, full quality in post — is one of the reasons we like a real-time green screen workflow over committing to a final pixel in-camera. The decisions you want to make in the chair shouldn’t be locked into a render that happened on set.
What We’re Watching
We’re paying close attention to a few things that change how this pipeline runs over the next year. Unreal Engine’s continued OCIO improvements mean fewer custom configs and more out-of-the-box compatibility with whatever a post house uses. The maturing ACES 2.0 output transforms promise better behaviour for highly saturated lights — useful for music videos and product shoots where the virtual environment leans into colour. And Blackmagic’s tighter integration of Resolve’s grading tools with on-set workflows is making the on-set-to-post handoff smoother every release.
None of this changes the basic discipline, though: shoot wide, work in linear, transform once at the end, and keep the maths honest.
If you’re planning a shoot and want to talk through how your assets, plates, and grade will travel through our pipeline, we’re happy to walk through it in person. Book a studio visit and we’ll show you the colour pipeline running live.
