What just happened? After turning the streaming market upside down, Netflix is now seemingly ready to bring yet another revolution in how video content is made. The company has developed a novel technique to bring chroma key compositing to quasi-real-time production scenarios.
Commonly known as green screen or blue screen filming, chroma key compositing is a visual-effects technique used to merge two or more video layers based on color hues. Actors in the foreground are filmed against a uniform background (usually green), which is later digitally replaced with other content in the post-production phase.
Green-screen filming is a standard production technique in the film industry and can also be applied automatically in real-time scenarios, such as weather forecasting or other TV productions. However, when used automatically, chroma keying doesn't provide the same level of accuracy as it does in film production. Therefore, the digitally replaced background may exhibit some artifacts, particularly when interacting with green clothing, transparent objects, or fine details like strands of hair.
Magenta Green Screen (MGS), the new chroma key technology developed by Netflix researchers, can seemingly provide an answer to all the issues of modern green screen techniques in both live and post-production applications. The new tech is designed to record the color image of a foreground actor and a "simultaneous high-quality alpha channel" in the background – without requiring a special camera or manual keying techniques.
In the new MGS technique, actors in the foreground are filmed against a backdrop of bright green LEDs, while being illuminated with red and blue LED sources. This setup creates a magenta glow against a green background, which can be easily and effectively replaced in real-time to display various video scenes. MGS can provide a realistic chroma-keying effect, even in challenging areas such as transparent bottles and hair.
Like many other recent technological innovations, MGS is equipped with machine learning-based, "intelligent" algorithms used to convert the magenta-tinted foreground actors into full-color objects. The AI utilizes previously shot photos of the actors in their natural colors as a reference. Notably, it performs its tasks efficiently, even though it does not operate in fully real-time scenarios.
Video specialists have shown interest in Netflix's new video technique, even though some are highlighting how MGS still involves a lot of complex steps which could not be that useful in big budget productions. Video producer Drew Lahat said that, from a practical perspective, the tech could work well "in a fully controlled space" but it would have to compete with "other new techniques like virtual production stages" in real-world applications.