Netflix's VOID: AI That Understands Physics in Video

Netflix has released its first open-source AI model for video editing, named VOID (Video Object Inpainting & Dynamics). Unlike previous tools that simply fill in missing areas of a video frame, VOID demonstrates a sophisticated understanding of physics and causality within a scene. This means it doesn’t just paint over removed objects; it calculates the physical interactions that remain and ensures realism.

For instance, if a person holding a guitar is removed from a video, VOID will intelligently make the guitar fall realistically, adhering to gravity and scene dynamics. It recalculates shadows, reflections, and the movement of nearby objects affected by the removed element, creating a much more seamless and believable result. This advanced capability sets VOID apart from simpler inpainting tools like Runway or ProPainter.

The model is released under the permissive Apache 2.0 license, allowing for commercial use. However, it comes with substantial hardware requirements, needing a graphics card with 40GB of VRAM or more for inference. Developed in collaboration with INSAIT, VOID is available on Hugging Face for those eager to explore its capabilities.