JoyAI-Video-Edit

JoyAI-Video-Edit is an open-source, instruction-guided system for editing live camera streams or uploaded videos as frames arrive. It processes frames causally without waiting for the complete video, requiring a predefined sequence length, or revisiting future frames, and supports subject and local edits, background replacement, style and motion changes, and reference-guided editing.

Its autoregressive diffusion architecture combines an MLLM-based condition encoder, a causal video VAE, and a 16-billion-parameter multimodal diffusion transformer. The deployment uses aligned autoregressive distribution-matching distillation, long-horizon optimization, bounded KV-state inference, and deployment-oriented scheduling to reduce train–inference mismatch and temporal drift during streaming generation. The repository reports 30 FPS end-to-end throughput at 720 × 1248 in its deployment benchmark.

The repository provides deployment code and model checkpoints, a local server with a browser interface, and instructions for CUDA-based inference. It is licensed under Apache 2.0.