AI product Open source
JoyAI-Video-Edit is an open-source, instruction-guided system for editing live camera streams or uploaded videos as frames arrive. It processes frames causally without waiting for the complete video, requiring a predefined sequence length, or revisiting future frames, and supports subject and local edits, background replacement, style and motion changes, and reference-guided editing.
Its autoregressive diffusion architecture combines an MLLM-based condition encoder, a causal video VAE, and a 16-billion-parameter multimodal diffusion transformer. The deployment uses aligned autoregressive distribution-matching distillation, long-horizon optimization, bounded KV-state inference, and deployment-oriented scheduling to reduce train–inference mismatch and temporal drift during streaming generation. The repository reports 30 FPS end-to-end throughput at 720 × 1248 in its deployment benchmark.
The repository provides deployment code and model checkpoints, a local server with a browser interface, and instructions for CUDA-based inference. It is licensed under Apache 2.0.
1 use taken from transcripts — each links to the moment in the video.
Edits video footage while it is still being recorded by processing incoming frames causally. It supports background swaps, subject changes, and style transfer using a 16-billion-parameter diffusion transformer.
1 in the library.