On September 15, Shengshu Technology released Vidu S2, a real-time video model, and opened it for online trials. The new version includes two models, Vidu S2-Avatar and Vidu S2-Editing, targeting real-time digital character interaction and real-time video stream editing respectively.

Where earlier models mostly generated a finished video from a prompt, Vidu S2 is built around processing a continuous video stream. Users can feed in new instructions and reference images while a video is being generated or played back, and the model carries on generating or revising the frames that follow.

Real-Time Digital Character Output Raised to 720P

Vidu S2-Avatar targets real-time interaction with digital characters. After uploading a photo of a person, users can talk to the generated character by voice and ask it to turn around, dance, pick up objects and perform other actions.

Compared with the previous generation, Vidu S1, S2-Avatar raises real-time output resolution from 540P to 720P and follows instructions involving large body movements more reliably.

The new version also lets users add reference images during an interaction. Send a picture of an outfit or an object, for instance, and the digital character can change into that clothing or pick up and show the specified item. The model starts adjusting subsequent frames from the moment the reference image arrives, while trying to keep the character's movement, lighting and on-screen content continuous.

Real-Time Wardrobe, Subject and Background Swaps in Video Streams

Vidu S2-Editing works on continuously incoming video streams, editing content in real time as it plays or is being filmed.

The features announced so far cover changing a person's clothing, replacing the subject, swapping the background and converting visual style. Rather than processing frame by frame or rendering a complete finished cut, as conventional video editing tools do, S2-Editing aims to keep modifying upcoming video content according to user instructions.

Potential uses include live streaming, virtual character performances and interactive content production. During a livestream, for example, the system can revise subsequent frames once a new clothing or scene request comes in; virtual characters can also switch appearance, background or visual style on viewer instruction.

In hands-on testing by QbitAI, the model held up fairly well on movement continuity, subject consistency and clothing deformation, though some frames still showed smeared color blocks and clipping between hair and hats.

Exploring Spatial Video for VR Headsets

Beyond real-time generation and editing, Shengshu Technology also showed off a spatial video capability.

Spatial video creates stereoscopic depth by presenting different views to the left and right eye, and generally needs to be viewed on a VR or mixed reality device that supports spatial computing. On ordinary displays, the same content still appears as flat two-dimensional video.

Built on S2-Avatar, the system can convert a digital character generated in real time into left- and right-eye video; built on S2-Editing, it can first adjust the people, clothing or background in a real-world video stream and then convert the result into spatial video.

Shengshu Technology said the spatial video feature is still exploratory. VR headsets are demanding on both resolution and response latency, and insufficient sharpness or lag can degrade the viewing experience. The team plans to move from fixed viewpoints to panoramic spatial video that viewers can look around freely.

Two-Stage Architecture Balances Speed and Image Quality

To reconcile generation speed with image quality, Vidu S2 uses a two-stage architecture consisting of a Backbone and a Refiner.

The Backbone first generates the main subject, motion and semantic content at lower resolution, establishing which elements appear in the scene and what the character is doing. The Refiner then fills in details such as the face, clothing and background, lifting the frame to 720P.

The two stages run as an asynchronous pipeline. While the Refiner works on details for the current frame, the Backbone can keep generating what comes next, cutting idle time between modules.

Vidu S2 also uses timestamps to align reference images with the video sequence. When a user adds a new reference image mid-interaction, the model can identify the point at which the new condition takes effect and apply it to subsequent frames.

Replay Training Curbs Error Accumulation in Longer Videos

Streaming video generation has to keep producing the next segment based on what came before. If early frames drift, the error can compound through everything generated afterwards.

To address this, Vidu S2 introduces a training method called Self-Replay Forcing. The model first generates a continuous stretch of video, then the output is chunked, noised and put through causal replay training, so that training feedback from later segments can shape the representation of earlier ones.

The mechanism is mainly meant to strengthen the model's ability to correct its own generation errors, keeping motion and character state consistent across longer continuous videos.

The team also reworked how training data is annotated around continuous interaction. Annotations follow the order in which events occur, recording when an action begins, what changes it produces and the state after it ends, so the model learns continuous processes such as "picking up an object," "continuing to hold it" and "continuing to speak."

A VLM Agent Tracks On-Screen State

Vidu S2 also adds a VLM Agent, a vision-language model agent that continuously analyzes frames already generated.

When a user issues a string of different requests, the VLM Agent works out what the current frame contains, what the character is doing and whether the previous instruction has been completed, then plans the next stage of generation and picks the moment to adjust generation conditions.

During training, the team also applied preference optimization methods such as DPO to improve image quality, motion naturalness, facial expression and audio-visual sync, and used reward optimization designed for streaming generation to strengthen instruction following.

Vidu S2 extends video models beyond one-shot generation of finished clips into real-time interaction and continuous editing. Its practical value will depend on latency, hardware cost, stability over long sessions and image quality in real production settings.