A Visual Guide to Causal, Bidirectional, and Hybrid Inference in Video Generation
How information flow changes across video time and denoising time
“Causal” and “bidirectional” sound like labels attached permanently to a video model. One model moves left to right; another sees the whole clip. Pick an architecture, inherit its behavior.
That picture is useful, but incomplete. A generator also needs a policy for deciding which parts of a noisy video may exchange information, and when. Once we look at that policy, causal and bidirectional generation become endpoints of a larger scheduling problem.
I want to build that idea with four chunks of an imaginary video. No benchmark is hiding in the diagrams. They show information availability, not speed or quality measurements.
Start with what each position is allowed to know
Suppose we split a video into four consecutive chunks: A, B, C, and D. A chunk is simply a group of neighboring video positions. It might contain one frame, several frames, or latent video tokens rather than pixels.
In causal generation, a position may use an available prefix: the current chunk and earlier chunks, but not later chunks that have not been generated. When producing C, information from A and B can flow forward; D cannot flow backward.
In bidirectional generation, positions across the clip may condition on one another while the clip is being refined. A rough state in A can influence D, and D can influence A. “Bidirectional” describes information flow, not playback direction. The finished video still plays from beginning to end.
The tempting model is therefore:
- causal means
A → B → C → D; - bidirectional means
{A, B, C, D}are refined together.
Nothing is wrong with those endpoints. The mistake is assuming there can be no useful schedule between them.
There are two kinds of time on the page
Diffusion adds a second clock. Video time tells us whether a chunk appears early or late in the clip. Denoising time tells us how far a noisy representation has been refined toward a clean one.
The grid below keeps these clocks separate. Columns run through the video. Rows move from a rough, noisy state toward a ready state. A row is not another moment in the story; it is another stage in the computation.
earliest
latest
rough partly refined ready
This separation matters because a schedule can change along either axis. It can group more or fewer neighboring video positions, and it can change those groups as denoising progresses.
Pause and predict: must D be completely unknown until A is ready? A strictly causal schedule says yes. A hybrid schedule does not have to.
The rigid endpoints
The tabs show two idealized reveal orders. Step through them from left to right. “Available” means the scheduler allows that region to participate; it does not mean the model has rendered a viewable frame at that instant.
All four chunks enter the computation together and can help establish a global plan. The price of this simple picture is that the full working set must be handled together; it also does not naturally emit a completed prefix while later chunks remain absent.
A is completed without future chunks. This makes a finished prefix possible, but A cannot use evidence that will appear only in C or D.
B may use A as context. C and D still cannot help. Repeating the pattern eventually completes the clip from left to right.
These diagrams deliberately exaggerate the contrast. Real systems differ in caching, attention masks, chunk size, and what counts as “ready.” The grid gives us a vocabulary for asking about those details instead of treating the labels as complete specifications.
A schedule can live between the endpoints
Flex-Forcing makes chunking flexible over both the temporal dimension and denoising steps. The same video diffusion model can operate under bidirectional, autoregressive, and hybrid generation regimes. Its project page describes larger chunks as closer to global bidirectional planning and smaller chunks as closer to autoregressive streaming.
Here is a conceptual four-chunk hybrid. It is an explanatory sketch, not the paper’s exact tensor layout or a reproduced experimental configuration.
All chunks receive a rough global context. Later positions are not finished, but they are not entirely absent either. This is the bidirectional part of the mental model.
A can now be completed while the rough plan for B–D remains available. Fine synthesis moves forward even though coarse planning did not.
B uses the completed prefix and the wider plan. Call this “autoregressive synthesis within the plan”: left-to-right completion—not that every internal operation must be strictly causal.
The result is still an ordinary video. What changed was the reveal order of intermediate information.
The paper’s claim is broader than this toy schedule. Flex-Forcing defines flexible chunking jointly over video positions and denoising steps, supports chunk sizes chosen for different device budgets, and permits any-order, any-timestep autoregressive generation without a strict causal constraint. The authors also report better video quality, long-video stability, and faster inference than rigid-schedule baselines across their experiments (paper abstract; project results). Those are results for the evaluated configurations, not a guarantee that every hybrid schedule improves every metric.
The project page offers one concrete example of why the schedule itself matters. For a five-second video represented by 21 latent frames, it reports that a coarse-to-fine chunk configuration [9, 6, 6] lies on its measured quality–speed frontier and outperforms uniform [7, 7, 7] and its Self-Forcing baseline on the displayed measures. The numbers describe chunk configurations in that experiment; they are not denoising-step counts or a universal recipe.
The policy changes the questions we should ask
Once generation becomes a schedule, “Is this model causal?” is too coarse. More useful questions are:
- Which video positions are grouped into a chunk?
- Can rough future positions influence the current chunk?
- Does chunk size change along the denoising trajectory?
- When is the first usable prefix available?
- Which intermediate states must remain in memory?
- Can a later chunk be edited without rebuilding the whole sequence?
Different answers can trade global planning, streaming behavior, memory use, and fine-grained control. The direction of those trade-offs is plausible from the information-flow pattern, but their size requires matched experiments on the same model, outputs, hardware, and quality target. A grid cannot produce a latency number.
Flex-Forcing also demonstrates why “inference policy” is a useful phrase rather than just a metaphor. Its project page describes editing at low-level refinement steps while keeping high-level planning fixed, and allowing a chunk to attend to both past and future tokens. The schedule determines which information can move and which parts can change.
LingBot exposes a different kind of inference policy
LingBot-Video is useful here as a contrast, not as another Flex-Forcing implementation. Its public repository and model card expose a user-facing workflow with prompt rewriting, base video generation, an optional refiner, and multiple serving paths. The documented MoE text-to-video command uses a 40-step base stage followed by an 8-step refiner stage.
That sequence is also an inference policy: it decides how a prompt is prepared and how generation is divided across two denoising stages. But it operates along different axes from Flex-Forcing’s joint temporal-and-denoising chunk schedule. 40 + 8 does not tell us whether information flows causally or bidirectionally across video chunks, and the public LingBot materials cited here do not claim Flex-Forcing’s mechanism.
The comparison therefore stops at a shared lesson: a model’s weights and architecture do not completely describe the behavior a user receives. Scheduling, refinement, conditioning, and serving choices are part of the generation path. The two systems expose different choices, so their counters cannot be placed on one speed scale without a matched evaluation.
A compact way to explain it back
Try drawing a grid with video position on the horizontal axis and denoising progress on the vertical axis.
A rigid bidirectional schedule refines a large region together. A rigid causal schedule completes regions from left to right, leaving future regions unavailable. A flexible schedule can make rough information available across a wide region, then complete finer regions in an autoregressive order. Flex-Forcing turns that two-axis chunking pattern into a training and inference framework for one model.
This mental model establishes how causal and bidirectional behavior can become scheduling choices. It does not establish the best schedule for a deployment, the memory required by an implementation, or a speed comparison with LingBot. Those answers need released implementations or matched measurements.
The next time a video model is called “causal” or “bidirectional,” the useful follow-up is not “which one is better?” It is: what information becomes available at each point in video time and denoising time?