The tempting model: a lookup table

The way most people hold einsum in their head is as a table of memorized entries:

• 'ij,jk->ik' is matrix multiply
• 'ij->ji' is transpose
• 'ii->' is trace
• 'i,i->' is dot product

The table has no entry for 'bhqd,bhkd->bhqk', and adding more entries doesn’t help, because the strings are not arbitrary codes. Each one is a compressed equation, and the compression scheme is a physics convention from 1916 called Einstein summation. Learn the scheme once and every string, including ones you have never seen, becomes legible.

Step 1: a dot product, and one rule

Start with the smallest thing that involves a sum. The dot product of two vectors:

The index is doing the walking: it visits every slot, multiplies the paired entries, and the sigma adds them up.

Einstein’s convention is a single observation about equations like this: whenever an index appears exactly twice in a term, it is always being summed over, so the sigma carries no information.

That is the entire convention. An index written twice means “sum over it, silently.” Everything that follows is this rule meeting larger objects.

Step 2: dummy indices and free indices

The dropped sigma creates two kinds of index, and telling them apart is the entire skill. Both have exact counterparts in code you already write, so let’s build the distinction there.

A dummy index is a loop variable inside a sum. Here is as the loop it abbreviates:

import numpy as np

v = np.array([1.0, 2.0, 3.0])
w = np.array([4.0, 5.0, 6.0])

total = 0.0
for j in range(3):          # j lives only inside this loop
    total += v[j] * w[j]
print(total)                 # the answer has no j in it

32.0

Everything you know about the variable j transfers to the index . It exists only inside the computation. It visits every value. And once the loop finishes, it is gone: the answer is 32.0, not something indexed by . Its name is irrelevant too. Rename j to k and it is the same computation, which is why and denote the same number. An index that appears twice is a loop variable, and spotting one should trigger the same reflex: this will be summed away and won’t exist in the output.

A free index is the opposite: it survives into the output and labels its slots. In code, it is the variable that indexes the result you are building. Take matrix-vector multiplication, :

A_small = np.arange(9.0).reshape(3, 3)

u = [sum(A_small[i, j] * v[j] for j in range(3))   # j: loop variable, consumed
     for i in range(3)]                             # i: labels the output slot
print(u)

[np.float64(8.0), np.float64(26.0), np.float64(44.0)]

Two nested roles, visually separated. The inner j is a dummy again, consumed by sum. The outer i is free: the output is a list indexed by i, three numbers, one per value it takes. A free index appears exactly once in each term of the equation, and the number of free indices tells you the output’s number of dimensions before you compute anything. Zero free indices, scalar. One, vector. Two, matrix.

One consistency law binds every valid equation: the free indices on the left must exactly match the free indices on the right. In loop terms: both sides must be built by the same outer for loops. A dummy on the right is fine, it is consumed by its own sum. A free index on only one side is like writing u[i] = ... where the right side loops over m instead: the program is malformed before it runs. This law catches most index mistakes before any arithmetic happens.

The analogy has one limit worth naming. A literal loop nest fixes an execution order, but the einsum equation does not: the sums can be regrouped and reordered freely, which is exactly the freedom optimizers exploit later. The loop picture is for reading the equation, not for how the machine runs it.

From here on, whenever an equation appears, I’ll mention the roles the same way: which letter is the loop variable, which letters shape the output.

Step 3: the strings are the equations

np.einsum takes the equation you would write on paper and asks you to type it, letters only:

• input tensors are described by their indices, comma-separated
• the arrow shows the free indices that survive into the output
• any index missing after the arrow is a dummy: summed

The dot product over two vectors and has no free index, so nothing follows the arrow:

print(np.einsum('j,j->', v, w))

32.0

Now flip one decision. Keep both indices free instead of matching them, , and no sum happens at all. Every component of meets every component of :

print(np.einsum('i,j->ij', v, w))

[[ 4.  5.  6.]
 [ 8. 10. 12.]
 [12. 15. 18.]]

This is the outer product, a 3×3 grid, and the string told you its shape before running anything: two free indices in, two dimensions out. Compare the two strings. 'j,j->' and 'i,j->ij' differ only in whether the letters match, and that single choice is the difference between collapsing to a scalar and expanding to a matrix.

Two more one-liners fall out. Transposing is just reordering free indices, no sum anywhere:

M = np.einsum('i,j->ij', v, w)
print(M.shape)

(3, 3)

And repeating an index within a single tensor walks its diagonal. means sum the entries where row equals column, the trace:

print(np.einsum('ii->', M))

32.0

Step 4: a matrix acts on a vector

A matrix-vector product, written per component, is a dot product between row and the vector:

This is exactly the loop nest from Step 2, now in notation: is the loop variable, consumed by its sum; names the output slot. One free index, so the output is a vector. The string is a direct transcription:

A_demo = np.arange(9.0).reshape(3, 3)
dotted = np.einsum('ij,j->i', A_demo, v)
print(dotted, dotted.shape)

[ 8. 26. 44.] (3,)

Matrix-matrix multiplication is the same sentence with one more free index along for the ride, : contract ’s columns against ’s rows over the shared , keep and to shape the result.

B_demo = np.ones((3, 3))
matmul = np.einsum('ij,jk->ik', A_demo, B_demo)
print(matmul, matmul.shape)

[[ 3.  3.  3.]
 [12. 12. 12.]
 [21. 21. 21.]] (3, 3)

Step 5: where a real matrix comes from

So far every matrix has been handed to us. The notation gets genuinely powerful when an equation produces a matrix you didn’t know was there. Here is the example that made this click for me: lets sayP> sunlight with direction casts a point onto the ground (the plane where the third coordinate is zero). Sliding along the ray until it hits the ground gives the shadow point

where picks out height and is the sun direction’s third component. The right side is linear in , so a matrix is hiding in it. Index notation digs it out in three moves, and to keep each move honest we’ll drag a dummy input through all of them: the sun at , so , and a point at , three units above the ground.

s = np.array([4/13, -3/13, -12/13])
r = np.array([6.0, 2.0, 3.0])
e3 = np.array([0.0, 0.0, 1.0])

Move 1: components. Write the -th component of each side, with the dot product in convention form:

Roles: is the loop variable, shapes the output, present once in every term on both sides. Well-formed.

Now watch actually loop and vanish. The piece expands to i.e. a dot product between the two vectors, the basis vector and the point , which is exactly the height of the point. Check it:

print(sum(e3[j] * r[j] for j in range(3)))

3.0

It collapses to , the height of the point, with no anywhere in the answer. The equation for the first component, , is then plain arithmetic: .

print(r[0] - s[0] * 3 / s[2])

7.0

So the first shadow coordinate is before any matrix exists. The remaining moves don’t change this number; they reorganize the computation so a matrix becomes visible.

Move 2: rewrite both terms using the identity matrix. Here is where we left off:

The goal of this move is simple: make both terms look alike, so the next move can factor them. Two small rewrites do it.

First, is the list exactly row 3 of the identity matrix. So we can write it as :

I = np.eye(3)
print(I[2])

[0. 0. 1.]

Second, the bare can be written as . Row of the identity has a single 1 in slot , so dotting it with just returns component . Check for : row against gives same for the remaining values meaning is valid for all :

print(sum(I[1, j] * r[j] for j in range(3)))

2.0

Why dress up a term that was already simple? Because factoring needs both terms to end in the same , and alone has no to offer.

Move 3: factor out . With both rewrites in place, every term ends in , and it pulls out:

The bracket has two free indices, and so it is a matrix. This is the matrix-vector pattern from Step 4, with

Check with the dummy input: row of the bracket is , and dotting it with should reproduce the from Move 1:

A = I - np.outer(s, I[2]) / s[2]   # A_ij = I_ij - s_i I_3j / s3
print(A)
print(sum(A[0, j] * r[j] for j in range(3)))

[[ 1.          0.          0.33333333]
 [ 0.          1.         -0.25      ]
 [ 0.          0.          0.        ]]
7.0

Same number, now produced by a matrix row. Note np.outer(s, I[2]) is exactly the term: two free indices, no shared letter the outer product from Step 3 in the wild.

The matrix itself is readable. is zero unless , so only column 3 differs from the identity: all the shadow action lives in one column. And at the two terms cancel, so row 3 is all zeros: every output has height zero the 3D-to-2D collapse written as a dead row.

Applying the full matrix finishes the example:

print(A @ r)

[7.   1.25 0.  ]

The shadow lands at : shifted because the sun direction has positive , shifted down in because it has negative , and flat on the ground.

One bonus check. A shadow of a shadow is itself, so :

print(np.allclose(A @ A, A))

True

Maps with this property are called projections, and this one is oblique: unless the sun is directly overhead, points travel diagonally to the plane rather than dropping straight down.

Step 6: the batch dimension appears

Now shadow four points at once. Stack them as columns of a matrix and write the transformation of the whole batch:

Read the roles. : appears twice, the loop variable, doing the actual contraction. : free, output component. : free, and it means which vector you are on. The crucial observation is what does not do. It never enters the sum. Nothing in the equation lets column 3 influence column 2. Batching works not by accident but because the batch index is free on both sides, and the notation makes that independence visible rather than asking you to trust it.

R = np.array([[5, -1, -3,  7],
              [4, -4,  1, -2],
              [9,  3,  0, 12]], dtype=float)

R_batch = np.einsum('ij,ja->ia', A, R)
print(R_batch)

[[ 8.   -0.   -3.   11.  ]
 [ 1.75 -4.75  1.   -5.  ]
 [ 0.    0.    0.    0.  ]]

The second check confirms it: the einsum equals four independent matrix-vector products laid side by side. This is the same mechanism by which a batch dimension in a deep learning framework broadcasts through a matmul without mixing examples.

Before scaling up further, a checkpoint. Write your answer before expanding.

print(np.einsum('ij,jk,ka->ia', A, A, R))

[[ 8.   -0.   -3.   11.  ]
 [ 1.75 -4.75  1.   -5.  ]
 [ 0.    0.    0.    0.  ]]

Step 7: reading the attention string

The opening line can now be read as a sentence:

attn = torch.einsum('bhqd,bhkd->bhqk', q, k)

Hunt for the loop variable first: which letter is missing after the arrow? Only , the feature dimension. So is the one that loops and vanishes: queries and keys are contracted along features, a dot product per pair, exactly the 'j,j->' from Step 1 wearing more clothes.

Then account for the free indices. (batch) and (head) appear in both inputs and the output in matching positions: they ride along untouched, exactly like did in Step 6, two batch dimensions instead of one. comes only from the first tensor and only from the second, and both survive: every query meets every key, the outer-product pattern from Step 3. The output is a grid of similarity scores for each batch and head.

rng = np.random.default_rng(0)
q = rng.standard_normal((2, 3, 4, 8))   # batch=2, heads=3, queries=4, features=8
k = rng.standard_normal((2, 3, 5, 8))   # batch=2, heads=3, keys=5, features=8

scores = np.einsum('bhqd,bhkd->bhqk', q, k)
print(scores.shape)

(2, 3, 4, 5)

The shape check alone shows the grammar working: the string predicted (2, 3, 4, 5) before the computation ran, and the einsum matches the conventional transpose-and-matmul spelling while stating its intent far more directly.

Where the mental model stops

First, reading an einsum tells you what is computed, not how fast. With three or more tensors, as in the checkpoint above, there is a choice of which pair to contract first, and for large tensors that choice can change the cost by orders of magnitude. NumPy’s optimize=True flag and the opt_einsum library exist to search for a good contraction order. The notation is about meaning; scheduling is a separate problem.

Second, not everything is an einsum. The convention expresses sums of products over matched indices: contractions, traces, transposes, outer products, and their batched combinations. Elementwise nonlinearities, softmax, max-pooling, anything with a comparison or a division by a computed sum, live outside it. When you see einsum in a codebase, you are looking at the multilinear skeleton of the computation, with the nonlinear flesh applied elsewhere. In the attention example, the softmax and the value-weighting that follow the score computation are separate lines for exactly this reason.

Four bits do not provide many choices

A floating-point layout is often named by how many exponent and mantissa bits it contains. E2M1 has one sign bit, two exponent bits, and one mantissa bit. Together, they make a four-bit payload, so E2M1 is one kind of FP4.

Ignoring the sign, its finite magnitudes are:

0, 0.5, 1, 1.5, 2, 3, 4, 6

The sign bit mirrors the nonzero values below zero. This is a very small codebook, meaning the set of values the bit patterns can select. NVIDIA documents this E2M1 payload as the element format used by NVFP4 (Transformer Engine documentation).

Now consider a block whose useful values cluster near 0.01, and another whose values cluster near 100. A bare E2M1 codebook cannot closely cover both neighborhoods. Its largest finite magnitude is 6, and the gaps between available values are large relative to many tensors.

Quantization solves this by treating the codebook values as numbers in a local unit rather than final tensor values. A scale defines that unit. The payload says “four units”; the scale says how large one unit is here.

The smallest example: the same payload, two real values

NVFP4 reconstructs a value with three factors:

x = x_e2m1 × s_block × s_global

Their scopes are more important than their names:

• x_e2m1 is the four-bit code stored for one element.
• s_block is a scale shared by a local block of 16 elements.
• s_global is a scale shared by the whole tensor.

Take two 16-value blocks and fix the tensor-wide scale at s_global = 0.25.

Tensor-wide scale: s_global = 0.25

Block A: s_block = 1.0       Block B: s_block = 1.5
         payload = 4                  payload = 4
              |                            |
              v                            v
      4 × 1.0 × 0.25 = 1.0        4 × 1.5 × 0.25 = 1.5

Both elements contain the same E2M1 payload. Yet one reconstructs to 1.0 and the other to 1.5, because each block uses a different local unit. The payload is not a standalone measurement; it is a code interpreted within two shared coordinate systems.

The explorer below lets you vary those three factors separately. Choose an exact E2M1 payload, then move the block scale or tensor scale. The block control changes the unit for one toy 16-value neighborhood; the tensor control changes the unit shared by every block. The blue result is the reconstructed value, not a quantization-error or model-quality measurement.

Try leaving the payload at 4 and changing only the block scale from 1.0 to 1.5. The selected four-bit value does not move, but the reconstructed result does. Changing the tensor scale then moves every local unit together. This is the division of labor that the phrase “two-level scaling” can otherwise hide.

Why two levels instead of one? The block scale adapts the codebook to a small neighborhood, where values are more likely to have a similar range. The global scale places those local scales within the range that their own storage format can represent. Transformer Engine describes the same hierarchy and derives the global scale from the tensor’s maximum absolute value before deriving local scales from block maxima (Transformer Engine documentation).

The format names can now be attached without hiding their roles:

• The element payload is E2M1, a four-bit floating-point code.
• Each group of 16 payloads shares an E4M3 scale. E4M3 is an eight-bit floating-point format with four exponent and three mantissa bits, so its scale can take fractional values between powers of two.
• The tensor shares an FP32 global scale, stored in ordinary 32-bit floating point.

This hierarchy is what two-level scaling means in NVFP4: a local scale per block, followed by a global scale per tensor. Per-block scaling adapts to variation among small groups. Per-tensor scaling provides one range-setting factor for the entire tensor. Neither phrase says that every operation or every piece of model state is stored at that precision.

“4-bit” is a payload count, not a memory measurement

Once the local scale is visible, a common storage shortcut breaks.

One block contains 16 four-bit payloads and one eight-bit local scale:

16 × 4 + 8 = 72 bits

Amortized across the block:

72 / 16 = 4.5 bits per value

So the basic payload-plus-local-scale representation is already 4.5 bits per value, not four. The tensor’s FP32 global scale adds a small amortized cost whose size depends on the number of elements in that tensor.

The next explorer makes that amortization visible. Move the slider to change the number of 16-value blocks in one toy tensor. It counts four payload bits per value, one eight-bit local scale per block, and one 32-bit global scale for the tensor. It intentionally excludes padding, alignment, duplicate layouts, high-precision components, and runtime buffers, so its result is a representation calculation rather than an application-memory forecast.

At the smallest setting, the one global scale is expensive because only 16 values share it. As the toy tensor grows, that one-time cost fades, while the local-scale cost remains fixed at 8 / 16 = 0.5 bit per value. The output approaches 4.5 bits per value but never turns the calculation into a full-memory estimate.

Even 4.5 bits is not a full runtime-memory prediction. Transformer Engine documents padded and swizzled scale layouts and a TN-only NVFP4 GEMM layout; training may also keep rowwise and columnwise representations of weights (Transformer Engine documentation). Alignment, padding, higher-precision layers, activations, optimizer state, KV cache, workspaces, and framework allocations all remain outside the small calculation.

Comparing payloads alone, four bits are one quarter of a BF16 payload and one half of an FP8 payload. Those ratios are useful for understanding the upper-level opportunity. They do not establish a fourfold reduction in total application memory.

Generic FP4, MXFP4, and NVFP4

“FP4” is not a complete deployment specification. It can identify a four-bit element format such as E2M1 without telling us how a tensor is scaled. Two systems can store the same E2M1 payload and reconstruct it differently.

MXFP4 makes the comparison concrete. MXFP4, the FP4 member of the OCP microscaling family, also uses E2M1 payloads. It shares one E8M0 scale across 32 values. E8M0 has eight exponent bits and no mantissa, so its scale is restricted to powers of two. NVFP4 instead shares an E4M3 fractional scale across 16 values and adds the FP32 scale for the tensor (NVIDIA’s NVFP4 technical overview).

The table is a comparison of representations, not model quality. NVFP4 gets two ways to fit a local distribution more closely: its scale covers half as many elements, and that scale can sit between powers of two. NVIDIA illustrates lower quantization error from these choices, but a representation-error example is not an end-to-end accuracy result (NVIDIA’s NVFP4 technical overview). Smaller blocks also mean more scale metadata and handling.

The better mental model is now precise: generic FP4 names a payload class; MXFP4 and NVFP4 specify different systems around an E2M1 payload.

A format is not a training recipe

If scaling were the whole story, low-precision training would amount to casting tensors and calling the same matrix multiplications. NVIDIA’s documented NVFP4 training path contains several additional mechanisms.

Two-dimensional weight scaling changes the grouping used for weights. Transformer Engine uses 16-by-16 tiles by default so rowwise and columnwise weight representations can support the forward and backward matrix multiplications. Activations and gradients use one-dimensional 16-value blocks.

Stochastic rounding chooses probabilistically between neighboring representable values instead of always taking the nearest one. For a gradient between two E2M1 values, the probabilities are chosen so the expected rounded value equals the original value. Transformer Engine enables this for gradient quantization. “Unbiased in expectation” does not mean an individual rounding is exact, and the added randomness has reproducibility implications.

A Random Hadamard Transform is an orthogonal mixing operation applied to selected operands before quantization. Its purpose here is to spread isolated large values across coordinates so that one outlier is less likely to set an unhelpful scale for an entire block. Transformer Engine currently documents this path for BF16 inputs and gradients used in selected weight-gradient operations (Transformer Engine documentation).

The broader recipe also retains higher precision for sensitive layers or operations. In other words, “NVFP4 training” is mixed-precision training built around NVFP4 matrix multiplications, not a claim that the whole training state and every computation use four bits.

NVIDIA reports pretraining a 12-billion-parameter model for 10 trillion tokens with loss and downstream accuracy comparable to an FP8 baseline using this combined method (NVFP4 training paper). This is evidence for the model, data, schedule, hardware, and recipe in that experiment. It is not evidence that naive E2M1 conversion works for arbitrary architectures, or that every model will match FP8.

Hardware support still needs a full sentence

NVFP4 is designed for native block-scaled matrix operations on NVIDIA Blackwell Tensor Cores. But “Blackwell supports NVFP4” leaves out which operation, device, software path, and tensor are involved.

Transformer Engine 2.16 documents NVFP4 training on compute capabilities SM100 and SM103, and inference on SM100 or later. It exposes the recipe to PyTorch and JAX through NVFP4BlockScaling, while also documenting layout and distributed-training constraints (Transformer Engine documentation). These are versioned software conditions, not permanent properties of every Blackwell deployment.

Inference has at least two separate quantization targets:

1. Weights and activations are operands used by model computations.
2. The KV cache stores attention keys and values from earlier tokens so autoregressive decoding does not recompute them.

TensorRT-LLM’s quantization matrix, inspected in the source package after its 2026-07-14 update, listed NVFP4 weights and activations on Blackwell SM100/103 and SM120. Its NVFP4 KV-cache support was narrower, limited to SM100/103, and the documented KV-cache combination required FP8 weights and activations (TensorRT-LLM quantization guide). Because this support changes with releases, the live matrix should be checked for the exact TensorRT-LLM version before deployment.

Training GEMMs, weight-and-activation inference, and KV-cache compression are therefore three different decisions. Each needs an exact compute capability, framework version, model and layout, calibration or training path, and quality test.

Where speed and accuracy claims can fail

Native Tensor Core execution creates a credible path to higher matrix throughput and lower operand bandwidth. End-to-end performance improves only if supported matrix multiplications dominate the workload.

Quantization and scale handling consume work. Attention, communication, unsupported operators, memory bandwidth, small tensor shapes, and irregular layouts may dominate instead. A faster GEMM can simply move the bottleneck elsewhere. Peak hardware throughput is not a substitute for measuring prefill latency, decode latency, generation throughput, or training step time at the intended shapes and batch sizes.

Accuracy needs a separate boundary. Finer local scaling can reduce the numerical error introduced while representing a tensor. Model quality also depends on where quantization is applied, how calibration is performed, whether training adapts to it, which computations stay at higher precision, and how errors propagate through the network. Lower error in a toy tensor does not prove equal language-model accuracy, much less equal behavior on rare or sensitive cases.

The strongest large-scale result cited above was produced by NVIDIA for a particular pretraining experiment. Independent behavior on a target model, task distribution, context length, and deployment stack remains unknown until it is measured.

A decision test for an NVFP4 claim

When a benchmark or deployment proposal says “NVFP4,” I would unpack it in this order:

1. Name the quantized object. Is it weights, activations, training GEMM operands, or the KV cache?
2. Count the representation. Include local and global scales, padding, duplicate layouts, and any higher-precision components instead of counting only E2M1 payload bits.
3. Verify the supported path. Record the compute capability, software versions, model family, tensor layout, and allowed precision combination.
4. Describe how the artifact was made. Distinguish a supported pre-quantized checkpoint, calibrated post-training quantization, and training with the full mixed-precision recipe.
5. Validate quality on the intended workload. Compare with BF16 or FP8 on held-out tasks, long contexts, outlier-heavy inputs, and rare behaviors that matter.
6. Measure the system boundary. Report checkpoint size, peak memory, KV-cache size where relevant, end-to-end latency or step time, throughput, and quality under the same conditions.

If those questions are unanswered, “4-bit” tells us the width of the element payload and little more.

NVFP4’s useful idea is not merely squeezing a floating-point value into four bits. It is giving the same tiny E2M1 codebook different local units, then putting those units into a tensor-wide range. That hierarchy can represent tensors more closely than a coarser power-of-two scheme, and Blackwell hardware can execute supported operations natively. The remaining memory, accuracy, and speed claims belong to the whole mixed-precision system and the workload running through it, not to the four payload bits alone.

Start with what each position is allowed to know

Suppose we split a video into four consecutive chunks: A, B, C, and D. A chunk is simply a group of neighboring video positions. It might contain one frame, several frames, or latent video tokens rather than pixels.

In causal generation, a position may use an available prefix: the current chunk and earlier chunks, but not later chunks that have not been generated. When producing C, information from A and B can flow forward; D cannot flow backward.

In bidirectional generation, positions across the clip may condition on one another while the clip is being refined. A rough state in A can influence D, and D can influence A. “Bidirectional” describes information flow, not playback direction. The finished video still plays from beginning to end.

The tempting model is therefore:

• causal means A → B → C → D;
• bidirectional means {A, B, C, D} are refined together.

Nothing is wrong with those endpoints. The mistake is assuming there can be no useful schedule between them.

There are two kinds of time on the page

Diffusion adds a second clock. Video time tells us whether a chunk appears early or late in the clip. Denoising time tells us how far a noisy representation has been refined toward a clean one.

The grid below keeps these clocks separate. Columns run through the video. Rows move from a rough, noisy state toward a ready state. A row is not another moment in the story; it is another stage in the computation.

This separation matters because a schedule can change along either axis. It can group more or fewer neighboring video positions, and it can change those groups as denoising progresses.

The rigid endpoints

The tabs show two idealized reveal orders. Step through them from left to right. “Available” means the scheduler allows that region to participate; it does not mean the model has rendered a viewable frame at that instant.

These diagrams deliberately exaggerate the contrast. Real systems differ in caching, attention masks, chunk size, and what counts as “ready.” The grid gives us a vocabulary for asking about those details instead of treating the labels as complete specifications.

A schedule can live between the endpoints

Flex-Forcing makes chunking flexible over both the temporal dimension and denoising steps. The same video diffusion model can operate under bidirectional, autoregressive, and hybrid generation regimes. Its project page describes larger chunks as closer to global bidirectional planning and smaller chunks as closer to autoregressive streaming.

Here is a conceptual four-chunk hybrid. It is an explanatory sketch, not the paper’s exact tensor layout or a reproduced experimental configuration.

The paper’s claim is broader than this toy schedule. Flex-Forcing defines flexible chunking jointly over video positions and denoising steps, supports chunk sizes chosen for different device budgets, and permits any-order, any-timestep autoregressive generation without a strict causal constraint. The authors also report better video quality, long-video stability, and faster inference than rigid-schedule baselines across their experiments (paper abstract; project results). Those are results for the evaluated configurations, not a guarantee that every hybrid schedule improves every metric.

The project page offers one concrete example of why the schedule itself matters. For a five-second video represented by 21 latent frames, it reports that a coarse-to-fine chunk configuration [9, 6, 6] lies on its measured quality–speed frontier and outperforms uniform [7, 7, 7] and its Self-Forcing baseline on the displayed measures. The numbers describe chunk configurations in that experiment; they are not denoising-step counts or a universal recipe.

The policy changes the questions we should ask

Once generation becomes a schedule, “Is this model causal?” is too coarse. More useful questions are:

• Which video positions are grouped into a chunk?
• Can rough future positions influence the current chunk?
• Does chunk size change along the denoising trajectory?
• When is the first usable prefix available?
• Which intermediate states must remain in memory?
• Can a later chunk be edited without rebuilding the whole sequence?

Different answers can trade global planning, streaming behavior, memory use, and fine-grained control. The direction of those trade-offs is plausible from the information-flow pattern, but their size requires matched experiments on the same model, outputs, hardware, and quality target. A grid cannot produce a latency number.

Flex-Forcing also demonstrates why “inference policy” is a useful phrase rather than just a metaphor. Its project page describes editing at low-level refinement steps while keeping high-level planning fixed, and allowing a chunk to attend to both past and future tokens. The schedule determines which information can move and which parts can change.

LingBot exposes a different kind of inference policy

LingBot-Video is useful here as a contrast, not as another Flex-Forcing implementation. Its public repository and model card expose a user-facing workflow with prompt rewriting, base video generation, an optional refiner, and multiple serving paths. The documented MoE text-to-video command uses a 40-step base stage followed by an 8-step refiner stage.

That sequence is also an inference policy: it decides how a prompt is prepared and how generation is divided across two denoising stages. But it operates along different axes from Flex-Forcing’s joint temporal-and-denoising chunk schedule. 40 + 8 does not tell us whether information flows causally or bidirectionally across video chunks, and the public LingBot materials cited here do not claim Flex-Forcing’s mechanism.

The comparison therefore stops at a shared lesson: a model’s weights and architecture do not completely describe the behavior a user receives. Scheduling, refinement, conditioning, and serving choices are part of the generation path. The two systems expose different choices, so their counters cannot be placed on one speed scale without a matched evaluation.

A compact way to explain it back

Try drawing a grid with video position on the horizontal axis and denoising progress on the vertical axis.

A rigid bidirectional schedule refines a large region together. A rigid causal schedule completes regions from left to right, leaving future regions unavailable. A flexible schedule can make rough information available across a wide region, then complete finer regions in an autoregressive order. Flex-Forcing turns that two-axis chunking pattern into a training and inference framework for one model.

This mental model establishes how causal and bidirectional behavior can become scheduling choices. It does not establish the best schedule for a deployment, the memory required by an implementation, or a speed comparison with LingBot. Those answers need released implementations or matched measurements.

The next time a video model is called “causal” or “bidirectional,” the useful follow-up is not “which one is better?” It is: what information becomes available at each point in video time and denoising time?