Chapter 18 · Trace vision and language into bounded robot action

Today in the field story

One problem, then the next

Represent the mug action as a discrete token, direct continuous value, diffusion-generated sequence, and flow-generated sequence on paper or small arrays. Compare resolution, multimodality, generation cost, horizon, re-observation, and physical decoding. None of these mathematical forms supplies joint limits, collision checks, freshness, or controller readiness. The mission asks which representation fits the task contract, not which research label sounds most advanced.

Why now: The pipeline can now compare output families against the same physical action meaning.
Ignore today: Ignore training these model families; reason about generation and execution contracts.
Unlocks next: A justified action representation and declared guarded decoder.

Understand

Build the physical picture first

An action representation is a shipping format for future motion: bins, numbers, denoised sequences, and learned flows trade resolution, ambiguity, and computation.

A discrete action representation chooses among symbols or bins. A mobile robot may select turn_left, straight, or turn_right; a VLA such as OpenVLA tokenizes normalized action dimensions so a language-model-style backbone can predict them. Discretization gives a finite vocabulary and a clear decoding table, but values within one bin become indistinguishable and independent per-dimension tokens may not preserve trajectory smoothness by themselves. Vocabulary size, bin edges, out-of-range handling, token order, and the physical inverse mapping are part of the deployed policy contract.

A continuous policy outputs floating-point action values or parameters of a continuous distribution. It avoids fixed quantization bins and can represent small numeric differences, but regression to one average can be poor when demonstrations contain several valid strategies. For example, moving around an obstacle on the left or right may produce an unsafe average path through the obstacle. A continuous value also remains only a representation: joint target, velocity, torque, gripper-frame delta, and base-frame pose have different controllers, stability implications, and limits even when all are stored as floats.

Diffusion and flow-based policies are generative ways to produce continuous actions, commonly as a sequence. A diffusion policy begins from noise and iteratively denoises toward an action trajectory conditioned on observations. Flow matching learns a vector field that transports a simple distribution toward the action distribution and samples it by following that field, often with a numerical integration or model-specific solver. Both can represent multiple plausible behaviors; neither guarantees safe, smooth, or fast motion. Solver steps, stochasticity, action statistics, inference latency, and validation remain measurable deployment choices.

An action chunk has shape [H, D]: H future times and D action components. Predicting a chunk can capture temporal structure and amortize a slow model call, but executing the whole chunk open-loop lets the world change underneath it. Receding-horizon execution consumes only a prefix, obtains a fresh observation, and predicts again. Choose observation horizon, prediction horizon, execution horizon, control rate, overlap, and fallback together. Validate each row and the sequence between rows; a safe-looking first action does not make a later discontinuity acceptable.

Words you need

Name each idea precisely

Quantization: Mapping a continuous range into a finite set of bins or symbols, losing distinctions smaller than the bin spacing.
Physical example:
Five steering tokens across [-1,1] cannot directly distinguish desired values 0.21 and 0.24 if both decode to the same centre.
Action chunk: A sequence of future action vectors predicted together, conventionally shaped as horizon by action dimension.
Physical example:
At 10 Hz, a chunk of four gripper-frame deltas describes 0.4 seconds of intended motion before execution policy decides how much to consume.
Diffusion policy: A conditional generative policy that iteratively removes noise to produce an action value or sequence consistent with the current observation.
Physical example:
Several denoising steps turn a noisy four-pose proposal into one sampled route around the left side of a tabletop obstacle.
Flow matching: A generative method that learns a time-dependent vector field carrying samples from a simple distribution toward the demonstrated action distribution.
Physical example:
An integrator follows learned action-space velocity estimates from an initial noisy chunk toward a continuous pick trajectory.
Action horizon: The future time span or number of steps represented by one policy output.
Physical example:
Six actions at 20 Hz cover 0.3 seconds, regardless of how long inference took to produce them.
Receding-horizon execution: Executing only an initial part of a predicted sequence, then re-observing and planning a new overlapping sequence.
Physical example:
The arm uses two actions from a six-action chunk, checks whether the cup slipped, and predicts a replacement chunk from the new scene.

Math, one line at a time

Work through today’s relationship

Prerequisite rescue · optionalToken, action, and latency budgets

Language reasoning must finish before the physical situation becomes stale.

T_total: end-to-end decision latencyUnit: milliseconds (ms)
f_control: safety/control update rateUnit: hertz (Hz)
H: action horizonUnit: steps or seconds

Perception takes 80 ms, model reasoning 220 ms, and skill dispatch 20 ms.
T_total = 80+220+20 = 320 ms.
At 1 m/s the robot moves 0.32 m during that delay; independent fast safety control must not wait for the LLM.

Programmer analogy

An LLM tool call resembles backend orchestration, but its latency must be converted into physical travel distance.

At 0.5 m/s, how far does a robot move during 400 ms?

400 ms = 0.4 s; distance = 0.5×0.4 = 0.2 m.

For $K=5$ evenly spaced action values over $[-1,1]$ , adjacent bin centres are separated by

\Delta a=\frac{1-(-1)}{K-1}=\frac{2}{4}=0.5.

Discretization loses distinctions smaller than the bin spacing unless the decoder provides another representation.

Compare four representations of one bounded x action

A simulator accepts relative gripper-frame x displacement in [-0.10, 0.10] m at 10 Hz. The desired first step is 0.03 m, and the policy predicts four steps.

For five evenly spaced discrete centres, calculate spacing (0.10 - (-0.10)) / (5 - 1) = 0.05 m, producing [-0.10, -0.05, 0, 0.05, 0.10].
Nearest-bin decoding maps desired 0.03 m to 0.05 m, creating 0.02 m absolute quantization error before any controller or physical error.
A direct continuous head can output 0.03 m, but record whether it predicts one value, a distribution, or a complete chunk and how it represents multiple valid routes.
A diffusion head starts from a noisy four-step sequence and applies its configured denoising process; a flow head starts from a simple sample and follows its learned vector field. Record samples and elapsed time rather than assuming either method is deterministic or faster.
Interpret chunk shape [H=4, D=1] at 10 Hz as 4 × 0.1 = 0.4 s of future action, then set execution horizon to two steps so the simulator re-observes after 0.2 seconds.
Reject every candidate with a non-finite value, per-step displacement outside ±0.10 m, inter-step jump above the declared bound, stale observation, predicted collision, or inference time beyond the action deadline.
Run identical seeded simulated scenes, report success, chosen route, quantization error, inference time, validator rejection, and intervention separately for each representation.

Result

The discrete version visibly loses 0.02 m resolution in the first step, while the continuous and generative versions avoid that fixed bin error but add their own mode, sampling, and latency questions.

What this proves

Representation decides what a policy can express; horizon, validation, controller semantics, and measured trials decide whether that expression is usable.

Physical examples

Where this appears in real life

Two valid routes around a bottle

Demonstrations move a gripper either left or right around a central bottle before reaching the cup; both modes succeed.

Look for:

A single averaged continuous path may cross the bottle, while a discrete or generative representation can preserve two modes only if sampling, conditioning, and evaluation actually separate them.

Cup slips during a predicted chunk

A policy predicts 0.5 seconds of lifting motion, but the cup moves inside the gripper after the first 0.2 seconds.

Look for:

Receding-horizon execution re-observes before the remaining actions, whereas open-loop chunk execution continues from a state that is no longer true.

Hands-on exercise

Make the idea observable

Use a one-dimensional simulated gripper marker and local functions that return tokens, floats, or fabricated action samples. Training a diffusion or flow model is outside this exercise.

Implement a five-bin encoder and decoder for [-0.10, 0.10] m; round-trip eleven test values and tabulate maximum and mean quantization error.
Implement a bounded continuous mock policy and two seeded generative mocks that choose left or right action sequences; label these as interface simulators, not trained-model evidence.
Represent every result as [H, D], attach control rate and frame, and assert that the claimed time horizon equals H / rate.
Execute one, two, and all four actions before re-observation; move the target after step two and record which execution horizon notices the shift before issuing stale actions.
Inject a NaN, one out-of-range row, an abrupt reversal, and a 150 ms inference delay against a 100 ms cycle; verify the sequence validator reports the responsible row and rule.
Write a comparison table covering resolution, multimodal behavior, determinism, sampling calls, measured latency, reactivity, failure modes, and the evidence your mocks cannot provide.

Observe

Finer numeric output is not automatically better behavior, and a longer chunk trades fewer model calls for more exposure to scene change unless execution deliberately re-observes.

Done when

The bin-error table is reproducible, action-horizon arithmetic is correct, all four injected faults are blocked, and the final comparison separates representation properties from unmeasured model quality.

Build today

Build a tiny language-conditioned policy interface over your existing simulator or dataset.

Evidence to save

DONE when the integrated “Discrete, continuous, diffusion, and flow actions” path is observable, cancelable, and leaves the prior baseline reproducible.

Common mistakes

Catch the wrong mental model

Wrong

Assuming discrete actions are crude and continuous actions are inherently precise.

Better

Measure bin error, sensor and actuator resolution, controller behavior, and distribution ambiguity; floating-point output can still average incompatible strategies or use the wrong semantics.

Wrong

Executing every predicted action because the model paid to compute the whole chunk.

Better

Choose a shorter execution horizon when the scene can change, re-observe on schedule, and define how overlapping chunks are reconciled without discontinuity.

Wrong

Treating diffusion or flow generation as an automatic trajectory-safety guarantee.

Better

Validate sampled values and the complete sequence for time, bounds, frames, kinematics, collisions, continuity, and controller acceptance every time.

Job connection

How this becomes employable evidence

Select and integrate a policy action head by matching robot control semantics, resolution, multimodality, chunk timing, inference budget, processor order, and sequence validation rather than choosing diffusion, flow, tokens, or regression by trend.

Relevant target roles

Robot Learning Deployment / Physical AI Integration Engineer
Robotics Software Engineer — ROS 2 / AMR
Robotics Deployment, Integration & Validation Engineer

Chapter 18 interview drill

Interview questions: Discrete, continuous, diffusion, and flow actions

Practise a 60–90 second answer: define the idea, connect it to a physical robot, state assumptions, frames, and units when relevant, then finish with the failure signal or evidence you would inspect.

Primary interview scenario

Compare tokenized, direct continuous, diffusion, and flow action outputs for a 20 Hz arm. Quantify one discretization tradeoff, define prediction and execution horizons, and explain how latency and re-observation change safe deployment.

Answer shape: clarify the situation → trace the physical and software path → test the most likely boundaries → name the evidence that would confirm the result.

Technical follow-up questions

Q1What quantization error occurs when `0.03 m` uses the nearest centre in the five-bin worked example?

Model interview answer

It decodes to 0.05 m, so the absolute representation error is 0.02 m before other system errors.

Q2How do diffusion and flow action generation differ at a high level?

Model interview answer

Diffusion iteratively denoises a sample, while flow matching learns a vector field and follows it from a simple distribution toward actions; actual solver steps and speed are implementation choices.

Q3Why predict four actions but execute only two?

Model interview answer

The chunk captures temporal intent, while a shorter execution horizon forces a fresh observation after 0.2 seconds so later actions are not blindly applied to a changed scene.