Chapter 14 · Collect demonstrations as synchronized, reviewable robot datasets

Today in the field story

One problem, then the next

Episode 014 stores a gripper move, but that phrase could mean joint targets, Cartesian deltas, velocities, or an action chunk. Write ordered component names, units, frames, rate, bounds, controller interpretation, and expiry for the selected handover action. Convert one example by hand and reject another whose dimensions match but semantics do not. Numeric shape alone cannot protect the future policy from a swapped command contract.

Why now: The recorder and learner must agree on what every action value physically means.
Ignore today: Ignore choosing the best policy representation; document and validate the collected action space.
Unlocks next: Machine-checkable action targets for training and replay.

Understand

Build the physical picture first

An action vector is a typed physical instruction, not a bag of numbers: its coordinate order, representation, frame, unit, limit, cadence, reference point, and execution horizon jointly determine what the robot will try to do.

Joint-position actions name a desired angle or displacement for each controlled joint, while joint-velocity actions name rates of change. Cartesian actions describe a tool pose, pose delta, twist, or waypoint in a declared frame. These representations are not interchangeable. Kinematics, calibration, controller behavior, constraints, and sometimes inverse kinematics sit between them. A value of 0.2 can mean radians, metres, radians per second, normalized policy output, or nothing useful until the contract says.

Each action feature needs an ordered component list, dtype, shape, units, frame where applicable, lower and upper limits, nominal rate, and semantics such as absolute versus delta. State explicitly whether a gripper value is width, joint position, current request, or a normalized open-close command. Preserve the unnormalized physical interpretation and the transform used for learning; otherwise an apparently compatible tensor can command the wrong axis or scale on replay.

Velocity commands also need duration and timeout semantics. A velocity is not a destination: continuing 0.1 m/s for 0.1 seconds differs from holding it for two seconds after network loss. Position targets need controller and completion semantics because writing a target does not prove the joint reached it. A safe consumer validates finiteness, shape, range, frame, freshness, mode, and enable state before forwarding any action.

Action chunking predicts several future action steps together. At 20 Hz, an eight-step chunk spans a nominal 0.4 seconds. Chunks can preserve a coherent short motion and reduce repeated inference, but later entries become stale when the world changes. Define how often a new chunk is produced, how overlapping predictions are combined, how many steps execute open-loop, and what cancels the remainder after new evidence, a mode change, or an abort.

Words you need

Name each idea precisely

Joint-position action: An ordered target for named joint coordinates, with angular or linear units and absolute or relative semantics.
Physical example:
The vector [0.3, -0.2] rad targets shoulder and elbow angles only when that order matches the controller contract.
Cartesian action: A tool pose, pose change, twist, or waypoint expressed relative to a named coordinate frame.
Physical example:
A +0.02 m x delta in tool0 moves along the tool's x-axis, not necessarily the table's x-axis.
Velocity action: A commanded rate whose physical effect depends on units, reference frame, duration, update cadence, and timeout.
Physical example:
A mobile base twist requests 0.10 m/s forward for only as long as fresh enabled messages satisfy its watchdog.
Action chunk: A time-ordered block of future actions predicted or stored together with a defined horizon and execution policy.
Physical example:
Eight seven-dimensional joint-and-gripper targets at 20 Hz describe 0.4 seconds of nominal future commands.
Execution semantics: The rules that turn an action record into behavior, including mode, timing, interpolation, clipping, completion, cancellation, and fallback.
Physical example:
A delta target is applied once from the measured pose, whereas an absolute target remains referenced to a fixed coordinate definition.
Action normalization: A reversible mapping between physical action values and a scaled learning representation, derived from documented training data or limits.
Physical example:
A joint range from -0.5 to 1.5 rad maps linearly to -1 to 1 only when inference applies the same versioned conversion before validation.

Math, one line at a time

Work through today’s relationship

Prerequisite rescue · optionalEpisode timing, normalization, and split leakage

A policy learns the dataset you actually recorded, including hidden leakage and timing errors.

z = (x−μ)/σ: standardized valueUnit: unitless
t: source timestampUnit: seconds (s)
N: number of independent episodesUnit: episodes

For x=14, dataset mean μ=10, and standard deviation σ=2, subtract: 14−10=4.
Divide: z=4/2=2, meaning two standard deviations above the mean.
Compute μ and σ from training data only, then keep whole scenes out of validation to prevent leakage.

Programmer analogy

It resembles a production event log, but camera, state, and action streams must describe the same instant.

What is z for x=8, μ=10, σ=2?

(8−10)/2 = −1.

For an $N=8$ step action chunk executed at $f=20\,\mathrm{Hz}$ ,

t_{\mathrm{horizon}}=\frac{N}{f}=\frac{8}{20}=0.4\,\mathrm{s}.

The result is a duration; it does not convert a joint command in radians into a Cartesian command in metres.

Decode and bound one Cartesian action chunk

A policy emits a chunk with shape [8, 7] at 20 Hz. Each row is [dx, dy, dz, dRx, dRy, dRz, gripper], where translation is metres in tool0, rotation is radians as a small rotation vector, and gripper is normalized from 0 closed to 1 open. Translation deltas are limited to ±0.02 m per step and rotation deltas to ±0.05 rad.

Read the schema before values: eight time steps, seven ordered components, tool-relative deltas, physical units for six components, and one dimensionless gripper command.
Calculate the nominal horizon as 8 / 20 Hz = 0.4 s; do not confuse that duration with a safe open-loop commitment.
Check the first row [0.01, -0.03, 0, 0, 0, 0.02, 0.7] for shape, finite values, frame availability, and current control mode.
Detect that dy = -0.03 m exceeds the per-step lower limit of -0.02 m; reject or apply the explicitly approved saturation policy rather than silently changing training labels.
Map the gripper value through the versioned physical conversion and validate the resulting actuator target independently; 0.7 is not itself a width or angle.
Define that only the first two steps may execute before a new observation, and that abort, stale input, frame loss, or mode change cancels all unexecuted rows.

Result

The chunk contains 56 values over 0.4 seconds, but its first row violates the y-delta contract and cannot be forwarded unchanged.

What this proves

Tensor shape and horizon are necessary bookkeeping; physical meaning and runtime gates decide whether an action is executable.

Physical examples

Where this appears in real life

The instruction “move five”

Give a partner a paper pointer and say only “move five,” then compare five degrees of rotation, five centimetres along the table, and five centimetres per second for two seconds.

Look for:

The same numeric value produces unrelated end states because type, unit, axis, frame, and duration were missing.

Prewritten route versus frequent replanning

Move a token toward a target using either an eight-arrow card committed in advance or a new single arrow after each move while a partner shifts an obstacle halfway through.

Look for:

The longer action chunk stays coherent but continues toward an outdated scene unless its remaining steps can be cancelled or replaced.

Hands-on exercise

Make the idea observable

Use a paper two-link arm, ruler, protractor, coordinate-frame sheet, and a small table or JSON file. No powered actuator is required.

Describe one paper-arm move four ways: absolute joint angles, joint-angle deltas, Cartesian tool delta, and joint velocities with an explicit duration.
For each representation, write ordered names, shape, dtype, units, frame, lower and upper bounds, rate, absolute-or-delta rule, and cancellation behavior.
Apply each valid action manually and record the resulting measured paper pose rather than copying the requested value into the state column.
Plant three defects—a swapped joint order, centimetres labelled as metres, and a stale timestamp—and verify that a schema checker rejects each with a distinct reason.
Create a four-step chunk, move the target after step one, and compare executing the remainder with cancelling and replanning from the new observation.
Save the action schema revision, example rows, rejection results, command-state comparison, and one unresolved conversion assumption for review.

Observe

Several representations can describe a related intention, but conversion depends on geometry and execution rules; malformed or stale actions can look numerically plausible until typed validation runs.

Done when

Another learner can execute the paper instructions without asking what any component means, and all three planted contract violations fail closed with the expected reason.

Build today

Create a small demonstration dataset with synchronized observations, actions, language, and quality labels.

Evidence to save

DONE when a deterministic “Action spaces: joint, Cartesian, velocity, and chunks” failure test reports expected versus actual behavior and passes after the documented fix.

Common mistakes

Catch the wrong mental model

Wrong

Documenting only action tensor shape and min/max values.

Better

Also define ordered component names, representation, physical units, frame, absolute-or-delta semantics, rate, normalization version, interpolation, timeout, and cancellation.

Wrong

Treating Cartesian and joint actions as equivalent arrays after matching their lengths.

Better

Use explicit kinematics and controller contracts; equal dimension does not align axes, units, feasibility, singularities, or collision constraints.

Wrong

Assuming every predicted chunk should execute completely.

Better

Bound open-loop execution and cancel remaining steps when observations, frames, modes, freshness, or safety state invalidate the original plan.

Job connection

How this becomes employable evidence

Specify and enforce the action boundary between a learned policy, ROS 2 integration layer, controller, and robot hardware so offline data, training normalization, inference tensors, and runtime limits share one versioned physical meaning.

Relevant target roles

Robotics Software Engineer — ROS 2 / AMR
Robotics Application / ROS 2 Integration Engineer
Robot Learning Deployment / Physical AI Integration Engineer

Chapter 14 interview drill

Interview questions: Action spaces: joint, Cartesian, velocity, and chunks

Practise a 60–90 second answer: define the idea, connect it to a physical robot, state assumptions, frames, and units when relevant, then finish with the failure signal or evidence you would inspect.

Primary interview scenario

A policy returns an 8 × 7 tensor. Describe every contract you need before executing it, calculate its horizon at 20 Hz, and explain how you handle bounds, overlapping chunks, stale observations, and cancellation.

Answer shape: clarify the situation → trace the physical and software path → test the most likely boundaries → name the evidence that would confirm the result.

Technical follow-up questions

Q1What is missing from `action = [0.2, 0.4]`?

Model interview answer

At least ordered component meaning, representation, dtype and shape, units, frame, absolute-or-delta semantics, limits, rate, normalization, duration, and execution rules.

Q2How long is an eight-step chunk at 20 Hz?

Model interview answer

Its nominal horizon is 8 / 20 = 0.4 seconds, though the system may intentionally execute fewer steps before replanning.

Q3Why store measured state separately from a position target?

Model interview answer

A target expresses the request; measured state shows delayed or limited physical response and whether the target was actually approached or reached.