Chapter 24

Train, optimize, evaluate, and freeze one bounded edge policy

Close the capstone policy loop without turning a promising training run into an unsupported robot claim. This chapter starts with a falsifiable hypothesis and runnable baseline, trains one policy while separating optimization from evidence, exports through ONNX and TensorRT with value-level drift tests, profiles a Jetson-class target under sustained load, opens an untouched held-out suite exactly once, directs the next improvement from a failure taxonomy without recycling final-test evidence, and freezes model v1 as a reproducible deployment bundle with explicit operating limits, fallback behavior, evaluation, and rollback.

Before you start

Complete Week 23 with a versioned dataset whose observation and action fields, units, frames, timestamps, episode groups, train/validation/held-out assignments, quality findings, and card are frozen. Keep a second development or reserve slice available if final-test failures later inspire another model.
Bring one already runnable policy or non-learned baseline, one training recipe supported by the selected policy implementation, a deterministic simulator or disconnected playback harness, and a command guard that checks finite values, shape, units, frames, freshness, absolute limits, rate limits, authority, and cancellation.
Be able to distinguish training loss, validation loss, offline action error, closed-loop task success, end-to-end latency, deadline misses, interventions, and prohibited outcomes. These measurements answer different questions and must not be merged into one unexplained score.
All learner work stays offline, in simulation, on recorded observations, or on an equivalent constrained computer. Any powered-robot rollout requires the commissioned platform, approved task and operating envelope, qualified supervision, physical safeguards, validated emergency provisions, and the site's authorization; a model, ROS 2 action, watchdog, or software fallback is not a safety function.

By the end

Write a falsifiable experiment charter that names the baseline, one intended change, data and code versions, primary metric, guard gates, minimum worthwhile improvement, run budget, and decision rule before seeing candidate results.
Train one policy from a pinned configuration, inspect training and validation curves without touching the held-out suite, select a checkpoint by a predeclared rule, and retain seeds, environment, logs, checkpoints, and failure evidence.
Export a checkpoint through an explicit preprocessing–ONNX–TensorRT contract, validate graph structure and shapes, compare source and optimized outputs on a representative golden corpus, and reject precision changes whose numerical or task drift crosses physical tolerances.
Profile the complete edge path under warm and sustained representative load, reporting input age, stage latency, p50 and tail latency, deadline misses, memory, power mode, temperature or throttling evidence, queue behavior, and a tested application fallback.
Run baseline and candidate on one frozen held-out manifest without adapting to revealed cases, preserve every denominator and terminal outcome, and issue the predeclared accept, revise, or reject result even when it is disappointing.
Turn final-test failures into hypotheses and new development data or system experiments while keeping the revealed suite historical, changing one causal lever at a time, and reserving fresh evaluation evidence for the next candidate.
Freeze a deployable model v1 bundle containing checksummed model and processor artifacts, action statistics, export and engine recipes, runtime and robot profiles, operating envelope, evaluation and latency reports, monitoring, fallback, rollback, approvals, and known limitations.

The field story

Policy Forge Model v1

Episode Foundry Dataset v1 arrives sealed at the Policy Forge. The train and validation manifests are available; held-out scene identities remain closed. Your mission is to produce Model v1 through a falsifiable experiment rather than a sequence of increasingly flattering runs. A simple baseline, one intended change, fixed data and code identities, primary metric, guard gates, run budget, and decision rule are written before training. Curves may guide development, but they cannot prove closed-loop task performance, deadline compliance, or safe physical behavior.

The candidate then passes through increasingly concrete gates. Training records seeds, checkpoints, curves, and failures. Export preserves preprocessing and decoding while ONNX or TensorRT outputs are compared on a representative golden corpus. A Jetson-class profile measures the whole path under sustained load without assuming every device behaves alike. Only after the candidate and decision rules are frozen does the held-out envelope open once. Revealed failures become hypotheses for a future dataset or model, never edits to this verdict. Model v1 freezes with processors, statistics, runtime, limits, fallback, rollback, and known limitations for Week 25.

Why this chapter now: Week 23 produced a sealed Dataset v1 with protected evaluation groups. The next step is one controlled policy experiment whose deployment claims remain bounded by parity, latency, and final evidence.
Ignore for now: Do not tune on held-out failures, equate lower loss with task success, assume export parity from a successful load, or claim Jetson performance without the exact target profile.
This unlocks: A versioned Model v1 and deployment bundle that Week 25 can place into SIL, HIL, controlled-hardware, observability, rollback, and frozen acceptance gates.
Proof you will leave with: A predeclared charter, baseline and candidate identities, training/validation curves, checkpoint lineage, golden parity rows, sustained edge metrics, untouched held-out manifest and outcomes, failure taxonomy, bundle checksums, fallback, and rollback record.

Environment contractrepository-supported Node.js 22.13.0 or newer and synthetic metric rows support the decision starter; actual training, ONNX, TensorRT, GPU, Jetson, and powered-robot execution are optional external environments and are not assumed.

Compatibility boundary: The starter models a release decision only. Real framework, exporter, precision, accelerator, driver, power-mode, thermal, processor, and robot compatibility must be pinned and tested on the exact candidate system.
Smoke check: Run node week-24-model-v1-decision.mjs; confirm the candidate passes its declared development gates, the planted prohibited outcome blocks release, and Model v1 remains rejected.
Contract reviewed: 2026-07-25
Runtime evidence: The dependency-free starter is executed by repository tests on the supported Node.js baseline. Chapter-specific ROS 2, Gazebo, model, dataset, checkpoint, and hardware environments are learner-created unless the repository supplies an explicit asset; run the smoke check and preserve its versions and output before claiming runtime compatibility.
Drift risk: high

Today in the field story

One problem, then the next

The Policy Forge opens with Dataset v1 hashes on the wall and the held-out envelope locked away. You state one falsifiable hypothesis, one runnable baseline, one intended change, code and data identities, primary development metric, command-guard gates, minimum worthwhile improvement, run budget, and decision rule. The baseline may be simple; its job is to expose whether added complexity earns measurable value. A result cannot redefine success after the candidate has already revealed its strengths and weaknesses.

Why now: A charter prevents experiment decisions from drifting toward the preferred candidate.
Ignore today: Do not train multiple families or inspect held-out scenes.
Unlocks next: A controlled comparison with precommitted evidence and stopping rules.

Understand

Build the physical picture first

An experiment is a fair race on one marked track: freeze the ruler, starting positions, finish rule, and safety disqualifiers before either policy runs.

Begin with the physical task and the simplest credible competitor. A baseline may be the last accepted checkpoint, a scripted grasp, nearest-neighbour action lookup, or an untouched pretrained policy, but it must actually run through the same observation, action, reset, guard, and scoring path as the candidate. Its job is to reveal whether the new work beats something real. A famous published number, a training loss from another dataset, or a policy that receives easier inputs is not a local baseline.

Write the experiment as a falsifiable sentence. Let B be baseline success rate, C candidate success rate, and Δ = C - B the absolute improvement on the declared development suite. A useful hypothesis is: “Changing only the policy from baseline revision b7 to ACT configuration a3, while keeping dataset d1 and every reset fixed, will make Δ ≥ 0.15 without any prohibited contact and with p95 end-to-end latency at or below 100 ms.” Define every symbol, unit, slice, and threshold beside the sentence.

Separate optimization evidence from final evidence now. D_train supplies gradient updates, D_val chooses settings or checkpoints, D_dev supports repeatable closed-loop diagnosis, and sealed D_test answers the final question once. The split unit is an episode family, scene, object instance, or collection session whenever related rows could leak familiarity. Compute normalization statistics only from allowed training data. Reading held-out outcomes to choose a learning rate, augmentation, threshold, or baseline turns that suite into development data.

Predeclare the decision rule and budget. Name the primary metric, mandatory guard gates, useful effect size, number of seeds, maximum training steps, checkpoint-selection rule, trial count, stop conditions, and how inconclusive results are handled. A small sample can show an engineering direction without proving a population claim, so preserve counts and uncertainty. No experiment authorizes motion: even a candidate that clears learning metrics still enters only the already approved simulator or guarded robot workflow.

Words you need

Name each idea precisely

Runnable baseline: A versioned reference method executed through the same inputs, resets, guards, and metrics as the candidate so the comparison has a physical meaning.
Physical example:
A scripted top-down grasp and a learned grasp both receive the same camera frame, object placements, timeout, collision rule, and gripper controller.
Experiment hypothesis: A testable prediction linking one declared change to a measured effect under frozen conditions, including a threshold that could be missed.
Physical example:
Replacing only the policy checkpoint is predicted to raise unassisted bin-placement success by at least 15 percentage points on fixed development starts.
Experimental unit: The independently assigned item counted in a comparison, chosen at the dependency level that prevents related observations from crossing conditions.
Physical example:
A complete reset-and-rollout episode is one unit; its thousands of camera frames are not thousands of independent trials.
Decision rule: The prewritten mapping from measured metrics and mandatory failures to accept, revise, reject, or inconclusive, applied after results are revealed.
Physical example:
A candidate advances only if success improves by at least 15 points, no prohibited contact occurs, and its p95 deadline is met.

Math, one line at a time

Work through today’s relationship

Prerequisite rescue · optionalLatency, memory, quantization, and acceptance deltas

A trained model is deployable only when it fits the edge device and preserves task quality.

FPS = 1/T: inferences per second from latency TUnit: frames/s
memory: runtime RAM or VRAM useUnit: MB or GB
Δmetric: optimized minus original metricUnit: metric unit

Inference latency is 50 ms = 0.05 s.
Maximum theoretical rate is 1/0.05 = 20 FPS.
Measure end-to-end rate under thermal load and compare the exact frozen success suite before and after ONNX/TensorRT or quantization.

Programmer analogy

Like profiling any application on its target device, optimize on the actual robot computer and protect behavior with regression tests.

What is the theoretical rate for 100 ms inference?

100 ms = 0.1 s; 1/0.1 = 10 FPS.

The target success improvement is

\Delta S=S_{\mathrm{new}}-S_{\mathrm{base}}=0.80-0.65=0.15.

Independently require $T_{\mathrm{total}}=T_{\mathrm{sense}}+T_{\mathrm{infer}}+T_{\mathrm{comm}}+T_{\mathrm{control}}\le100\,\mathrm{ms}$ .

Turn an attractive capstone idea into a bounded experiment

A tabletop policy must place a foam block in a bin. Baseline b7 succeeds on 12 of 20 frozen development starts, has p95 pipeline latency of 72 ms, and produces no prohibited contact. The team proposes ACT configuration a3 on unchanged dataset d1.

Calculate baseline success as B = 12 / 20 = 0.60; retain the count 12/20 because the denominator and individual case outcomes matter more than the percentage alone.
Define the one intended change as policy: b7 → a3; pin dataset, split manifest, processors, action statistics, simulator, robot profile, reset cases, command guard, hardware target, and scoring code as controlled variables.
Set the worthwhile effect to Δ_min = 0.15, so the candidate target is C ≥ 0.75; with 20 trials that means at least 15 successes, subject to the other gates.
Add mandatory gates of zero prohibited contact, zero unhandled inference exception, and p95 ≤ 100 ms; specify that a gate failure rejects advancement even if task success reaches 15 of 20.
Choose validation loss for checkpoint selection, three recorded seeds for training variability, and the same 20-case development rollout for diagnosis; keep the final held-out manifest unread.
Sign the charter before training and label its conclusion scope: this experiment can select a candidate for final evaluation, but cannot establish real-robot safety, broad generalization, or production readiness.

Result

The team now has a baseline, one-variable hypothesis, numeric effect threshold, guard gates, finite budget, and evidence boundary that can produce a real failure instead of a movable success story.

What this proves

A hypothesis becomes useful when another engineer can run the same comparison and knows in advance which observations would disprove it.

Physical examples

Where this appears in real life

Two grasp policies on unequal starting marks

Policy A receives centered cubes under one lamp, while Policy B receives randomized cubes with shadows; A reports 18 successes in 20 trials and B reports 13.

Look for:

The scores do not rank the policies because reset distributions differ. Replay one frozen case manifest through both policies, preserve case order and scoring, and pair outcomes by case ID.

A ruler that bends after the result

A rover policy misses the original 20-second docking limit, so the team changes the limit to 30 seconds after watching the final runs and calls the model successful.

Look for:

The changed timeout may define a future product decision, but it cannot rewrite the original result. Record the miss, version a new requirement, and evaluate a future candidate on fresh evidence.

Hands-on exercise

Make the idea observable

Use the Week 23 dataset card, a prior checkpoint or deterministic script, and a spreadsheet or Markdown experiment charter. Stay on recorded observations or the disconnected simulator.

Name one physically measurable capstone outcome, timeout, allowed interventions, prohibited outcomes, and terminal robot state; attach the exact scorer and units.
Run or replay the baseline on a small frozen development manifest, preserving case IDs, per-case outcomes, stage failures, and latency rather than copying a headline benchmark.
Write D_train, D_val, D_dev, and sealed D_test lineages, then verify that preprocessing statistics, near-duplicates, object instances, and collection sessions respect those boundaries.
Write one hypothesis that changes one primary factor and includes a minimum useful effect, mandatory guard gates, seed or repeat budget, and checkpoint-selection rule.
Ask a peer to identify any condition that could be changed after results appear; freeze or explicitly mark it exploratory before launching training.
Save the charter beside exact code, configuration, baseline artifact, dataset manifest, environment, and scorer identifiers, then compute checksums for the files that define the comparison.

Observe

Most vague claims collapse into several separate questions: whether training fits recorded actions, whether rollouts complete the task, whether latency fits the loop, and whether any disqualifying behavior occurs.

Done when

The baseline is locally runnable, every symbol and metric is defined, only one intended change remains, final evidence is sealed, and the signed decision rule can reject the candidate.

Build today

Fine-tune one policy, export and profile it on Jetson-class hardware or an equivalent constrained target, deploy behind a safe ROS 2 action, and evaluate held-out scenes.

Evidence to save

DONE when the learning log explains “Select a baseline and write the experiment hypothesis” in five precise points and a checked example produces the predicted output.

Common mistakes

Catch the wrong mental model

Wrong

Using the previous run's training loss as the baseline for a robot task.

Better

Run a versioned reference policy or controller on the same observations, resets, command path, outcomes, and latency measurements; loss and task success are different evidence.

Wrong

Writing “the new policy will improve performance” without a magnitude or failure rule.

Better

Name the changed factor, metric, minimum worthwhile effect, mandatory disqualifiers, finite trial budget, and decision produced when the threshold is missed.

Wrong

Looking at held-out trials to choose the baseline because that seems fairer.

Better

Choose and validate the baseline from prior or development evidence, then freeze it before opening the held-out suite; otherwise final cases have influenced the comparison.

Job connection

How this becomes employable evidence

Create an experiment charter for a customer-facing robot-learning change, reproduce the current baseline on exact cases, protect data boundaries, negotiate latency and behavior gates with integration owners, and deliver a decision rule that survives a disappointing result.

Relevant target roles

Robot Learning Deployment / Physical AI Integration Engineer
Robotics Deployment, Integration & Validation Engineer

Chapter 24 interview drill

Interview questions: Select a baseline and write the experiment hypothesis

Practise a 60–90 second answer: define the idea, connect it to a physical robot, state assumptions, frames, and units when relevant, then finish with the failure signal or evidence you would inspect.

Primary interview scenario

You inherit a policy claimed to be 20 percent better than baseline. Explain how you would verify the baseline, experimental unit, split lineage, controlled change, effect threshold, safety gates, trial budget, and untouched final evaluation before accepting that claim.

Answer shape: clarify the situation → trace the physical and software path → test the most likely boundaries → name the evidence that would confirm the result.

Technical follow-up questions

Q1What does `Δ = C - B` mean in this chapter?

Model interview answer

B is baseline success rate, C is candidate success rate on the same declared cases, and Δ is their absolute difference in percentage-point form.

Q2Why are ten thousand frames from one rollout not ten thousand independent experimental units?

Model interview answer

Those frames share one reset, scene, policy history, and physical trajectory, so treating them as independent would greatly overstate the amount of evidence.

Q3When may the final held-out suite influence hyperparameters?

Model interview answer

Never while it remains the claimed final suite; once its outcomes influence a choice, it becomes development evidence and a fresh untouched evaluation is required.

Chapter references

Hugging Face LeRobot — Imitation Learning on Real-World RobotsMaintainer workflow for versioned robot data, policy training, checkpoint creation and resumption, pinned policy revisions, rollout modes, and evaluation; used as one concrete implementation path rather than a universal training recipe or safety authority.
PyTorch — ReproducibilityFramework-owned guidance on controlling random sources and deterministic operations, plus the important boundary that exact reproducibility is not guaranteed across releases, commits, platforms, or CPU and GPU execution.
ONNX — Checker and model consistencyProject-owned API contract for checking whether an ONNX model is structurally legal and internally consistent; this chapter keeps that check separate from semantic parity, numerical drift, latency, task performance, and physical safety.
NVIDIA TensorRT — Accuracy ConsiderationsVendor documentation for reduced-precision range and precision tradeoffs, rounding and clamping error, quantized types, and accuracy-debugging concerns that motivate representative calibration and source-versus-engine differential tests.
NVIDIA Jetson Linux — Platform Power and PerformanceRelease-scoped vendor reference for Jetson power modes, clocks, fan control, thermal sensing, throttling, shutdown, and monitoring; used to define what must be observed during a sustained target-device profile without assuming identical behavior across modules or software releases.