Chapter 13 · Build a trustworthy PyTorch experiment from robot-shaped data

Today in the field story

One problem, then the next

Train the scout with a loss that matches millimetre error, a declared optimizer, schedule, and checkpoint rule. Separate training from validation mode, prevent gradients during evaluation, and prove that save, fresh-process load, and resume preserve the intended state. A falling training curve is one observation, not the finish line; keep per-component errors and rejected non-finite batches visible.

Why now: The selected model needs a correct, restartable optimization path before generalization can be judged.
Ignore today: Ignore final-test results and deployment; establish training mechanics and checkpoint lineage.
Unlocks next: Comparable candidate checkpoints chosen by a predeclared rule.

Understand

Build the physical picture first

Training is a controlled descent: the loss marks local height, gradients point uphill, the optimizer chooses a step downhill, and a checkpoint records the exact campsite.

A loss function converts prediction error into the scalar training signal differentiated by autograd. Choose it from the target meaning, noise, and cost of errors. Mean squared error squares numeric residuals, so large regression errors receive extra weight and its unit is the square of the target unit. Cross-entropy expects class logits and class targets under its documented contract. A loss is not automatically a deployment metric: an image-to-position model may train with normalized MSE while reviewers need centimetre error, tail error, rejection rate, and task success in physical units.

One ordinary training step has a strict evidence path. The model predicts a batch, the loss compares predictions with targets, gradients are cleared, backward computes parameter gradients, and the optimizer updates parameters. PyTorch gradients accumulate, so an accidental missing clear operation changes the effective update across batches. Inspect finite loss, gradient norms, and parameter updates before celebrating a downward curve. Clipping may contain an extreme gradient but can also hide wrong scaling or corrupt data; diagnose the cause before treating clipping as the fix.

The learning rate controls update size, while an optimizer defines how gradients and stored history become parameter changes. Plain stochastic gradient descent makes the relationship easy to inspect; momentum and Adam keep additional state. A schedule changes the learning rate according to a declared rule, such as after an epoch or validation plateau. Step it at the intended frequency and log the actual value. Validation runs with the model in evaluation mode and gradients disabled, and its examples must never call the optimizer or select themselves into the training set.

An inference checkpoint can be only model parameters plus the architecture and preprocessing needed to use them. A resumable training checkpoint needs more: model state, optimizer state, scheduler state, completed epoch or step, random-generator states where controlled, configuration, data and split identity, code and environment versions, and recent metric state. Loading must use the intended device and mode. Prove resumption by comparing the next batch, learning rate, loss, and parameter result with an uninterrupted reference; successfully opening a file proves only serialization.

Words you need

Name each idea precisely

Loss function: A differentiable scalar objective that tells training how current predictions disagree with targets under a declared error rule.
Physical example:
Squared position error penalizes a ten-centimetre miss four times as much as a five-centimetre miss before averaging.
Optimizer: An algorithm and stored state that convert parameter gradients into updates.
Physical example:
SGD moves a weight opposite its gradient by learning rate times gradient, while Adam also tracks moving statistics.
Learning rate: A scale controlling how far an optimizer changes parameters at each update.
Physical example:
A very large step can jump across a low-loss region, while a tiny step may make little progress within the available training budget.
Schedule: A declared rule that changes learning rate or another training hyperparameter as steps, epochs, or monitored metrics change.
Physical example:
After five epochs, a schedule reduces a 0.1 learning rate to 0.05 and the run log records the new value.
Checkpoint: A versioned snapshot of the state needed either for inference or for continuing a training process.
Physical example:
A resume file stores model, Adam buffers, scheduler position, completed step, data version, configuration, and controlled random states.
Evaluation mode: The model mode used for inference or validation so modules such as dropout and batch normalization follow evaluation behavior.
Physical example:
A loaded vision model calls eval() before latency and held-out error are measured, while training later calls train() again.

Math, one line at a time

Work through today’s relationship

Prerequisite rescue · optionalTensor shapes and gradient descent

Most robot-learning bugs are shape, scale, split, or optimization mistakes.

B×T×D: batch, time steps, and feature dimensionsUnit: counts
L: loss measuring prediction errorUnit: task-dependent
η: learning rateUnit: unitless scale

A batch has B=8 episodes, T=20 time steps, and D=12 features.
The tensor contains 8×20×12 = 1,920 numbers.
An optimizer updates a weight with w_new = w_old − η∂L/∂w; inspect shape and finite values before training.

Programmer analogy

Tensor shapes are explicit multidimensional array contracts: every axis has a declared meaning and size.

How many values are in a 4×10×6 tensor?

4×10×6 = 240 values.

The SGD update is

\theta_{\mathrm{new}}=\theta-\eta g=1.0-(0.1)(0.4)=0.96.

For one prediction $2$ and target $3$ , $\mathrm{MSE}=(2-3)^2=1$ . Lower training loss does not prove that the optimizer, schedule, or checkpoint generalizes.

Trace one SGD update and its scheduled next step

A scalar model weight is θ = 1.0. Backward produces gradient g = 0.4. Plain SGD uses learning rate η = 0.1, and an epoch schedule halves that rate before the following epoch.

Confirm the gradient belongs to the current batch and parameter, is finite, and was computed after old gradients were cleared.
Apply the SGD rule: θ_new = θ - ηg = 1.0 - 0.1 × 0.4.
Calculate the update magnitude: 0.1 × 0.4 = 0.04, so θ_new = 0.96.
Advance the schedule at its documented epoch boundary: the next learning rate becomes 0.1 × 0.5 = 0.05.
Record the checkpoint after the update with θ = 0.96, optimizer state, scheduler position, completed step, random states, and run identities.
Resume and verify that the next known batch sees learning rate 0.05 and produces the same next loss and parameter as the uninterrupted reference within tolerance.

Result

The inspected update moves the weight from 1.0 to 0.96, and the resumable state makes the next step use 0.05 rather than accidentally repeating or skipping the schedule.

What this proves

A decreasing loss is not enough; the update order, learning rate, modes, saved state, and next-step replay must agree with the declared experiment.

Physical examples

Where this appears in real life

Measured steps down a paper hill

Draw a U-shaped height curve, place a token on one side, and compare moving opposite the local slope with a large, medium, and tiny step.

Look for:

The large step can overshoot, the tiny step progresses slowly, and no single local slope reveals the whole terrain.

Saved workshop bench

Pause a multi-step paper assembly and photograph not only the object but also tool settings, remaining instructions, material labels, and the exact completed step.

Look for:

The object alone resembles model weights; continuing identically also needs optimizer-like tool state, schedule position, inputs, and procedure version.

Hands-on exercise

Make the idea observable

Train a tiny CPU model on the synthetic or recorded Week 13 dataset. Keep validation read-only and keep all outputs disconnected from motion.

Choose a loss whose input shape and target meaning fit the task; calculate one batch loss by hand in physical or normalized units and compare with PyTorch.
Implement the explicit step order, logging batch identity, loss, learning rate, gradient norm, finite checks, and one parameter value before and after the update.
Run a separate validation function with eval() and no gradient tracking, then assert that no optimizer step or training-transform randomness occurs in that path.
Add one simple schedule, log its value every epoch, and deliberately move the schedule call to the wrong boundary so a regression assertion detects the shifted sequence.
Save a general checkpoint containing model, optimizer, scheduler, completed position, controlled random states, configuration, preprocessing, and code/data/split identities.
Compare an uninterrupted two-epoch run with a one-epoch-save-load-resume run on the same ordered data; retain next-loss and parameter differences plus the tolerance.

Observe

The first update matches hand arithmetic, validation cannot mutate parameters, and complete state produces a matching next step while weights-only loading does not promise that continuation.

Done when

A reviewer reproduces the 0.96 update, inspects the learning-rate sequence, proves validation is read-only, and resumes to the same next result within the declared numerical tolerance.

Build today

Train a small image-to-pose or observation-to-action network with a reproducible experiment report.

Evidence to save

DONE when the integrated “Loss functions, optimizers, schedules, and checkpoints” path is observable, cancelable, and leaves the prior baseline reproducible.

Common mistakes

Catch the wrong mental model

Wrong

Calling backward on each batch without clearing the previous gradients.

Better

Clear at the intended accumulation boundary, log gradient norms, and compare one update with hand arithmetic before extending the run.

Wrong

Using validation loss in an optimizer step because the value is already available.

Better

Keep validation in evaluation mode with gradients disabled and no optimizer mutation; use it only for declared selection or stopping decisions.

Wrong

Saving only model weights and calling the file a resumable checkpoint.

Better

Include optimizer, scheduler, training position, controlled random states, configuration, preprocessing, and code/data lineage, then test the next step.

Job connection

How this becomes employable evidence

Own a robot-model training runner that separates optimization from validation, logs gradient and schedule health, saves portable inference weights and full resume state, and proves checkpoint continuation with a deterministic regression case.

Relevant target roles

Robot Learning Deployment / Physical AI Integration Engineer
Robotics Software Engineer — ROS 2 / AMR

Chapter 13 interview drill

Interview questions: Loss functions, optimizers, schedules, and checkpoints

Practise a 60–90 second answer: define the idea, connect it to a physical robot, state assumptions, frames, and units when relevant, then finish with the failure signal or evidence you would inspect.

Primary interview scenario

A resumed run diverges immediately although model weights loaded successfully. Enumerate missing state, train/eval and data-order checks, then design a next-batch equivalence test.

Answer shape: clarify the situation → trace the physical and software path → test the most likely boundaries → name the evidence that would confirm the result.

Technical follow-up questions

Q1Why is a training loss not automatically the deployment metric?

Model interview answer

It is chosen to provide an optimization signal and may use normalized or squared units; deployment also needs physical error, tail failures, latency, rejections, and task outcomes.

Q2What three core operations normally occur around backward in one optimizer step?

Model interview answer

Clear old gradients, compute backward on the current scalar loss, then let the optimizer update parameters; the exact clearing boundary changes for deliberate accumulation.

Q3What is the strongest simple proof that a checkpoint can resume training?

Model interview answer

On fixed data and environment, its next learning rate, loss, and updated parameters match an uninterrupted reference within a declared tolerance.