Chapter 20 · Build a bounded language-to-ROS 2 task executor

Today in the field story

One problem, then the next

A place request times out after the controller accepted it, leaving Courier-17 unsure whether the card moved. Repeating the action could cause a second grasp or collision. You retain the operation identity, reconcile controller and observed state, and choose compensation, safe abort, or human handoff according to the specific skill. Cancellation acceptance and stopped motion remain separate evidence. The mission packet now includes what was requested, what is known, what remains uncertain, and exactly which decision a human must make.

Why now: Outcome uncertainty is where naive retries turn a recoverable fault into a new hazard.
Ignore today: Ignore generic rollback promises and automatic retries without reconciliation.
Unlocks next: Idempotent requests, bounded compensation, and safe handoff for ambiguous outcomes.

Understand

Build the physical picture first

When an action's outcome is uncertain, freeze new effects, reconcile what happened, then choose exactly one safe retry, compensation, handoff, or abort path.

A timeout means the orchestrator's waiting bound expired; it does not prove the robot did nothing or that the skill failed. Preserve request and goal IDs, stop issuing dependent or conflicting work, request cancellation when defined, and query action status plus fresh world state. Classify the outcome as confirmed success, confirmed no effect, confirmed partial or wrong effect, still active, or unknown. Recovery begins from that classification, not from the assumption most convenient for automation.

Idempotency is operation-specific. “Set indicator off” can produce the same final state when repeated, while “move forward 10 cm,” “toggle gripper,” and “dispense one item” can duplicate an effect. A request ID and execution ledger prevent a server from starting the same logical request twice, but they do not make an inherently relative physical operation safe. Re-observe current state and express desired-state commands where possible.

Rollback in the physical world is usually compensation, not time reversal. Releasing a software reservation, returning an unused tool, or parking after a failed mission may be safe compensating steps; placing a dropped fragile object back or reversing through an occupied aisle may not be. Each compensation has its own preconditions, hazards, timeout, result, and postcondition. Never build a generic “undo” tool that the planner may invoke without operation-specific review.

Human handoff is a designed terminal workflow, not an error message. First place the robot in the declared safe or paused state and resolve active command authority. Then present goal, plan and skill versions, current verified state and its age, last successful effect, active or uncertain goal IDs, failure and attempts, hazards, remaining budget, camera or log evidence, and permitted operator actions. Require an explicit owner and resume decision; stale automation must not restart merely because a network connection returns.

Words you need

Name each idea precisely

Outcome-unknown: A state in which the orchestrator cannot yet determine whether a requested physical effect occurred, did not occur, or remains active.
Physical example:
The place action connection times out after release, and neither action status nor the first camera view can locate the foam block.
Idempotency key: A caller-chosen identity for one logical request that lets the receiver return or reconcile the same execution instead of starting a duplicate.
Physical example:
Two network retries with request R-72 map to one set_dock_latch(open) attempt and one stored outcome.
Reconciliation: Checking execution records, active-goal status, and fresh physical state to determine what occurred before another effect is allowed.
Physical example:
After a timeout, the executor checks goal G9, gripper state, and the target bin before deciding whether place already completed.
Compensation: A separately validated action intended to reduce or repair a prior effect when literal rollback is impossible or unsafe.
Physical example:
A failed delivery releases the reserved station and returns the cart to a named holding point if the route and payload state are verified.
Handoff package: The concise state, evidence, authority, hazard, attempt, and resume context transferred to a named human operator.
Physical example:
The console shows that goal G9 is canceled, block B-17 is not visible, the arm is parked, one attempt remains forbidden, and inspection is required.

Math, one line at a time

Work through today’s relationship

Prerequisite rescue · optionalTask graphs, timeouts, and retries

An autonomous task is a state machine with measurable guards, not one long prompt.

t_deadline: latest allowed completion timeUnit: seconds (s)
N_retry: maximum retry countUnit: attempts
P(success): observed success frequencyUnit: probability

A grasp skill gets a 5 s timeout and at most 2 retries.
The maximum planned attempt time is 3×5 = 15 s, excluding recovery.
After each failure, re-observe and check a typed precondition before retrying; do not replay a stale command.

Programmer analogy

It resembles a workflow engine with typed APIs, except retries require fresh perception of a changed world.

One initial try plus three retries, each capped at 4 s, permits how much attempt time?

4 attempts × 4 s = 16 s.

An idempotent operation satisfies

f(f(x))=f(x).

For example, applying set OFF twice leaves the same OFF state. A toggle operation is not idempotent because applying it twice changes state twice.

Reconcile a placement timeout without duplicating motion

The task is to place foam block B-17 in bin C. Request R72 maps to ROS goal G9. The network times out after feedback says release_started; the gripper camera is temporarily unavailable, and one bin camera remains online.

Enter OUTCOME_UNKNOWN, freeze the dependent park-and-finish transition, and prohibit a second place request. Preserve R72, G9, plan P12, input snapshot S61, last feedback, and the exact timeout clock.
Ask the action server for G9 status and request cancellation if it is still active. Treat an accepted cancel request as canceling, then wait for canceled, aborted, succeeded, or the handoff reconciliation deadline.
Acquire fresh evidence: gripper opening, arm pose, bin-C view, source-tray view, and scene version. Suppose G9 reports succeeded and B-17 is clearly observed inside C while the gripper is empty.
Evaluate the original postcondition against the new snapshot. Because in_zone(B-17, C) and gripper_empty are confirmed with sufficient identity and freshness, close R72 as reconciled success without repeating physical motion.
Consider the alternate branch: if B-17 remains clearly on the source tray and G9 is terminal with no effect, a retry may be eligible only if the place contract, remaining budget, and refreshed preconditions explicitly allow it under the same logical request.
If B-17 is missing or an active goal cannot be resolved by the deadline, command only the preapproved nonconflicting safe state, such as stop and park if verified feasible; release software reservations only through their compensation contracts.
Create a handoff package naming the unresolved object, current safe state, goal status, observations and ages, attempts, forbidden automatic retry, hazards, evidence links, and the operator decisions inspect, abort task, or authorize new plan.

Result

The nominal timeout branch recognizes a completed placement without duplicating it, while no-effect and unknown branches have different bounded rules and transfer unresolved physical uncertainty to a prepared human workflow.

What this proves

Reliable recovery asks “what effect exists now?” before “should I call the function again?” and treats uncertainty as a first-class state.

Physical examples

Where this appears in real life

Timed-out foam-block placement

A simulated place call loses its response just after the gripper opens. Repeating the same trajectory could strike the bin or move a block that is already there.

Look for:

The executor blocks a retry, reconciles the goal and fresh bin/gripper state, accepts an already-satisfied postcondition, or parks and hands off if object state remains unknown.

Jammed drawer handoff

A low-force simulator drawer skill aborts twice with RESISTANCE_LIMIT, and its policy forbids more force or a third automatic attempt.

Look for:

The robot releases force and parks, cancels or terminates the exact goal, presents current drawer and tool state, and requires a named operator to inspect and authorize any resume.

Hands-on exercise

Make the idea observable

Use cards, switches, or a simulator. Include desired-state operations, relative operations, one reversible software reservation, and one physical operation whose outcome can be hidden.

Classify six operations as naturally idempotent, idempotent only with a request ledger, non-idempotent, or unsafe to retry while outcome is unknown; write the physical reason for each.
Add request IDs and a ledger with received, dispatched, active goal, terminal result, postcondition, and compensation state. Replay a duplicate network request and verify it cannot start a second execution.
Inject a timeout before effect, after effect, and during effect. For each, collect action status and fresh physical evidence before selecting a branch.
Write one operation-specific compensation with its own preconditions, limits, result, postcondition, and failure path; reject a generic inverse command.
Define the robot safe or paused handoff state and prove active command authority is resolved before the operator is asked to approach or resume.
Build the handoff card or console view with goal, plan, current state and age, last verified effect, uncertain goals, attempts, remaining budget, hazards, evidence, owner, and allowed decisions.
Run all three timeout positions plus exhausted recovery, and verify each ends in reconciled success, confirmed no effect, safe compensation, declared abort, or owned handoff without an unapproved repeat.

Observe

The same timeout produces different safe decisions depending on whether the effect is absent, complete, partial, active, or unobservable; a retry policy that ignores these states duplicates physical work.

Done when

Duplicate requests map to one execution, every timeout is reconciled before new effects, compensation is operation-specific, and handoff transfers a verified safe state plus enough context for an operator to decide deliberately.

Build today

Build an agent that converts a natural-language goal into inspectable ROS 2 actions and recovers from one failure.

Evidence to save

DONE when a 60–120 second uncut “Timeouts, idempotency, rollback, and human handoff” demo links to its command, logs or plots, result count, and honest failure note.

Common mistakes

Catch the wrong mental model

Wrong

Assuming a timeout means the action did not execute and immediately sending the same physical command.

Better

Enter outcome-unknown, reconcile goal status and fresh world state, and permit a retry only when the effect is confirmed absent and the operation's contract allows it.

Wrong

Calling a request idempotent solely because it carries an idempotency key.

Better

Use the key to deduplicate one logical execution, while separately analyzing whether repeating or resuming the physical effect is safe from current state.

Wrong

Showing “operator assistance required” without first resolving motion, authority, and evidence.

Better

Reach the defined safe or paused state, identify active or uncertain goals, transfer hazards and current observations, name an owner, and require an explicit resume decision.

Job connection

How this becomes employable evidence

Design mission and operator workflows that survive lost acknowledgments without duplicate robot motion, distinguish canceling from canceled, compensate only reversible effects, and transfer unresolved goals to a safe state with complete evidence and authority.

Relevant target roles

Robot Fleet Backend / Platform Engineer
Robot HMI / Control & Monitoring Engineer
Robotics Deployment, Integration & Validation Engineer
Robotics Application / ROS 2 Integration Engineer
Robot Learning Deployment / Physical AI Integration Engineer

Chapter 20 interview drill

Interview questions: Timeouts, idempotency, rollback, and human handoff

Practise a 60–90 second answer: define the idea, connect it to a physical robot, state assumptions, frames, and units when relevant, then finish with the failure signal or evidence you would inspect.

Primary interview scenario

A robot place action times out after the gripper may have opened. Explain outcome uncertainty, goal reconciliation, idempotency limits, compensation, when retry is forbidden, and the exact information and authority needed for human handoff.

Answer shape: clarify the situation → trace the physical and software path → test the most likely boundaries → name the evidence that would confirm the result.

Technical follow-up questions

Q1What does a timeout prove about a physical action?

Model interview answer

Only that the orchestrator's waiting bound expired; the effect may be absent, complete, partial, still active, or unknown and must be reconciled.

Q2Why does an idempotency key not make `move forward 10 cm` safe to repeat?

Model interview answer

The key can prevent duplicate server executions, but the relative motion itself changes physical state each time and remains unsafe if prior completion is uncertain.

Q3What must a human receive before taking ownership of an unresolved task?

Model interview answer

A verified safe or paused state, resolved command authority, goal and plan identity, current state and age, last effect, uncertainty and attempts, hazards, evidence, permitted actions, and an explicit resume or abort decision.