Chapter 05 · Build and debug a real ROS 2 system
Today in the field story
One problem, then the next
The operator asks the Aisle Seven robot to inspect a shelf, a task too long for a blocking service. Model it as an action with goal identity, feedback, timeout, terminal result, and cancellation handshake. Trace cancel request, acceptance, command inhibition, observed simulated stop, and final canceled state; keep uncertain retries idempotent rather than duplicating physical intent.
- Why now
Long-running robot work needs explicit progress and terminal semantics before the watchdog can supervise it.
- Ignore today
Ignore navigation-specific action fields; use a harmless numeric task.
- Unlocks next
Cancelable missions for Nav2, manipulation, and fleet orchestration.
Understand
Build the physical picture first
Use a notice board for an ongoing stream, an information desk for one quick question, and a tracked delivery job for work that takes time. The tracked job needs an identifier, progress reports, a final outcome, and a real cancellation handshake.
Topics, services, and actions represent different lifetimes. A topic is for ongoing asynchronous data. A service is a named request-response interaction expected to finish quickly: ask for calibration status, reset a harmless simulation counter, or calculate a small result. A client waits for a server response, so a slow or missing service can tie up the calling logic. Timeouts, availability checks, and idempotency still matter; the word 'service' does not make network calls reliable or side effects safe.
An action represents long-running work that needs a goal, periodic feedback, a terminal result, and cancellation or preemption. Navigation, docking, arm motion, and a multi-second perception scan are common examples. Each accepted goal has an identity and state. The server may reject a goal before execution, report progress while running, and finish as succeeded, canceled, or aborted. A result should state task outcome, not merely that a function returned.
Requesting cancellation is not the same as stopping physical motion. The action server must receive and accept the cancellation request, the execution loop must notice it, the trusted controller must bring the simulated or physical system to its declared safe state, and the action must report a terminal canceled result. Measure time from cancel request to quiet/safe output. For real hardware, an independent safety layer remains necessary because ROS action cancellation is an application protocol, not an emergency stop.
A service used for a long robot task creates poor behavior: no natural feedback, awkward client timeouts, and no standard goal cancellation. A topic used as an improvised request can create duplicate or uncorrelated commands unless IDs, ownership, acknowledgement, and expiry are invented. Choose the interface from semantics first. Then define retry rules, unique goal/request IDs where needed, deadlines, terminal states, and whether repeating a request is safe.
Client code must remain responsive while work is in progress. Prefer asynchronous calls inside callbacks. Do not block an executor thread waiting for a response that requires another callback on the same blocked execution path; Day 33 will reproduce that deadlock. Even with the right interface type, the system needs bounded waits and observable transitions for server unavailable, goal rejected, feedback missing, cancel rejected, timeout, abort, and late result.
Words you need
Name each idea precisely
- Service
A short request-response interface between a client and a server.
Physical example:Ask a localization node whether its map is loaded and receive one status response.
- Action
A long-running goal interface with feedback, terminal result, and cancellation support.
Physical example:Request a rover to move two metres while receiving distance-remaining feedback.
- Goal handle
The client's/server's reference to one accepted or rejected action goal and its state.
Physical example:Two simultaneous docking requests must not share one anonymous progress record.
- Feedback
Intermediate progress data emitted while an accepted action is executing.
Physical example:A docking action reports current phase and remaining distance.
- Terminal result
The final outcome associated with a goal, such as succeeded, canceled, or aborted plus task data.
Physical example:The robot reports aborted because the dock marker was lost, rather than simply 'finished'.
- Idempotency
The property that repeating a request has the same intended effect as performing it once.
Physical example:Retrying 'set light to off' is naturally safer than retrying 'toggle light'.
Math, one line at a time
Work through today’s relationship
Prerequisite rescue · optionalMessage rates, queues, and latency
ROS nodes form a distributed timing system; rates and queue depth decide freshness.
- λ
- messages arriving each secondUnit: messages/s
- μ
- messages processed each secondUnit: messages/s
- latency
- receive time minus source timestampUnit: milliseconds (ms)
A camera publishes λ = 30 messages/s while a node processes μ = 20 messages/s.
The backlog grows by λ − μ = 10 messages each second.
A depth-5 queue fills in about 0.5 s; choose a QoS policy based on whether freshness or completeness matters.
ROS pub/sub resembles backend messaging, but an old robot message can command the wrong physical state.
Input is 50 Hz and processing is 40 Hz. How fast does backlog grow?
10 messages per second.
Cancellation latency is
At , the feedback period is , so stopping takes about feedback periods. That timing measurement alone does not guarantee safety.
Trace a cancellation from request to terminal state
A simulated move action publishes feedback at 20 Hz. The client requests cancellation at 1.200 s, the server accepts it at 1.225 s, simulated velocity reaches zero at 1.280 s, and the canceled result arrives at 1.310 s.
Calculate feedback period: T = 1/20 = 0.05 s = 50 ms.
Calculate cancel-request-to-acceptance latency: 1.225 - 1.200 = 25 ms.
Calculate cancel-request-to-zero-output latency: 1.280 - 1.200 = 80 ms.
Express the zero-output latency in feedback periods: 80/50 = 1.6 periods.
Calculate cancel-request-to-terminal-result latency: 1.310 - 1.200 = 110 ms.
Keep the three measurements separate: protocol acceptance, physical/simulated output stop, and action terminal reporting are different events.
The cancellation was accepted after 25 ms, output became quiet after 80 ms, and the client observed the terminal canceled result after 110 ms.
A cancel button is only an input. Credible cancellation evidence follows the request through acceptance, bounded safe output, and a terminal result.
Physical examples
Where this appears in real life
Status check versus docking
An operator asks whether the charging dock is available, then commands a robot to dock.
Availability is a quick service response; docking is a long action with acceptance, phase feedback, cancellation, and a task-level result.
Printer queue
Reading toner level is immediate, while printing 100 pages takes time and may be canceled after page 12.
A short query and a tracked long job need different contracts even though both involve the same machine.
Hands-on exercise
Make the idea observable
Implement a harmless Python action called SimMove whose goal is a distance, feedback is distance remaining, and result contains final state and distance completed.
Define goal, result, and feedback fields with units and reject zero, negative, non-finite, or over-limit goals.
Implement an action server that advances only a simulated numeric position at a fixed rate and emits timestamped progress.
Implement a client that records goal sent, accepted/rejected, each feedback sample, cancel requested, cancel response, and terminal result.
Run one normal goal to success and verify monotonically decreasing distance remaining.
Run a second goal, request cancellation halfway, check for the request in the server loop, set simulated output to zero, and return a canceled terminal result.
Run server-unavailable, rejected-goal, and feedback-stall cases with explicit timeouts; never connect this exercise to motors.
A responsive execution loop can notice cancellation between updates. A blocking loop or unchecked sleep delays cancellation even though the client sent it immediately.
Logs prove success, rejection, timeout, and cancellation paths; cancellation has measured request-to-zero and request-to-result latency, and no run leaves the simulated command non-zero.
Build today
Build a Python/C++ telemetry, command, and watchdog system; then reproduce QoS, discovery, executor, lifecycle, and cancellation failures.
Evidence to save
DONE when a deterministic “Services versus actions and cancellation semantics” failure test reports expected versus actual behavior and passes after the documented fix.
Common mistakes
Catch the wrong mental model
Implementing a 30-second, preemptible robot motion as a blocking service.
Use an action with feedback, timeout, cancel handling, and a task-level terminal result.
Treating 'cancel request sent' as proof that output stopped.
Observe cancel acceptance, safe/quiet output, bounded latency, and terminal canceled state separately.
Retrying a side-effecting service automatically after an uncertain timeout.
Use an idempotent command or request ID and reconcile the actual server state before retrying.
Returning action success because the execution loop ended.
Evaluate the physical task acceptance criterion and distinguish succeeded, canceled, rejected, and aborted outcomes.
Job connection
How this becomes employable evidence
A teleoperation or mission UI sends a navigation goal and exposes cancel. The engineer must show goal identity, current state, stale feedback, rejection, and final outcome, while ensuring application cancellation cannot be mistaken for an emergency-stop guarantee.
Relevant target roles
- Robot HMI / Control & Monitoring Engineer
- Robotics Deployment, Integration & Validation Engineer
- Robotics Application / ROS 2 Integration Engineer
- Robotics Software Engineer — ROS 2 / AMR
Chapter 05 interview drill
Interview questions: Services versus actions and cancellation semantics
Practise a 60–90 second answer: define the idea, connect it to a physical robot, state assumptions, frames, and units when relevant, then finish with the failure signal or evidence you would inspect.
Primary interview scenario
When would you choose a service rather than an action for robot calibration? Describe the failure and cancellation behavior for both a quick status query and a 30-second calibration motion.
Answer shape: clarify the situation → trace the physical and software path → test the most likely boundaries → name the evidence that would confirm the result.
Technical follow-up questions
Q1Which interface fits a continuous camera stream, a quick map-name query, and a navigation goal?
A topic for camera data, a service for the quick query, and an action for long-running cancelable navigation.
Q2What four observations make an action cancellation credible?
The request was sent, the server accepted it, output reached the declared safe/quiet state within a bound, and the goal reached a canceled terminal result.
Q3Why is a unique goal or request identifier useful after a network timeout?
It lets client and server reconcile whether the original work was accepted or completed instead of accidentally creating duplicate work.