Chapter 08 · Map, localize, navigate, and recover with Nav2
Today in the field story
One problem, then the next
Freeze Medicine-Cart Route Trial scenarios before calculating success, time, path efficiency, clearance, final error, recovery, intervention, and failure taxonomy. Keep denominators and difficult cases. Compare estimator covariance only as its internal confidence claim and use time-aligned Gazebo truth for measured error; Chapter 11 will test whether those uncertainty claims are statistically honest.
- Why now
The final demo needs metrics that expose weak conditions rather than reward one lucky route.
- Ignore today
Ignore advanced uncertainty calibration until Chapter 11, but never relabel covariance as error.
- Unlocks next
A scenario-level scorecard and predeclared release gates.
Understand
Build the physical picture first
A navigation scorecard is a school report for many fixed journeys: it shows which conditions were attempted, what failed, how long recovery took, and whether help was needed.
Start evaluation by freezing scenarios, not by choosing a flattering metric. Record world version, map and parameter versions, start and goal poses, obstacle schedule, random seed where applicable, simulation speed, and acceptance rules. Include ordinary, tight-clearance, temporary-blockage, permanent-blockage, cancellation, bad-initial-pose, and stale-data cases. A result without the number and type of attempted trials cannot support a reliability claim.
Task success needs a precise terminal definition. Autonomous success may require the correct action result, final pose within tolerance, no collision, no manual intervention, and completion before a timeout. Report successes over total attempts and split the result by scenario. A human dragging the robot icon to a better pose or clearing a costmap can be useful diagnosis, but that trial is assisted and must not be counted as autonomous success.
Path and time metrics explain efficiency, not safety by themselves. Record planned and executed path length, path efficiency against a declared reference, planning time, total goal time, time stopped, and recovery time. Use medians and percentiles or worst cases rather than only the mean, because one very slow recovery may matter operationally. Minimum clearance and collision-monitor events help explain whether a fast path consumed safety margin.
Localization needs its own evidence. AMCL covariance describes the estimator's uncertainty belief; it is not measured pose error. In simulation, compare estimated pose with ground-truth pose only after transforming them into the same frame and time. On hardware, use surveyed landmarks, fiducials, motion-capture, total station, or another documented reference appropriate to the site. Report position and orientation error, loss events, relocalization time, and cases where confidence was small but error was large.
Recovery metrics preserve the failures inside successful tasks. Count failure events, recovery attempts, distinct recovery types, resolved failures, time to detection, time to resume, retry count, operator interventions, and terminal aborts. Group failures by actionable cause such as localization, transform timing, map/costmap, planner, controller, command chain, or scenario design. Do not delete warm-up or failed bags; mark exclusions before analysis and keep representative traces.
Words you need
Name each idea precisely
- Scenario
A fully described test condition with fixed setup, event schedule, and expected outcome.
Physical example:Start at loading bay A, send goal shelf B, place a pallet at 8 s, and require reroute within 3 s.
- Autonomous success rate
The number of trials meeting every autonomous acceptance rule divided by all relevant attempted trials.
Physical example:15 unassisted passes out of 20 frozen trials gives 75%, with scenario breakdown still required.
- Localization error
The difference between estimated pose and a documented reference pose in the same frame and time.
Physical example:AMCL is 0.18 m from simulator ground truth after both are transformed to
mapat the same timestamp.- Recovery rate
Resolved eligible failure events divided by the failure events for which recovery was attempted.
Physical example:Six of eight obstruction recoveries resume the mission, so the observed recovery rate is 75%.
- Failure taxonomy
A set of causal, actionable categories used to group failed trials.
Physical example:Three failures are tagged transform timeout, two controller infeasibility, and one test-fixture error.
Math, one line at a time
Work through today’s relationship
Prerequisite rescue · optionalOccupancy probability and path cost
Navigation converts uncertain map cells into a collision-aware route.
- p(occupied)
- belief that a map cell contains an obstacleUnit: probability from 0 to 1
- g(n)
- cost already travelled to cell nUnit: cost or metres
- h(n)
- estimated remaining costUnit: same as g
For an A* node, suppose g = 4 m and admissible h = 3 m.
Total priority f = g + h = 7 m.
The planner compares f values, but the final path must also clear the inflated robot footprint.
It resembles shortest-path routing in a network, but each node represents physical space and the robot has width.
What is f when g = 2.5 m and h = 1.5 m?
f = 4.0 m.
Success rate is
For the three paths, mean efficiency is
Calculate a navigation trial scorecard without hiding hard cases
A suite contains 20 trials: 8 nominal, 8 temporarily blocked, and 4 bad-initial-pose cases. Results are 8, 6, and 1 autonomous successes respectively. Eight recovery attempts occur and six resolve the triggering failure.
Add autonomous successes: 8 + 6 + 1 = 15.
Divide by every attempted trial: 15 / 20 = 0.75, so overall autonomous success is 75%.
Keep scenario rates visible: nominal 8/8 = 100%, blocked 6/8 = 75%, and bad-initial-pose 1/4 = 25%.
Calculate observed recovery rate using recovery events: 6 / 8 = 75%; do not use 6 / 20 because most trials did not attempt recovery.
Report the two unresolved recovery events and all five unsuccessful trials in the failure taxonomy; one trial may contain more than one event, so event and trial counts must not be mixed.
State the decision: the 100% nominal result does not compensate for 25% bad-initial-pose success if relocalization is a required operating condition.
The suite has 75% overall autonomous success and 75% observed recovery success, but the scenario split exposes a severe initial-localization weakness.
Totals show scale; stratified scenarios show where the system is actually weak.
Physical examples
Where this appears in real life
Two delivery routes
Route A is shorter but clips the inflation boundary and fails twice; Route B is longer and completes every fixed trial without intervention.
Shortest path length is not the same as best operating outcome; reliability, clearance, and recovery belong beside distance.
Class test with hidden missing questions
A student reports nine correct answers but omits that the test had twenty questions and skips the hardest section.
A success count without denominator and scenario coverage hides reliability just as badly.
Hands-on exercise
Make the idea observable
Create a small scenario manifest and runner around the Gazebo/Nav2 launch. Keep raw bags, structured results, map/config hashes, and unedited outcome labels.
Define at least ten trials across nominal, narrow passage, temporary obstruction, permanent obstruction, cancellation, and incorrect initial pose, with pass rules written before execution.
Record goal identity, start/goal poses, scenario event times, action result, intervention, collision or near-boundary event, and final pose for every trial.
Calculate planned and executed path length, planner time, total time, recovery time, final pose error, minimum clearance proxy, and transform or command freshness where available.
Separate autonomous success, assisted completion, safe failure, unsafe or invalid outcome, and test-infrastructure failure; never silently rerun and replace an unfavorable row.
Produce scenario-level rates plus median and worst-case timing, then link at least one nominal and one failure row to their bag and event timeline.
Change one navigation parameter only after the baseline report, rerun the same manifest, and explain both improvements and regressions.
The dataset should retain denominators, difficult cases, interventions, failure events, and evidence links so another engineer can challenge the conclusion.
Every manifest row has a reproducible result and linked evidence, aggregate calculations match raw counts, and the report names the largest unresolved navigation risk.
Build today
Launch one pinned Gazebo, SLAM Toolbox, AMCL, and Nav2 stack; survive cancellation, obstruction, stale localization, and bounded recovery; then freeze the world, seed, bag, map, graph, configuration, transform snapshot, scenarios, and raw results.
Evidence to save
DONE when a 60–120 second uncut “Navigation metrics: success, path, time, localization error, and recovery” demo links to its command, logs or plots, result count, and honest failure note.
Common mistakes
Catch the wrong mental model
Reporting only successful runs after rerunning failed scenarios.
Keep every attempted frozen trial, mark infrastructure exclusions explicitly, and show scenario denominators.
Using AMCL covariance as measured localization error.
Treat covariance as the estimator's uncertainty claim and compare pose against an independent, time-aligned reference to measure error.
Averaging all route times without showing failures or tail latency.
Report success state, scenario split, median, percentile or worst case, recovery time, and exclusions together.
Counting a human-assisted completion as autonomous success.
Record assistance as an intervention and keep autonomous, assisted, and failed outcomes separate.
Job connection
How this becomes employable evidence
Build the release dashboard and regression manifest for an AMR navigation change, preserving scenario coverage, intervention flags, localization and recovery evidence, and drill-down bags for field failures.
Relevant target roles
- Robotics Deployment, Integration & Validation Engineer
- Robotics Software Engineer — ROS 2 / AMR
- Robot Fleet Backend / Platform Engineer
- Robot HMI / Control & Monitoring Engineer
Chapter 08 interview drill
Interview questions: Navigation metrics: success, path, time, localization error, and recovery
Practise a 60–90 second answer: define the idea, connect it to a physical robot, state assumptions, frames, and units when relevant, then finish with the failure signal or evidence you would inspect.
Primary interview scenario
A team says Nav2 is 95% successful. List the denominator, scenario, assistance, localization, path, timing, recovery, safety, and evidence questions you need before accepting that claim.
Answer shape: clarify the situation → trace the physical and software path → test the most likely boundaries → name the evidence that would confirm the result.
Technical follow-up questions
Q1Why is 8 successful runs not enough to state a success rate?
The total number of attempts, scenario mix, pass definition, and intervention policy are missing.
Q2What is the difference between AMCL covariance and localization error?
Covariance is the estimator's belief about uncertainty; localization error is measured against an independent pose reference in the same frame and time.
Q3Why should recovery events stay visible inside successful trials?
They reveal fragility, delay, retry cost, and failure types that a final success label would otherwise hide.