Chapter 06 · Give the robot an inspectable body, frame tree, and control boundary
Today in the field story
One problem, then the next
Replay an image captured while the Warehouse Arm Identity Audit was moving. Request the camera-to-base transform at the image time, then produce connected, disconnected, past-extrapolation, and future-extrapolation cases. Preserve stamp, source, target, buffer bounds, and timeout; never replace a failed lookup with identity or the latest pose merely to keep the pipeline running.
- Why now
Correct frame names can still produce false geometry when the transform belongs to another instant.
- Ignore today
Ignore clock synchronization across real devices; use one declared simulated or recorded clock.
- Unlocks next
Time-valid sensor plumbing for Gazebo, vision, calibration, and state estimation.
Understand
Build the physical picture first
tf2 is a stack of frame-tree photographs taken over time; a sensor measurement must use the photograph from when the light, force, or motion was actually observed.
A moving robot does not have one timeless transform. A camera image stamped 10.15 s describes what the camera observed at 10.15 s, so transforming its detections with a base pose from 10.00 s mixes two physical moments. tf2 keeps a bounded history of dynamic transforms and looks up the requested relationship at a timestamp. Static transforms do not need a changing history, but every dynamic edge along the requested path must cover the needed time.
Transform failures carry useful meaning. A lookup failure may mean the named frame is unknown. A connectivity failure means both frames exist but no tree path joins them. Extrapolation into the past means the requested time is older than available history; extrapolation into the future means the requested time is newer than the newest required transform. A timeout only says the requested data did not become available within the wait. Catch these errors to report and handle them, not to silently substitute an identity transform.
Using “latest” can be correct for a live status display whose contract explicitly asks for the newest pose. It is usually wrong for old sensor data because the body may have moved since measurement. Clock consistency matters too: all cooperating nodes in simulation must use simulation time, and timestamps from devices or computers need a declared synchronization strategy. At startup, a listener may legitimately have no history yet. Design startup, replay, and clock-reset behaviour rather than classifying every early exception as a software defect.
Diagnosis should preserve four facts: sensor stamp, requested target and source frames, newest and oldest available transform times, and the allowed wait or age. Add motion context because 150 ms has different consequences on a parked platform and a fast arm. Logs such as “transform unavailable” are too weak. A useful failure record says which edge lagged, by how much, what data was dropped or deferred, and whether the system entered a bounded degraded state.
Words you need
Name each idea precisely
- Transform buffer
A time-bounded history of dynamic frame relationships available to a tf2 listener.
Physical example:Recent base poses are kept so a delayed camera detection can be transformed at capture time.
- Measurement timestamp
The time when the underlying physical observation was made or is defined to apply.
Physical example:The camera exposure time in an image header.
- Extrapolation
A request outside the time range tf2 can support for a required edge.
Physical example:Asking for a base pose at 10.15 s when the newest pose is from 10.00 s.
- Connectivity failure
Source and target frames exist but are in disconnected trees.
Physical example:A camera subtree was launched without its fixed connection to base_link.
- Transform timeout
The maximum time a lookup is allowed to wait for the required transform to arrive.
Physical example:A perception node waits 50 ms, then drops a frame instead of blocking forever.
Math, one line at a time
Work through today’s relationship
Prerequisite rescue · optionalTransforms, joint limits, and command interfaces
A robot model must keep geometry, state, and commands consistent.
- q
- joint positionUnit: rad or m
- q̇
- joint velocityUnit: rad/s or m/s
- limit
- allowed minimum or maximumUnit: same as the value
A joint accepts q from −1.0 rad to +1.0 rad.
A planner asks for 1.2 rad, which exceeds the maximum by 0.2 rad.
Reject it by default before the hardware interface. Clip only when a separate, explicit controller contract authorizes clipping inside a validated envelope and records both the proposal and applied command.
Treat hardware interfaces like typed function contracts with validated ranges, except a broken contract can damage a mechanism.
Is q = −0.8 rad valid for limits [−0.5, 1.0] rad?
No. It is 0.3 rad below the minimum.
The time mismatch is
which exceeds the tolerance. Although spatially, that transform is not valid at the requested time.
Quantify one future-extrapolation risk
A lidar scan is stamped 10.15 s. The newest required base transform is stamped 10.00 s. The integration contract allows at most 0.10 s mismatch, and the base may travel at 0.8 m/s.
Compute the time gap: Δt = 10.15 s − 10.00 s = 0.15 s.
Compare 0.15 s with the 0.10 s contract; the gap exceeds the bound by 0.05 s.
Classify the request as future extrapolation for that transform history, not as an unknown-frame error.
Estimate the possible translation during the full gap: 0.8 m/s × 0.15 s = 0.12 m.
Reject or defer the transform according to the data policy; do not silently combine the scan with the old pose.
Record the delayed edge, both stamps, requested frames, wait duration, and chosen degraded behaviour.
A seemingly small 150 ms mismatch can place an obstacle about 12 cm away from its time-correct location at the stated speed.
Timestamp errors become spatial errors, so time validity is part of geometry rather than an optional logging detail.
Physical examples
Where this appears in real life
Moving conveyor camera
A network-delayed image arrives after the conveyor and robot wrist have both moved.
Use exposure time, not processing-arrival time, or the grasp point can shift even though every individual value looks reasonable.
Phone video with delayed subtitles
The words and picture are both correct but refer to different moments, so the combined experience is wrong.
Connect this to a correct camera detection combined with a correct—but newer—robot pose.
Hands-on exercise
Make the idea observable
Use two paper timelines or a tf2 tutorial with a controllable broadcaster. Keep the case small enough that every timestamp is visible.
Draw or publish one static sensor mount and one dynamic base transform at a known rate.
Request a transform at a time inside the available history and record the successful source, target, and returned stamp.
Request a time before the oldest data, after the newest data, and across a disconnected tree; classify each failure separately.
Delay the dynamic publisher and compare sensor age, newest transform age, wait timeout, and data-drop behaviour.
Try a latest-time lookup only for a clearly labelled live-display case, then explain why it cannot repair an old measurement.
Create one structured failure report that contains enough timestamps and frame names for another engineer to reproduce the problem.
The same pair of frame names can succeed or fail depending on time; changing to latest may hide the exception while creating a physically false pose.
You can produce and correctly name connected, disconnected, past-extrapolation, and future-extrapolation cases and show the exact bounded response for each.
Build today
Model a mobile manipulator in URDF/Xacro/SRDF, wire it to ros2_control, and verify frames, controllers, state, and lifecycle in RViz.
Evidence to save
DONE when the integrated “Timestamped data and transform lookup failures” path is observable, cancelable, and leaves the prior baseline reproducible.
Common mistakes
Catch the wrong mental model
Catching every transform exception and returning zero translation and identity rotation.
Preserve the failure class and enter an explicit data-drop, wait, or degraded-state policy; invented geometry is unsafe.
Transforming delayed sensor data with the newest robot pose.
Request the transform at the measurement time unless the product contract explicitly asks for latest state.
Mixing wall time, simulation time, and device time without a clock contract.
Declare the time source and synchronization strategy, then test startup, replay, and clock resets.
Job connection
How this becomes employable evidence
Diagnose a customer-site overlay that drifts only under network load by correlating camera stamps, tf2 buffer coverage, clock source, robot speed, and drop/defer policy.
Relevant target roles
- Robot HMI / Control & Monitoring Engineer
- Robotics Deployment, Integration & Validation Engineer
- Robotics Application / ROS 2 Integration Engineer
- Robotics Software Engineer — ROS 2 / AMR
Chapter 06 interview drill
Interview questions: Timestamped data and transform lookup failures
Practise a 60–90 second answer: define the idea, connect it to a physical robot, state assumptions, frames, and units when relevant, then finish with the failure signal or evidence you would inspect.
Primary interview scenario
Both frames appear in the tree, but lookupTransform throws an extrapolation error. Explain why existence is insufficient, what timestamps you need, and when using latest would be unsafe.
Answer shape: clarify the situation → trace the physical and software path → test the most likely boundaries → name the evidence that would confirm the result.
Technical follow-up questions
Q1Why can a lookup fail even when both frame names exist?
They may be disconnected, or one required dynamic edge may not have data covering the requested time.
Q2What is wrong with using a 10.00 s pose for a 10.15 s image on a moving robot?
It combines different physical moments and can convert time lag into a wrong spatial result.
Q3What four facts make a transform failure report useful?
The source and target frames, measurement/request time, available transform time range, and wait or age policy.