Chapter 26 · Convert verified engineering work into role-matched hiring evidence
Today in the field story
One problem, then the next
The capstone case study follows one request from operator or API through model, validator, ROS 2 boundary, controller or simulator, observed state, and terminal evidence. You defend one architectural tradeoff and reconstruct one incident as fact, inference, and unknown. The correction must target the evidenced mechanism, and verification must replay the failure plus neighboring timeout, cancellation, restart, concurrency, or rollback risks. Residual hardware, scale, clock, and site uncertainties remain explicit instead of disappearing behind a polished diagram.
- Why now
Architecture and incident reasoning reveal senior engineering judgment more clearly than feature lists.
- Ignore today
Do not invent root cause, production scale, or real-hardware verification.
- Unlocks next
A defensible system-design and debugging interview story.
Understand
Build the physical picture first
A case study is a cutaway machine beside its repair log: show where information moved, where it failed, what proved the cause, and what risk remains.
Start the architecture story with a system boundary, not a cloud of boxes. Name the user or operator, robot or simulator, edge process, ROS 2 graph, service boundary, storage, external dependency, and the authority that may issue or cancel motion. For every arrow, write the payload or action, direction, protocol, ownership, freshness rule, failure result, and observability point. A box labeled backend hides the questions an interviewer needs answered; POST /missions with mission_id, idempotency rule, accepted state, and cancellation owner makes the boundary inspectable.
A tradeoff is a deliberate choice between competing properties under constraints, not a retrospective defense of whatever was built. Use the form decision | alternatives | constraint | benefit | cost | evidence | revisit trigger. For example, retaining PostgreSQL as the mission source of truth while using Redis for transient coordination may improve recoverability but adds consistency and outage questions. The story must show why that balance fit the tested workload, what measurement informed it, and which condition would force a different design.
Build the incident section from evidence in time order. Separate fact—a trace, row, timestamp, packet, or observed motion—from inference—the current explanation—and unknown—missing proof. “The client timed out” can be factual; “the network caused a duplicate mission” needs correlation across request IDs, retries, writes, and queue events. Root cause is the evidenced mechanism that explains the failure, not the component closest to the symptom. If the actual cause remains uncertain, publish the strongest bounded hypothesis and the decisive experiment still needed.
Verification must challenge the correction through the same failure path plus neighboring risks. Reproduce the defect first, make one causal change, rerun the fixed case, add a regression test, and exercise timeout, cancellation, restart, concurrency, and rollback boundaries relevant to that change. Report before and after denominators without treating a small observed reduction as universal proof. Finish with residual risk: what still depends on real hardware, longer duration, more load, clock behavior, customer configuration, or an unavailable fault. An incident story is stronger when it explains what the fix did not establish.
Words you need
Name each idea precisely
- System boundary
The declared line around components owned by the project, including the interfaces through which actors, devices, services, and dependencies cross.
Physical example:A teleoperation boundary separates browser intent, authenticated gateway, ROS 2 action server, local command guard, controller, and measured wheel motion.
- Interface contract
The payload, units, direction, timing, state, error, ownership, and compatibility rules that both sides of a connection must obey.
Physical example:A velocity request carries metres per second, frame, sequence, expiry, lease identity, and a terminal rejection reason.
- Tradeoff
A choice that improves one desired property while accepting a named cost or constraint, supported by evidence and a revisit condition.
Physical example:A shorter command lease reduces stale-motion exposure but requires more frequent renewals and becomes more sensitive to network jitter.
- Incident
An observed event that degraded or threatened the expected system outcome and deserves a reconstructable timeline, impact, containment, and follow-up.
Physical example:A mission acknowledgment timed out, the client retried, and two queue entries appeared for one requested delivery.
- Root cause
The evidenced mechanism whose correction prevents or controls the reproduced failure within the tested conditions, distinguished from symptom and speculation.
Physical example:Two requests with one mission identity reached an insert path lacking a uniqueness guard, producing duplicate accepted rows under replay.
- Residual risk
A remaining failure possibility after the selected correction and verification, including conditions not represented by the available test environment.
Physical example:The retry test passes in simulation, but prolonged edge disconnection and controller reboot on commissioned hardware remain unevaluated.
Math, one line at a time
Work through today’s relationship
Prerequisite rescue · optionalEvidence matrices and application funnels
Job readiness is demonstrated by traceable artifacts and measured application feedback.
- I/A
- interviews divided by targeted applicationsUnit: percent
- coverage
- job requirements backed by evidenceUnit: percent
- N
- batch size before changing one application variableUnit: applications
A role lists 10 important requirements and your projects provide evidence for 7.
Evidence coverage is 7/10 = 70%; name the three gaps honestly.
If 3 of 20 targeted applications reach interview, conversion is 15%; change one positioning variable for the next comparable batch.
Treat the résumé like an API response: every claim should resolve to a repository, diagram, metric, test, video, or incident analysis.
Four interviews from 25 targeted applications gives what conversion?
4/25 = 0.16 = 16%.
Before the fix, incident rate is ; after the fix it is . The observed change is
Different denominators and conditions mean the trace-backed cause and frozen verification still matter.
Write a trace-backed duplicate-mission incident case
In a hypothetical simulator run, three of twelve retry cases create duplicate mission rows. The service has request logs and database timestamps but no physical robot was connected.
Draw the bounded path
client → mission API → PostgreSQL → dispatcher → simulated robot, then annotate mission identity, request identity, retry timeout, accepted response, queue event, and cancellation ownership.Build the timeline from retained facts: request
r1arrives, rowm-41/acommits, response exceeds the client timeout, requestr2arrives with the same mission identity, and rowm-41/bcommits.List hypotheses separately: duplicate client submission, message redelivery, transaction retry, or missing idempotency guard; do not call any one the cause until evidence distinguishes it.
Replay both requests against the same revision and observe two accepted rows; inspect the schema and write path to confirm there is no unique mission identity or conflict-return behavior.
Add one atomic uniqueness rule and deterministic existing-result response, then rerun the original twelve cases plus concurrent retries, restart, cancellation, and rollback checks.
Report
3/12duplicates before and0/20in the expanded post-fix suite, while stating that the different denominators and simulated environment do not prove all networks, loads, or robot controllers safe.
The case study links architecture, incident timeline, decisive reproduction, bounded correction, regression evidence, and residual hardware and scale risk without blaming an unproven network cause.
The persuasive unit is not the fix alone; it is the unbroken chain from observed symptom through decisive evidence to a correction tested against the failure mechanism.
Physical examples
Where this appears in real life
Duplicate mission after an ambiguous timeout
A simulated fleet client sends mission_id=m-41; the service commits the mission but its reply is delayed, so the client retries and two queue entries appear.
Trace one identity through both requests, database writes, dispatcher events, and acknowledgments. Distinguish the delayed reply from the missing idempotency enforcement that allowed the duplicate state.
Operator display stays green after telemetry expires
A Web or Flutter status card preserves the last idle value when a simulated robot stops publishing, so the screen looks healthy although the sample is five seconds old.
Show timestamp ownership, age calculation, stale threshold, disconnected state, command consequences, and the test proving the corrected UI does not infer fresh physical state from cached data.
Hands-on exercise
Make the idea observable
Choose one incident that actually occurred in a course project or one clearly labeled teaching reconstruction. Use only artifacts you are allowed to publish and preserve the original failure evidence.
Define the system boundary and annotate every important interface with data or command, units, direction, authority, timing rule, failure result, and observability point.
Select three consequential decisions and fill
alternatives | constraint | chosen option | benefit | cost | evidence | revisit triggerwithout describing ordinary implementation as a tradeoff.Reconstruct one incident timeline from logs, traces, state snapshots, trial rows, or video; label every entry
fact,inference, orunknownand redact sensitive content.Reproduce the failure on the preserved baseline or explain the exact blocker, then design the smallest measurement that separates the leading causal hypotheses.
Document the correction and rerun the original case plus relevant boundary, cancellation, restart, concurrency, rollback, and regression checks with exact revisions and denominators.
Publish the impact, containment, evidence-supported cause or remaining hypothesis, verification result, residual risk, and unsupported real-hardware or production conditions.
Architecture prose becomes clearer when every unexplained arrow is treated as an unanswered contract, and incident certainty becomes narrower when facts are forced into a separate column from interpretations.
A reviewer can follow one identity through the architecture and timeline, reproduce or inspect the failure, see why alternatives were rejected, verify the correction, and name at least one remaining risk.
Build today
Publish a robotics portfolio, role-targeted résumé, architecture case study, and a 30-day application sprint.
Evidence to save
DONE when “Capstone architecture, tradeoffs, and incident case study” runs from one documented command and the nominal plus boundary outputs are attached.
Common mistakes
Catch the wrong mental model
Drawing product names and arrows without payload, time, ownership, or failure behavior.
Annotate every consequential interface with contract, authority, freshness, terminal states, compatibility, observation point, and the safe or bounded failure result.
Calling the first plausible explanation the root cause.
List competing hypotheses, identify a discriminating measurement, reproduce the mechanism, and retain uncertainty when the available evidence cannot select one cause.
Ending the incident story with “added retries and it worked.”
Explain why retry is valid for the operation, how duplicate effects are prevented, what timeout and cancellation mean, which cases verified the change, and what retry can still amplify.
Job connection
How this becomes employable evidence
A Robot Fleet Backend / Platform Engineer must defend mission identity, state transitions, ordering, observability, and recovery; a Robotics Application / ROS 2 Integration Engineer must expose protocol and lifecycle boundaries; a Robotics Deployment, Integration & Validation Engineer must make the failure reproducible and acceptance-linked; a Robotics Software Engineer — ROS 2 / AMR must reason about the behavior reaching the robot. One evidence chain lets each role inspect its boundary.
Relevant target roles
- Robot Fleet Backend / Platform Engineer
- Robotics Deployment, Integration & Validation Engineer
- Robotics Application / ROS 2 Integration Engineer
- Robotics Software Engineer — ROS 2 / AMR
Chapter 26 interview drill
Interview questions: Capstone architecture, tradeoffs, and incident case study
Practise a 60–90 second answer: define the idea, connect it to a physical robot, state assumptions, frames, and units when relevant, then finish with the failure signal or evidence you would inspect.
Primary interview scenario
Draw the capstone from operator request to observed robot or simulator outcome, then defend one tradeoff and one incident. Which statements are facts, what evidence isolates the root cause, why is the correction causally appropriate, which regression cases ran, and what residual risk prevents a broader claim?
Answer shape: clarify the situation → trace the physical and software path → test the most likely boundaries → name the evidence that would confirm the result.
Technical follow-up questions
Q1What information turns an architecture arrow into an inspectable interface?
Name the data or command, schema and units, direction, protocol, owner, authority, timing or freshness rule, compatibility, observable state, errors, and bounded failure result.
Q2When may an incident case study call a mechanism the root cause?
When evidence distinguishes it from credible alternatives, the mechanism reproduces the observed failure, and a targeted correction controls that mechanism under the stated test conditions.
Q3Why does `0/20` duplicate missions after a fix not prove universal reliability?
It is evidence for those twenty cases, revision, environment, timing, and load only; hardware behavior, larger samples, different networks, concurrency, and long-duration operation may remain untested.