Case Studies in Failure
The five case studies that follow are not merely cautionary tales — they are the evidence base that justifies every engineering principle taught in this course. When you are asked to justify an engineering practice (defence in depth, integration testing, phased rollouts, single points of failure, audit trails), you are expected to name the matching case study and explain the causal mechanism, not just assert the principle in the abstract.
Case Study 1: Therac-25 (1985–1987)
Background. The Therac-25 was a medical linear accelerator for radiation therapy, built by AECL (Atomic Energy of Canada Limited). It was the successor to the Therac-20.
The fatal cost-cutting decision. The Therac-20 had hardware interlocks — physical switches and circuits that prevented the high-power electron beam from firing unless the targeting shield was physically in place. AECL, to reduce cost, removed these hardware interlocks from the Therac-25 and entrusted safety entirely to software.
The bug. The control software used a shared variable to track operator input. An operator might type “X” to select X-ray mode, realise it was a mistake, then quickly press “Up” arrow and “E” to correct it to Electron mode. If this sequence occurred within a specific narrow timing window, a race condition in the software would configure the high-power beam without deploying the physical shield.
The UI failure. When the system detected an anomaly, it displayed a cryptic error — “Malfunction 54” — with no explanation. Operators could simply press “P” (Proceed) to dismiss the error and fire again.
The cost. At least six patients received massive radiation overdoses — up to 100 times the intended dose. Several died from radiation poisoning. AECL initially denied any software fault, attributing the injuries to patient sensitivity.
Engineering lessons:
- Defence in depth — never rely on a single software check for human safety. Hardware interlocks are essential because they fail in predictable, visible ways and cannot be bypassed by a race condition.
- Concurrency is dangerous — shared mutable state in multi-threaded systems must be managed with formal rigour. A race condition that is harmless 99.99% of the time will eventually kill someone.
- UX is safety — error messages must be actionable and hard to blindly dismiss. “Malfunction 54” is not fit for purpose in a life-critical system.
Case Study 2: Mars Climate Orbiter (1999)
Background. A $327 million NASA/JPL probe (with Lockheed Martin as contractor) to study the Martian climate. On 23 September 1999, during Mars orbit insertion, the orbiter’s signal was lost. It had entered the atmosphere too low and disintegrated.
The root cause: a unit mismatch. Lockheed Martin (Colorado) wrote the thruster-impulse software using Imperial units (pound-force-seconds). NASA JPL (California) wrote the navigation software expecting metric units (Newton-seconds). Every thruster firing command was off by a factor of ~4.45 (1 lbf·s ≈ 4.45 N·s). Over the course of the interplanetary journey, this tiny per-burn error accumulated into a 170 km altitude error at Mars — well inside the atmosphere.
Why it was not caught. The Software Interface Specification (SIS) explicitly required metric units. Lockheed Martin did not adhere to it. More critically, the teams tested their components in isolation with no end-to-end integration testing — each subsystem’s own tests passed, but no one ran a joint test that would have exposed the unit mismatch.
Engineering lessons:
- Type safety — strongly typed languages with typed units (e.g.
newtonsvspound_secondsas distinct types that cannot be implicitly converted) can catch such errors at compile time. Where typed units are not available, explicit naming conventions and code review are essential. - Integration testing is not optional — isolated component testing proves nothing about the system. The boundaries between teams and organisations are precisely where failures propagate.
- Process adherence over blame — this was a systemic process breakdown (the contractor ignored the interface specification), not one programmer’s arithmetic error. The engineering process failed to enforce the agreed contract.
Case Study 3: London Ambulance Service CAD (1992)
Background. In October 1992, a new Computer-Aided Dispatch (CAD) system was rolled out to automate ambulance dispatch across London. The previous system had operated with paper and radio.
The doomed procurement. The contract was awarded to a company with no prior experience building emergency dispatch systems. The company won by significantly underbidding. Timelines were compressed by political pressure to show results.
The hostile user base. Ambulance locations were tracked via imperfect radio triangulation. Paramedics felt micromanaged and distrusted the system. Because they had received minimal training, they often pressed incorrect status buttons on their in-vehicle terminals — feeding the CAD system corrupt availability data (ambulances marked “available” when they were not, “busy” when they were free).
The system overload. A memory leak in the server caused progressive slowdown over the course of the shift. As response times grew, citizens re-called repeatedly, flooding the system with duplicate call records. The CAD began dispatching multiple ambulances to the same address while other genuine emergencies were ignored.
The collapse. Within 36 hours of launch, the system locked up entirely. The control room reverted to pen and paper. Patients died waiting for ambulances that never arrived.
Engineering lessons:
- This is a socio-technical system — software does not exist in isolation from human psychology, organisational culture, and hardware. The CAD did not fail purely for technical reasons; it failed because of a combination of procurement failure, user distrust, inadequate training, unrealistic timelines, and insufficient testing.
- Phased rollouts — never do a “Big Bang” deployment for a critical system. The CAD should have been introduced in stages: one borough at a time, running in parallel with paper dispatch with a manual fallback.
- Load testing — test under extreme, anomalous, bursty load, not just the happy path of normal evening traffic. The memory leak that ultimately destroyed the system would have been detected under a sustained stress test.
Case Study 4: Post Office Horizon Scandal (from 1999)
Background. The Horizon IT system, built by Fujitsu for the UK Post Office, automated accounting for thousands of local post office branches. Subpostmasters began reporting inexplicable financial shortfalls. The Post Office insisted that Horizon was “robust” and “effectively infallible,” and prosecuted subpostmasters for theft and false accounting.
What was really happening. The Horizon system had multiple critical bugs that could silently create, modify, or delete financial transaction records. Fujitsu’s own developers were aware of some of these bugs. Fujitsu engineers could remotely access branch terminals and alter data — including without the subpostmaster’s knowledge.
The cost. Over 700 subpostmasters were prosecuted. Many were imprisoned, went bankrupt, or lost their homes. Several took their own lives. This was the largest miscarriage of justice in British legal history.
Engineering lessons:
- Audit trails — in any financial or legal system, every automated action must be transparent, auditable, and attributable to a specific human actor or process. The ability for engineers to remotely alter data with no visible trace is indefensible.
- Engineering integrity — developers who identified bugs were ignored or overruled by management. Technical truth must override business convenience or reputational defence. Engineers have a professional obligation to escalate concerns beyond their immediate chain of command if necessary.
- Presumption of infallibility — the law at the time effectively presumed computer evidence to be correct unless proven otherwise. Engineers must challenge this “myth of infallibility” wherever it surfaces. Computers are no more inherently reliable than the humans who programmed them — less so, in many cases, because humans can exercise judgement.
Case Study 5: Boeing 737 MAX — MCAS (2018–2019)
Background. Boeing needed to fit larger, more fuel-efficient engines to the 737 airframe. To achieve the necessary ground clearance, the engines were mounted further forward and higher on the wing, altering the aircraft’s aerodynamics. The new configuration was prone to pitching up under certain conditions, risking a stall.
The “fix”: MCAS. The Manoeuvring Characteristics Augmentation System (MCAS) was a software system that, on detecting a high angle of attack, would automatically push the nose down to prevent a stall. Critically, MCAS relied on data from a single Angle-of-Attack (AoA) sensor — a single point of failure.
The failure. If that one sensor failed (as it did on two flights — Lion Air 610 and Ethiopian Airlines 302), the system would read an erroneously high AoA and repeatedly command nose-down trim. Pilots, unaware of MCAS’s existence (Boeing had deliberately “hidden” it to avoid costly pilot retraining certification), fought against the system but could not overcome it. 346 people died.
Engineering lessons:
- Single point of failure — never design a critical safety system that relies on data from only one sensor. Use redundancy (at least three sensors with voting logic) for any input that can command irreversible physical action.
- Complexity hiding — software must not act autonomously without the operator’s knowledge. Pilots must know what systems exist, what inputs they depend on, and — critically — how to disable them.
- The “software fix” trap — do not use software to patch a fundamental hardware or aerodynamic design flaw. If the airframe is aerodynamically unstable without software intervention, the risk profile is radically different from a conventionally stable airframe. Software should augment safety, not be the sole safety mechanism.
Why These Case Studies Matter for the Exam
These are not trivia. Each illustrates a distinct engineering failure mechanism:
| Case study | Primary failure mode | Engineering principle |
|---|---|---|
| Therac-25 | Race condition; no hardware interlock | Defence in depth; concurrency |
| Mars Climate Orbiter | Unit mismatch; no integration testing | Type safety; integration testing |
| London Ambulance | Socio-technical; Big Bang deployment | Phased rollout; load testing |
| Post Office Horizon | No audit trail; ignored engineers | Audit trails; professional integrity |
| Boeing 737 MAX | Single AoA sensor; hidden complexity | Redundancy; operator transparency |
When asked to justify an engineering principle, name the case study and explain the causal chain — this grounds your answer in evidence and is rewarded by examiners.
Past Paper Questions
These case studies are directly referenced in Example Sheet 1, Question 1 (the health system vs the to-do app — how risk profiles differ) and inform every scenario-based Tripos question where safety, testing, or process reliability is at stake.