Between 1985 and 1987, a radiation therapy machine called the Therac-25 delivered massive overdoses to at least six patients in the United States and Canada. Some of them died. The machine's manufacturer, Atomic Energy of Canada Limited, had built earlier models with hardware interlocks: physical mechanisms that made certain dangerous states impossible no matter what the software believed. The Therac-25 removed most of those interlocks and gave their job to software. The software had a race condition, a bug whose appearance depended on timing, in this case on how quickly a skilled operator typed. Fast, experienced operators were the ones most likely to trigger it. The machine would display a cryptic malfunction code, sometimes reporting that no dose had been delivered, when in fact the electron beam had fired at full power without the safeguards that made it a treatment instead of a weapon.
Read that again, slowly, because it inverts everything intuition says about safety. The safer machine was the older one, not because its software was better, but because its designers did not ask software to be the only thing standing between a patient and a lethal state. Hardware interlocks are stupid and gloriously incorruptible. A physical switch cannot be raced. It cannot overflow. It does not care how fast the operator types. When the interlocks were removed, the system did not become more modern. It became more confident, which is a different thing, and in safety engineering confidence is not a control.
The investigations, most famously the analysis by Nancy Leveson and Clark Turner published in 1993, found layers beneath the bug. The software was reused and adapted from earlier machines whose hardware had silently covered for its flaws, so the code arrived with a reputation it had not earned on its own. There was no independent verification of the safety-critical code. Incident reports from hospitals were not investigated with the urgency they deserved; the manufacturer initially could not reproduce the failures and suggested operator error, a suggestion that looks, in hindsight, like the most dangerous sentence in the whole story. The failure was not one programmer's mistake. It was a system that had arranged, decision by decision, to have exactly one layer between a timing bug and a human body, and then declined to believe the layer had failed.
Every field believes it has moved past Therac-25, and every field is running its own version of the experiment. The pattern is always available: a proven safeguard that looks redundant, a rewrite that removes it in the name of elegance or cost, reused code whose reputation was earned inside a cocoon that no longer exists, and early warnings explained away as user error because the system, by design, cannot be wrong. In our world of agents and automation, the translation writes itself. A human review step that slows the pipeline is a hardware interlock. An approval gate that feels like friction is a hardware interlock. A rule that no worker promotes its own result is a hardware interlock. The pressure to remove them will always arrive dressed as efficiency, and it will always be partly right: interlocks do cost speed. What they buy is the property that a whole class of software beliefs cannot become physical facts.
Our factory keeps certain gates deliberately un-automated, and this essay is the reason they will stay that way. Machines measure, the ear decides. No swarm promotes its own result. Irreversible actions wait for a named human. These are not nostalgia. They are the physical switches we refuse to let software talk us out of.
The patients in this story trusted a machine that had been designed to deserve trust and then redesigned in a way that spent it. The least the rest of us can do is learn the exchange rate: a safeguard removed is a bet placed, and the bet is always larger than it looks, because it is placed against every future bug at once, including the ones nobody has written yet.
Sources, for the record: Leveson and Turner's "An Investigation of the Therac-25 Accidents," IEEE Computer, July 1993, remains the canonical analysis, documenting the race condition, the removed hardware interlocks, the reuse of software from the Therac-20, and the accident sequence. FDA records and subsequent safety literature corroborate the six known massive overdose accidents between 1985 and 1987.