Your Self-Driving Car Has a Very Old Problem
Photo: Joseph Nash, Public domain, via Wikimedia Commons
Every few months, another panel convenes in Washington to grapple with the 'novel' ethical questions posed by artificial intelligence. Should an autonomous vehicle prioritize its passenger? Who is accountable when an algorithm allocates scarce medical care? Can a machine make a life-or-death call, and if it does, who answers for it?
The framing is always the same: unprecedented, uncharted, a brave new world of moral complexity.
It isn't. And the pretense that it is might be the biggest obstacle to actually governing this stuff.
The Thought Experiment Is the Wrong Tool
The trolley problem — the one where you divert a runaway trolley to kill one person instead of five — was formalized in 1967 by Philippa Foot and became the default lens for autonomous vehicle ethics. It's a fine teaching device. It's also a terrible guide to reality, because it strips out everything that matters: uncertainty, accountability, social context, and consequences that arrive years later.
For a better dataset, don't consult the seminar room. Consult the historical record. Because humans have been making trolley-problem decisions with real bodies for as long as we've had ships, cities, and wars — and unlike philosophers, the people making those calls had to live in the aftermath.
The Sea Has Always Run This Experiment
Maritime tradition had a doctrine called jettison: when a ship is sinking, cargo goes overboard, and sometimes people. Medieval sea law — the Rolls of Oléron, the Laws of Wisby — didn't pretend the dilemma away. It regulated it. Who decides? (The master, in consultation.) What's the liability afterward? (Compensation rules for lost goods.) What happens to the decision-maker? (Sometimes nothing. Sometimes a great deal.)
The point of these codes wasn't to find the 'correct' answer to the dilemma. There isn't one. The point was to establish, in advance, who holds the authority, what procedure they must follow, and who answers afterward. That's the actual ethical technology: not the choice, but the governance around the choice.
Famine Magistrates and Battlefield Commanders
History is dense with officials who had to allocate scarcity. Magistrates managing siege cities rationed grain, deciding in effect who ate and who didn't. Quarantine officials in plague-era Venice decided which ships docked and which turned away — knowing that every decision killed someone either way, just differently. Battlefield triage, formalized in the nineteenth century, sorted the wounded into treatable and not, with a categorization system that is functionally an algorithm: here are the inputs, here is the sorting rule, here is who walks away.
Notice what these people did that we're not doing. They wrote the rules down before the crisis. They argued about the procedure, not just the outcome. And they built in accountability — a name attached to every decision, a record, a reckoning. The worst versions of these systems (and there were plenty) were the ones where the decision was diffuse, unrecorded, and unaccountable. That's not a philosophy insight. That's a pattern across centuries of data.
What We're Actually Getting Wrong About AI
When we treat algorithmic triage or autonomous vehicle behavior as a novel philosophy puzzle, we ask the wrong question: 'what should the machine choose?' But the machine's choice is downstream of choices humans made — what data went in, what objective function was set, who signed off. The real questions are the ones the Rolls of Oléron were built to answer: Who holds authority? What procedure must they follow? Who answers when it goes wrong?
Current US debates over AI governance keep circling back to the machine's dilemma because it's dramatic and photogenic. But a self-driving car's trolley problem is decided years before the crash, in a product requirements document, by a committee, under commercial pressure. The historical lesson is blunt: societies that tolerated unaccountable decision-makers in life-and-death roles — privateers, blockade runners, famine contractors — paid for it in legitimacy. Societies that demanded procedure, documentation, and reckoning did better, slowly and imperfectly.
The trolley problem has no answer. It never did. What it has always had is a governance problem, and governance problems have answers — procedural, boring, essential answers that ship captains and plague magistrates worked out the hard way so we wouldn't have to. We should read their file before we hand the trolley lever to a machine and call the situation unprecedented.
