All writings

When Safety Becomes the Danger

What happened in Wuhan on March 31, 2026 is not an accident. It is a systemic failure we have been looking away from.

It is just after nine in the evening in Wuhan. Mr. Lu, 26 years old, livestreamer, ordinary Tuesday commute, is sitting in the back seat of an Apollo Go robotaxi when the car slows and stops. Not at a bus stop. Not at a red light. In the middle of the Third Ring Road, an elevated expressway with no hard shoulder, no sidewalk, no way to leave on foot.

Trucks pass on both sides.

He presses the SOS button. Nothing happens. He calls customer service. The car disconnects the call. He tries again. And again. And again. Twelve attempts over thirty minutes, and he reaches no one.

The screen on the back seat displays a message: "Driving system malfunction. Staff are expected to arrive in 5 minutes."

No one arrived for almost two hours.

What is happening outside is worse. One hundred Apollo Go vehicles, Baidu's self-driving taxi fleet in Wuhan, have stopped simultaneously. Same car, same system, same moment. City traffic collapses. An orange SUV rear-ends a stationary robotaxi. Passengers are trapped on elevated expressways with no way to evacuate themselves. There is no hard shoulder to walk along. There is no human driver to take them anywhere.

Baidu's customer service eventually answers and tells Lu to stay in the vehicle.

That is the only advice they can give.


We have been asking the wrong question

The global debate about autonomous vehicles almost always gets stuck on the same question: are they safer than humans? We compare accident statistics per million miles driven. We read press releases about how many millions of miles have been covered without a fatal crash. We listen to promises that the technology will soon save the 1.35 million people who die in traffic every year.

That is not a bad question. But Wuhan reveals it is the wrong one.

The right question is not how often does a self-driving car crash? The right question is what happens when a hundred self-driving cars stop working at the same second?

This is a fundamentally different category of risk. Traditional vehicles break down randomly and independently of each other. Self-driving fleets are controlled by shared backend systems, common software, common network infrastructure. They can fail collectively, simultaneously, for the same reason, in the same way.

A single car stopping is an obstacle. A hundred cars stopping at once is a traffic system event.

The safety mechanism that creates the danger

To understand why the cars stop, we need a concept that the industry's communications departments never highlight: the Minimal Risk Condition, MRC for short.

The principle is simple and, in theory, sensible. When an autonomous vehicle encounters a situation it cannot handle, when the network drops, when the system loses its bearings, when the software cannot compute a safe path forward, it is supposed to bring the vehicle to a state of minimal risk. The best outcome is that the vehicle navigates to a safe location and parks. The worst, but technically permitted, outcome is to stop in the travel lane with hazard lights on.

That is the mechanism that activated in Wuhan. And this is precisely where engineering in a vacuum collides with reality.

Hannah Pelikan, a researcher at Linköping University who studies human-machine interaction in traffic, identifies the core problem: machines are programmed to stop as soon as they become uncertain, because that is considered the safest option from a technical perspective. But simply expressing uncertainty by stopping can itself create new dangers. Humans operate under time pressure and social obligation. Machines lack that dimension entirely. They have no timeout mode, they stop and wait for a response that may never come.

A car that stops and waits in an empty parking lot is safe. A car that stops and waits on an elevated expressway in rush-hour traffic is a lethal obstacle. MRC is designed for a single vehicle in an isolated crisis, not for a hundred vehicles activating it on the same highway, at the same time, for the same reason.

That is what happened in Wuhan.

The pattern we refuse to see

Wuhan is not an exception. It is the latest point in a pattern where the MRC principle is transformed, again and again, from a protective mechanism into a traffic hazard.

On December 20, 2025, a power outage knocked out electricity across a third of San Francisco. Traffic lights went dark. Waymo's vehicles, programmed to treat dark signals as four-way stops and contact the operations center for confirmation, sent an avalanche of simultaneous requests to a remote team that was not scaled for fleet-wide crises. Dozens of vehicles froze at intersections. Police received reports that Waymo cars were blocking emergency responders at two fires. Waymo suspended service Saturday evening and restarted Sunday. The software update was shipped on Christmas Eve.

On August 11, 2023, the Outside Lands festival drew 200,000 visitors to San Francisco. They overwhelmed the mobile networks. Ten Cruise vehicles lost contact with their backend systems and froze in traffic. Despite each vehicle carrying four SIM cards from four separate carriers, specifically to avoid this kind of situation, the redundancy was not enough. City Supervisor Aaron Peskin put the problem plainly: "It's scary as heck if you think about the fact that moving these vehicles out of traffic requires cell service."

On January 1, 2026, a Zoox vehicle stopped in the middle of a busy Las Vegas intersection. It moved slightly forward and backward but could not complete the crossing.

In San Francisco, from January to June 2023, the fire department documented 39 separate incidents where Cruise and Waymo vehicles blocked emergency responders, driving through police tape, blocking firehouse driveways, running over fire hoses. In one incident on August 14, 2023, a stalled Cruise vehicle blocked an ambulance transporting a critically injured pedestrian. The patient later died.

In every case the underlying pattern is the same: a fleet sharing common infrastructure encounters a common problem, activates MRC collectively, and creates a danger proportional to the size of the fleet, not to any individual vehicle.

The hidden human infrastructure

On March 31, 2026, the exact same day as the Wuhan incident, Senator Ed Markey published the results of his investigation into autonomous vehicle companies and their use of human remote operators.

The findings were remarkable.

All seven major American AV companies investigated acknowledged that their vehicles are, when needed, assisted by human operators working remotely. Not one of those companies was willing to disclose how often that assistance is required. Waymo is the only company using remote operators in another country, the Philippines. Tesla acknowledged that its operators can take direct control of vehicles. No federal standards govern operator qualifications, response times, or offshore staffing.

The senator's comment: "Autonomous vehicle companies make big promises about the safety of their self-driving cars, but it turns out that human workers still play a critical role with this technology."

This is not a detail. It is a fundamental gap between how the technology is marketed and how it actually works. Every time a vehicle encounters a situation it cannot handle, which happens with a frequency no company is willing to disclose, there is a human on the other side of a screen tasked with resolving it. The system is not autonomous. It is a vehicle with a distributed human staff.

And when the hundred vehicles in Wuhan activated MRC simultaneously, that remote team was too small, too slow, and too far away.

The statistics that don't hold up

Waymo publishes the most rigorous safety data in the industry. Their figures are externally reviewed and show genuinely impressive results: 88 percent fewer property damage claims compared to human drivers, 92 percent fewer bodily injury claims, 170 million miles driven without a human behind the wheel.

But there is an asterisk.

Waymo operates in a handful of cities, under strict geographic and weather conditions, with detailed digital maps updated to millimeter precision. The safety data is for that environment, not for elevated expressways without emergency infrastructure in Wuhan, not for power outages that knock out a third of a city's traffic lights, not for music festivals that overwhelm mobile networks. And a study published in Nature Communications found that autonomous vehicles are five times more likely to crash at dawn and dusk, when camera sensors struggle with glare and rapidly shifting shadows, precisely the conditions at the boundary of what the systems can handle.

Tesla's statistics are an entirely different problem. Elon Musk promised full autonomy in 2015, 2017, 2019, 2020, 2021, 2022, 2023, and 2024. A court ruled in 2024 that Musk's predictions constitute legally meaningless corporate puffery. Tesla counts only crashes that trigger airbag deployment, a definition that excludes an estimated 82 percent of all police-reported accidents. Autopilot runs almost exclusively on highways, statistically the safest road type, but is compared against averages across all road types. In March 2026, NHTSA opened an investigation into 3.2 million Tesla vehicles for a specific flaw: the system fails to warn the driver when cameras are blinded by sun glare, fog, or dust, a deficiency linked to at least one fatal pedestrian crash whose report Tesla filed seven months late.

And the fundamental statistical problem remains. In RAND Corporation's statistical scenario, demonstrating a 20 percent lower fatality rate than human drivers, with 95 percent confidence and 80 percent statistical power, would require roughly 11 billion miles driven. No company is anywhere close.

The social problem that is hardest to solve

Beyond the flawed statistics, the technology faces a fundamental barrier in how it understands the world.

Barry Brown, professor at the Department of Computer and Systems Sciences at Stockholm University, describes a structural problem with current AV architecture that is rarely discussed in technical forums.

A self-driving car constructs a simplified picture of the world based on sensor data that ignores enormous amounts of detail from the real, social world. Traffic is not only a physical phenomenon, it is a social system. Drivers negotiate with eye contact. They read each other's body language. They understand when an oncoming driver is about to make a risky decision not because they calculate a trajectory but because they are interpreting a social situation in real time.

Brown's assessment of when the technology will actually be ready: not in two years, as the industry has continuously claimed for a decade. More like 30 to 40 years.

That is not technophobia. It is a calibration against the actual difficulty of the problem.

Jack Stilgoe, Professor of Science and Technology Policy at University College London, adds a dimension that connects directly to Wuhan: there will never be a perfectly safe technology, and it is the edge case, the moment when something new happens, that determines the outcome. Not the average. The edge.

Philip Koopman, professor at Carnegie Mellon and one of the world's leading voices on AV safety, is more direct: there are significant incentives for companies to cut corners on safety, to the detriment of other road users.

Three questions no one is answering

The Wuhan incident is not an argument against autonomous vehicles. It is an argument against how the industry is currently choosing to deploy them.

There are three concrete questions that need to be asked, and answered, before more cities open their roads to autonomous fleets at scale.

  • How is correlated MRC handled? The current answer, stop with hazard lights on, is designed for individual vehicles in isolated situations. It is not a protocol for a hundred simultaneous emergencies on elevated expressways. Regulators and municipalities must require companies to specify their protocols for fleet-wide crises and scale their remote operations accordingly.
  • What is the actual frequency of human intervention? Senator Markey's report shows that not one of seven companies investigated was prepared to answer that question. That is unacceptable. If a vehicle calls itself autonomous but requires human assistance every ten miles, it is not autonomous. It is a vehicle with a distributed human workforce whose scalability has never been publicly tested.
  • In which environments is the technology actually validated? Waymo's figures are impressive in Phoenix and San Francisco under normal conditions. They say nothing about what happens on elevated expressways without evacuation infrastructure, in cities without detailed digital maps, or in countries without regulated operator protocols.

Mr. Lu sat trapped on the Third Ring Road for almost two hours.

That is not an acceptable outcome from a system marketed as a revolution in transport safety. And it is not an exception, it is the MRC principle working exactly as designed, in a reality it was not designed for.

: The RAND mileage estimate has been corrected and its statistical assumptions specified.

Carry forward

What happened in Wuhan on March 31, 2026 is not an accident. It is a systemic failure we have been looking away from.