Safety, Trust and Reliability

On the night of 2 October 2023 a hit-and-run driver struck a woman crossing Market Street in San Francisco and threw her into the path of a driverless car operated by Cruise, General Motors' robotaxi subsidiary. The car braked hard. It then stopped, judged that it had been in a side collision and tried to pull over to the kerb, travelling about 20 feet at up to about 7 mph with the pedestrian beneath it (Cruise and investigators, reported). Cruise had won approval for round-the-clock paid service in the city ten weeks earlier and ran about 400 driverless vehicles (reported). Its first public statements stressed the human driver's role. When it met the California Department of Motor Vehicles on 3 October it played video of the incident, but regulators later said the video did not show the dragging, or that Cruise did not point it out; the DMV learned of it from another government agency (DMV, reported).

On 24 October 2023 the DMV suspended Cruise's driverless permits, saying its cars were "not safe for the public's operation" and that Cruise had "misrepresented" information about safety (DMV order, quoted in press). Cruise paused driverless operations nationwide on 26 October and recalled 950 vehicles on 8 November to change their post-collision logic. Its co-founder and chief executive, Kyle Vogt, resigned on 19 November, and about 900 people, about a quarter of the staff, were laid off on 14 December. An outside report by the law firm Quinn Emanuel (25 January 2024) found no intent to mislead but criticised leadership's "fixation" on correcting a media narrative (as quoted in press). The federal penalties followed: $1.5 million from the National Highway Traffic Safety Administration (NHTSA) for incomplete crash reports (30 September 2024), and a deferred prosecution agreement with a $500,000 fine for a false report (14 November 2024). On 10 December 2024 GM ended funding for robotaxis, after spending more than $10 billion on Cruise since 2016 (reported).

Regulators acted more for the incomplete disclosure than for the incident itself. This chapter explains why, in AI, safety is less a cost centre than a licence to operate, why reliability is judged on rare failures rather than averages, and why the first large consequences came from consumer harm rather than catastrophe.