Autonomous vehicle companies operating in California are currently required to report the number of disengagements its autonomous test prototypes experience – that being the number of times the safety driver has had to take the reigns from the autonomous system.
Disengagement data will sometimes be used to determine how advanced a particular companies’ autonomous vehicle platform is, with companies being judged on the number of miles logged between system disengagements. Kyle Vogt, co-founder of General Motors‘ Cruise autonomous driving subsidiary, recently said the metric is not an accurate representation of how capable an AV is, though, as not all AV companies test in the same way. Cruise, for example, logs all of its miles in the congested city center of San Francisco, whereas others may test vehicles mainly on highways or roads with less traffic and congestion. An AV is more likely to experience a disengagement in a city center due to pedestrians, bicycles, erratic drivers and congestion, which is precisely why Cruise tests in downtown San Francisco.
Waymo van
It’s not just Vogt that thinks California’s disengagement reporting is a futile effort, Reuters reports. After the state released its 2019 Autonomous Vehicle Disengagement Report this week, Waymo published a series of tweets criticizing the disengagement metric, saying it “does not provide relevant insights into the capabilities of the Waymo Driver or distinguish its performance from others in the self-driving space.” For Waymo, this is mainly because its most advanced prototypes, such as those operating under its Waymo One ride-hailing service in the Phoenix area, are its most advanced. The test miles it logs in California, which is where its headquarters is based, are mostly with test prototypes that are in the early stages of development.
“At this stage our real-world driving in California is predominately engineering development, and not production releases,” Waymo also said.
Cruise Automation prototype
Cruise and Waymo had the greatest number of test miles logged between disengagements in 2019, the California DMV found. Cruise separated its disengagement reports into two parts after making a serious of major upgrades to its AV platform halfway through 2019. It reported 43 disengagements in 328,285 total miles driven in the first half of the year, equal to 7,635 miles per disengagement, while it logged only 25 disengagements in 502,755 miles in the second half of the year.
Cruise prototypes
Despite this strong performance in the disengagement report, the company still says there is no clear way to tell how advanced a company’s AV system is without looking at their own internal data. Other smaller self-driving companies echoed Google and Waymo’s sentiment toward disengagement reporting when speaking to Reuters, including Zoox and Aurora.
Subscribe to GM Authority for and around-the-clock GM news coverage.
Source: Reuters
Comment
The reporting regulations need fixing fast without any delay.