Toyota Tests Backseat-Driver Software That Could Take Control in Dangerous Moments

Making a turn across oncoming traffic is one of the most dangerous maneuvers drivers undertake every day. Researchers at Toyota think it’s one of the situations in which a software guardian angel built into your car could save lives.

In trials at private testing grounds in the U.S., left turns are one of the first scenarios Toyota has used to test the concept of a system it has dubbed “Guardian,” which judges whether a human is about to make a dangerous mistake.

Radar and other sensors on the outside of the car monitor what’s happening around the vehicle, while cameras inside track the driver’s head movements and gaze. Software uses the sensor data to estimate when a person needs help spotting or avoiding a hazardous situation.

So far Toyota is just testing the ability of software to understand the hazards around a car and whether a person has spotted them, but the company plans to eventually make Guardian capable of taking action if a person doesn’t look ready to do so already.

“Imagine going through an intersection and you’re going to get T-boned—the right thing for the car to do is accelerate you out of it,” says Ryan Eustice, VP of autonomous driving at Toyota Research Institute, which was established in 2015 to work on robotics and automated driving (see “Toyota’s Billion Dollar Bet”). The group first said it would start developing Guardian last year (see “Toyota Joins the Race for Self-Driving Cars with an Invisible Co-Pilot”).

Eustice argues that the Guardian effort could have a widespread impact on public safety earlier than cars that fully remove driving duties from humans. Toyota is working on such technology, along with competitors like Alphabet, Ford, and Uber. But despite high-profile testing programs on public roads, Eustice and his counterparts at other companies say that truly driverless vehicles are still some years from serving the public and will initially be limited to certain routes or locales.

“We see an opportunity to deploy it sooner and more widely,” says Eustice of the backseat-driver approach. That’s because unlike full autonomy, it won’t be reliant on hyper-detailed maps and could be easily packaged into a conventional vehicle sold to consumers, he says. However, he declines to predict how soon Guardian might be ready for commercialization.

Steven Shladover, a researcher at the University of California, Berkeley, says the claim that Guardian could save lives sooner than fully autonomous vehicles makes sense. “If the driver has a 99 percent chance of detecting hazards and the automation system also has a 99 percent chance of detecting hazards, that gives the combination of the driver and system a 99.99 percent chance,” he says. “But this is much simpler and easier than designing a fully automated system that could reach that 99.99 percent level by itself.”

Getting the relationship between Guardian and humans right will be critical, though. Any mistakes it makes, such as intervening or sending a warning when a person has correctly interpreted a situation, would undermine a person’s trust in the system and could even lead to new kinds of accidents, says Shladover.

Eustice says Toyota is well aware of those challenges. “There will have to be a lot of studies to understand human acceptance,” he says. One idea he’s considering is enabling the system to talk with a driver about incidents on the road.

“If the car does intervene, it will be important for the car to explain to you why it did that, or to later say, ‘Hey, I didn’t intervene back there, but that was actually a close call,’” he says.