How Safe Are Self-Driving Cars After the First Fatal Accident?

[Dr. Courtney]

http://www.foxnews.com/us/2018/03/19/self-driving-uber-car-kills-arizona-pedestrian-police-say.html

Considering the small experimental installed base, this gives me serious concerns with the technology. Not ready for prime time.

 

[Borek]

Considering the small experimental installed base

Do you know how large the base is and how it does compare to the deaths/mile for cars driven by humans?

 

[Spinnor]

"A self-driving vehicle made by Uber has struck and killed a pedestrian. It’s the first such incident and will certainly be scrutinized like no other autonomous vehicle interaction in the past. But on the face of it it’s hard to understand how, short of a total system failure, this could happen, when the entire car has essentially been designed around preventing exactly this situation from occurring."

From, https://techcrunch.com/2018/03/19/h...ving-cars-are-supposed-to-detect-pedestrians/

From, https://news.google.com/news/story/dWZXEd51GqIuoLMy_mmpiTM-SCwsM?ned=us&hl=en&gl=US

That sucks.

 

[Choppy]

One of the potential bright sides to this tragedy, as opposed to most human-caused motor vehicle fatalities, is that you can have a team of engineers review the incident and data collected about it in detail with a much higher likelihood of determining the principle causes and then get to work on strategies for avoiding similar situations in the future. That will make not just this individual autonomous vehicle better, but potentially *ALL* autonomous vehicles better. In that way something useful can come out of the tragedy.

Something else to remember is that not all motor vehicle accidents are completely unavoidable. If a car is traveling along at the speed limit and someone jumps out in front of it from a concealed position, its options are limited by physics and in some cases ethical decisions.

 

[Vanadium 50]

There are 93 vehicles (as of October). Assuming 8 hours a day at 30 mph for the whole year, that's about 8 million miles, or 12.5 deaths per 100 million miles. The rate of pedestrians and cyclists killed by living drivers is 1.25 per 100 million. So there's a factor of 10 different (at least).

 

[HAYAO]

My condolences to the family who lost one of their important members.

I am very concerned about how it happened. Was it a bug? Bad programming? Fundamental flaw in the technology? Or simply an unavoidable accident (for example the pedestrian tripped over something and fell on the road)? If it is the first two, then we have a simple fix. However, if it is some fundamental flaw, then we need to use different kind of technology for this. If the last one, then it just cannot be helped unfortunately. I will not be quick to judge that a technology is good or bad simply based on this incident because no information as to how it happened was provided.

I am also concerned about the legal aspect of this. How do laws view this incident?

 

[BillTre]

I could have been some kind of pedestrian error.
Today I was driving down a residential road, turned the corner and was faced the a guy riding no-hands on a bike, holding his phone in two hands and doing something with it while he was looking down and not where he was going.
Sometimes people do stupid things.

 

[Bystander]

of pedestrian error.

According to news, "jaywalking."

 

[russ_watters]

One of the potential bright sides to this tragedy, as opposed to most human-caused motor vehicle fatalities, is that you can have a team of engineers review the incident and data collected about it in detail with a much higher likelihood of determining the principle causes and then get to work on strategies for avoiding similar situations in the future. That will make not just this individual autonomous vehicle better, but potentially *ALL* autonomous vehicles better. In that way something useful can come out of the tragedy.

This is why American airliners don't crash anymore: every crash makes the next one less likely.

Something else to remember is that not all motor vehicle accidents are completely unavoidable. If a car is traveling along at the speed limit and someone jumps out in front of it from a concealed position, its options are limited by physics and in some cases ethical decisions.

Yes - at the outset we know that there is fault to be found in the victim here; she was not in a crosswalk. That doesn't necessarily make the accident unavoidable, but it at least makes it less avoidable.

 

[russ_watters]

Considering the small experimental installed base, this gives me serious concerns with the technology. Not ready for prime time.

We'll see what caused this, but either way I have a fundamental moral objection to beta testing technology that can kill you on the public - including people who didn't choose to be part of the test. This would never be allowed with airplanes.

 

[rootone]

According to news, "jaywalking."

Yes apparently the killed person crossed the road away from a designated crossing. (though not very far away).
I am sure a human driver would recognize that kind of situation easily.
It might annoy them considerably. but they definitely would stop

 

[StoneTemplePython]

This is why American airliners don't crash anymore: every crash makes the next one less likely.

Yes, though in terms of incentives, it's worth pointing out that if there's a crash, pilots go down with the plane. The issue of people 'in charge' externalizing losses completely on the general public, is an issue, which I think ties into...

We'll see what caused this, but either way I have a fundamental moral objection to beta testing technology that can kill you on the public - including people who didn't choose to be part of the test.

- - - -
In general I'm a fan of self-driving cars, but it's perilous, delicate road ahead. There are a lot of concerns to be had, including these.

 

[sandy stone]

In particular, if the accident is found to be avoidable, who goes to jail for involuntary manslaughter? This leads back to the whole "corporations are the same as people" garbage.

 

[gleem]

There are 93 vehicles (as of October). Assuming 8 hours a day at 30 mph for the whole year, that's about 8 million miles, or 12.5 deaths per 100 million miles. The rate of pedestrians and cyclists killed by living drivers is 1.25 per 100 million. So there's a factor of 10 different (at least).

I don't think this data can say much about the probable rate of accidents from autonomous cars vis-a-vis driven cars since the projected uncertainly in the autonomous car accident rate does not exclude the driven car rate at this point.

As far as determining the cause one would think that considering these cars are experimental there should be equipment on board like dashcams and black box recorders monitoring everything from speed to accelerations to computer functions. Maybe they do.

 

[TeethWhitener]

From Bloomberg:
https://www.bloomberg.com/news/arti...tepped-suddenly-in-front-of-self-driving-uber
The victim was walking her bike by the center median at around 10pm at night and stepped suddenly out into the roadway. The car had a backup driver but was in autonomous mode. The backup driver didn't see the victim either. The car was going about 3 mph above the speed limit.

 

[berkeman]

From Bloomberg:
https://www.bloomberg.com/news/arti...tepped-suddenly-in-front-of-self-driving-uber
The victim was walking her bike by the center median at around 10pm at night and stepped suddenly out into the roadway. The car had a backup driver but was in autonomous mode. The backup driver didn't see the victim either. The car was going about 3 mph above the speed limit.

Great find, thanks @TeethWhitener -- finally it starts to make sense. Wonder if it was a suicide...

 

[Borek]

Seeing comments in my internet bubble I am starting to think future for autonomous cars looks bleak. Each casualty will be reported by media and blown out of proportion for the sake of hype/viewership/ratings/clickability/whatever and in time it will build a skewed opinion about how these cars are inherently unsafe, no matter what the numbers will say. Similar social mechanism that fuels anti-vaccine movements because of rare vaccine injuries.

 

[russ_watters]

Seeing comments in my internet bubble I am starting to think future for autonomous cars looks bleak. Each casualty will be reported by media and blown out of proportion for the sake of hype/viewership/ratings/clickability/whatever and in time it will build a skewed opinion about how these cars are inherently unsafe, no matter what the numbers will say. Similar social mechanism that fuels anti-vaccine movements because of rare vaccine injuries.

I believe you are correct, and again, planes provide the analogue: for perception reasons, right or wrong, planes need to be MUCH safer than cars or people will be afraid to fly them.

And from a legal perspective there may actually be more culpability in a self driving car than a human driven car due to the fact that humans are known/allowed to be flawed, but machines are required to be perfect.

 

[russ_watters]

In particular, if the accident is found to be avoidable, who goes to jail for involuntary manslaughter? This leads back to the whole "corporations are the same as people" garbage.

Along that vein, the AP is reporting that the driver ("operator") was a convicted felon. It is against the law for convicted felons to be rideshare drivers.

https://www.google.com/amp/www.foxn...ona-pedestrian-was-felon-report-says.amp.html

Since reporters don't have thinking caps, despite using the right terminology, the report does not pick up on the fact that he wasn't a 3rd party "driver", but an employed "operator" and may not be covered by that law.

In either case, it is too rare but corporate heads do sometimes go to jail for the crrimes of the company. What makes it difficult is pinpointing which person casud the crime. In the Enron scandal, many were pinpointed and went to jail and in addition the accounting firm was given a corporate "death sentence" and killed-off by government mandate (Enron died on its own).

 

[Windadct]

Approx 15 pedestrian fatalities per day, short of inventing mind reading to let the car know when someone decides to step into traffic ( I had to drive down a strip mall at walking speed the other day, as the pedestrian kept turning to step out, see me step back, but would not look at me or waive, I was just guessing - 10 steps, turn, turn back, 10 steps, turn... arg) . This situation is practically unavoidable, and not all auto-accidents are the fault of the driver. I am still convinced Self-driving is far safer, and will become more safe over time as weakness can be addressed and rolled out to the whole "population" as an update. You can not re-program human drivers.

And the whole "stand-by" emergency driver piece is just a warm fuzzy, there is very little chance that a human, who is not engaged in the act of driving, can instantly step in and make a proper timely decision, they will not have complete situational awareness, IMO potentially making the problem worse.

 

[Borg]

People can be really careless around cars. They frequently think that they can be seen when they can't, don't look before entering the road, or just expect that the driver will have to avoid them like it's some kind of game. I've had to avoid each of these more than I can count.

I came very close to a situation like the Uber accident once but fortunately it was daylight and I saw that the person was clueless. I was driving southbound on this stretch of road where the speed limit is 55 mph (traffic is usually going faster than that). A girl was walking on the edge of the road with her back to traffic where the red arrow is. I saw that as a very dangerous situation for her and kept focused on her as I approached. Just before I got to her position, she abruptly turned right and stepped into my lane without looking. She barely stepped back as I swerved away from her. If I had been paying as little attention as she had, she would have been killed for sure.

 

[TeethWhitener]

People can be really careless around cars. They frequently think that they can be seen when they can't, don't look before entering the road, or just expect that the driver will have to avoid them like it's some kind of game. I've had to avoid each of these more than I can count.

I came very close to a situation like the Uber accident once but fortunately it was daylight and I saw that the person was clueless. I was driving southbound on this stretch of road where the speed limit is 55 mph (traffic is usually going faster than that). A girl was walking on the edge of the road with her back to traffic where the red arrow is. I saw that as a very dangerous situation for her and kept focused on her as I approached. Just before I got to her position, she abruptly turned right and stepped into my lane without looking. She barely stepped back as I swerved away from her. If I had been paying as little attention as she had, she would have been killed for sure.

View attachment 222387

Hey, I know that road! NOVA in the house!

 

[cosmik debris]

From Bloomberg:
https://www.bloomberg.com/news/arti...tepped-suddenly-in-front-of-self-driving-uber
The victim was walking her bike by the center median at around 10pm at night and stepped suddenly out into the roadway. The car had a backup driver but was in autonomous mode. The backup driver didn't see the victim either. The car was going about 3 mph above the speed limit.

How can any backup driver help in these instances? A driver has to be fully engaged with the environment, I find it difficult to believe that a backup driver can remain concentrated and respond to emergency situations in time. We do research on micro-sleep here and even driving drivers fall asleep for short spells.

Cheers

 

[gleem]

And of course the ubiquitous smart phone will be assisting in the distraction.

 

[Windadct]

"of course the ubiquitous smart phone will be assisting" While I am confident this is statistically valid in pedestrian fatalities - no report of this in this case. Personally I will fight tooth and nail to call out this speculation, as I have said, AV is (IMO) far safer than human drivers.

This is NOT to say I want to take away driving as an activity, if you want to drive - go ahead ( a fully engaged driver may be as good as an AV. But I think an AV oversight of a human driver, is much different then an human driver oversight of an AV one.

Humans - next to a hazard ( like traffic) - will always find a way to get hurt.

 

[Windadct]

"And of course the ubiquitous smart phone will be assisting in the distraction." - however on a related note, my brother, a rather demented college prof - has proposed, they remove the man-hole covers on campus to filter out the phone-zombies... They have golf carts on campus, and can literally come to a stop as they approach one, and these people walk right into the GC.

 

[HAYAO]

I believe you are correct, and again, planes provide the analogue: for perception reasons, right or wrong, planes need to be MUCH safer than cars or people will be afraid to fly them.

And from a legal perspective there may actually be more culpability in a self driving car than a human driven car due to the fact that humans are known/allowed to be flawed, but machines are required to be perfect.

I just want to point out that fatal accidents with commercial planes are significantly more devastating than cars considering how hundreds of people can die at once, which is probably why planes have much higher standards in safety than cars. While numbers say that death rates with cars are lopsidedly higher than planes, fatal accidents per each unit tend to be more severe for planes than cars. But you are still right that the general public's perception, even if it is irrational, is what determines their will to use planes, which forces the safety standards to be more strict.

 

[russ_watters]

I just want to point out that fatal accidents with commercial planes are significantly more devastating than cars considering how hundreds of people can die at once, which is probably why planes have much higher standards in safety than cars. While numbers say that death rates with cars are lopsidedly higher than planes, fatal accidents per each unit tend to be more severe for planes than cars. But you are still right that the general public's perception, even if it is irrational, is what determines their will to use planes, which forces the safety standards to be more strict.

At least from a perception perspective that's all true, but I don't necessarily agree that it's right. An agreeing example to highlight why I disagree:

I just performed a risk analysis of an industrial campus electrical system in Puerto Rico. Failures are scored according to severity and likelihood, with full power outages for more than a day getting the same, maximum score. Below a day, outages or voltage dips ranging from miliseconds (can often be plowed-through) to seconds (it takes 30 seconds for a generator to start) to minutes (it can take a few minutes to reset a tripped circuit breaker) have different severities.

Hurricane Maria, however, was an unprecedented emergency situation, and generators that were designed to run for hundreds of hours a year instead ran for thousands last year. This requires a new level of "severity" not previously considered.

Unlike power outages, which are not all the same severity despite looking the same at the moment they happen, a death is a death. They are all the same. And in a country where individual rights are paramount, they are all weighed equally. So a fair risk assessment should say that a plane crash that kills 150 people is exactly equal to 50 car crashes that each kill 3. And multiplying that by the probability of each gives the relative risk.

...though "probability" is not clear-cut because the transportation is used in different ways/for different purposes and as such there are different choices that can be made for the measurement in some cases. I can consistently compare miles or trips to make a decision on how to get from Philly to Boston (and get different answers if I use "miles" or "trips"), but I can't really compare them for a flight to San Juan, since you can't drive there. That makes direct comparison a little tough.

 

[russ_watters]

How can any backup driver help in these instances? A driver has to be fully engaged with the environment, I find it difficult to believe that a backup driver can remain concentrated and respond to emergency situations in time.

I agree. Humans don't have a ready-standby mode where they can sit idle and monitor tasks and then engage instantly when needed. Our brains are always searching for something to do and will assign themselves a different task if not fully engaged in one. Switching back takes time at best or worse the other task causes them to miss something.

This is exacerbated by the fact that the automation levels are complicated (there are 5 levels), drivers aren't trained on it (not many even read the manual) and even when they are it is often impossible to react fast enough to overcome a computer failure: since humans have slower reaction times, if the computer fails, the person won't necessarily know it until they've past the point of no return.

 

[Spinnor]

The car was going about 3 mph above the speed limit.

That is crazy! If anything the damn car should have been going slower because of the new technology and slower because it was night. Uber, guilty. Pay up many millions. And do like they do in Asian countries, the CEO should get down on his knees and beg forgiveness to the community and the women's surviving family.

 

[Borg]

That is crazy! If anything the damn car should have been going slower because of the new technology and slower because it was night. Uber, guilty. Pay up many millions. And do like they do in Asian countries, the CEO should get down on his knees and beg forgiveness to the community and the women's surviving family.

For 3 mph over the limit and an accident that was likely the pedestrian's fault? Do you think that a vehicle that is going significantly slower than the rest of the traffic is less or more of a hazard? It's more.

 

[Spinnor]

For 3 mph over the limit and an accident that was likely the pedestrian's fault?

The car was in clear violation of the rules of the road. Uber, guilty.

 

[Borg]

The car was in clear violation of the rules of the road. Uber, guilty.

Are you going to lock up every driver on the road then?

 

[Spinnor]

Are you going to lock up every driver on the road then?

No. But this is new tech. Why the hell is it going over the speed limit? Guilty. Pay up Uber. What is the stinking rush.

 

[Borg]

No. But this is new tech. Why the hell is it going over the speed limit? Guilty. Pay up Uber.

As I stated before, a vehicle going much slower than the rest of the traffic is a hazard. I don't know if there was additional traffic but driving 3 mph over the speed limit is the last thing that I think that anyone would be alarmed over.

 

[Spinnor]

Do you think that a vehicle that is going significantly slower than the rest of the traffic

Someone passed me while I was carrying a large trailer load of tile and I was going quite significantly slower then the rest of traffic. Some guy behind me was in such a hurry to pass me, 4 lane road, he caused an accident. People need to slow down.

 

[russ_watters]

No. But this is new tech. Why the hell is it going over the speed limit? Guilty. Pay up Uber.

I'm with Borg on this one: I'd rather the automated car mach the low-end of the flow of traffic. Excessively/unusually low speed is itself a hazard.

Someone passed me while I was carrying a large trailer load of tile and I was going quite significantly slower then the rest of traffic. Some guy behind me was in such a hurry to pass me, 4 lane road, he caused an accident. People need to slow down.

You're arguing against your point.

 

[Spinnor]

driving 3 mph over the speed limit is the last thing that I think that anyone would be alarmed over.

It is breaking the law!

 

[Spinnor]

You're arguing against your point.

People are idiots, if the guy delayed his trip by less then a minute he could have safely passed me, people are idiots.

 

[Borg]

Someone passed me while I was carrying a large trailer load of tile and I was going quite significantly slower then the rest of traffic. Some guy behind me was in such a hurry to pass me, 4 lane road, he caused an accident. People need to slow down.

You're making my point. However, you expect that everyone else should match your speed on a road that has a marked speed limit. Most roads (escpecially freeways) have minimum limits as well because of just this type of accident.

 

[russ_watters]

People are idiots, if the guy delayed his trip by less then a minute he could have safely passed me, people are idiots.

Agreed, but you're still arguing against your point! In order for an automated car to maximize safety, it must avoid doing things that trigger idiots to act like idiots.

[edit] And frankly, this conversation matches my perception of truck drivers not caring if they are in the way, or worse being obstinate about it.

 

[Spinnor]

However, you expect that everyone else should match your speed on a road that has a marked speed limit.

It was a 4 lane road, he could have passed me safely with only a slight delay of his trip.

 

[Spinnor]

Most roads (escpecially freeways) have minimum limits as well because of just this type of accident.

The road I was on had a 45 speed limit, there was no minimum.

 

[Spinnor]

automated car to maximize safety

But it should also follow the rules of the road. New tech, over the speed limit, I can guarantee you Uber will be found negligent.

 

[Borg]

It was a 4 lane road, he could have passed me safely with only a slight delay of his trip.

I really do understand your frustration but there are stupid drivers on the road. I have a dash cam and could make a full length Michael Bay movie of the stupity that it has recorded in just the last year. The only reason I didn't have any accidents or indirectly cause them is that I expect others to do stupid things and do my best to avoid them.

 

[russ_watters]

But it should also follow the rules of the road.

Fair enough. My opinion is different; that it should instead always act in a way that maximizes safety, even if that causes a violation of the law.

Let me ask you this: if the car had noticed the biker and calculated it couldn't stop in time to avoid the collision, but instead swerved into the opposing lane of traffic (which is illegal), would you have the same position? Would you rather the car decide to kill someone in order to avoid a minor violation of traffic law?

New tech, over the speed limit, I can guarantee you Uber will be found negligent.

If Uber is found neglegent, I doubt it will be because of being over the speed limit.

 

[Spinnor]

The only reason I didn't have any accidents or indirectly cause them is that I expect others to do stupid things and do my best to avoid them.

You are a science advisor, you are smart and probably drive smart . Being as careful as possible on the road takes a lot of attention. I do not always give 100 percent attention while on the road, shame on me, but I strive to be a safe driver.

 

[Spinnor]

If Uber is found neglegent, I doubt it will be because of being over the speed limit.

Maybe not but I'm sure even a crappy lawyer will get a big payout from Uber. What does the software tell the car to do, forget the speed limit, go with the flow?

 

[russ_watters]

What does the software tell the car to do, forget the speed limit, go with the flow?

No, I'm sure there is an algorithm that starts with the speed limit and then adds an adjustment for traffic conditions. And if there's no traffic it may have a preset fraction or value above that it chooses.

And could you please respond to my question about whether your position on this goes to the extreme that you'd rather the car choose kill someone than commit a minor traffic law violation.

 

[Borg]

You are a science advisor, you are smart and probably drive smart . Being as careful as possible on the road takes a lot of attention. I do not always give 100 percent attention while on the road, shame on me, but I strive to be a safe driver.

Thank you but I'm not as careful as that. I am probably too observant of other drivers to the point of getting upset at having to continually avoid them. I keep thinking that I should get a bumper sticker that says "Caution, I drive like you".