How Safe Are Self-Driving Cars After the First Fatal Accident?

[Spinnor]

And could you please respond to my question about whether your position on this goes to the extreme that you'd rather the car choose kill someone than commit a minor traffic law violation.

Do neither. Don't kill, obey the law. That is one rule I guess, obey the law.

 

[russ_watters]

Do neither. Don't kill, obey the law.

That isn't one of the choices. Have you never had to swerve to avoid hitting someone/something?

 

[Spinnor]

Have you never had to swerve to avoid hitting someone/something?

City driving, yes. Now that I think of it, many times.

 

[russ_watters]

City driving, yes.

Have you ever crossed the centerline in doing so? Would you?

 

[Spinnor]

Have you ever crossed the centerline in doing so? Would you?

In one particular case, I had to.

 

[russ_watters]

In one particular case, I had to.

Ok, so then the answer is yes; you do believe there are cases where violating the law is needed to increase safety and is preferred over choosing to cause an incident by not violating the law.

 

[Spinnor]

violating the law.

If the Uber car needs to violate the speed limit in order to keep of with the flow of traffic which is also violating the law then Uber should not be on the road and all the drivers should have transponders in their cars that report their speed violations to big brother.

What was the speed limit in this case, do you know?

 

[Borg]

What was the speed limit in this case, do you know?

The car was most likely going about 38 miles (61 kilometers) per hour, Moir said. The speed limit where the accident occurred is 35 mph, police spokeswoman Lily Duran said.

 

[Spinnor]

That is not very fast. Humans are pretty damn fragile.

There has got to be a safer way to teach robots to drive?

Would limiting the cars to daylight hours of operation help? Do they lose some of their sensing "abilities" at night? Surely the human backup driver would have had a better chance of seeing the women during the day?

 

[Borg]

From TeethWhitener's link on page one, the woman stepped in front of the vehicle so fast that it was unlikely that anyone could have avoided that accident. This really isn't a case of excessive speed.
https://www.bloomberg.com/news/arti...tepped-suddenly-in-front-of-self-driving-uber

 

[Spinnor]

From TeethWhitener's link on page one, the woman stepped in front of the vehicle so fast that it was unlikely that anyone could have avoided that accident. This really isn't a case of excessive speed.
https://www.bloomberg.com/news/arti...tepped-suddenly-in-front-of-self-driving-uber

Going slower would have reduced the injuries a bit. If it was day time and you were approaching this women you would have probably had her in your sights and might have been cautious near her. You said yourself you watch out for stupid people. I would like to think you would have avoided hitting her during the day, you would have been alert and swerved out of the way, at night less likely.

 

[russ_watters]

If the Uber car needs to violate the speed limit in order to keep of with the flow of traffic which is also violating the law then Uber should not be on the road...

So you can violate the law, but Uber can't? Really?

...and all the drivers should have transponders in their cars that report their speed violations to big brother...

Well that's just silly. You're really digging in here, but I think you've probably gotten the point by now that your logic doesn't work/doesn't produce the best outcome. So I think we can drop this rather than keeping getting sillier and sillier.

 

[Spinnor]

Well that's just silly.

Why is obeying the speed limit funny? People speed in front of my house and I detest it. Tech could basically eliminate speeding.

 

[russ_watters]

From TeethWhitener's link on page one, the woman stepped in front of the vehicle so fast that it was unlikely that anyone could have avoided that accident. This really isn't a case of excessive speed.
https://www.bloomberg.com/news/arti...tepped-suddenly-in-front-of-self-driving-uber

Maybe. We'll have to wait for the results of the investigation, but one thing I suspect computer drivers still aren't good at is recognizing and reacting to threats before they happen (such as anticipating whether a pedestrian at the side of the road is about to try to cross). Whether or not that could have prevented this specific case, I suspect it is still a considerable issue in other cases. Part of that is that drivers and pedestrians communicate with each other in ways that autonomous vehicles are not yet able to do.

 

[russ_watters]

Why is obeying the speed limit funny?

Please stop. I wasn't saying obeying the speed limit is funny, I was saying your suggestion to put transponders on cars to enforce it was silly (not funny). And it is unnecessary for you to keep trying to argue this point, since you already acknowledged that you are willing to break the law to enhance safety. So there is no reason for you to continue arguing as if you don't believe that.

 

[Spinnor]

Stopping now.

 

[StoneTemplePython]

...though "probability" is not clear-cut because the transportation is used in different ways/for different purposes and as such there are different choices that can be made for the measurement in some cases. I can consistently compare miles or trips to make a decision on how to get from Philly to Boston (and get different answers if I use "miles" or "trips"), but I can't really compare them for a flight to San Juan, since you can't drive there. That makes direct comparison a little tough.

The idea from finance here would be a replication (read: arbitrage) argument. In math it would be either dominance or an exchange argument. (I actually think you could look at this, nicely, in terms of majorization of probability distributions as well.)

Looking at safety in terms of mortality per person per mile (aka per "person miles") is a fairly common metric. As a simple first cut, you could decompose your trip to San Juan by driving to Florida then taking a boat. Commercial flying in the US is so shockingly safe that I'm quite confident it will come out ahead in terms of safety per person mile over any amount by car and, I suspect, per person mile by ferry / commercial boat.

Getting good data on public marine transport seems to be a bit patchy, though a quick look on mortality rates on cruise ships puts them as being in the neighborhood of airlines on a per passenger amount (I think it gets much worse once mileage comes into play, and industry doesn't have great reporting requirements here).

---
Note trains are another land option, though they too are dominated by planes in terms of safety evaluated on per 'person mile' morality .
---

The idea is that commercial flying dominates all other transport methods in terms of safety, so any convex combination of other transportation options, to get the same person miles done, is dominated by the commercial flying approach. (Note that flying generally allows approximately direct routes in a way that cars can't -- they are at mercy of the road system-- and certainly trains are limited by where the train tracks run, and I suspect there are similar issues with ships, so we could throw in triangle inequality as icing on the cake.)

 

[Borg]

Maybe. We'll have to wait for the results of the investigation, but one thing I suspect computer drivers still aren't good at is recognizing and reacting to threats before they happen (such as anticipating whether a pedestrian at the side of the road is about to try to cross). Whether or not that could have prevented this specific case, I suspect it is still a considerable issue in other cases. Part of that is that drivers and pedestrians communicate with each other in ways that autonomous vehicles are not yet able to do.

Agreed. While I try not to judge events through their news stories, this really looks like a case of someone stepping into the road without looking. Of course the family will sue and they will probably get something - lawyers will always be chasing ambulances.

While it's beyond my skillset as a programmer, I agree with your suspicions about threat recognition. However, I think that the companies that have been designing and building these cars have done a remarkable job so far. The accident rate has been much smaller than I expected. And I'm REEEALY GLAD that this isn't a government, lowest-bidder project.

 

[OmCheeto]

About an hour ago, the Tempe police department released a video of the accident:

 

[gleem]

If the standby driver had had his foot on the break he might I say "might" have kept the car from killing her.

 

[gleem]

And if he was paying attention to a possible collision too. How long can you expect a person to remain in that state of readiness unless he is actually engaged in driving the car.

 

[dipole]

I feel bad for the woman, but I think she was clearly at fault here.

Though I wonder how much the automated car relies on visual cues. Seems like radar or sonar systems should be able to avoid those kind of scenarios

 

[russ_watters]

About an hour ago, the Tempe police department released a video of the accident:

Yikes. It seems initial reports were wrong: neither the car nor the standby driver were paying attention at the time of the accident, and they could/should have prevented it.

[and I'm shocked that this was released]

If the standby driver had had his foot on the break he might I say "might" have kept the car from killing her.

And looking up at the road instead of into the cockpit.

I feel bad for the woman, but I think she was clearly at fault here.

Though I wonder how much the automated car relies on visual cues. Seems like radar or sonar systems should be able to avoid those kind of scenarios

I too would have expected the sensors to detect this and the car to be able to avoid this -- assuming it had either sonar or radar. Note though that the woman was crossing at the worst possible spot: the very edge of a street light's coverage (on the wrong side of it). That makes cameras less effective... if this car used cameras for that purpose. But the driver - had he been looking - should still have been able to see her, since our eyes are much more sensitive to high dynamic ranges.

 

[OmCheeto]

About an hour ago, the Tempe police department released a video of the accident:
...

Late entry:

Tempe Police Department; "Tempe Police Vehicular Crimes Unit is actively investigating the details of this incident that occurred on March 18th. We will provide updated information regarding the investigation once it is available."​

Btw, I was tipped off to the video by Stephen King.

Stephen King; "The pedestrian fatality in Tempe, Ariz. is very sad, but the immediate assumption that the self-driving car was at fault--an assumption that now seems erroneous--is fascinating. Food for sociologists and psychologists."
4:07 PM - 21 Mar 2018 [PDT]​

Stephen has written at least one book about autonomous homicidal cars, before there were autonomous cars.

 

[Vanadium 50]

According to news, "jaywalking."

Even so, I don't think Jaywalking deserves the death penalty.

I have a fundamental moral objection to beta testing technology that can kill you on the public

There's billions at stake. You can't make an omelet without breaking a few eggs. History suggests that the people who run and ran Uber - can I call them the Ubermenschen? - are not the sort to let doing what's right stand in the way of making a buck.

Presumably they have a group to do risk management. I'd love to be able to find out the answers to these and similar questions:

• Was the probability of a fatality calculated? If not, why not?
• What is the acceptable fatality rate?
• Was there any management pressure for Risk to calculate an arbitrarily low accident/fatality rate?
  
• As accidents have occurred, such as the one a year ago in Tempe, how is the model's predictive power compared with data? Is the model adjusted in light of new information? If so, by whom? Who decides whether the new risks are acceptable?

 

[gleem]

What is not apparent in the video is if the car was braking at the time of the impact. If not then there is a problem. The sensors should have picked up an object moving into the path of the car.

 

[berkeman]

Wonder if it was a suicide...

From this video, it does appear so to me. She could see the approaching lights of the car. She was not in the lights of the car until the last second. I wonder if she left a note...

http://www.latimes.com/business/la-fi-uber-death-video-20180321-story.html

 

[gleem]

By my estimate from the video it seems the car was less than 100 feet from the time that the person was visually apparent. I do not know the speed limit but it has two lanes in one direction so 55 mph is not unreasonable. From the Government Fleet website the car would have gone about 80 ft before the brakes where applied for an average driver. So the car might not have slowed down too much before impact. Considering the sensors should have picked up the person before visual sighting and the fact that the reaction time for the computer is essentially zero it is likely that the car still would have impacted the person at a significant speed.

 

[nsaspook]

The poor woman on the bike looked like a homeless person doing the standard look down, ignore the world walk. I'm unimpressed by the automated vehicle but IMO, 100% on the bike lady for what happened.

That said:
There is a nice paved walkway that just invites (Yes, there is a sign that say's don't so the person didn't have right of way) people to cross there instead of at the intersection.
https://www.google.com/maps/@33.436...4!1sseIHdIkV5FyYzACTGokvBg!2e0!7i13312!8i6656
https://www.google.com/maps/@33.436...4!1sE4V6_Cyyoq-K3JKD56PPyA!2e0!7i13312!8i6656
https://www.google.com/maps/@33.436...4!1svgBgLp4aEODKKcwDOGl-tg!2e0!7i13312!8i6656

 

[rootone]

From this video, it does appear so to me. http://www.latimes.com/business/la-fi-uber-death-video-20180321-story.html

She appears to walking with a bicycle (however carelessly).
A suicide attempt would not require a bicyle

 

[russ_watters]

Presumably they have a group to do risk management. I'd love to be able to find out the answers to these and similar questions:

• Was the probability of a fatality calculated? If not, why not?

A: "Let's just do whatever we want and let the lawyers sort it out - it's always worked before."

 

[russ_watters]

By my estimate from the video it seems the car was less than 100 feet from the time that the person was visually apparent.

"Visually apparent" to us in the video does not necessarily correspond with "visually apparent" to the driver or a radar/sonar/lidar on a car.

I do not know the speed limit but it has two lanes in one direction so 55 mph is not unreasonable.

38 in a 35 zone.

 

[nsaspook]

"Visually apparent" to us in the video does not necessarily correspond with "visually apparent" to the driver or a radar/sonar/lidar on a car.

38 in a 35 zone.

Unless it was recently changed the limit was 45 as marked.
https://www.google.com/maps/@33.435...4!1sx-K4_17J8MVthFRapvIa2A!2e0!7i13312!8i6656

 

[russ_watters]

Unless it was recently changed the limit was 45 as marked.
https://www.google.com/maps/@33.435...4!1sx-K4_17J8MVthFRapvIa2A!2e0!7i13312!8i6656

Not sure where that is in relation to the accident, but a source was provided on page 1 saying 38 in a 35.

 

[OmCheeto]

Odd. The northbound lanes of Mill Avenue speed limit is 45 mph, while the southbound lanes are set at 35 mph.
Perhaps that's where some of the confusion is coming from.
The vehicle was traveling north.

 

[Borg]

A: "Let's just do whatever we want and let the lawyers sort it out - it's always worked before."

Ironically, this seems to also be the attitude of pedestrians who don't pay attention when they cross the street.

https://www.google.com/maps/@33.436...4!1svgBgLp4aEODKKcwDOGl-tg!2e0!7i13312!8i6656

If you look down the road on this one, you can see two pedestrians crossing the road in front of oncoming traffic (that had to break for them) with no crosswalk or sidewalk where they crossed even though there is a much wider sidewalk on the side that they were originally on.
https://www.google.com/maps/@33.436...4!1slC6iG7fn3TmsKGAC65MJnA!2e0!7i13312!8i6656

 

[nsaspook]

Not sure where that is in relation to the accident, but a source was provided on page 1 saying 38 in a 35.

38 in a 35 makes it sound like the car was over-driving the conditions and that could reduce normal reactions times to below what was safe for the road.

 

[Ryan_m_b]

Watching the video it looks like the headlights were not at the appropriate angle or intensity. The pedestrian didn't just step out from behind a blind spot or corner yet was only visible in the very last moments.

 

[HAYAO]

At least from a perception perspective that's all true, but I don't necessarily agree that it's right. An agreeing example to highlight why I disagree:

I just performed a risk analysis of an industrial campus electrical system in Puerto Rico. Failures are scored according to severity and likelihood, with full power outages for more than a day getting the same, maximum score. Below a day, outages or voltage dips ranging from miliseconds (can often be plowed-through) to seconds (it takes 30 seconds for a generator to start) to minutes (it can take a few minutes to reset a tripped circuit breaker) have different severities.

Hurricane Maria, however, was an unprecedented emergency situation, and generators that were designed to run for hundreds of hours a year instead ran for thousands last year. This requires a new level of "severity" not previously considered.

Unlike power outages, which are not all the same severity despite looking the same at the moment they happen, a death is a death. They are all the same. And in a country where individual rights are paramount, they are all weighed equally. So a fair risk assessment should say that a plane crash that kills 150 people is exactly equal to 50 car crashes that each kill 3. And multiplying that by the probability of each gives the relative risk.

...though "probability" is not clear-cut because the transportation is used in different ways/for different purposes and as such there are different choices that can be made for the measurement in some cases. I can consistently compare miles or trips to make a decision on how to get from Philly to Boston (and get different answers if I use "miles" or "trips"), but I can't really compare them for a flight to San Juan, since you can't drive there. That makes direct comparison a little tough.

That's a cool assessment. Great insight.

I am also curious about more rigorous statistical risk analysis on this issue. That is, the "fairest" comparison between two different transportation methods, namely planes and cars. Just like you said, probability based on deaths/events and probability based on deaths/distance is going to yield some different result. Furthermore, as you mentioned, the purpose of using certain transportation method changes the weight of each accidents. I wonder how that plays out.

 

[nsaspook]

Watching the video it looks like the headlights were not at the appropriate angle or intensity. The pedestrian didn't just step out from behind a blind spot or corner yet was only visible in the very last moments.

They look bad like most SUV headlights systems on a dark object at night.

A very good driver with the best equipment might have bailed her out after the disregard of approaching traffic while jaywalking.

 

[Ryan_m_b]

They look bad like most SUV headlights systems on a dark object at night.

Here in the UK IIRC the regulation for dipped headlights is that they must illuminate a distance of 50m (~165 feet) in clear weather conditions. I imagine the US has similar laws but the video and your statement seem to suggest otherwise. Looking back at the video the headlights are only illuminating two road marks ahead of the vehicle, which isn't even 10m.

I think there are a couple of serious questions to be answered here.

1) Given that Elaine had reached the far lane while crossing at a slow/steady pace and the vehicle had not turned a corner or any other blindspot was she picked up by the LIDAR? If so at what distance, what corrective measures (if any) did the car attempt to make and if she was not then why not.

2) Again given that Elaine didn't jump out suddenly and the car had not turned the corner why did the driver not notice her? What measures have Uber taken to ensure their drivers understand the need to constantly be aware of the situation.

Given the video I suspect that if the car was being driven by a human and they had their full beams on (given that it's a dark road with no oncoming traffic) they would have seen Elaine at a distance sufficient to slow down, and Elaine would have had a visual warning of the car coming at a greater distance. If the car doesn't need the headlights to drive itself then there seems to be a double failure here, 1) whatever caused the car to not see her in sufficient distance and 2) the driver/car failing to put on high beams, possibly due to complacency wrt to the autopilot's capability.

 

[256bits]

the driver/car failing to put on high beams, possibly due to complacency wrt to the autopilot's capability.

Probably not.
High beams blinds the driver in the car ahead of you, who then can't see as well.
Rear view and side mirrors and all that.
So everyone in the city drives with low beam.
Imagine high beam behind a police car, not wise.

 

[256bits]

Would limiting the cars to daylight hours of operation help? Do they lose some of their sensing "abilities" at night?

Uber is in the taxi business.
Curtail the business at night in unrealistic, if these cars are going to be allowed on the road.
The cars will be driving at night, so they have to be tested at night.

 

[LiamSmith]

Car driving is the most dangeres thing in our world. Be care of uresefl guys

 

[Spinnor]

Uber is in the taxi business.

They are also teaching robots to drive. Restricted to day time driving there is a good chance that an alert backup driver would have prevented this death. What was the women backup driver looking down at?

 

[BillTre]

Perhaps the back-up driver should have been a robot!
No wandering attention.

 

[256bits]

They are also teaching robots to drive. Restricted to day time driving there is a good chance that an alert backup driver would have prevented this death. What was the women backup driver looking down at?

Backup driver-
A human driver avoiding this incident we will never know.
With nothing to do but be there, look out the window and daydream a lot I would suspect. Attention span in that type of job...
Bored kids would say, "There's nothing to do!".
How long is a "testing run" between breaks to get out and stretch ones legs.
An inactive person with little stimulation becomes lethargic after the routine sets in.
I am actually surprised the person hadn't fallen asleep, especially since the sun was down.

I have not seen any testing on human backup drivers, and their value, it is just a given that one is needed.
Is that just a bureaucratic decision without any basis, but built on false assumptions?
Certainly there are recorded anecdotal incidents where the driver took over control of the vehicle.
But in any "experiment", which is what this is, there is no control group for comparison of what the driverless vehicle would do on its own, what a human driver would do, and what a human-driverless-vehicle would do. The testing is on the fly in the real world which is what you are getting at.

Restricted to day time driving -
Right. In hindsight, the technology for this vehicle was not ready for night driving.
But they do have to test at night to see what happens, since the cars will be used at night.
And it could be not even ready for day time driving.
The program, the sensors, something did not live up to expectations.
For all the hype that surrounds the "safer than human drivers", this was a complete failure.
If the lady has a family, I hope they sue for $500 million bucks, just to get this thing on the right track, instead of the what it seems to be the first one past the starting gate wins.
The promotional material is speaking to the converted.In the future, when these cars will be licensed for the road, I will predict that the mix of 10%, 20% etc will produce different outcomes than just 1 amongst many.
Ever hear of road rage.
What's better than to shoot the tires out from under a self-driving car that doesn't know how to drive.
Stay tuned.

 

[Ryan_m_b]

There seems to be some dispute circulating online regarding Uber's dashcam footage, which might explain why the headlights look to be defective/illegally modified. The road in the dashcam footage is pitch black with the cyclist appearing last minute. However there are regular citizens posting pictures and videos of that road at night to show that it is in fact very well illuminated. Either Uber's dashcam is of extremely poor quality or the footage has been altered somehow to make it appear darker than it was and give the impression Elaine jumped out of nowhere.

This video was taken by a member of the public at roughly the same period of night, the spot of the accident is passed about the 30 second mark.

 

[nsaspook]

I don't think the crash video was defective or modified, just poor. If we remove the automation could a reasonable person avoid that accident under these conditions (dark object in the dark part of the road).


#1 Pedestrian failed to cross safely or properly. In most parts of the world that's case closed.

Do we really need red flags and two safety drivers for driver-less cars for bad 'corner cases' ("Anything can go wrong will go wrong") ?

 

[Ryan_m_b]

I don't think the crash video was defective or modified, just poor. If we remove the automation could a reasonable person avoid that accident under these conditions (dark object in the dark part of the road).

Going by that video I would say yes absolutely. The area of the crash is visible for approximately ten seconds in that video. Aside from the fact this shows the safety driver should have seen Elaine and intervened the Uber had Radar and LIDAR, neither of which should have been affected by the lighting of the road. Yet despite it being a straight road for a significant stretch between the car and Elaine it did not appear to make any attempt to slow down.

#1 Pedestrian failed to cross safely or properly. In most parts of the world that's case closed.

Err I'm not so sure on that. IIRC there are only two nations in the world with jaywalking laws, in the others drivers are expected to do to give way to pedestrians in a safe manner. In the UK for example pedestrians are legally considered to have right of way on the roads excepting those that specifically ban pedestrian use (like motor ways). In the event of an accident partial or full blame is only placed on the pedestrian if seeing and avoiding them can be shown to be impossible in the time given, assuming the driver is otherwise driving safely (i.e. not speeding, lights at appropriate setting, not on phone etc).