How Safe Are Self-Driving Cars After the First Fatal Accident?

[nsaspook]

There is no excuse for Uber's car to not be able to handle such a straightforward/common accident scenario. To me, this is a homicide case. And to me, Tesla's fatal accident wasn't far behind this.

I completely agree this is a homicide case where we (the police and legal system) decide responsibility for the death.

Is there evidence that 'your 'driving directly contributed to the death of the individual and was the standard of your driving below or well below the standard of what is expected from a reasonable driver in those circumstances. Evidence that suggests the pedestrian stepped/moved/was detected in front of you in conditions leaving you little or no chance to avoid or stop must be given great weight on who to blame.

A similar case with no automation.
http://www.daily-chronicle.com/2018/03/04/genoa-pedestrian-39-hit-and-killed-on-route-72/amriqy8/

Smith said his department was called about 7:35 p.m. to the scene of the crash, where officers determined Michael J. Price, of the 800 block of Wilshire Drive, was crossing the road when Cindy P. Napiorkowski, 38, also of Genoa, was driving west and couldn't stop before hitting Price.

Smith said two members of the DeKalb County Coroner's Office were at the scene, where the man was pronounced dead.

Smith said the man was not crossing at an intersection.

"It isn't very well-lit where he was crossing," Smith said.

He said no citations were issued, the weather was clear, and no foul play is suspected.

 

[russ_watters]

I don't believe it's actually true that modern cameras are inferior to the human eye for dynamic range.

I'm not trying to be condescending here, but do you do much photography beyond just basic snapping? It's a huge and well known problem. Take a photo outside of something backlit - you'll notice the subject is near totally black in the photo yet you have no trouble seeing them yourself. It's why people use flashes and professionals use massive reflectors to illuminate their subjects outside -- and that's during the daytime!

Automatic adjustment when you have oncoming headlights mixed with darkness is a difficulty, however.

Yes, and in this case, the AI has to decide which it wants to see well: the brightly lit area near the car or the darker area further away and adjust accordingly.

Note though that this part of our discussion probably doesn't matter much. As far as we know, this was a dashcam, and not part of the car's self-driving system.

 

[russ_watters]

Not only did the vehicle have LIDAR and Radar but the maker of the LIDAR system has come out and said they can't understand how she wouldn't have been detected by it.

Ok, I know that, but what I asked is whether you know it was using the LIDAR. I'll take that as a no. I'm suggesting that because this seems like an easy accident to avoid for LIDAR, perhaps the car wasn't using it but was rather using/testing another system.

 

[olivermsun]

I'm not trying to be condescending here, but do you do much photography beyond just basic snapping? It's a huge and well known problem.

No offense, but you are being a little condescending here. I've been a reasonably serious photographer for a few decades now, including paid work for some years. I'm professionally interested in sensor design and data processing, so if you have some better sources I am all ears (eyes?).

Take a photo outside of something backlit - you'll notice the subject is near totally black in the photo yet you have no trouble seeing them yourself. It's why people use flashes and professionals use massive reflectors to illuminate their subjects outside -- and that's during the daytime!

A big part of that problem is metering. Use a modern digital sensor and "develop" the raw data correctly, you will realize that you have more dynamic range available than your output medium is capable of displaying. Hence the old -1.7 stop fill flash trick is much less a "thing" now than it was in the slide film days.

 

[russ_watters]

From the little information I've been able to find the safety driver was looking down at the object detection (that combined all sensors including LiDAR) display on a laptop or similar computer as it was her job to monitor the system.

("his") Yes, that's what I was getting at. I'll add a caveat to my previous statement though: the law may dictate that the person in the drivers' seat is legally responsible. Otherwise, it's Uber who set him up with a task to complete that took his attention away from being a true safety back-up system.

Her bike crossing timing as a detection target in relationship to the background and angle of approach of the car might have reduced the unique human signature as it blended with a bike with various sizes of plastic bags strung over it. I wonder how many pictures of homeless people walking laden bikes are in the images databases for a high confidence target classification?

I agree with @gleem that the job of the AI is to avoid collisions with objects and it doesn't need that level of identification to do so. I don't know if a floating grocery bag is visible on radar, but anything bigger needs to be avoided.

Classification of objects as a boulder or a moose is very important beyond detection of an object because the response should be different for 'benign' objects seen by the detection system. Executing a emergency stop for all objects detected is dangerous too.

Huh? Can you name an object besides a floating grocery bag or newspaper that the car should choose to hit instead of avoiding? If I'm approaching an object that's got a cross section of 12 square feet, I'm stopping no matter what it is!

 

[nsaspook]

https://jalopnik.com/lidar-maker-velodyne-blame-to-uber-in-fatal-self-drivin-1824027977

She said that lidar has no problems seeing in the dark. “However, it is up to the rest of the system to interpret and use the data to make decisions. We do not know how the Uber system of decision-making works,” she added.

Recognizing pedestrians continues to be a challenge for autonomous technology, which will be part of the focus of the investigation. Thoma Hall suggested that those answers will be found at Uber, not Velodyne.
...
That jibes with comments from experts who study autonomous cars. Earlier this week, University of South Carolina law professor Bryant Walker Smith told Jalopnik that Uber’s equipment “absolutely” should’ve detected Herzberg on the road.

The issue, he said, is that the tech quite likely “classified her as something other than a stationary object.”

 

[russ_watters]

Is there evidence that 'your 'driving directly contributed to the death of the individual and was the standard of your driving below or well below the standard of what is expected from a reasonable driver in those circumstances. Evidence that suggests the pedestrian stepped/moved/was detected in front of you in conditions leaving you little or no chance to avoid or stop must be given great weight on who to blame.

Agreed, but I think in this case investigators will say that both the "safety driver" and car should have easily been able to avoid this collision, placing the blame heavily on them (Uber).

A similar case with no automation.
http://www.daily-chronicle.com/2018/03/04/genoa-pedestrian-39-hit-and-killed-on-route-72/amriqy8/

Similar, yes, but there is not enough information there for us to say how similar.

 

[rootone]

The issue, he said, is that the tech quite likely “classified her as something other than a stationary object.”

.
Well software which classifies a pedestrian in the same category as a windblown plastic bag needs something improved alright.

 

[russ_watters]

No offense, but you are being a little condescending here. I've been a reasonably serious photographer for a few decades now, including paid work for some years. I'm professionally interested in sensor design and data processing, so if you have some better sources I am all ears (eyes?).

I didn't know, so I asked - but that is shocking to me. Here's some sources:
http://clarkvision.com/imagedetail/eye-resolution.html
https://photo.stackexchange.com/que...-human-eye-compare-to-that-of-digital-cameras
https://www.cambridgeincolour.com/tutorials/cameras-vs-human-eye.htm#sensitivity

A big part of that problem is metering. Use a modern digital sensor and "develop" the raw data correctly, you will realize that you have more dynamic range available than your output medium is capable of displaying.

I'm aware of this, but it is highly unlikely a dashcam is shooting raw 14bit (per channel) images. More likely 8bit.

Please note: one of the big advantages of cell phones - which is probably what was used in that demonstration - is that they are little computers and as a result can do on-the-fly post processing that stand-alone cameras often can't (though they are getting better). I just took a few photos/videos with a quality point and shoot (Panasonic Lumix DMC-ZS50) and my Samsung Galaxy S8 and the difference is big:

The Lumix has the far superior camera and lens, but the Galaxy is clearly post-processing the video to boost the brightness (albeit making it noisy) before writing it to disk...and even then, the scene from the Canon is far inferior to what I could see with my eyes.

Given your experience, I suspect you are viewing this from a perspective of high-end equipment that doesn't match well with what we are dealing with here.

 

[olivermsun]

I didn't know, so I asked - but that is shocking to me.

I'm confused. Which part is shocking to you?

Here's some sources:
http://clarkvision.com/imagedetail/eye-resolution.html
https://photo.stackexchange.com/que...-human-eye-compare-to-that-of-digital-cameras
https://www.cambridgeincolour.com/tutorials/cameras-vs-human-eye.htm#sensitivity

I am not sure what I am to take away from those sources. We are talking about instantaneous dynamic range, I assume, not the dynamic range of the eye-brain system allowing for tens of minutes of accommodation. What do your sources say? What about actual studies of the human visual system?

I'm aware of this, but it is highly unlikely a dashcam is shooting raw 14bit (per channel) images. More likely 8bit.

If a video camera his being used for vehicle navigation/collision avoidance, it should probably be better than 8-bit. This is an experimental self-driving car that probably costs more than $150. I have no idea what is the source of the video we're being shown publicly, but the vehicle better have something more than an 8-bit backup camera sensor.

Please note: one of the big advantages of cell phones - which is probably what was used in that demonstration - is that they are little computers and as a result can do on-the-fly post processing that stand-alone cameras often can't (though they are getting better). I just took a few photos/videos with a quality point and shoot (Panasonic Lumix DMC-ZS50) and my Samsung Galaxy S8 and the difference is big:

View attachment 222622 View attachment 222623

The Lumix has the far superior camera and lens, but the Galaxy is clearly post-processing the video to boost the brightness (albeit making it noisy) before writing it to disk...and even then, the scene from the Canon is far inferior to what I could see with my eyes.

Given your experience, I suspect you are viewing this from a perspective of high-end equipment that doesn't match well with what we are dealing with here.

Plenty of modern cameras offer on-camera dynamic range adjustment, compression, and even editing. We're not talking ultra-high-end equipment, but current < $500 system cameras.

But to return to my earlier point, I can't see why a self-driving car would have a less capable camera and image processor than a cell phone (which these days is pretty darned good) or a cheap system camera. That would seem to be a poor engineering decision, given the total costs of the system and the goal of the demonstration.

 

[nsaspook]

(
I agree with @gleem that the job of the AI is to avoid collisions with objects and it doesn't need that level of identification to do so. I don't know if a floating grocery bag is visible on radar, but anything bigger needs to be avoided.

Huh? Can you name an object besides a floating grocery bag or newspaper that the car should choose to hit instead of avoiding? If I'm approaching an object that's got a cross section of 12 square feet, I'm stopping no matter what it is!

Don't assume a Lidar computer vision system is similar to what the human eye or most video cameras sees.
The problem when using abstract sensors like Lidar is that you often have large possible false detection areas that must be filtered or classified using Feature extraction with a neural network like classification system.

8:00 12:00 in the video for examples.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3230992/

A camera image samples the intensity of a scene at roughly uniform angular intervals. Individual pixels have no notion of range (and therefore of the shape of the surface they represent), but the intensity of the pixels is assumed to be approximately invariant to viewpoint and/or range. As a consequence, the appearance of a feature is reasonably well described by a set of pixel values.

LIDARs also sample the scene at uniform angular intervals, but each sample corresponds to a range measurement. Critically, unlike cameras, the value of each “range pixel” is profoundly affected by the position and orientation of the sensor. As a result, it becomes non-trivial to determine whether two features encoded as a set of <angle, range> tuples match.

Because of these fundamental differences between cameras and LIDARs, there are some challenges if we want to extract features from LIDAR data using extractors from the computer vision field.

 

[HAYAO]

https://jalopnik.com/lidar-maker-velodyne-blame-to-uber-in-fatal-self-drivin-1824027977

Perhaps the system can recognize a single person, a bike, or a person on a bike, but not yet a pedestrian pushing a bike. That sounds like a solvable problem if so. But of course, I wouldn't know.

 

[russ_watters]

I'm confused. Which part is shocking to you?

That you haven't noticed the dynamic range problem that digital cameras have vs our eyes.

I am not sure what I am to take away from those sources. We are talking about instantaneous dynamic range, I assume, not the dynamic range of the eye-brain system allowing for tens of minutes of accommodation.

Agreed. I'm not seeing a problem in the sources.

What do your sources say?

From the third link:

Dynamic range* is one area where the eye is often seen as having a huge advantage. If we were to consider situations where our pupil opens and closes for different brightness regions, then yes, our eyes far surpass the capabilities of a single camera image (and can have a range exceeding 24 f-stops). However, in such situations our eye is dynamically adjusting like a video camera, so this arguably isn't a fair comparison.

Eye Focuses on Background Eye Focuses on Foreground Our Mental Image
If we were to instead consider our eye's instantaneous dynamic range (where our pupil opening is unchanged), then cameras fare much better. This would be similar to looking at one region within a scene, letting our eyes adjust, and not looking anywhere else. In that case, most estimate that our eyes can see anywhere from 10-14 f-stops of dynamic range, which definitely surpasses most compact cameras (5-7 stops), but is surprisingly similar to that of digital SLR cameras (8-11 stops).

On the other hand, our eye's dynamic range also depends on brightness and subject contrast, so the above only applies to typical daylight conditions. With low-light star viewing our eyes can approach an even higher instantaneous dynamic range, for example.

If the "arguably is not a fair comparison" part is what you are referring to, there are two things to take away:
1. Fair or not, it is real. Our eyes rapidly adjust and enable us to see a vastly higher dynamic range than video cameras. Perhaps the video camera could be programmed to take different exposures and combine them or post-process and boost the dark areas like my cell phone did, but clearly the video in question didn't.
2. Even including the "not a fair comparison" part, the eye is still much better than the camera instead of vastly better.

What about actual studies of the human visual system?

To be honest, I see this as such a mundane, everyday issue that it hadn't occurred to me to look for scientific studies of it. I'm not even sure what one would try to say with a study, since it might end up looking like a product comparison against a specific camera. I don't think that's a subject for scientific research and a little bit of googling came up empty. But if you have any sources that you think are more relevant, I'd be happy to read them.

If a video camera his being used for vehicle navigation/collision avoidance, it should probably be better than 8 bit.

Agreed. Please note, the quote of mine you responded to was discussing the dashcam footage. I would hope the navigation/collission avoidance cameras are better.

This is an experimental self-driving car that probably costs more than $150. I have no idea what is the source of the video we're being shown publicly, but the vehicle better have something more than an 8-bit backup camera sensor.

It says "dashcam", so I assume it is a commercially available dash cam. No doubt it cost more than $150 (not sure where that number comes from), but I highly doubt even higher-end dash cams record in anything higher than normal HD, at 10bits color depth (and compressed). The file sizes would be unwieldy.

Plenty of modern DSLRs offer on-camera dynamic range adjustment, compression, and even editing.

Ok. Clearly, the dashcam used in the video we are discussing was not a DSLR and didn't do that sort of processing.

 

[olivermsun]

That you haven't noticed the dynamic range problem that digital cameras have vs our eyes.

Russ, you seem be very assertive/aggressive even on topics even where you are not well informed. This isn't the first thread we've seen this.

That you haven't noticed the dynamic range problem that digital cameras have vs our eyes.

I have noticed that problem and followed it from the slide film era up till modern digital sensors. We've gone from roughly 7 stops to 15 in usable dynamic range. The problem today is representing a scene in a perceptually "correct" way even when the display medium cannot match that range. Furthermore you are talking about a video camera, with stacks of frames to work with, so HDR processing is completely possible.

Agreed. I'm not seeing a problem in the sources.

Seemingly you did not read them honestly and with a critical mind.

From the third link:

If the "arguably is not a fair comparison" part is what you are referring to, there are two things to take away:
1. Fair or not, it is real. Our eyes rapidly adjust and enable us to see a vastly higher dynamic range than video cameras. Perhaps the video camera could be programmed to take different exposures and combine them or post-process and boost the dark areas like my cell phone did, but clearly the video in question didn't.

The linked source says: "If we were to instead consider our eye's instantaneous dynamic range (where our pupil opening is unchanged), then cameras fare much better. This would be similar to looking at one region within a scene, letting our eyes adjust, and not looking anywhere else. In that case, most estimate that our eyes can see anywhere from 10-14 f-stops of dynamic range, which definitely surpasses most compact cameras (5-7 stops), but is surprisingly similar to that of digital SLR cameras (8-11 stops)."

The dynamic range needed to process the scene is a near-instantaneous dynamic range, not a range with allowance for adjustment of the human visual system over minutes or tens of minutes.

The video camera's metering would be expected to adjust over a period of seconds, not minutes. Of course the video camera should process the image to return a reasonable "brightness." What the heck do you think an autometering/autoexposure system does?

The dynamic range that is being quoted is also outdated.

2. Even including the "not a fair comparison" part, the eye is still much better than the camera instead of vastly better.

Again, selective and dishonest reading of the sources.

To be honest, I see this as such a mundane, everyday issue that it hadn't occurred to me to look for scientific studies of it. I'm not even sure what one would try to say with a study, since it might end up looking like a product comparison against a specific camera. I don't think that's a subject for scientific research and a little bit of googling came up empty. But if you have any sources that you think are more relevant, I'd be happy to read them.

Yes, why the heck on a science forum would one look up scientific evidence before posting an opinion?

 

[russ_watters]

Russ you seem very ignorant but assert

Agreed. I'm not seeing a problem in the sources.

Your reply includes a broken double-quote, and the above is all the content I see from you. Perhaps you are editing it, but anyway, if you have an explanation of your position to offer, I'm all ears (eyes).

 

[OmCheeto]

https://jalopnik.com/lidar-maker-velodyne-blame-to-uber-in-fatal-self-drivin-1824027977

... Recognizing pedestrians continues to be a challenge for autonomous technology, ...

Perhaps Uber made a bad choice in picking Arizona:

Arizona has highest rate of pedestrian deaths in the U.S., report says
published March 1, 2018, 2 weeks before this accident.

 

[russ_watters]

Don't assume a Lidar computer vision system is similar to what the human eye or most video cameras sees.

I'm not - I didn't even mention LIDAR in this part of the discussion. This is about what the human driver saw (should have seen) vs the dashcam footage. When people initially saw the dashcam footage, they concluded from it - incorrectly - that it showed that the human driver would not have been able to see the pedestrian. And I'm explaining why the dashcam footage is so poor and why videos that other people have uploaded show a brighter scene.

 

[nsaspook]

Perhaps Uber made a bad choice in picking Arizona:

Arizona has highest rate of pedestrian deaths in the U.S., report says
published March 1, 2018, 2 weeks before this accident.

I see it as a positive. Maybe they will stop jaywalking like manics after this unfortunate incident.

 

[russ_watters]

I'll re-reply to add:

8:00 12:00 in the video for examples.

I just watched from about 7-13 in the video and it's very interesting. Much I've seen before, but not to that level of detail. It's amazing to me that the cars can do as well as they do in these extremely complicated situations. But this accident was not a complicated situation at all -- indeed, the scenario presented at 12:15 was far more complex (albeit lower speed) than the one we're discussing. I did note that he showed the car braking to avoid a flying bird -- not a choice I would make, and I wonder how it does react to a plastic bag.

 

[OmCheeto]

I see it as a positive. Maybe they will stop jaywalking like manics after this unfortunate incident.
...

I don't have a problem with Jaywalkers. I do have a problem with people who don't look both ways before crossing the street.

Videos taken from the inside of our local 50 TON commuter trains:

 

[nsaspook]

I don't have a problem with Jaywalkers. I do have a problem with people who don't look both ways before crossing the street.

I have a major problem with Jaywalkers. What is a 'Jay'.
http://www.todayifoundout.com/index.php/2012/07/origin-of-the-term-jaywalking/

Contrary to popular belief, the term jaywalking does not derive from the shape of the letter “J” (referencing the path a jaywalker might travel when crossing a road). Rather, it comes from the fact that “Jay” used to be a generic term for someone who was an idiot, dull, rube, unsophisticated, poor, or simpleton. More precisely, it was once a common term for “country bumpkins” or “hicks”, usually seen incorrectly as inherently stupid by “city” folk.

 

[Vanadium 50]

the "safety driver" and car should have easily been able to avoid thi

I don't think they had a safety driver. To really be a backup, she needs to be driving all the time, with the inputs disabled, and she needs to be able to reengage at a moment's notice. If she's sitting in the front seat doing other duties, she's not a safety driver.

was a convicted felon who spent time in jail for armed robbery...

Sounds like they fit the corporate culture perfectly.

 

[OmCheeto]

I have a major problem with Jaywalkers. What is a 'Jay'.
...

...it comes from the fact that “Jay” used to be a generic term for someone who was an idiot,
...​

Ok. You got me there. I was trying to be polite.

 

[nsaspook]

...it comes from the fact that “Jay” used to be a generic term for someone who was an idiot,
...​

Ok. You got me there. I was trying to be polite.

I do agree they should at least look.

This self driving car case shows that history does repeat or at least rhyme.

https://www.vox.com/2015/1/15/7551873/jaywalking-history

https://99percentinvisible.org/episode/episode-76-the-modern-moloch/

 

[OmCheeto]

Some people have claimed the "notasafetydriver" was looking down, and was therefore part of the problem.
I think it would be an interesting experiment, for everyone to place a camera pointing at their face while driving, even a moderate distance.

I came up with this notion yesterday, when I drove the 2000 feet to my corner convenience store, to pick up some essentials.
At least twice, I had panic attacks, realizing I had been distracted by things to my side (I still don't have a cell phone), taking my eyes off the road for at least a second, and some "Jay"walker might be in front of me!

According to wiki's article on "Braking distance"; "A perception-reaction time of 1.5 seconds, and a coefficient of kinetic friction of 0.7 are standard for the purpose of determining a bare baseline for accident reconstruction and judicial notice; most people can stop slightly sooner under ideal conditions."

As far as I can tell, had I been the driver in the Uber car, I would probably also have smooshed that lady, even in non-autonomous mode.

 

[OmCheeto]

That said, I am Trevor Noah's mom.

My younger brother once claimed that I drove like a paranoid schizophrenic.
Last time I rode with him, I decided he was a Jay-driver.

 

[nsaspook]

Some people have claimed the "notasafetydriver" was looking down, and was therefore part of the problem.

This GM car won't need any of that.
http://media.chevrolet.com/media/us...Pages/news/us/en/2018/jan/0112-cruise-av.html

General Motors filed a Safety Petition with the Department of Transportation for its fourth-generation self-driving Cruise AV, the first production-ready vehicle built from the start to operate safely on its own, with no driver, steering wheel, pedals or manual controls.

 

[OmCheeto]

This GM car won't need any of that.
http://media.chevrolet.com/media/us...Pages/news/us/en/2018/jan/0112-cruise-av.html

As an old person, that's kind of uncomfortably "freaky".

Though, having lived through:

1960's: Computers will be cool
through
2010's: Computers are incomprehensibly cool​

and having sat in the seat while my sister, another Jaydriver, drove me to the coast.

I think I would be more comfortable getting in the GM car.

 

[BillTre]

My younger brother once claimed that I drove like a paranoid schizophrenic.
Last time I rode with him, I decided he was a Jay-driver.

Big deal.
A friend of mine said I drive like Mario Andretti!

 

[nsaspook]

Big deal.
A friend of mine said I drive like Mario Andretti!

I hope not.