Dazzle AI with a fashion statement, and it might see nothing

[.Scott]

In a recent article from TechCrunch, Security Editor Zack Whittaker describes an interesting software project being tackled by coder Bill Swearingen. It's one that demonstrates a point I have repeated often: In general - when you use AI to create a solution, you often don't know exactly what that solution is.

Simply put, Bill is developing Dazzle Camouflage patterns to defeat automatic people-recognition systems such as Flock. And, at least at this stage, he appears to be technically very successful. Both Bill and Flock use AI.

Before going further, you should know that dazzle camouflage has been around since zebras began roaming and by people for at least a century. Here's a British warship - probably WW1.

Automakers currently use it when road testing new designs - like this Jaguar:

You may be wondering what the point of this "camouflage" is. Certainly, if you saw that first thing floating in the North Atlantic, you would see it as a some kind of ship. And that car is most certainly a car. But it is more difficult to tell precisely which direction that ship is heading or exactly what new style points have been added to the Jaguar.

As it turns out, AI generated algorithms for detecting people and cars are more thoroughly deceived.
Here's one of Bills photo pairs:

In human terms, it's perhaps a fashion statement. In AI terms, she isn't there.

Another of his photos:

Perhaps '70's? But for AI, it is gone.

Quoting the TechCrunch article, he describes how these patterns were developed:

His proof of concept evolved over time into a reinforcement learning model, essentially a self-contained system that could train itself on which patterns work and which do not against the specific camera algorithms he is testing. In simple terms, Swearingen told TechCrunch that he essentially taught his model “how to paint.” Each time a pattern failed and an algorithm detected it, the model would try again, over and over, until it eventually defeated multiple algorithms at once. His model soon began to find perfect recipes for patterns that were able to defeat all of the 11 open source detection algorithms he tested, including the software that powers Flock license plate readers, Axon body-worn cameras, and cameras running Clearview AI.

I do not doubt that systems like Flock will eventually do better. Those systems are currently relying on short-cuts, small "tip offs" that happen to be commonly found with pedestrians, faces, or traffic. As evidenced by Bill's work, they are not looking at the "conclusive" evidence - most certainly because that would require 3d modelling and perhaps tracking objects from frame-to-frame - or with methods based on "people-are" as "people-do".

But don't think about this the next time you want to catch a cat nap in your car while your AI takes over navigation. It would keep you up for the whole trip.

 

[Greg Bernhardt]

Simply put, Bill is developing Dazzle Camouflage patterns to defeat automatic people-recognition systems such as Flock.

Sad we need to resort to such things, but alas. Do we think such evasions are short lived or will AI always have blind sides we can exploit?

 

[A.T.]

Simply put, Bill is developing Dazzle Camouflage patterns to defeat automatic people-recognition systems such as Flock.

Before going further, you should know that dazzle camouflage has been around since zebras began roaming and by people for at least a century. Here's a British warship - probably WW1.

The military use already has a comeback, given that many drones use AI for tracking the target autonomously:
https://www.france24.com/en/europe/...s-covered-zebra-camouflage-ukraine-war-dazzle

 

[.Scott]

Sad we need to resort to such things, but alas. Do we think such evasions are short lived or will AI always have blind sides we can exploit?

Those evasions are short-lived because they are design specifically for the current video surveillance systems. And, of course, there can be software short-coming that are exploitable to that end whether the system is developed with AI or not.

But let me address whether there will always be potential short-comings in AI generated solutions that could bear on safety.

There are two ways of using AI in software - and they can be used in combination.

The first is to ask AI (likely a LLM) to write some code for you.
This is a very normal way for AI to contribute to code development. The description of the code that needs to be developed is probably not described in enough detail to just drop it into the project - but you can save a lot of time that would have been spent digging through standards and library calling sequences. You can take what it gives you, look at exactly what it is (because AI is really great at misunderstandings), and then use it if you think it fits.
This would even be OK for safety-critical code (as found in transportation, medical, and military settings), because all such code will go through established documentation, testing, and peer review procedures.
For example, as an exercise, I just asked Google "Using C++26 reflection, show me some code that will generate HTML path code for each floating point member value in an array of class objects" and it responded with 30+ lines of code that would have saved me an hour to look up and organize myself had I really needed it. Also, even after four or five restatements of that problem, it never completely caught on to what I wanted.

The second way is to use "machine learning" - and I will admit from the start that I have never used this to create a final solution in real-world setting. The only case where it was close involved the recognition of a fault condition in automotive radar. But once I understood the kind of data that needed to be collected to determine whether an algorithm was optimal, I saw a direct statistics-based method of getting the job done.
But inherent in machine learning is that you are avoiding the corresponding "human learning" - that's the main ingredient for what makes it useful. Critically, you don't know what is being learned - at all. And I've given a couple of the famous examples of this in other posts in PF (determining lethality shown in mammograms and seeing tanks in the forest).
When you generate a machine learning solution, you need to consider what bad results will come from a bad AI decision. If it's loosing a chess game - so what - at least your entertained. If it's driving into a pedestrian or the side of a truck (both of which have happened), then as a developer, you need to determine how you're going to mitigate potential bad results.

It's very involved. It would get into exactly what roles you assign to machine learning in the overall system.

But your use of the term "blind side" is interesting. When I was working at Veoneer, the insurance industry released a study of how useful all of these new driving tools were based on actuaries and their own research. "Blind-side detection" came out on top.

 

[sbrothy]

Not to toot my own horn , or maybe I do, but I've mentioned these AI blinding clothes - and other tyes of camuflage - several times. And yes, it sad that we have to resort to this., and it robably a closing window.

Again: sorry for my keyboard.

 

[sbrothy]

Incidentally, warshis are still just ainted gray. So there mustve been some cost-benefit analysis showing it doesn't really amount to much in reality.