The
usual story here is that someone puts an AI in charge of a paperclip factory, they tell it to figure out how to make as many paperclips as possible, and the AI (being superhumanly intelligent) realizes that it can invent some molecular nanotechnology that will convert the whole solar system into paperclips.
If you have an AI that can invent this kind of nanotechnology, then you don't have paperclip factories anymore to put AI's in. Besides, paperclip factories only have the facilities to make paperclips. They don't have the ability to invent super advanced nanotechnology. This is true for ALL manufacturing facilities. The supreme law of, "I don't want to spend money on stuff I don't need to make my product" pretty much prevents factories that make ball bearings or aluminum cans from making weapons of mass destruction or magic nanotech.
It also prevents people from spending money on AI's that they don't need. If we get to the point where a factory is paying for a general purpose AI that can do millions of tasks that the factory doesn't need it to do, including advanced research and development, then we're to the point where everyone has similar AI that can probably be used to prevent other AI's from doing bad things.
I'm sorry but destroying the world is hard. It takes a
lot of effort. Of the potential world-ending weapons I can think of off the top of my head:
- Nuclear weapons have huge infrastructure chains supporting their research, manufacture, deployment, maintenance, and use. No one, not even an AI, is just going to take them over and wipe us out in a day. Even the launching of existing nukes is easily avoidable by not giving control over them to an AI or any single source (which is what we already do).
- Bioweapons are potentially easy for other advanced AI's to reverse engineer a cure for and are subject to many potential issues with surviving and spreading once released. The recent Covid pandemic shows a (admittedly flawed) way of handling such things. I suspect that by the time we reach the point that we understand how to artificially create super-pathogens, we will also understand how to quickly analyze and stop them.
- 'Nanotech' is a magic word used to describe fantasy technology that will probably never exist. Going from raw materials to even a simple finished product takes a huge amount of work and, often, other raw materials and finished products. Going from raw ore to a refined metal by itself is a rather delicate process that requires various chemicals, high-temperatures, LOTS of energy, and the right conditions to do it correctly. Squirting some grey goo on a rock in Alaska is not going to make a bunch of paperclips.
- Armed robots are countered by other armed robots controlled by other AI's working for humans and not against them. And there's probably going to be a lot more on our side.
anorlunda said:
But a bigger flaw is the most basic. Find a definition of AI that everyone agrees to. Unless people agree as to the subject of a debate, the debate is doomed.
I think the biggest flaw is the complete and utter lack of understanding of how the real world works that most people debating this issue have.
"Oh no! Scary AI will kill us all if we let it!"
"Ok. So don't let it. "
"But it's so much smarter than us!"
"But we're the ones who invented it, created and run the infrastructure that runs it, and maintains its systems."
"But what if it gets control of everything?! Then we're doomed!!"
"Ok. So don't let it do that."
"But it's so much smarter than us!"
...
...
