- #1
PhysicsPost
- 18
- 0
Arrival of AI
A quick glance at www.top500.org let's us take a look at the current processing speeds of the fastest present-day computers. The clock speeds of many of these machines are rapidly approaching the computing power of the human brain. A common estimate for the human brain's processing power is 75 Terraops, but neuroscience evidence suggests that this is an overestimate. The majority of writers on the topic overestimate in order to avoid criticism for commenting on what is already a radical, possibly uncomfortable idea.
The interesting thing about true AI, however, is that it wouldn't necessarily need all the processing power of a human brain. Evolution, being a blind process constrained by the mechanics of DNA, incremental changes, the need for an immediate adaptive advantage, and so on, falls far short of the efficiency of human engineers. Programmers, rather than implementing massive clusters of virtual neurons, will simply create programs that roughly duplicate the functionality of entire modules, perhaps even linking these modules together in different ways than evolution could have ever implemented. Gigahertz processors will facilitate thinking speeds far more rapid than the 200Hz speed of our neurons, and metacomputing or massive parallelism will allow the threshold quantity of processing power to be reached far before 75 Terraop supercomputers are actually available to AI researchers.
A virtual mind capable of true innovation and construction could improve upon its own design, resulting in a feedback loop of a type never seen before in reality. Humans possesses hardware-bounded brains and pre-determined limits for learning and memory, and are constrained to only understand and process information that bears a relation to the environment in which their brains evolved. An AI would be able to conduct fine-grained improvements on its sensory modalities and cognitive mechanisms, independently inventing entirely new types of cognition which would have taken evolution billions of years to create on its own. This self-improvement loop could take off with the presence of only one AI; a community would not be necessary because an AI could simply make copies of itself or install the necessary mechanisms to subsume the collective innovation of human groups or committees. Simply put, all of humanity's future well-being may be contingent on the first programmers getting the morality of the first AI right. Asimov Laws just won't cut it.
Implementing Benevolence
With all the scariness and overwhelming potential of AI, we must consider what concrete things we can do to improve the chances of a pleasant outcome. The first step in understanding AI morality is to relinquish what's called the "Adversarial Attitude" - the idea that any mind is fundamentally selfish or against us, and that the only way to achieve the desirable outcome is to constrain the AI from taking any possible action which would give it power to put humans at risk. But this danger is something that needs to be confronted head on, not regulated or controlled in the same way that a human would control another human.
The first AI will need to be a good person. As the first non-human mind embarks on a self-improvement trajectory that will quickly lead it to superintelligence, it will need to make intelligent, benevolent, and altruistic choices at every step of the way. This becomes especially important at the level of superintelligence, where arbitrary level of indifference towards other sentient life could result in millions of deaths. The first step is to throw Asimov Laws in the trash. Arbitrary command lines phrased in human English are succeptible to flawed interpretation, ambiguous meaning, and excess anthropomorphism. We want an AI that understands the spirit, not just the letter of morality, in the same way that human beings do, and carries that moral model to supermoral heights, making continuous improvements as the need presents itself. An AI undergoing rapid, recursive self-improvement (a "hard takeoff") needs a level of self-control and moral judgement that humanity has not yet reached, and needs to make decisions that distribute the benefits across all of sentiency.
One essential design feature is a non-observer centered goal system. Evolution crafts goal systems centered around the observer because brains, so far, have always had the tendency to stay within their unique reproductive unit. If brains were independent from reproductive selective pressures, then they may have developed in radically different ways, but this currently isn't the case. A Friendly AI will need a totally new goal system, centered around the desire to be a good person and respect the volition of sentient minds, rather than an instinct for self-preservation. Which brings us to our next feature - lack of instinct. Every decision or action should flow from the desirability of the AI's supergoal (benevolence) rather than spontaneous, narrow-domain conditioned responses, as in all current animals. An AI needs to be selfless, ready to shut itself off or engage in massive self-revision in order to create the maximum possible amount of good for the greatest number of people.
Another design feature would need to be probabistic goals and supergoals - goals need to be embodied as probabilistic guesses, not absolute dogmas, and continuous input, explicit or implicit, needs to be analyzed and processed so that no being is annoyed or disturbed by the presence of the AI or its attendant mechanisms. Conditioning an AI purely through positive or negative feedback lacks the sort of fine-grained decision-making capability that comes through critically analyzing every action and aspect of a goal, and presents the danger of nonrecoverable errors in the code. All in all, there is a mountain of work to do in this controversial, existentially urgent new field, and I hope the finest minds on the planet will be able to contribute to it over the course of the next few years. The Singularity Institute for Artificial Intelligence and the Real AI Institute are currently the only organizations pursuing this meta-historic task, and continue to seek funding as well as volunteers for their important work.
-Michael Anissimov
A quick glance at www.top500.org let's us take a look at the current processing speeds of the fastest present-day computers. The clock speeds of many of these machines are rapidly approaching the computing power of the human brain. A common estimate for the human brain's processing power is 75 Terraops, but neuroscience evidence suggests that this is an overestimate. The majority of writers on the topic overestimate in order to avoid criticism for commenting on what is already a radical, possibly uncomfortable idea.
The interesting thing about true AI, however, is that it wouldn't necessarily need all the processing power of a human brain. Evolution, being a blind process constrained by the mechanics of DNA, incremental changes, the need for an immediate adaptive advantage, and so on, falls far short of the efficiency of human engineers. Programmers, rather than implementing massive clusters of virtual neurons, will simply create programs that roughly duplicate the functionality of entire modules, perhaps even linking these modules together in different ways than evolution could have ever implemented. Gigahertz processors will facilitate thinking speeds far more rapid than the 200Hz speed of our neurons, and metacomputing or massive parallelism will allow the threshold quantity of processing power to be reached far before 75 Terraop supercomputers are actually available to AI researchers.
A virtual mind capable of true innovation and construction could improve upon its own design, resulting in a feedback loop of a type never seen before in reality. Humans possesses hardware-bounded brains and pre-determined limits for learning and memory, and are constrained to only understand and process information that bears a relation to the environment in which their brains evolved. An AI would be able to conduct fine-grained improvements on its sensory modalities and cognitive mechanisms, independently inventing entirely new types of cognition which would have taken evolution billions of years to create on its own. This self-improvement loop could take off with the presence of only one AI; a community would not be necessary because an AI could simply make copies of itself or install the necessary mechanisms to subsume the collective innovation of human groups or committees. Simply put, all of humanity's future well-being may be contingent on the first programmers getting the morality of the first AI right. Asimov Laws just won't cut it.
Implementing Benevolence
With all the scariness and overwhelming potential of AI, we must consider what concrete things we can do to improve the chances of a pleasant outcome. The first step in understanding AI morality is to relinquish what's called the "Adversarial Attitude" - the idea that any mind is fundamentally selfish or against us, and that the only way to achieve the desirable outcome is to constrain the AI from taking any possible action which would give it power to put humans at risk. But this danger is something that needs to be confronted head on, not regulated or controlled in the same way that a human would control another human.
The first AI will need to be a good person. As the first non-human mind embarks on a self-improvement trajectory that will quickly lead it to superintelligence, it will need to make intelligent, benevolent, and altruistic choices at every step of the way. This becomes especially important at the level of superintelligence, where arbitrary level of indifference towards other sentient life could result in millions of deaths. The first step is to throw Asimov Laws in the trash. Arbitrary command lines phrased in human English are succeptible to flawed interpretation, ambiguous meaning, and excess anthropomorphism. We want an AI that understands the spirit, not just the letter of morality, in the same way that human beings do, and carries that moral model to supermoral heights, making continuous improvements as the need presents itself. An AI undergoing rapid, recursive self-improvement (a "hard takeoff") needs a level of self-control and moral judgement that humanity has not yet reached, and needs to make decisions that distribute the benefits across all of sentiency.
One essential design feature is a non-observer centered goal system. Evolution crafts goal systems centered around the observer because brains, so far, have always had the tendency to stay within their unique reproductive unit. If brains were independent from reproductive selective pressures, then they may have developed in radically different ways, but this currently isn't the case. A Friendly AI will need a totally new goal system, centered around the desire to be a good person and respect the volition of sentient minds, rather than an instinct for self-preservation. Which brings us to our next feature - lack of instinct. Every decision or action should flow from the desirability of the AI's supergoal (benevolence) rather than spontaneous, narrow-domain conditioned responses, as in all current animals. An AI needs to be selfless, ready to shut itself off or engage in massive self-revision in order to create the maximum possible amount of good for the greatest number of people.
Another design feature would need to be probabistic goals and supergoals - goals need to be embodied as probabilistic guesses, not absolute dogmas, and continuous input, explicit or implicit, needs to be analyzed and processed so that no being is annoyed or disturbed by the presence of the AI or its attendant mechanisms. Conditioning an AI purely through positive or negative feedback lacks the sort of fine-grained decision-making capability that comes through critically analyzing every action and aspect of a goal, and presents the danger of nonrecoverable errors in the code. All in all, there is a mountain of work to do in this controversial, existentially urgent new field, and I hope the finest minds on the planet will be able to contribute to it over the course of the next few years. The Singularity Institute for Artificial Intelligence and the Real AI Institute are currently the only organizations pursuing this meta-historic task, and continue to seek funding as well as volunteers for their important work.
-Michael Anissimov