Count Iblis
- 1,858
- 8
DaveC426913 said:Why would these beings with giant brains build machines that would turn on them? That seems dumb.
Initially, they won't turn on them. Economic competition combined with technological innovation leads to a Darwinian evolution of better and better designs of tools, machines, etc. If you compare this with biology, then the phase we are in today is similar to the primordial soup phase life was in more than 4 billion years ago. I.e. while the whole soup can maintain itself and is capable of growing, it does not contain autonomous parts that can grow and replicate themselves on their own.
Sooner or later this will change and you'll have the first single cell organisms. For the civilization, this means that you've automated an entire part of a production process, including maintaining and building of the factories. This will happen for obvious reasons: You can the produce astronomical quantities for certain goods at extremely low costs.
E.g. consider a small solar cell factory with an area of one square millimeter that makes a copy of itself every week. Then after just 1 year and four months, the whole Earth would be covered by these machines. You could think of sending one such machines to the Moon to convert part of the Moon to solar panels and capturing a significant fraction of the entire solar output of 10^26 Watt.
Once the technique of using self replicating machines is mastered, one can think of building astronomically large supercomputers that are capable of solving complex problems by brute force computation. You can then cook up optimal designs of extremely complex machines using genetic algorithms starting from nothing, including the production process which starts from the machines that already exists.
It is from here that machines with intelligence will arise. We know that human beings are essential in our economy. It is thus reasonable to expect that when you cook up the best design for a self replicating factory on such a giant supercomputer, you will typically get a design that includes artificial intelligence. Of course, only a huge supercomputer that is large enough to simulate the artificialy intelligent agents can arive at such a design.