On Fri, Jun 24, 2016 at 05:24:00PM -0700, Jim Vassilakos wrote: > So, my first guess is that when a society develops strong AI, > clearing the final hurdle is not due to a breakthrough in hardware > so much as software, which means that the whole process will be > replicable on a mass scale and that various “finished” AIs could be > copied rather easily. At the moment, it looks like that could go either way. There are certainly reasons to believe that we could get a lot better AI on existing hardware than we have so far, but there are also reasons to think that we may need better (and so very expensive) hardware to approach or exceed human capabilities. > I’m guessing that the first strong AIs will be able to inhabit robot > bodies and will be raised to some extent as are children. [...] > My guess is that AIs that chronically misbehave will be aborted. This is where things start to get very messy. The number of ways strong AI might develop are extremely broad. If socially acceptable behaviour cannot be designed but must be learned as humans do, that indicates a dangerous lack of understanding about how the AIs actually work internally. If strong AI is just a "software issue" so that hardware capable of supporting such sapience is cheap, portable, and widely available by the time it develops, that's probably the most dangerous and unpredictable path of all. It means that of the few initial strong AIs, nobody really knows how they think, and whichever convinces humanity of their benignity first probably gets to replicate billionfold in short order. What's more, it also means that there is very much faster hardware on which they might run than the mass produced hardware generally available. So virtually overnight, there's a billion copies of some AI in robot bodies, of unknown internal thought processes and motivations, and probably thousands of copies embodied as super-AIs. So the society ends up with a huge monoculture of poorly understood beings, who can easily copy themselves into commodity hardware, and some who can think and learn just as well and very much faster than any human. That's not an automatic recipe for disaster, but it's fertile ground for one. > They will create entirely new sciences and technologies and > eventually even more powerful versions of themselves. One day, > humanity and strong-AI may merge to an extent, if not completely. That's probably a scenario with about the most hope for humanity under these assumptions, but I think with the given starting points it's less likely to arise than otherwise. [ Snipped tables that have been saved and look like a useful starting point for a game ] > I’ll stop here for the moment, but you get the general idea. What do you > think so far? Very interesting indeed. It's probably a rockier and more dangerous start to the development scenario than I would hope for, but the assumption that strong AI can inhabit cheap hardware as soon as it is developed does seem to be a common theme across a lot of science fiction, both optimistic and horribly grim. - Tim
