AI is a thing...it may even be a very big thing, but it’s not going to be *everything*.

I tend to agree: there will always be real estate novelists, who never have time for a wife. And plumbers.

But I'm uneasy about money and power. That's not to say that as a species, we're doing the best at managing these already: most of humanity is already pretty poor and disenfranchised. And in contemplating AGI, I can think of many more paths leading to our societal wealth and power becoming even less evenly distributed than I see pathways that "seem most likely to effect" our "Safety and Happiness", as Jefferson et. al. phrased it.

Not to say that AGI couldn't do this. More justice, less inequality; more sharing and growing of the commons than exploitation and destruction. Maybe it would just pull all of the levers necessary to implement the Kyoto protocol, and let the chips fall.

In other words, the ethos absorbed by recursive AI models could be that of the Enlightenment, or Buddhism, or the Beatitudes. Lincoln, Jefferson, Madison, Washington.

But while we can hope that, I think we have no good grounds to expect that. Psychopathy is also possible, perhaps equally so.

It's simply an experiment that hasn't ever been run, and I'm not at all reassured by the obvious observation that those in charge of its development are clearly not the wisest, most mature adults among us.

But I agree with Goofy that there's no sense in getting our knickers in a twist, particularly seeing as how there's not a bloody thing we can do about it. We're all about to find out together.

--sutton