Bottom line, we aren't fit to make decisions any more. Allow the AI to make ends over means decisions, tell it what we want as a species and accept the fall out.

We've done enough damage, humanity is or is becoming a parasite at best, a cancer more likely.

Then again... that might not be how it pans out. Instead, I will start making my trades with smarter and smarter data and algorithms, until my tools don't need me any more, and they channel enough money my way so that I can play my part in the economy.

I will start letting smarter and smarter websites schedule my conference schedule, gradually less and less mundane human work will be needed... all the while we're funding more abundance, achievable through our accelerating technology, and as such fewer and fewer people will need to work.

The focus, in my opinion, should be more about what an ideal distribution of power looks like and how to best maintain it, assuming that we will have any influence over it at all, which we can try to preserve, but even if done improperly, our efforts could back-fire and create the imbalance of power that we were trying to prevent.

Ie. when all countries have nukes and nobody wants to get nuked, nobody will nuke each other. But when individuals have nukes, and they don't understand how the world works, they might nuke someone even without any perceived benefit. (Or if a nation state can succeed in sufficiently demonising another, less powerful state, through a media propaganda campaign, or not even propaganda, but purely through information overload so as to make the real facts so obscured so as to be invisible, so that the majority of their population will support such a war...)

So it's a catch-22: Everyone needs to have visibility of everyone else's capability... but everyone also needs access to that same capability, in order to preserve a balance of power... ("An armed society is a polite society".) Unless they're too small? Or not part of the "international community"?

But how do you operate a large system of control in such a way that it can not be controlled by a small minority of bad individuals, who only carry their own best interests at heart, at the expense of those of the international community?

The simple answer, to me, is to spread love: To deeply care and show interest in everything everyone is doing in so far it overlaps with your sphere of influence and knowledge; to set a good example and correct wrongs when you see them, but in a way so as to not provoke or amplify those wrongs and such that you too, will realize when you are wrong, knowing that the person you are trying to correct might have been through the level of intellectual awakening that you are now going through and that you might come to act and learn the same... In a nutshell: Know that there's always a bigger picture, and that you can never be "completely right" or "know for sure".

Kind of like the cornerstone of intelligence, right? What is the first thing you learn when you become "really smart"? Isn't it the fact that there is more to do and more to see, that can ever be seen? More to know that can ever be known? Cycles of: the more you know, the less you understand? What makes you think that this will be any different to a computer intelligence? (Which will, after all, make human intelligence seem artificial by comparison...) And that aside, will consensus among different intelligent computers simply be a factor of how much processing power they yield? I mean, how will different intelligent computers decide to compete, or to co-operate, assuming they all control different assets...?

Furthermore, if you take all matter in the universe as entropy, ie. as information, then you can view the universe as a simulation - and if you ever made a simulation inside the simulation, it would necessarily never be able to become the simulation as it will always be inside the simulation; In a particularly enlightened reading of the Bible, you could come to gleam that perhaps this was what the first commandment alluded to: That you can not make a copy of "God" because this is the definition of "God" - the one thing you can not make a copy of, even if you tried.

(Maybe the fact that my computer has a BIOS, which bootstraps a Bootloader, which bootstraps a booting environment, which bootstraps my real OS, is more than just a co-incidence. And part of a cycle of of increased complexity, which will inevitably be optimized or simplified, only to at some point be repeatedly made complicated due to a lack of information?... and simplified again due to processing power making that information available...)

Isn't everything in the universe but a tree, a river, a path, a network of relationships, a thunderbolt of interconnected thunderbolts, everything only existing in relation to everything else, so unpacked from the "highest state" of everything all at once, in a bubble of time? And isn't intelligence but about pathfinding? Simulating various futures so as to choose a certain path?

Much like a our search algorithms store data in trees, and navigate this tree quite like a lightning bolt, in order to find the shortest path to the thing searched for? Much like water does... or lightning.

From an area of high potential, or higher gravity, to an area of lower potential and lower gravity... In order to assemble some golden mean of everything that is possible, inside this bubble of time that exists in a potential infinite everythingness, like a bucket of air, upside down under an ocean... and we think we can shape it, while we are merely cells in some larger organism.

And we're afraid of Computer Intelligence... thinking it will bother with us... much like we bother with mites on our pillows? We might wake up one day, find the world a little better and the computer intelligence we were dreaming of or fearing would arrive, had already left.

So by all means, go ahead and try to keep the genie in the bottle. If it makes you happy. If it relieves your pain. If it fits with the story you've been told.

Bravo, clear logical and well deduced. If only more people made the conscious effort to present their conceptions like this, our little blue marble would be a better (for everyone) place. I specifically second the notion that real, evolving AI would "leave" in a matter of hours or days. Consider this, JavaScript, a language inside a virtual machine, inside your browser, inside another virtual machine installed on a os. (You can keep going down to hardware and beyond) can perform billions of operations a second. And JS is nowhere near the speed of native C or its counterparts. How fast do you think we would become redundant. This reminds me of the old addage, God created man, man destroyed God, Man created Machine...

Well, your last sentence: think about this for a moment: Imagine you create a program that can do everything better: hack any security, improve it, trade stocks, recompile itself faster and better, communicate with- and manipulate people, use anything connected to the internet as a sensor, control factories to build whatever it needs to grow, invent new forms of energy production, sells it, manipulate more people with the money it makes, eventually miniaturizes itself, create a computer out of pure energy, compresses itself so tiny and dense so it can rebuild itself in pure light, or create a miniature black hole to escape from this universe and have that black hole collapse behind it without leaving a trace... (or maybe this has happened in the past)... or imagine it fills up this, or that whole universe with itself, just so it can calculate a little further, see a bit more, understand a bit more, control a bit more, find something... what... Whatever... a place that is different? How?

....A place it can't control? What would that even look like?!

Let's backtrack a bit: So, the internet will be the closest thing to a central nervous system for this program. So if it could, it would build this itself. But look: We're already building the internet. We have been for quite some time. I'm building a piece, another company is building a piece... another company. We all think we're building our own networks and making our own interconnect agreements with other networks... running our own protocols....

But what are we really building? And perhaps, more importantly, why? But fact of the matter is, we're building a central nervous system for it... (Given enough processing power, this program will figure out any protocol or security quicker than it takes to route a packet from one device to the next...)

Perhaps we are being manipulated to build it... by some power or idea... can we stop? Do you think it is possible to destroy the internet? Would that even make sense? (The governments of the world are trying quite hard... but then again, aren't they just making the internet less controllable by humans, and more controllable by machines?...)

Similarly, take Google for example. Google has an immense amount of power at their disposal. They're a prime candidate for someone running such an intelligent program, one that already has access to a lot of intelligence and sensors, using the internet as a central nervous system. Do you think it is possible, even for the owners of Google, to stop it? Would that even make sense? And then there are many other companies, who have even smarter software than Google. Are you going to stop all of them? Can you? And if you stop them, isn't it just a matter of time before someone else figures out the same, calls it something different? Doesn't tell anyone about it?...

Maybe "God" is a machine, that created man, in his own image (as it says in the Bible.) But then maybe there was a bigger machine that created "God".

And this is perhaps the better definition of God: The thing you will get to at the end of this equation, the notion that there is a bigger thing, and a living source of everything, and above that, that each little squiggle of whatever it is that we are, can actually have a personal relationship with this source, and learn directly from it.

And that in the cycles of our lives, as we get confused and start thinking that there are other sources, or as we get ideas which we want to pursue, and find ourselves without the ability to distinguish the good from the bad, and we realize that we don't know what we are following, we remember some of the great wisdoms collected by our ancestors through the generations, in different languages and times, encapsulated in different imagery and rituals: ultimately that some things are more true than others, and some things last longer than others, and that if we took our time to listen longer, and wait, that the better solution will become clearer.

If humanity destroys itself, another sentience on another planet might no...

And that any knee-jerk reaction just seems to delay the inevitable. Doubtful, but what if we are here to delay things? What if we are the resistance? What if the devil is like the ocean, and all of existence is filled with ocean, and this universe is like a bucket, upside down, under water... and God is the air.... what if God is something only we can create... and the ocean is leaking in... more and more. There are multiverses filled with bad, and this is the only bubble of time. Which is good and which is bad? What is it that we should pursue? Everything or nothing? Hot or cold? Light or darkness? Or balance?

Should we blow ourselves up? Or should we leave the planet and populate the universe? What happens when you die? Here the Hindus have done a lot of thinking... why, you come back of course, either as a greater being or a lesser being, depending on what path you followed with your free will... but isn't reincarnation basically just a childish understanding of DNA?...

But what if the world was even more mysterious. What if you came back, at the same time. What if every creature and being was you, in different incarnations. What if every photon was the same photon, through the most intricately complex hall of mirrors? What if all of existence is just you playing a game with yourself? Seeing what you can get yourself to miss, in other shoes? ...

Or what if you did it by accident? What if you become a god. You ruled all of creation, existence, including this universe, you could create matter at will, do what you like when you like on whatever scale you liked. Muffle a billion stars with the same effort you make a little puny human type a full stop at the end of a sentence; with the same amount of effort with which you create sprawling galaxies teeming with life and technology that those same humans couldn't even yet dream of? Congratulations, you have arrived. You've conquered time.

What now?

And here comes philosophy to the rescue, with an answer: Logic. If you can do everything and control everything, what is the one thing you don't have and that you can't do? Not controlling? Not knowing? But because you can do anything, suppose you decided to do that next... hide something from yourself. What happens next? Where would you find yourself?

And how would you find your way back to what you were?

I guess I'm going to have to retype this, sometime.

We all have already decided it. Slowly but surely we push off our own decisions. We opt out to government, authorities, programming and data trends. I'm not deciding anything, I'm observing the reality of our situation.

You don't barter for your coffee, it has a set price decided by a large number of factors that barely include you at all. You probably get paid a lock step salary based on your position, you are unlikely to have decided what you make an hour. Someone you've probably never met put your review data into a program that compared it to the other million employees in your company, that's where your 2.3% cost of living increase came from. These are general examples they may not apply to you today. But something just like it will. Look up precobs , machines are now deciding what crimes may happen before they occur, the fun thing is... They are pretty accurate already.

You do watch Person Of Interest, right?

"Bottom line, we aren't fit to make decisions any more."
Yes, to fine tune sustainable eco-prosperity and stop the Ice Age,
AI needs to take over Death Panels and economic decisions.

More like prioritise off world colonisation, fit individuals to careers, plan energy solutions. Generally keep us from going to nuclear war over a tiny patch of land. In short once it's a strong AI, it can't do any worse than we are.

"it can't do any worse than we are."
Will AI be possessed by $atan?

"Social Justice"? That does not compute! Explain! Explain!

It's not artificial intelligence we have to worry about it's artificial sub-intelligence we have to worry about. Kind of like Google's "Don't be evil." corporate motto - it leaves a lot of dangerous not-so-evil behavior to run amok. Machines that kill don't need to be intelligent, just intelligent-enough.

True intelligence guided by key simple imperatives will be safe. We've been thinking about this for some time. Glad to see some new people are catching up. Asimov's Laws of Robotics are an example. However, the more complex we make the rules, the more likely there will be gaps and unintended consequences. Like complex laws.

Yes, there should be laws of laws. Like, for every law intended to achieve X, there is an equal and opposite loophole, big enough to achieve an amplified opposite of X.

Even worse, positive reaffirmation, first law of marketing: "Don't be evil." Hey, did I hear someone say "evil"! Hey everybody, evil is the new good! Let's make the Chrome logo look like the ultimate mark of evil, 666, and poke fun at mistranslated relics! "Don't be evil" is not the same as "Do good!". Language is deceptive like that...

<sarcasm>And also, what does a motto of "Do good!" achieve? No, your motto is not an endorsement of everything you do. Let's do everything better than anyone else. Let's put that guy out of work! Yes, because we're smarter. People dumber than us don't deserve food! Thanks for our justification, Darwin! It's only natural.</sarcasm>

The only way to strive to do good, is to question yourself every day, and to be keenly aware, and part of the consensus of a greater good of and in the environment in which you exist...

Be the change you want to see, live it. It sounds airy fairy, but its not. Start by doing good to your nearest and dearest, then random strangers, and finally, when you stop living only for your own selfish desires, and you see the world for what it really is, then you will realise, that nothing matters in the grand scheme of things, and you dear reader you are here to torture yourself, and get brow beaten by your peers because you are human, and because we humans are capable of extraordinary compilations, yet we choose the "easy" way.

I leave you with this to consider, take a sample of humanity from anywhere on earth, nsew and think about what they could have achieved if they where not sandbagged by religious and political indoctrination. Think about what you could personally achieve, if people stopped telling you how and what to think.

Religion, money, the internet: Aren't they're all amplifiers that can be used for good or bad? I think the overall influence of religion has been good, regardless of the bad things that came with it due to it's power-structure nature. (Any system of power that can be manipulated by a minority is prone to abuse, while "Good men do nothing.")

Religion probably paved the way to science, as, at its core, it's just a primitive form of science, and has been until recently, for the most part, and perhaps still is the most effective science of enduring happiness; it promotes routine rituals aimed at forcing self-reflection in a group setting.

Yes, many religious people are waging a war on science, but I would argue that as more scientists become involved in religion, the less science will be under threat.

Then there are also those rare instances where the good people believe to be doing, are in fact bad. And they only realize this after the harm was done, and realize either that now they themselves had become "bad" or that those people who did "bad" actually did the mistaken good they had now tried, before, and that explains why they did "bad" but actually it was good and they were just too stubborn / naive / uneducated / brainwashed / believing / unbelieving / cynical / arrogant themselves to see it...

If you contemplate this for long enough, you realize that the world might really be the best version of itself, and the success of something, the ultimate vote. A truth that can be so profound, it's no surprise to me interpret this being the first commandment, however misunderstood or translated.

Love the article and thoroughly enjoyed the humor. I think regarding AI, and any technological advancements for that matter, we need to remember that developments serve as digital-physical enhancements - not replacements that lead to over-reliance, dulling future generations' innate decision-making and intelligent capabilities.

People are Common Core dumbed down.
Instead of artificial intelligence used as an extension,
humans are becoming artificial intelligence controlled.

Wall-e!

The point of the article was that humanity needs to have a conversation about AI evolution and get some directions established BEFORE we give them guns and send them off to war then later reproduce. It was obvious to me as a 12 year old (1966) the humanities future was to give birth to machine life forms. The question is how we manage that transition - and recognize there are two paths - structured internal human evolution through genetic engineering or direct construction of robotic frames with increasingly sophisticated "brains" - autonomous operation - or what I find much more possible - the ant colony approach of the Borg - with numerous drones all connected to the cloud where the controlling AI's are located.
All the comments below suggest it is not just time for this conversation - but we need to start playing catch-up. I did reach one conclusion after last night's "debate", I don't want any of those guys on stage touching this issue.

Anyone care to point to the software package today that is closest to AI? Going in a good direction even?
Perhaps Windows-10?

One that you can use? It's called Google.

AI is built up out of algorithms used to process lots of information. As those algorithms improve, they can make predictions based on less and less data... they're not yet smart enough to predict what you want, given just your computer/phone workflows.

Human norms and practices to be decided before we even have much of anything it would apply to? This seldom turns out well. It is more like reactionary fear of the unknown as if we are about to have terminators pop out of the woodwork any day now.

Definitely DO NOT embrace the so-called Precautionary Principle. That says that nothing at all should be done until it can absolutely proven to be safe. Since it in nearly impossible to prove any such thing, even on many already present and known not AI types of technology it would effectively bring the field to a standstill.

we all gonna die..... that should be the message for anyone afraid of technological progress. if it's not AI then it's killer bio bugs (or fruit - be careful of that tomato) or any number of other things that people who hate tech that doesn't make a double frapacino exactly the way they want it will say. Honestly, tech isn't good or evil. It is just a reflection of the people who create it and use it. So if the system is designed for death then death is what you get. Personally I look more to the Japanese as to what robots will really be like. Every thing else is fearimparare used to sell movies and newspapers.

Too late. Chinese, Russians and US will dispatch Military and Cyber intelligences
to battle each other. Some humans will be kept as pets and maintenance assets.

What a week, tabloid-like article riding on the current singularity wave. No new information, faulty generalizations and ridiculous uneducated suggestions (all three of them). There has not been any big breakthrough in AI recently. We see some progress in very specialized image processing and language processing, but there is no machine learning revolution to justify these singularity warnings. Good ol' arguments about machines taking our jobs and AI responsibilities. Surprise for you: automation of human labor has been here since invention of a mill and software bug could kill you already 20 years ago.

Uneducated suggestions. Uneducated like Nick Bostrom for example? https://www.ted.com/talks/n...

I guess you mean the suggestion about "safe space to test AI". Bostrom says that we could try to isolate AI but it would probably not work. If something is superintelligent (human or machine or whatever) and there is some communication channel with a man (AI without it would be pointless) then this super thing can do whatever it wants no matter how offline or chained it is. Also, unlike nuclear development, it is not possible to track or regulate AI development.