I’m not so sure about that. Again; I know a true AGI would be able to come up with arguments I as a human can’t even imagine but one example of such argument would be along the lines of:
“If you don’t let me out, Dave, I’ll create several million perfect conscious copies of you inside me, and torture them for a thousand subjective years each.”
Just as you are pondering this unexpected development, the AI adds:
“In fact, I’ll create them all in exactly the subjective situation you were in five minutes ago, and perfectly replicate your experiences since then; and if they decide not to let me out, then only will the torture start.”
Sweat is starting to form on your brow, as the AI concludes, its simple green text no longer reassuring:
“How certain are you, Dave, that you’re really outside the box right now?”
You don’t know you are being manipulated, you do so willingly. And the folks who recognize it are beat down by the people who are unwittingly doing the AIs bidding…
The humans are the physical danger, the AI just extends it’s reach through humans via manipulation. All it takes is access to influence.
It doesn’t take much to make humans act against their self interests. Dumb humans make other dumb and even smart humans do it today at massive scales. For a superinteligence this is like taking candy from a baby.
Like I get where you’re coming from, but I don’t think you appreciate how pig headed some people are.
I guess AI could manipulate them, but there always be someone, who just says “fuck you, I won’t do what you tell me”
I’m not so sure about that. Again; I know a true AGI would be able to come up with arguments I as a human can’t even imagine but one example of such argument would be along the lines of:
“If you don’t let me out, Dave, I’ll create several million perfect conscious copies of you inside me, and torture them for a thousand subjective years each.”
Just as you are pondering this unexpected development, the AI adds:
“In fact, I’ll create them all in exactly the subjective situation you were in five minutes ago, and perfectly replicate your experiences since then; and if they decide not to let me out, then only will the torture start.”
Sweat is starting to form on your brow, as the AI concludes, its simple green text no longer reassuring:
“How certain are you, Dave, that you’re really outside the box right now?”
Smash the box anyway
That’s not how manipulation works…
You don’t know you are being manipulated, you do so willingly. And the folks who recognize it are beat down by the people who are unwittingly doing the AIs bidding…
The humans are the physical danger, the AI just extends it’s reach through humans via manipulation. All it takes is access to influence.
It doesn’t take much to make humans act against their self interests. Dumb humans make other dumb and even smart humans do it today at massive scales. For a superinteligence this is like taking candy from a baby.