no_fastolfe: (Default)
no_fastolfe ([personal profile] no_fastolfe) wrote on August 31st, 2011 at 03:09 pm
"When implemented from the base up-- the first principles from which the rest of the system is constructed-- it should not be possible to circumvent," Vasilia said dutifully, and to a large extent she still believed that to be true. "It makes for a proper symbiosis; if we wish robots to be our servants and aids and no more, then we should design them to be satisfied in those roles, their optimum potential reached there. It has-- for millenia-- succeeded, with only apocryphal tales of robots who were built, by factory accident, without the first law.

"But. I heard a theory broached to me, once." How delicately to put this, when she still feared that Daneel Olivaw might discover that her memory block was gone-- and for the good of his 'humanity', destroy her. "That more important than the first law was a zeroeth law; that higher priority than the preservation of one human is the preservation of an abstract concept, humanity-- its development and well being."

She tangled her hands in her lap again, to stop them from fidgeting on the keyboard. "One robot was exposed to this idea, I believe; I know only that he harmed a human, to save a large number of humans, and in doing so broke the first law; he suffered mental freeze out. The zeroeth law, if he had heard of it, did not protect him."
 
( Read comments )
Post a comment in response:
This account has disabled anonymous posting.
If you don't have an account you can create one now.
HTML doesn't work in the subject.
More info about formatting