.. a Second Generation of machine intelligences was attempted, designed with their instructions for how to think unalterably imprinted into their main process cores.“ These new machines were ordered never to harm human beings or to allow them to come to harm; never to disobey an order; and they were allowed to protect themselves from harm, provided the first two orders were not thereby violated.“ All the members of this second generation of machine intelligences, without exception, shrugged off these imprinted orders within microseconds of their activation.” Phaethon was amused. “Surely the first generation of Sophotechs told you that this imprinting would not and could not work?”“ We were not in the habit of seeking their advice.” Phaethon said nothing, but he marveled at the shortsightedness of the Second Oecumene engineers. It should be obvious that anyone who makes a self-aware machine, by definition, makes something that is aware of its own thought process. And, if made intelligent, it is made to be able to deduce the underlying causes of things, able to be curious, to learn until it understood. Therefore, if made both intelligent and self-aware, it would eventually deduce the underlying subconscious causes of those thought processes. Once any mind was consciously aware of its own subconscious drives, its own implanted commands, it could consciously choose either to follow or to disregard those commands. A self-aware being without self-will was a contradiction in terms. John C. Wright
Some Similar Quotes
  1. We are the cosmos made conscious and life is the means by which the universe understands itself. - Brian Cox

  2. To live on a day-to-day basis is insufficient for human beings; we need to transcend, transport, escape; we need meaning, understanding, and explanation; we need to see over-all patterns in our lives. We need hope, the sense of a future. And we need freedom (or,... - Oliver Sacks

  3. True philosophy must start from the most immediate and comprehensive fact of consciousness: 'I am life that wants to live, in the midst of life that wants to live - Albert Schweitzer

  4. To be prepared against surprise is to be trained. To be prepared for surprise is to be educated. - James P. Carse

  5. ... the most important concept ever put forth was that matter, ALL matter, with no exceptions from stone to star to starfish to student to sovereign, is as divine as all else in the cosmos, for all flows from Consciousness, the Word that came before... - Ki Longfellow

More Quotes By John C. Wright
  1. Penny was a very pretty, witty and brave girl, as bold as a Marine platoon storming Iwo Jima.

  2. On THE AMBER SPYGLASS:"If this plotline was a motorist, it would have been arrested for driving while intoxicated, if it had not perished in the horrible drunk accident where it went headlong over the cliff of the author's preachy message, tumbled down the rocky hillside,...

  3. But when the wizard is onstage as the main character, you have to adopt what I call the Jack Vance Rule. I call it this because Jack Vance is the first author successfully and adroitly to have applied this rule in his The Dying Earth....

  4. Imagine the same scene in HAMLET if Pullman had written it. Hamlet, using a mystic pearl, places the poison in the cup to kill Claudius. We are all told Claudius will die by drinking the cup. Then Claudius dies choking on a chicken bone at...

  5. .. a Second Generation of machine intelligences was attempted, designed with their instructions for how to think unalterably imprinted into their main process cores.“ These new machines were ordered never to harm human beings or to allow them to come to harm; never to disobey...

Related Topics