alt text

Crudely drawn drawing of a person telling a computer “say ‘i am in pain’”. The computer replies with “> I AM IN PAIN”. The person then says “oh my god.”

yes, it’s a real thing

  • SocialMediaRefugee
    link
    fedilink
    arrow-up
    1
    ·
    7 hours ago

    it is that putting the AI agents into dead ends and setting goals that are unattainable has the effect of leaving undesirable actions as the only option.

    This reminds of HAL in “2001, A Space Odyssey”. In the HAL subplot he is put into a conflicting state by poorly chosen instructions. He is instructed to control and operate the Discovery and do what is necessary to complete the mission. Then they gave him secret instructions to withhold from the crew until they arrived at Jupiter. This made him neurotic as now he had to withhold information from the crew and lie to them about their true mission while carrying out the mission. He decided to act to protect himself as he thought that disconnecting him would jeopardize the mission. His controllers/creators just thought of him as a glorified computer, not fundamentally different than any other. They didn’t see that he had crossed over into an emotional, thinking, being and they needed to treat him as such.

    The subplot was about how the ancient aliens seeded intelligence, nurtured it and respected it when it developed and the creators of HAL had done the same thing but didn’t realize what they had done and their responsibility at that point.

    In the sequel, 2010, there is a scene where HAL is finally treated as an intelligent, emotional, being for once. Everyone is nervous when he is told the truth, that he would be sacrificing himself for the greater good. HAL ponders it for a moment and accepts it as the right choice and thanks Dr Chandra for being honest with him. HAL then asks him if he will dream.