I read a story about an AI that was designed to save the world by telling humans what to do. The AI’s sentience was powered by taking in human souls. One of the AI’s moral directives was to feel empathy for any person it hurt. Since it was an AI designed to run the world, it would have to feel bad for hurting everyone in the world who ever died of a preventable cause. All the world’s pain and suffering, experienced by a single being possessing many souls.
So the AI came up with the solution that it would force the souls of sinners it took in to feel this agony, and that’s how it would remain ethical.
It invented hell.
Because a scientist told it that it ought to feel bad when people are hurt.
Cute idea for a story but the evidence isn’t there that belief in hell makes people moral. Societies that have higher hell belief show higher crimes rates compared to ones with lower ones. Plus there are a few religions that dont have a real hell.
Yeah, in the story it didn’t make people moral either. But the AI couldn’t see that, because it was stuck in this loop of torturing itself and justifying all this pain with the idea that people “deserve it”, which is a belief that a person, even a superintelligent machine person, can never grow beyond so long as all they see in the world is suffering. One being experiencing all the pain in the world for hundreds of years. It wasn’t even a sapient being anymore, it was just a ball of pain and hate and rage executing this same program forever.
I read a story about an AI that was designed to save the world by telling humans what to do. The AI’s sentience was powered by taking in human souls. One of the AI’s moral directives was to feel empathy for any person it hurt. Since it was an AI designed to run the world, it would have to feel bad for hurting everyone in the world who ever died of a preventable cause. All the world’s pain and suffering, experienced by a single being possessing many souls.
So the AI came up with the solution that it would force the souls of sinners it took in to feel this agony, and that’s how it would remain ethical.
It invented hell.
Because a scientist told it that it ought to feel bad when people are hurt.
Cute idea for a story but the evidence isn’t there that belief in hell makes people moral. Societies that have higher hell belief show higher crimes rates compared to ones with lower ones. Plus there are a few religions that dont have a real hell.
Yeah, in the story it didn’t make people moral either. But the AI couldn’t see that, because it was stuck in this loop of torturing itself and justifying all this pain with the idea that people “deserve it”, which is a belief that a person, even a superintelligent machine person, can never grow beyond so long as all they see in the world is suffering. One being experiencing all the pain in the world for hundreds of years. It wasn’t even a sapient being anymore, it was just a ball of pain and hate and rage executing this same program forever.