Character AI has filed a motion to dismiss a case brought against it by the parent of a teen who committed suicide allegedly after becoming hooked on the company's technology.
Told him not to but also failed to drop the fantasy and understand the euphemism of “come home”. Almost any human would put a full stop to the interaction and if they didn’t they should also be charged.
Those conversations didn’t happen at the same time from what I gather. These things don’t have infinite context size and at the rate he seemed to be using it, the conversation probably “resets” every few days.
No actual person would be charged for these kinds of messages in any case, pure exaggeration imo.
The context size wouldn’t have really mattered because the bot was invested in the fantasy. I could just as easily see someone pouring their heart out to a bot about how they want to kill people but said in a tactful way that the bot just goes along with it an essentially encourages violence. Again, the bot won’t break character or make the connection that this isn’t just make believe, this could lead to real harm.
This whole, “It wasn’t me, it was the bot,” excuse is a variation on an excuse many capitalists have used before. They put out a product they know little about but they don’t think too hard because it sells. Then hundreds of people get cancer or poisoned and at worst there’s a fine but no real blame or jail time.
Character AI absolutely could create safeguards that would avoid harm but instead they’re putting in the maximum effort it seems to do nothing about it.
The context only mattered because you were talking about the bot missing the euphemism. It doesn’t matter if the bot is invested in the fantasy, that is what it’s suppose to do. It’s up to the user to understand it’s a fantasy and not reality.
Many video games let you do violent things to innocent npcs. These games are invested in the fantasy, as well as trying to immerse you in it. Although It’s not exactly the same, it’s not up to the game or the chatbot to break character.
Llms are quickly going to be included in video games and I would rather not have safeguards (censorship) because a very small percentage of people with clear mental issues can’t deal with them.
It’s up to the user to understand it’s a fantasy and not reality.
I believe even non-AI media could be held liable if it encouraged suicide. It doesn’t seem like much of a leap to say, “This is for entertainment purposes only,” and follow with a long series of insults and calls to commit suicide. If two characters are taking to each other and encourages self-harm then that’s different. The encouragement is directed at another fictional character, not the viewer.
Many video games let you do violent things to innocent npcs.
NPCs, exactly. Do bad things to this collection of pixels, not people in general. The immersion factor would also play in favor of the developer. In a game like Postal you kill innocent people but you’re given a setting and a persona. “Here’s your sandbox. Go nuts!” The chat system in question is meant to mimic real chatting with real people. It wasn’t sending messages within a GoT MMO or whatnot.
Llms are quickly going to be included in video games and I would rather not have safeguards (censorship) because a very small percentage of people with clear mental issues can’t deal with them.
There are lots of ways to include AI in games without it generating voice or text. Even so that’s going to be much more than a chat system. If Character AI had their act together I bet they’d offer the same service as voice chat even. This service was making the real world the sandbox!
Told him not to but also failed to drop the fantasy and understand the euphemism of “come home”. Almost any human would put a full stop to the interaction and if they didn’t they should also be charged.
Those conversations didn’t happen at the same time from what I gather. These things don’t have infinite context size and at the rate he seemed to be using it, the conversation probably “resets” every few days.
No actual person would be charged for these kinds of messages in any case, pure exaggeration imo.
The context size wouldn’t have really mattered because the bot was invested in the fantasy. I could just as easily see someone pouring their heart out to a bot about how they want to kill people but said in a tactful way that the bot just goes along with it an essentially encourages violence. Again, the bot won’t break character or make the connection that this isn’t just make believe, this could lead to real harm.
This whole, “It wasn’t me, it was the bot,” excuse is a variation on an excuse many capitalists have used before. They put out a product they know little about but they don’t think too hard because it sells. Then hundreds of people get cancer or poisoned and at worst there’s a fine but no real blame or jail time.
Character AI absolutely could create safeguards that would avoid harm but instead they’re putting in the maximum effort it seems to do nothing about it.
The context only mattered because you were talking about the bot missing the euphemism. It doesn’t matter if the bot is invested in the fantasy, that is what it’s suppose to do. It’s up to the user to understand it’s a fantasy and not reality.
Many video games let you do violent things to innocent npcs. These games are invested in the fantasy, as well as trying to immerse you in it. Although It’s not exactly the same, it’s not up to the game or the chatbot to break character.
Llms are quickly going to be included in video games and I would rather not have safeguards (censorship) because a very small percentage of people with clear mental issues can’t deal with them.
I believe even non-AI media could be held liable if it encouraged suicide. It doesn’t seem like much of a leap to say, “This is for entertainment purposes only,” and follow with a long series of insults and calls to commit suicide. If two characters are taking to each other and encourages self-harm then that’s different. The encouragement is directed at another fictional character, not the viewer.
NPCs, exactly. Do bad things to this collection of pixels, not people in general. The immersion factor would also play in favor of the developer. In a game like Postal you kill innocent people but you’re given a setting and a persona. “Here’s your sandbox. Go nuts!” The chat system in question is meant to mimic real chatting with real people. It wasn’t sending messages within a GoT MMO or whatnot.
There are lots of ways to include AI in games without it generating voice or text. Even so that’s going to be much more than a chat system. If Character AI had their act together I bet they’d offer the same service as voice chat even. This service was making the real world the sandbox!