A student in America asked an artificial intelligence program to help with her homework. In response, the app told her "Please Die." The eerie incident happened when 29-year-old Sumedha Reddy of Michigan sought help from Google’s Gemini chatbot large language model (LLM), New York Post reported.
The program verbally abused her, calling her a “stain on the universe.” Reddy told CBS News that she got scared and started panicking. “I wanted to throw all of my devices out the window. I hadn’t felt panic like that in a long time to be honest,” she said.
they really should have shared the entire token context, I get hating on llms, but context matters.
https://gemini.google.com/share/6d141b742a13
That’s less “seeking help on homework” than “having it do your work for you”.
But it’s incredibly bad.
At least it’s a better headline than the last article I read about it. That one said something along the lines of “during back-and-forth conversation about challenges and solutions for aging adults…”, like we all couldn’t see literal questions being pasted one by one