Literally had a coworker, who unfortunately is a bit above me in the hierarchy, do this.
A system that I basically admin/manage was having issues in a way related to the system she manages. She reported an issue to me and I explained how the systems were related and asked her if she had any ideas on solutions from her end. She responded with fairly detailed instructions about where to find a setting in my system that would solve the problem.
Except the where was kind of simultaneously vague and specific at the same time. The description and name of the setting was very specific (and unfamiliar to me) but the location of the setting was one level too high in the hierarchy - there were no settings there… all settings were one level deeper in the hierarchy in subgroups. So I trolled through them all looking for the setting and couldn’t find it anywhere - I even ran a search and couldn’t find it.
I asked her where she found this setting was documented because it was having trouble finding it… and she responded with the name of a popular AI Chatbot, and three more similar “solutions” that also wasted my time.
chatbots always seem to be like this. Yesterday it was a specific website (for a paid service) with help offered by a chat bot built in to the UI. It was telling me where to find settings with specific instructions that just didn’t match the UI.
One of it’s answers was longwinded instructions on how to build an app that would change the settings via their API. There were a few sentences at the end suggesting a “low code” option where I could change the setting in the web ui.
I think what I was doing is a fairly common use case, a guide explaining how to achieve what I wanted would have been much quicker.
I have had this happen pretty often. I think even the wording used in the “prompt” can also influence whether the chatbot hallucinates a solution.
I remember recently asking a third party’s mandatory chatbot support system (if it can’t help, it then forwards you to a human) whether I was able to change the mapping of some fields in an upload to them without the need for a support agent. It walked me through the process but it didn’t work. Eventually, after three failed chatbot modifications to the instructions, I was forwarded to a human that confirmed I was not, in fact, able to do it myself. Such a fucking waste of time.
What gets me is that despite how often LLMs hallucinate a completely incorrect answer like this, for some reason people still blindly trust them and regurgitate their output without telling you where it came from.
You would think after the first time of encountering an output that is verifiably false people would at least have some hesitancy in the output, but they still keep blindly trusting the language guessing machine.
She does not have a level of access to the system to implement or even view the settings in the “solution” but she does have access to the support documentation. I was still suspicious from the beginning.
The documentation is also not publicly available so I am guessing the chatbot just used words that would sound feasible from other similar systems’ documentation.
Literally had a coworker, who unfortunately is a bit above me in the hierarchy, do this.
A system that I basically admin/manage was having issues in a way related to the system she manages. She reported an issue to me and I explained how the systems were related and asked her if she had any ideas on solutions from her end. She responded with fairly detailed instructions about where to find a setting in my system that would solve the problem.
Except the where was kind of simultaneously vague and specific at the same time. The description and name of the setting was very specific (and unfamiliar to me) but the location of the setting was one level too high in the hierarchy - there were no settings there… all settings were one level deeper in the hierarchy in subgroups. So I trolled through them all looking for the setting and couldn’t find it anywhere - I even ran a search and couldn’t find it.
I asked her where she found this setting was documented because it was having trouble finding it… and she responded with the name of a popular AI Chatbot, and three more similar “solutions” that also wasted my time.
chatbots always seem to be like this. Yesterday it was a specific website (for a paid service) with help offered by a chat bot built in to the UI. It was telling me where to find settings with specific instructions that just didn’t match the UI.
One of it’s answers was longwinded instructions on how to build an app that would change the settings via their API. There were a few sentences at the end suggesting a “low code” option where I could change the setting in the web ui.
I think what I was doing is a fairly common use case, a guide explaining how to achieve what I wanted would have been much quicker.
I have had this happen pretty often. I think even the wording used in the “prompt” can also influence whether the chatbot hallucinates a solution.
I remember recently asking a third party’s mandatory chatbot support system (if it can’t help, it then forwards you to a human) whether I was able to change the mapping of some fields in an upload to them without the need for a support agent. It walked me through the process but it didn’t work. Eventually, after three failed chatbot modifications to the instructions, I was forwarded to a human that confirmed I was not, in fact, able to do it myself. Such a fucking waste of time.
What gets me is that despite how often LLMs hallucinate a completely incorrect answer like this, for some reason people still blindly trust them and regurgitate their output without telling you where it came from.
You would think after the first time of encountering an output that is verifiably false people would at least have some hesitancy in the output, but they still keep blindly trusting the language guessing machine.
Next time ask her to screenshot the solution for you or ask her to show you.
She does not have a level of access to the system to implement or even view the settings in the “solution” but she does have access to the support documentation. I was still suspicious from the beginning.
The documentation is also not publicly available so I am guessing the chatbot just used words that would sound feasible from other similar systems’ documentation.