cross-posted from: https://lemmy.blahaj.zone/post/46082903
In fairness, the argument that AI is a cult/false religion/false god has been made many times before, however I believe that AI and the AI community most closely aligns with a cult, and here’s why:
Indoctrination techniques:
Generally, techniques that indoctrinate users into the cult tend to follow similar trends, which are notable in many of the AI models having similar effects on the users who subsequently become frequent users of the technology. Broken down, I’d say it’s something like this:
-Innocuous beginnings: Cults are notorious for beginning the indoctrination process with something simple and seemingly harmless on the surface (I.e. conferences, videos, films, games, etc.) these are all real techniques that various cults have used in the past, to somewhat success. Likewise, AI chatbots, image and video generators present just this: an invitation to engage with something fun and seemingly harmless, something that engages the audience and draws their attention.
-Love bombing: Perhaps the most notorious hook of cults is love bombing — giving heaps of praise and complements to the joining members. Likewise, AI chatbots constantly tell you how good your point is or how you’re always on the right track (“you’ve hit the nail on the head”, “that is an amazing point” are but a few I have read of Gemini) This is undoubtedly a form of love bombing.
-Promise of more: Cults often work by enticing their audience with something good, but promising something great (something better) later. Likewise, AI tech companies are always promising more, more, more if you subscribe/follow their work. What better reward for AI users can there be but the promise of bigger and better AI? This is exactly how both cults and AI operate.
Beyond the indoctrination similarities, there are also many, many peripheral similarities (by which I mean subtle, not less important similarities) which include:
-reach and scope: historical cults have often managed to expand their reach across multiple countries around the world. Need I explain how this is similar to the reach and scope of AI?
-withholding/hiding information
-leaders who turn out to be crooks/criminals
-leaders who amass fortunes
And many more.
What do you think? Are the similarities superficial, or could AI actually be a worldwide cult hidden in plain sight?
It’s just a product of optimizing for engagement maximization at all costs, like tons of other products from Big Tech.
Much of the sycophancy and love-bombing you see in modern LLMs comes from RLHF:
https://en.wikipedia.org/wiki/Reinforcement_learning_from_human_feedback
They’re engineered as technical tools, and outputs humans prefer are selected in training.
…The problem is that Big Tech wrapped what’s really glorified autocomplete or a coding/text processing bot, optimized to please humans with its output, in extensive scaffolding to make it look like an omniscient oracle. Every word that comes out of Altman’s mouth is textbook cult language, and they engineer that “vibe” into the system prompt and RAG system.
Take the raw LLM weights by themself, run in a purely technical context… and it’s obviously not a god. Even to the average person, its more like fancy spellcheck; a tool.
It’s all the dressing-up, hype, deliberate obfuscation, the positive feedback loops and such that turn them into a machine that can perpetuate a cult.
What I’m trying to say is it’s not “runaway technology.”
It’s people.
It’s people doing this on purpose.
“Blaming” the LLM weights instead dilutes their responsibility. Every ounce of blame is on people like Altman or Modi, but THEY want you to think it’s out of their control, hence the whole quasi religious angle.
I know. I’m agreeing with you. All I’m saying is I don’t think AI is 100% innocent in this, even though the driving force will always be it’s corporate owners, and it most certainly still has a role in the mess