Other platforms too, but I’m on lemmy. I’m mainly talking about LLMs in this post
First, let me acknowledge that AI is not perfect, it has limitations e.g
- tendency to hallucinate responses instead of refusing/saying it doesn’t know
- different models/models sizes with varying capabilities
- lack of knowledge of recent topics without explicitly searching it
- tendency to be patternistic/repetitive
- inability to hold on to too much context at a time etc.
The following are also true:
- People often overhype LLMs without understanding their limitations
- Many of those people are those with money
- The term “AI” has been used to label everything under the sun that contains an algorithm of some sort
- Banana poopy banana (just to make sure ppl are reading this)
- There have been a number companies that overpromised for AI, and often were using humans as a “temporary” solution until they figured out the AI, which they never did (hence the gag, “AI” stands for “An Indian”)
But I really don’t think they’re nearly as bad as most lemmy users make them out to be. I was going to respond to all the takes but there’s so many I’ll just make some general points
- SOTA (State of the Art) models match or beat most humans besides experts in most fields that are measurable
- I personally find AI is better than me in most fields except ones I know well. So maybe it’s only 80-90% there, but it’s there in like every single field whereas I am in like 1-2
- LLMs can also do all this in like 100 languages. You and I can do it in like… 1, with limited performance in a couple others
- Companies often use smaller/cheaper models in various products (e.g google search), which are understandably much worse. People often then use these to think all AI sucks
- LLMs aren’t just memorizing their training data. They can reason, as recent reasoning models more clearly show. Also, we now have near frontier models that are like 32B, or 21B GB in size. You cannot fit the entire internet in 21GB. There is clearly higher level synthesizing going on
- People often tend to seize on superficial questions like the strawberry question (which is essentially an LLM blind spot) to claim LLM’s are dumb.
- In the past few years, researchers have had to come up with countless newer harder benchmarks because LLMs kept blowing through previous ones (partial list here: https://r0bk.github.io/killedbyllm/)
- People and AI are often not compared fairly, for isntance with code, people usually compare a human with feedback from a compiler, working iteratively and debugging for hours to LLMs doing it in one go, no feedback, beyond maybe a couple of back and forths in a chat
Also I did say willfully ignorant. This is because you can go and try most models for yourself right now. There are also endless benchmarks constantly being published showing how well they are doing. Benchmarks aren’t perfect and are increasingly being gamed, but they are still decent.
You must log in or register to comment.
People don’t have a problem with the capabilities, they have a problem with the ethics.
And intellectual property theft
I’m not arguing over ethics. There are lots of people here who do have problems with its capabilities
But those same people have no issue with others expanding their own knowledge by reading books written by someone else and then using that knowledge to make a profit. They don’t apply the same standards to humans that they do for AI.
How is a human expanding his knowledge by studying and adapting the things they learned in their own way the same as a computer blatantly copy-pasting intellectual property?
“Blatantly copy-pasting” is not what LLMs do either.
I tend to agree - for whatever reason tech heavy internet communities tend to be filled with Luddites.
My organization has found LLMs really useful for interdisciplinary collaboration. Subject matter experts can generate and check code examples, while software engineers can ask questions about what the code is meant to be doing. It really lowers the friction in these kinds of interactions
Imagine looking at the state of our society and calling for less thinking and more technology.
I do not know where you got that idea. I am asking for more thinking regarding the technology.
I personally find AI is better than me in most fields except ones I know well. So maybe it’s only 80-90% there, but it’s there in like every single field whereas I am in like 1-2
I generally agree with your points here. The issue, though, is that you only notice an LLM’s shortcomings when discussing a topic you deeply understand. When you’re less familiar with the subject, it might seem like the LLM is performing better - but that’s only because you can’t detect its mistakes. It’s similar to the Toupee Fallacy: it’s obvious when an LLM is blatantly wrong, but there are also times when it’s confidently bullshitting you, yet doing it so well that the errors fly under your radar.
This is true, I am generalizing based on results I have confirmed. But I have no reason to think it might be otherwise for other fields
I personally find AI is better than me in most fields except ones I know well. So maybe it’s only 80-90% there, but it’s there in like every single field whereas I am in like 1-2
Hard disagree. Every time I’ve asked ChatGPT and similiar a question about things I barely understand its answer sounds logical at first. But after digging deeper into it it’s clear it doesn’t know shit and just spouts things that sound logical
I am amazed by the possibilities machine learning (not your topic, i know) can bring, and I’m aware what LLMs are capable of in some regards. What I do not like is getting shoved LLMs into everything without alternatives.
LLMs in my profession however, are way over hyped by management and the industry, and (at the moment) are poisoning what entry-level programmers do.
Im not in the loop about what people are expecting of ai or what state of the art models can do. But here’s my review.
Ive found chatgpt useful for code snippets, rewording paragraphs, writing emails, fun images. It saves time, but I still need to adjust things. I also use an llm for code suggestions, which I love.
I can not use it for things I don’t already understand well. Whenever I try to diagnose issues in my Linux computer, i feel like I get dragged down tangents and I get confused and after many messages it completely forgets what we’re doing. I would need to already know how things work to be able to navigate this.
It doesn’t do well with niche information. I haven’t been able to have it make me a functional EDH-deck. I have not been able to get basic information about my not-so-niche field of research. It gets things wrong too often for me to trust it as a source of information.
Overall ive found it very useful for the things I’ve found it useful. I understand why it fails at the other tasks. But I assume that someone better equipped than me could prompt-engineer, or adjust the model somehow, to make it useful for those tasks as well.
But ive also heard of really impressive uses, like alphago and alphafold. Im sure there are more recent examples.
But I honestly don’t understand what people are envisioning that these ai are supposed to do. Im probably just ignorant about the state of the art, but it seems absurd to me that an ai could tell you how to make government efficient by just throwing existing data at it, to use a topical example.
I don’t necessarily hate the capabilities of LLMs and generative AI (well, I kind of do in a lot of ways but it’s more of a personal issue about our relationship with art so we can move past it).
What I really foundationally have an issue with is the politics and the ownership of AI that comes with the territory of it being promulgated by venture capitalists. They like to extract all the value possible from any source of labour, and they see all AI as a modern, ethically cleaner source of essentially free labour from digital chattel slaves.
We are all going to be put in the position of either knuckling under for a new trillionaire class, or becoming modern renditions of the John Henry mythos, the way things are going. I see it this way: either all AI needs to be expropriated for the common good, or the genie needs to be let out of the bottle at every stage so that everyone has equal computational powers. Neither is a perfect solution by any means, but they’re both better by far than the default scenario we’re heading towards.
Whenever I try to diagnose issues in my Linux computer…
Ironically as someone who has now idea how to operate Linux, ChatGPT feels like a personal tech support person for me. It walks me through the necessary steps to solve most issues I’ve had so far. The fact that I can just take a picture of my terminal window and have it read it and tell me what to do next is incredibly helpful. Now, I’m under no illusion that what it tells me to do is the best way to do it but it makes quite the difference when the alternative is to just stare at the screen with zero clue where to even begin.
I might be willfully ignorant. I find LLMs to be too specific where it doesn’t matter and incorrect where it does. If you can’t get specificity or accuracy out of a tool, you need to do the research yourself anyway for any serious pursuit.
Because of the rapid degeneration of traditional search tools though, more and more I’ve been turning to LLMs to start research.
