Gamers and the general consumer electronics customers have been told for the past two years that GPUs cannot be offered at a reasonable price anymore due to unprecedented AI data center demand. Gl…
…LLMs don’t “know” anything about their training. Beyond (sometimes) a stock greeting intentionally finetuned in.
I’ve brought this up dozens (hundreds) of times, for years, every time someone thinks they’ve identified a stealth model or identified some clue because they’ve gotten a model to “admit” it. It’s all just blind improvisation, it’s literally what they do.
If you want a more clear example, run Deepseek in Mikupad with the raw syntax, ask it your “meta” question, look at the logprobs for the critical decision token, then continue different answers from there, and it will confidently improvise any answer because it is roleplaying.
Or you could just consider what information the model itself has access to, and realize that kind of insight is physically impossible.
Do Chinese companies have a history of skirting IP? Sure. That’s literally history.
Did Deepseek (and other Chinese companies) harvest Claude answers for training data?
Oh yes. It’s obvious. They pulled Claude absolutely shamelessly.
If you want actual proof, see papers on it, or EQBench’s “slop family tree” graphs:
But other LLM companies around the world have no leg to stand on. They all train on each other’s output, they all train on the same legally questionable data anyway.
And I reiterate:
You can’t “investigate” an LLM by asking it for information. They just make stuff up. You can get any model to say it’s training data came from anywhere with the right initial seed, and many will ape Claude or ChatGPT unprompted because they trained on so much of it across the internet.
…LLMs don’t “know” anything about their training. Beyond (sometimes) a stock greeting intentionally finetuned in.
I’ve brought this up dozens (hundreds) of times, for years, every time someone thinks they’ve identified a stealth model or identified some clue because they’ve gotten a model to “admit” it. It’s all just blind improvisation, it’s literally what they do.
If you want a more clear example, run Deepseek in Mikupad with the raw syntax, ask it your “meta” question, look at the logprobs for the critical decision token, then continue different answers from there, and it will confidently improvise any answer because it is roleplaying.
Or you could just consider what information the model itself has access to, and realize that kind of insight is physically impossible.
Do Chinese companies have a history of skirting IP? Sure. That’s literally history.
Did Deepseek (and other Chinese companies) harvest Claude answers for training data?
Oh yes. It’s obvious. They pulled Claude absolutely shamelessly.
If you want actual proof, see papers on it, or EQBench’s “slop family tree” graphs:
https://eqbench.com/creative_writing_longform.html
But other LLM companies around the world have no leg to stand on. They all train on each other’s output, they all train on the same legally questionable data anyway.
And I reiterate:
You can’t “investigate” an LLM by asking it for information. They just make stuff up. You can get any model to say it’s training data came from anywhere with the right initial seed, and many will ape Claude or ChatGPT unprompted because they trained on so much of it across the internet.
Wow, you think you know so much more than you do! It’s impressive!
Let’s see, who should I believe? The guy actually referencing research and data or the guy going “believe me bro, I did the research”.
Hmmmm…
I’m not asking you to believe anyone. Set up the experiment yourself if you’re capable (doubt). I owe you nothing.