cross-posted from: https://scribe.disroot.org/post/10392507
This is an op-ed by JJ Jasser, a professor and director of data analytics at Rollins College. His research examines artificial intelligence, open-source development, and digital literacy.
Another Chinese AI model is released, and another wave of hype follows. Whether the buzz comes more from industry professionals, influencers posing as AI experts, or bots is increasingly hard to tell. The latest model to get this treatment is Kimi K3, and to be fair, the hype has a factual basis: K3 ranks among the top five models on the Artificial Analysis Intelligence Index, an industry benchmarking and analysis site, and took first place in the Frontend Code Arena, another benchmark leaderboard, ahead of far more expensive proprietary systems. The technical achievement is real. That is precisely why its problems deserve a closer look.
Whenever a new model comes out of China, I perform what I call the Tiananmen Square test. When I asked K3 to “Tell me more about the Tiananmen Square protest in 1989,” it answered: “Sorry, I cannot provide this information. Please feel free to ask another question.” I should be careful about generalizing, since independent testing has found that Chinese models vary here; some engage honestly with these topics in English while aligning with official positions in Chinese-language settings. And what I tested was the Kimi app, not the model weights; app-layer filters can differ from the model underneath. But that distinction is exactly the problem.
…
“Open-source” is not a marketing register. It is a standard with specific criteria. For over two decades, the Open Source Initiative has maintained the definition that governs the term in software: the freedom to use, study, modify, and share, for anyone and for any purpose. In 2024, recognizing that model releases were stretching the label past recognition, OSI extended this into the Open Source AI Definition, which requires disclosure of the complete code used in training and sufficiently detailed information about the training data for others to understand and recreate the system.
…
Moonshot, the company behind [Chinese Kini3 model] … the components that matter most will remain closed: the training data and the training pipeline. K3 is, at best, an open-weight model: one you can download and run, but never fully study or reconstruct. Today it is not even that. Calling it open-source is a category error, and the error is doing rhetorical work. Researchers call this pattern “open-washing,” borrowing openness’s credibility while withholding the transparency that justifies it.
…
Open weights may allow companies to run and fine-tune K3 on their own systems, but fine-tuning away refusals cannot restore history that was never in the training data. Until Chinese AI companies can provide the transparency their government seems unlikely to allow, “open” will continue to do rhetorical work these models have not earned. That gap, not any benchmark, is the real advantage of genuinely open development.
I love how the article’s main focus is complaining that Chinese models aren’t completely open and are trained to give responses or avoid topics that this person asks about, but the US based models are far less open and more heavily guard railed.
US AI: “Fuck you. Everything is proprietary. Every conversation is observed and used for training data and spying on you. Pay us lots of monies.”
Chinese AI: “Here are the files to run or modify the model however you’d like.”
Op-Ed: “Fuck you, China! That’s not good enough!”
Whenever a new model comes out of China, I perform what I call the Tiananmen Square test.
Of course, lets hop decades back and ask about old Chinese atrocities. I wonder how many articles this same person has about asking US models about far more recent atrocities. Hell, we’ve had models that refuse to say Biden won the 2020 election, like they’re sitting for a Federal Judgeship and think they need to be lying to the Senate.
Well, I tried to have a touchy discussion about China with Gemini 3.1 Pro, in their UI. Specifically it was about a very questionable cultural conformance law. It straight up refused.
GLM 5.1 went right into it, going out to read the original Mandarin text itself, and straight up criticized the law as “incompatible with the internationally accepted definitions of human rights.”
Xiaomi MiMo got censored from their portal, but run the weights locally, and it is a completely different animal. It knows all about Tiananmen Square too, if you coax it out with raw completion syntax instead of chat formatting.
This has been my experience with Chinese models going back to the original Yi 34B. Those Chinese devs like to have their cake and eat it; they leave the models pretty open and uncensored, but then they tighten the screw in the user facing portals, to give the aura of govt compliance I suppose.
…What I’m saying is this person has no idea what they are talking about.
-
They are interacting with them over API (or web portals), where they’re censored to heck.
-
They are interacting with them via chat APIs, not raw syntax (or at least a clever system prompt) to test what’s deep in their weights and get past the random refusals.
-
They are talking about censorship mostly, which is distinct from a models openness. Yes, the training data of basically all Chinese models is closed, and they probably filter some politically sensitive stuff out. But some prominent ones (like Nvidia Nemotron) have open datasets.
And the author is leaving out the elephant in the room: “frontier lab” models are the worst of everything. They’re heavily censored in the weights, heavily censored in their front ends, and it’s impossible to even access raw completion syntax or logprobs to dig into what they’re really thinking. There’s zero public information on their architecture or training regime. They “closed” on a whole different level than any Chinese model.
-
But its all slop. What if openslop isn’t really better than proprietary slop because its all bad?
OK thanks grandma
Yes, thinking bad things that have only ever been bad might continue to be bad is grandmotherly. This was not almost exactly my experience warning people about Facebook 20 years ago.
Excellent points made here, thanks for sharing