The AI industry is loud right now. Mostly about safety. Mostly about control. Specifically, there is a buzz about whether American tech giants want the U.S. government to ban open-weight models from China or open-weight models in general.
Dario Amodei, Anthropic’s founder and CEO, finally said what was obvious to anyone who actually read his blog posts.
He doesn’t want a ban. He just doesn’t like where the technology might go next.
“Anyone who has read my past writing should see that I don’t regard such bans [on open weights] as a useful measure… Anthropic has never advocated for a ban.”
This clarification came on Monday afternoon. It was a direct response to an open letter posted on X (formerly Twitter) by Jensen Huang. Huang, founder of Nvidia, was sharing that letter for the first time.
The letter was signed by a long list of companies: Hugging Face, Meta, Microsoft. Mistral. Nvidia itself. They argued against “premature restrictions” on open-weight models.
The subtext wasn’t subtle. The industry is worried about Chinese labs. They think those labs are catching up fast. Or stealing.
Distillation is one method. One AI bombs another with prompts. It learns how the first one works. The U.S. has even threatened sanctions over intellectual property theft involving U.S. models.
But Amodei sees the world differently.
The specific fear behind Anthropic’s safety push
Amodei draws a hard line in the sand. Open-weight models without dangerous capabilities are public goods. They are cheap. Developers use them. Businesses use them. That part is fine.
The problem isn’t the code. It’s the intent.
He worries about authoritarian governments building models that are stronger than anything the U.S. can make. Permanent military superiority. Or total control over their own people.
He named names. The Chinese Communist Party isn’t the only one. It’s just the “most capable” one.
He also raised a darker scenario. Biological attacks.
In his view, open-weight models make this easier. You can’t put guardrails on something once it’s out there. You can’t monitor it. A UK AI Security Institute report confirms this: once open-weights are released, you can’t take them back.
Open-source advocates argue the opposite. They say access helps defenders. They say you need to know the enemy’s tools to fight them. Amodei isn’t convinced.
What Anthropic actually wants done instead
Amodei didn’t just complain. He listed actions. Real ones.
First, keep restricting access to powerful chips. The U.S. has done this for a while. It makes sense.
Second, crack down on distillation. Formal crackdowns. Stop the intellectual property siphoning.
But the big idea is global testing.
Amodei supports a model safety testing organization. He wants it global. Including China. Even if that sounds impossible.
He cited the Trump administration’s recent moves in this direction. He also mentioned industry proposals that apply testing to the most capable models. Open or closed. No exceptions.
Less capable models? Startups? Academia? Those are exempted entirely.
“I think this idea is actually close to a… consensus… limited cooperation around preventing AI biological weapons may be in China’s interest.”
That’s a weird sentence. “Limited cooperation” with the CCP on biological weapons. But he thinks it’s possible. Why? Because even authoritarian regimes want to survive. And open-weight models make survival harder for them if everyone else has better shields.
The nuance most people miss
The narrative right now is simple. Silicons Valley vs. China. Corporate control vs. open access.
Amodei’s position is messier.
He isn’t anti-open weight. He’s anti-uncontrolled dangerous capability.
If the model is dumb, let it fly. Let everyone use it. Let the developers build on it. That’s a public good.
But if the model is smart enough to help design a new pathogen or outmaneuver military defenses, the rules change. Then you need global standards. You need everyone on board.
Does he trust China to play fair? No. But he trusts their self-interest more.
They don’t want their population nuked by a rogue AI. They don’t want their power grid taken over by a rival state.
So, maybe they’ll submit to testing. Maybe they will.
It’s a fragile hope. But it’s better than banning everything and hoping for the best.
The debate continues. The models get bigger. The questions get harder. And the line between public good and public danger keeps blurring.
