Nist's Deepseek "evaluation" Is a Hit Piece

Now I'm not sure legality is on-topic any more.

That's easy (well, possible) to detect. I'd go the opposite way - sift the code that is submitted to identify espionage targets. One example: if someone submits a piece of commercial code that's got a vulnerability, you can target previous versions of that codebase.

I'd be amazed if that wasn't happening already.

There's zero reason or even technical feasibility for them to skip in backdoor that would be easily detected and destroy their market share

None of the security benchmarks or audits show that any Chinese models write insecure code

Note that the value of $current_administration changes over time. For some reason though it is currently fashionable in tech circles to disagree with it about ICE and H1B visas. Maybe it's the CCP's doing?

Then, you introduce the bias into relative unknown concepts that no one prompts for. Preferrably, obscure and unknown words that are very unlikely to be checked for ideologically. Finally, when you want the model to push for something, you introduce an idea in the general population (with a meme, a popular video, maybe even an expression) and let people interact with the model given this new information. No one would think the model is biased for that new thing (because the thing happened after the model launch), but it is, and you knew all along.

The way to avoid this kind of influence is to be cautious with new popular terms that emerge seemingly out of nowhere. Basically, to avoid using that new phrase or word that everyone is using.

It's funny because recently I wanted to learn about the history of intellectual property laws in China. DeepSeek refused the conversation but ChatGPT gave me a narrative where the WTO was essentially a colonial power. So right now it's the American AI giving the pro China narratives while the Chinese ones just sit the conversation out.

If one takes a few minutes to review the NIST report [1], one will indeed find evidence and references detailed in the footnotes.

Without judgment, I ask: How did you miss this? I'm certainly not asking you to defend yourself, which would likely trigger rationalization [2]. I am asking you to sincerely try to figure it out for yourself. Under what conditions do you have the ability to admit that you are wrong, even if only to yourself?

[1]: https://www.nist.gov/system/files/documents/2025/09/30/CAISI...

[2]: https://www.logicallyfallacious.com/logicalfallacies/Rationa...

> Just because you already afraid or want others to be afraid.

We need to be more careful with our thinking and writing. [1] The word "just" indicates the commenter landed only on one explanation, but there are other plausible explanations, including, for example:

- others have different experiences -- but if this experience was communicated and understood, another person would incorporate the new information modify their assessment somewhat

- others have various values and preferences (some overlapping, some phrased differently, some in tension)

- others are using different reasoning (and if one's goal is to learn, it would be better to ask and clarify rather than over simplify and accuse them of having nefarious motives.)

Second, the commenter above is speculating. They don't know me, nor have they engaged in a sincere, constructive, meaningful discussion to understand what I'm saying.

Third, to me, the comment above comes across as unnecessarily abrasive, to the point of being self-defeating and degrading the quality of a shared discussion.

[1] If one is writing privately (e.g. a journal), I care relatively less about logical fallacies such as motivated reasoning. [2] But here in public, wayward reasoning has more negative externalities. Our time would be better spent if more people took the time to write thoughtfully. I don't think most people here are lacking in sufficient computational ability. However, they must choose to respect their audience -- their time, their diversity of values, their different experiences -- and remember that one's hasty comment (authored in say 2 minutes) might be read by hundreds of people (wasting say 100+ minutes). Lastly, it is nice to see when a person has the character to say "thank you for the correction", but this is uncommon on HN.

[2] But I still care because society is highly interconnected and the downstream effects matter to me. Put another way, "Your Rationality is My Business" as explained here: https://www.lesswrong.com/posts/anCubLdggTWjnEvBS/your-ratio...

This links to a video titled "(why facts dont [sic] change minds (goose explains)"). First, I am not seeing the connection to the comment above -- the comment is hard to make sense of for various reasons, including grammatical errors, vague language, and easily refuted claims (see my other comment).

Second, as I watch the video, I am thinking as follows: "Yes, there are some good points here. I wish the person who posted the video would watch the video again with an eye towards self-reflection, as it reveals some areas for improvement."

My overall take on the video: it oversimplifies, even gets some things wrong. There is some value to be found if one knows how to extract the good from the bad. I would not recommend it, not even as an introduction. There are better sources.

I am not going to dignify this with a response.

Please review the hacker news guidelines.

Not to mention Anthropic says Claude will eventually automatically report you to authorities if you ask it to do something "unethical"

You are confused about what the NIST report claimed. Please review the NIST report and try to find a quote that matches up with what you just said. I predict you won’t find it. Prove me wrong?

Please review the claims that the NIST report actually makes. Compare this against Eric Hartford’s article. When I do this, Hartford comes across as confused and/or intellectually dishonest.

"smear" as understood by most people, means "to damage the reputation of (someone) by false accusations; slander: someone was trying to smear her by faking letters." (Apple dictionary)

If there are no false accusations, there is no smearing. Are you claiming the report makes false accusations? Where?

Disagreeing with emphasis or prioritization isn't sufficient. Not engaging with the reasoning (or not understanding it) isn't a valid basis for claiming "false accusations". I reply more fully at [1]: https://news.ycombinator.com/item?id=45493266

In case people will feel better knowing that I'm not on the team of their enemies, I can assure you I'm opposed to corrupt and authoritarian behavior anywhere. I'm sickened by who Trump is, what he has done, how he has confused so many Americans, and how he is a conduit for some of the worst beliefs and impulses of Americans. Some of these tendencies are rooted in confused ethics and bad reasoning.

> (antonvs) If only that were true.

Using a charitable reading of your comment, it seems you are actually talking about the effectiveness of NIST, not about its mission. In so doing, you were not replying to my actual claim. If you read my sentence in context, I hope it is clear that I'm talking about the implicit values baked into the report. When I write that NIST promotes certain ideals, I'm talking about its mission, stated here [1]:

> To promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve our quality of life.

This is explained using different words in a NIST FAQ [2]:

> Everything in science and technology is based on measurement. Everything we use every day relies upon accurate measurements to work. NIST ensures the measurement system of the U.S. meets the measurement needs of every aspect of our lives from manufacturing to communications to healthcare. In science, the ability to measure something and determine its value — and to do so in a repeatable and reliable way — is essential. NIST leads the world in measurement science, so U.S. businesses can innovate in a fair marketplace. We use measurement science to address new challenges ranging from cybersecurity to cancer research.

It is clear NIST's mission is a blend of scientific rigor and promotion of western values (such as free markets, free ideas, innovation, etc). Reasonable people can disagree on the extent to which NIST achieves this mission, but I don't think reasonable people can deny that NIST largely aims to achieve this mission.

My take on the Trump and his administration: Both are exceptionally corrupt by historical standards. They have acted in ways that undermine many of the goals of NIST. But one has to be careful to distinguish elected leaders and appointees from career civil servants. We have to include both (and their incentives, worldviews, and motivations) when making sense of what is happening.

[1]: https://www.nist.gov/about-nist

[2]: https://www.nist.gov/nmi

1. You accept the definition: "Xenophobia is fundamentally about irrational fear or hatred of people based on their foreign origin or ethnicity. It targets people and operates through stereotypes, dehumanization, and often cultural or racial prejudice."

2. You claim this sentence from the NIST report is an example of irrational fear: "the expanding use of these models may pose a risk to application developers, consumers, and to US national security."

3. As irrational fear isn't sufficient for xenophobia, you still need to show that it is "based on their foreign origin or ethnicity".

4. You don't provide any evidence from the report of #3. Instead, you refer to Trump's comments as evidence of his xenophobia.

5. You directly quote my question "Can you please share where you see BS and/or xenophobia in the original report?" In your response, you imply that Trump's xenophobic language is somehow part of the report.

My responses to the above (again, which I think is an accurate but clearer version of your argument): (1) Good; (2) I disagree, but I'll temporarily grant this for the sake of argument; (3) Yes; (4) Yes, Trump has used xenophobic language; (5) Since we both agree that Trump's language is not part of the report, your example doesn't qualify as a good answer to "Can you please share where you see BS and/or xenophobia in the original report?".

Your claim only shows how a xenophobic Trumpist would interpret the NIST report.

My take: Of course the Trump administration is trying to assert control over NIST and steer it in more political directions. This by definition will weaken its scientific objectivity. To what degree it has eroded so far is hard for me to say. I can't speak to the level of pressure from political appointees relating to the report. I can't speak to the degree to which they meddled with it. But this I can say: when I read the language in the report, I don't see xenophobia.

Yes, that contains a quote from the executive summary. First (perhaps a minor point), I wouldn't frame this a fear, I would call it a risk assessment. Second, it is not an irrational assessment. It seems you don't understand the reasoning, in which case disagreement would be premature.

> There’s no support for that claim in the report, just vague handwaving at the fact that a freely available open source model doesn’t compare well on all dimensions to the most expensive frontier models.

I'm going to put aside your unfounded rhetoric of "vague handwaving". You haven't connected the dots yet. Start by reviewing these sections with curiosity and an open mind: 3.3: Security Evaluations Overview (pages 15-16); 6.1: Agent Hijacking (pages 45-57); 6.2: Jailbreaking (pages 48-52); 7: Censorship Evaluations (pages 53-55)

Once you read and understand these sections, the connection to the stated risks is clear. To spell it out: when an organization deploys a DeepSeek model, they are exposing themselves and their customers to higher levels of risk. Risks to (i) the deploying organization; (ii) the customer; and (iii) anything downstream, such as credentials or access to other systems.

Just in case I need to spell it out: yes, if DeepSeek is only self-deployed (e.g. via Ollama) on one's local machine, some risks are much lower. But a local-deployment scenario is not the only one, and even it has significant risks.

Lastly, it is expected (and not unreasonable) for government agencies to invoke national security when cybersecurity and bioterrorism are involved. Their risk tolerance is probably lower than yours, because it is their job.

Next, I will ask you some direct questions:

1. Before reading Hartford's post, what were your priors? What narratives did you want to be true?

2. Did you actively try to prove yourself wrong? Did you put in at least 10 uninterrupted minutes trying to steel-man the quote above?

3. Before reading the NIST report, would you have been able to e.g. explain how hijacking and jailbreaking are different? Would you have been able to explain in your own words how they fit into a threat model?

Of course you don't have to tell us your answers. Some people have too much pride to admit they are uninformed or mistaken even privately, much less in public. To many, internet discussions are a form of battle. Whatever your answers are, strive to be honest with yourself. For some, it takes years to get there. I'm speaking from experience here!

Yes, it seems that Trump considers anyone who loudly disagrees with him to be an enemy. So when he looks at Portland, Chicago, and Washington DC, he views them as filled with enemies. On this I think we agree.

But you are confused if you think this tendency of Trump means that this particular NIST report is irredeemably twisted and manipulated. You seem to believe that Trump's derangement has percolated NIST to the point where nearly every word in the report is in service of his whims or agenda (which changes so often that even his supporters have to find ways to cope with the chaos).

No. I haven't seen you demonstrate much understanding of NIST or U.S. government agencies in general. I've seen you commit many errors and much motivated reasoning.

You aren't using the words "absolute" [1], "charitable" [2], and "smear" [3] in the senses that reasonable people expect. I think you are also failing to use your imagination and holding onto one possible explanation too tightly. I think it would benefit you to relax your grip on one narrative and think more broadly and comprehensively.

[1] Your use of "absolute" is rhetorical not substantive.

[2] You use the word "charitable" but I don't see much intellectual flexibility or willingness to see other valid explanations. To use another phrase, you seem to be operating in a 'soldier' mindset rather than a 'scout' mindset. [5]

[3] Here is the sense of smear I mean from the Apple dictionary: "to damage the reputation of (someone) by false accusations; slander: someone was trying to smear her by faking letters." NIST is not smearing DeepSeek, because smearing requires false claims. [4]

[4] If you intend only to claim that NIST is overly accentuating negative aspects of DeepSeek and omitting its strengths, that would be a different argument.

[5] https://en.wikipedia.org/wiki/The_Scout_Mindset

The flaw in it is, of course, that capitalism is supposed to be all about competition, and there are plenty of good reasons for capitalists to want that, like "Commoditize Your Complement" where companies like Apple, Nvidia, AMD, Intel, AWS, Google Cloud, etc. benefit from everyone having good free models so they can pay those companies for systems to run them on.

   Short answer: it’s contested. Major human-rights bodies 
   say yes; Israel and some legal scholars say no; no court 
   has issued a binding judgment branding “Israel” an 
   apartheid state, though a 2024 ICJ advisory opinion 
   found Israel’s policies in the occupied territory 
   breach CERD Article 3 on racial segregation/apartheid. 

   (Skip several paragraphs with various citations)

   The term carries specific legal elements. Whether they 
   are satisfied “state-wide” or only in parts of the OPT 
   is the core dispute. Present consensus splits between 
   leading NGOs/UN experts who say the elements are met and 
   Israeli government–aligned and some academic voices who 
   say they are not. No binding court ruling settles it yet.

Chinese models, conversely, are aligned with explicit, mandatory guardrails to exalt the CCP and socialism in general. Unless you count prohibitions against adult material, drugs, explosives and the like, that is simply not the case with US-based models. Whatever biases they exhibit (like the Grok example someone else posted) are there because that's what their private maintainers want.

And that's OK, because nobody in the government forced him to set it up that way.

There’s also the issue that practically nobody actually uses LLMs to criticize political entity XYZ. Let's face it, the vast majority of use cases are somewhere else, yet a tiny minority is pretending like the LLM not giving them the responses they want for their political agenda is the only thing that matters. When it comes to censorship areas that matter to most use cases, many people have found that many Chinese LLMs do better than western LLMs simply because most use cases never touch Chinese political stuff. See thorough discussions by @levelsio and his followers on Twitter on this matter.

Token price on 3.2 exp is <5% what the US LLMs are and it's very close in benchmarks. Which we know that ChatGPT, Google, Grok and Claude have explicitly gamed to inflate their capabilities

>> TLDR for others...

Facepalm.

Like who? Meng Wanzhou?

No there isn't. China revoked their passport to keep them prisoners not to keep them safe.

"On a good faith basis, one would think China banning people from leaving is a good thing"

Why would anyone think imprisoning someone like this is a good thing?