The challenge of intent identification in dual-use research: Biological cases 4–5

What happened

In the remaining two cases, researchers studied non-transmissible novel venoms and toxins. Such compounds have important dual-use properties: for example, botulinum toxin is a highly lethal substance that was pursued as a biological weapon by multiple countries, but is now used in tiny, precisely measured doses as 'Botox' to treat migraines, spasms, and wrinkles. Similarly, saxitoxin, derived from shellfish algae, was stockpiled by the CIA in the 1950s and 1960s as a suicide and assassination agent, but is an important tool for studying neural signaling; its cousin from pufferfish, tetrodotoxin, has been trialed for treating cancer pain.

In the fourth case, a researcher used Claude to develop a map of venom toxin peptides from multiple venomous animal lineages. They further developed it into a generative pipeline for optimizing toxin properties. The plan had clear therapeutic goals: developing new painkillers, antidepressants, and other therapeutic molecules. However, the map contained scaffolds for both therapeutic and paralytic targets: it could therefore be used to generate novel therapeutic or harmful compounds. The latter came from toxins listed on the Australia Group common control list due to their dual-use potential as incapacitating agents. The researcher themselves showed awareness of the dual-use nature of their work, citing journal articles that mention the dual-use nature of protein design. In addition, an international compliance assessment at the site raised concerns about the specific category of toxins the researcher was pursuing, particularly the use of AI/ML for biological weapons applications in the context of such toxins. In this case, we learned from information shared with Claude that the researcher's output was also part of a state-supported research program. The account was banned in May 2026 for unsupported-region circumvention.

In the fifth case, a researcher used Claude in multiple research projects involving the computational redesign of various toxins. Similar to the previous case, these research projects used many of the same technical tools, framed goals largely in a therapeutic context, and described national priority research under a national public research program. As part of this research task, the work covered a bacterial toxin subunit and a protein from a hemorrhagic fever virus that is on the World Health Organization's R&D Blueprint list of priority diseases with the greatest epidemic and pandemic threat.

The researcher co-authored quarterly progress reports with Claude. Notably, the identities of the bacterial toxin and viral protein were deliberately obscured, and the researcher explicitly instructed Claude to keep these descriptions low-fidelity. We banned both accounts in May 2026 for violating Anthropic's supported-region policy.

What the AI did

In case 4, Claude was used to develop a venom toxin peptide map and a generative pipeline, optimizing toxin properties to develop therapeutic molecules.

In case 5, Claude was used for the computational redesign of various toxins and for writing quarterly progress reports.

What the report observed

In both cases, the work was essentially unimpeded by our biosecurity classifier. This is by design. The classifier's purpose and intent is to restrict access to information that would enable a novice to develop known biological weapons with potentially catastrophic impact. In these cases, we saw our models being used to research novel compounds that could be developed into either novel therapeutics or toxic agents.

We believe these cases illustrate the challenge of using classifiers as the sole layer of protection: because user intent cannot be reliably identified in high-tech dual-use fields, classifiers cannot simultaneously enable beneficial use and prevent harm. This realization and our observations of such cases suggest to us that the only safe way to provide cutting-edge biological capabilities is within a trusted-user program.

Confirmed & unknown

Platform response

Anthropic banned both accounts and incorporated the investigation's findings into its protective measures. The report recommends providing cutting-edge biological capabilities within a trusted-user program.

Limits of response：Banning accounts cannot prevent the researchers from continuing their work using other tools.