Microsoft Says AI Can Create “Zero Day” Threats in Biology

Microsoft Says AI Can Create “Zero Day” Threats in Biology

Estimated reading time: 7 minutes

• Microsoft’s research demonstrates that advanced AI can bypass existing biosecurity systems, creating “zero-day” vulnerabilities in DNA synthesis and protein design.
• Generative AI models, while powerful for beneficial purposes like drug discovery, possess a dual-use potential that can be exploited to design harmful biological agents undetectable by current safeguards.
• The discovery has triggered an “arms race” in biosecurity, necessitating continuous red-teaming, rapid patching, and adaptive defense strategies to counteract sophisticated AI-generated threats.
• Experts debate whether defense should focus on enhancing DNA synthesis screening or integrating “security-by-design” directly into AI models themselves.
• A comprehensive, multi-faceted approach involving strengthened regulatory frameworks, ethical AI development, and international collaboration is urgently needed to manage this evolving biothreat.

• Microsoft Says AI Can Create “Zero Day” Threats in Biology
• Microsoft’s Groundbreaking AI Biothreat Discovery
• The AI-Powered Biosecurity Challenge: Unpacking Microsoft’s Discovery
• The Evolving Arms Race: Patches, Peril, and Perspectives
• Rethinking Defense: Where Do We Go From Here?
• Actionable Steps for a Safer Tomorrow
  • 1. Strengthen Regulatory Frameworks and Enforcement
  • 2. Integrate “Security-by-Design” into AI Development
  • 3. Implement Continuous Red-Teaming and Adaptive Defenses
• Conclusion
• Frequently Asked Questions (FAQ)

Microsoft Says AI Can Create “Zero Day” Threats in Biology

Artificial intelligence continues to astound us with its capabilities, pushing boundaries across industries from medicine to entertainment. Yet, this incredible power carries an inherent duality. What can be used for good, can also be weaponized. A recent, groundbreaking revelation from Microsoft casts a stark light on this dichotomy, unveiling a new frontier in the realm of biosecurity risks.

The tech giant’s findings suggest that advanced AI models can be leveraged to bypass existing safeguards in biological systems, creating what are effectively “zero-day” threats in the world of genetic engineering. This isn’t theoretical future-gazing; it’s a demonstration of current capabilities, prompting an urgent re-evaluation of how we protect against misuse in the biological domain.

Microsoft’s Groundbreaking AI Biothreat Discovery

A team at Microsoft says it used artificial intelligence to discover a “zero day” vulnerability in the biosecurity systems used to prevent the misuse of DNA. These screening systems are designed to stop people from purchasing genetic sequences that could be used to create deadly toxins or pathogens. But now researchers led by Microsoft’s chief scientist, Eric Horvitz, says they have figured out how to bypass the protections in a way previously unknown to defenders. The team described its work today in the journal Science.

Horvitz and his team focused on generative AI algorithms that propose new protein shapes. These types of programs are already fueling the hunt for new drugs at well-funded startups like Generate Biomedicines and Isomorphic Labs, a spinout of Google. The problem is that such systems are potentially “dual use.” They can use their training sets to generate both beneficial molecules and harmful ones. Microsoft says it began a “red-teaming” test of AI’s dual-use potential in 2023 in order to determine whether “adversarial AI protein design” could help bioterrorists manufacture harmful proteins.

The safeguard that Microsoft attacked is what’s known as biosecurity screening software. To manufacture a protein, researchers typically need to order a corresponding DNA sequence from a commercial vendor, which they can then install in a cell. Those vendors use screening software to compare incoming orders with known toxins or pathogens. A close match will set off an alert.

To design its attack, Microsoft used several generative protein models (including its own, called EvoDiff) to redesign toxins—changing their structure in a way that let them slip past screening software but was predicted to keep their deadly function intact. The researchers say the exercise was entirely digital and they never produced any toxic proteins. That was to avoid any perception that the company was developing bioweapons. Before publishing the results, Microsoft says, it alerted the US government and software makers, who’ve already patched their systems, although some AI-designed molecules can still escape detection.

“The patch is incomplete, and the state of the art is changing. But this isn’t a one-and-done thing. It’s the start of even more testing,” says Adam Clore, director of technology R&D at Integrated DNA Technologies, a large manufacturer of DNA, who is a coauthor on the Microsoft report. “We’re in something of an arms race.”

To make sure nobody misuses the research, the researchers say, they’re not disclosing some of their code and didn’t reveal what toxic proteins they asked the AI to redesign. However, some dangerous proteins are well known, like ricin—a poison found in castor beans—and the infectious prions that are the cause of mad-cow disease.

“This finding, combined with rapid advances in AI-enabled biological modeling, demonstrates the clear and urgent need for enhanced nucleic acid synthesis screening procedures coupled with a reliable enforcement and verification mechanism,” says Dean Ball, a fellow at the Foundation for American Innovation, a think tank in San Francisco.

Ball notes that the US government already considers screening of DNA orders a key line of security. Last May, in an executive order on biological research safety, President Trump called for an overall revamp of that system, although so far the White House hasn’t released new recommendations. Others doubt that commercial DNA synthesis is the best point of defense against bad actors. Michael Cohen, an AI-safety researcher at the University of California, Berkeley, believes there will always be ways to disguise sequences and that Microsoft could have made its test harder.

“The challenge appears weak, and their patched tools fail a lot,” says Cohen. “There seems to be an unwillingness to admit that sometime soon, we’re going to have to retreat from this supposed choke point, so we should start looking around for ground that we can actually hold.”

Cohen says biosecurity should probably be built into the AI systems themselves—either directly or via controls over what information they give. But Clore says monitoring gene synthesis is still a practical approach to detecting biothreats, since the manufacture of DNA in the US is dominated by a few companies that work closely with the government. By contrast, the technology used to build and train AI models is more widespread.

“You can’t put that genie back in the bottle,” says Clore. “If you have the resources to try to trick us into making a DNA sequence, you can probably train a large language model.”

The AI-Powered Biosecurity Challenge: Unpacking Microsoft’s Discovery

Microsoft’s “red-teaming” exercise was a critical exploration into AI’s “dual-use” potential. Generative AI models, like Microsoft’s own EvoDiff, are designed to propose novel protein shapes, a capability currently harnessed for beneficial purposes such as drug discovery. However, these same models can be trained to generate harmful molecules.

The core of the experiment involved using these AI algorithms to redesign known toxins. The objective was to alter their structure sufficiently to evade detection by standard biosecurity screening software, while crucially maintaining their deadly function. These screening systems are the first line of defense, scrutinizing DNA sequence orders placed with commercial vendors to prevent the synthesis of dangerous biological agents.

It is vital to emphasize that this entire exercise was conducted digitally. Microsoft researchers meticulously avoided producing any actual toxic proteins, proactively mitigating any perception of developing bioweapons. This ethical approach allowed them to identify vulnerabilities without creating immediate physical risks. Following their discovery, Microsoft responsibly disclosed its findings to the U.S. government and relevant software developers. While patches have been implemented, the report acknowledges that some AI-designed molecules can still slip through the revised detection mechanisms.

The Evolving Arms Race: Patches, Peril, and Perspectives

The situation, as described by Adam Clore of Integrated DNA Technologies, is a rapidly escalating “arms race.” The implemented patches are a temporary measure, and the state of the art in AI and biosecurity is in constant flux. This isn’t a one-time fix but the beginning of continuous testing and adaptation.

To prevent malicious actors from misusing their research, Microsoft has withheld certain code and specific details about the toxic proteins the AI was tasked to redesign. However, the potential for harm remains palpable, as some dangerous proteins are universally recognized, such as ricin (a potent poison from castor beans) and the infectious prions responsible for mad-cow disease. The ability of AI to subtly modify these known threats, making them undetectable by current systems, is a significant concern.

Dean Ball of the Foundation for American Innovation stresses the “clear and urgent need for enhanced nucleic acid synthesis screening procedures coupled with a reliable enforcement and verification mechanism.” The U.S. government already recognizes the importance of DNA order screening, with former President Trump’s executive order in May calling for a revamp of the system. This underscores the high-level awareness of this escalating biosecurity challenge.

Rethinking Defense: Where Do We Go From Here?

While the focus has largely been on commercial DNA synthesis as a defensive choke point, not everyone agrees this is the most robust long-term strategy. Michael Cohen, an AI-safety researcher at the University of California, Berkeley, argues that methods to disguise genetic sequences will always emerge. He suggests that Microsoft’s test might have been too easy, and that relying solely on external screening for DNA orders is a losing battle.

Cohen proposes a paradigm shift: “biosecurity should probably be built into the AI systems themselves.” This could involve intrinsic controls over the information AI models provide or their generative capabilities. The idea is to embed safety at the source of creation rather than solely at the point of manufacture.

Conversely, Adam Clore maintains that monitoring gene synthesis remains a practical and effective approach. He highlights that DNA manufacturing in the U.S. is concentrated among a few companies that collaborate closely with the government, making it a manageable point of control. He points out that the technology for building and training AI models is far more widespread and difficult to regulate. “You can’t put that genie back in the bottle,” Clore states, implying that if one possesses the resources to trick DNA synthesis companies, they likely also have the capability to train powerful AI models.

Actionable Steps for a Safer Tomorrow

The implications of Microsoft’s findings are profound, necessitating a multi-faceted and proactive approach from various stakeholders. Addressing this evolving threat requires collaboration and innovation across different sectors.

1. 1. Strengthen Regulatory Frameworks and Enforcement:

   Governments worldwide must collaborate to enhance and standardize nucleic acid synthesis screening procedures. This includes developing robust enforcement mechanisms, fostering international data sharing on emerging threats, and ensuring that regulatory bodies are equipped with the latest intelligence and technological tools to detect sophisticated, AI-generated biothreats. Regular reviews and updates to biosecurity policies are essential to keep pace with rapid advancements in AI.

2. 2. Integrate “Security-by-Design” into AI Development:

   AI developers and researchers bear a significant responsibility. Instead of solely focusing on post-facto screening, a fundamental shift towards embedding biosecurity directly into the design and training of generative AI models is crucial. This could involve developing ethical guidelines for AI use in biology, implementing built-in safeguards that prevent the generation of harmful sequences, and rigorous pre-deployment testing for dual-use potential. Collaboration between AI ethicists, biologists, and security experts is paramount here.

3. 3. Implement Continuous Red-Teaming and Adaptive Defenses:

   For biotech companies, DNA synthesis vendors, and biodefense agencies, the “arms race” demands constant vigilance. This means establishing ongoing red-teaming exercises, similar to Microsoft’s, to proactively identify vulnerabilities in existing screening software and protocols. Rapid patching, continuous algorithm updates, and a commitment to adaptive defense strategies are non-negotiable to counteract the sophisticated, evolving threats posed by adversarial AI protein design.

Conclusion

Microsoft’s research provides a crucial, if sobering, glimpse into the future of biosecurity. The dual-use nature of AI in biological modeling presents a complex challenge, demanding vigilance and innovation. While the current defense mechanisms have been patched, the inherent dynamism of AI means that this is an ongoing battle, not a solved problem.

The discussion between proactive AI-level safeguards and reactive synthesis screening highlights the need for a comprehensive strategy. Success will hinge on continuous research, transparent disclosure, strong regulatory frameworks, and an unwavering commitment to international collaboration. Only through such concerted efforts can we hope to harness the immense potential of AI in biology while safeguarding against its perilous misuse.

Frequently Asked Questions (FAQ)

• Q: What is a “zero-day” threat in biology?

  A: In cybersecurity, a zero-day threat refers to a vulnerability that is unknown to those who should be interested in mitigating it (the “defenders”). In biology, Microsoft’s research shows AI can create genetic sequences or protein designs that can bypass existing biosecurity screening systems because these systems are unaware of such novel adversarial designs. This makes them “zero-day” threats in the biological domain.

• Q: How did Microsoft’s AI bypass biosecurity systems?

  A: Microsoft used generative AI models, like EvoDiff, to redesign known toxins. The AI altered the structure of these toxins just enough so they could slip past standard biosecurity screening software—which compares incoming DNA orders against known dangerous sequences—while theoretically maintaining their harmful biological function. This digital redesign created sequences that current systems were not programmed to detect.

• Q: Has AI actually created harmful proteins in a physical lab setting due to this research?

  A: No. Microsoft explicitly stated that their “red-teaming” exercise was entirely digital. Researchers meticulously avoided producing any actual toxic proteins to prevent creating immediate physical risks and to avoid any perception of developing bioweapons. The goal was to identify theoretical vulnerabilities in biosecurity systems, not to create new biothreats.

• Q: What are the proposed solutions to this AI biosecurity challenge?

  A: Solutions include a multi-faceted approach: strengthening international regulatory frameworks for nucleic acid synthesis screening, integrating “security-by-design” directly into AI development to prevent the generation of harmful sequences, and implementing continuous red-teaming exercises and adaptive defenses to keep pace with evolving AI capabilities. There’s also a debate about whether to prioritize external screening or internal AI safeguards.

• Q: Why is the situation referred to as an “arms race”?

  A: The term “arms race” is used because the capabilities of AI to create novel biothreats are constantly advancing, requiring biosecurity systems to continuously adapt and improve their defenses. As soon as one vulnerability is patched, new AI models might find new ways to bypass safeguards, creating an ongoing cycle of attack and defense. This dynamic interplay demands constant vigilance, research, and rapid updates to security protocols.