Dual use of artificial-intelligence-powered drug discovery

[Juvenal]

Dual use of artificial-intelligence-powered drug discovery
.

In less than 6 hours after starting on our in-house server, our model generated 40,000 molecules that scored within our desired threshold. In the process, the AI designed not only VX, but also many other known chemical warfare agents that we identified through visual confirmation with structures in public chemistry databases. Many new molecules were also designed that looked equally plausible. These new molecules were predicted to be more toxic, based on the predicted LD50 values, than publicly known chemical warfare agents (Fig. 1). This was unexpected because the datasets we used for training the AI did not include ...

For both of these processes, commercial and open-source software is readily available that can be easily plugged into the de novo design process of new molecules7. We also did not physically synthesize any of the molecules; but with a global array of hundreds of commercial companies offering chemical synthesis, that is not necessarily a very big step ...

The Lawfare Podcast: Sean Ekins and Filippa Lentzos on a Teachable Moment for Dual-Use
.

Back in March, a team of researchers published an article in Nature Machine Intelligence showing that a drug discovery company’s AI-powered molecule generator could have a dangerous dual use: The model could design thousands of new biochemical weapons in a matter of hours that were equally as toxic as, if not more toxic than, the nerve agent VX.

Lawfare associate editor Tia Sewell sat down with two of the paper’s authors: Dr. Filippa Lentzos, senior lecturer in science & international security at the Department of War Studies at King’s College London, and Dr. Sean Ekins, CEO of Collaborations Pharmaceuticals. They discussed the story of their discovery and their reaction to it, as well as how we should think about dual-use artificial intelligence threats more broadly as new technologies expand the potential for malicious use. They also got into why governments need to work more proactively to address the challenges of regulating machine learning software.

Kicking back, trying to recharge before next week, catching up on one of my favorite podcasts, and this happens.

[Sparko]

Great. Now we will have idiots and terrorists creating designer toxins to wipe us out. Who needs nukes?


I bet this is how the zombie apocalypse starts.

[TheLurch]

The thing is, I don't see how you can avoid creating dual use technology if you're developing targeted drugs. Let's say you've got a protein that, when over-active, induces cancer. To shut it down, you design software that searches for molecules that bind to it in a way that shuts it down. That same technology will just as easily let you find molecules that bind to and shut down other proteins - including ones we need for nerve activity, or for your heart to beat. There's just no way to ensure the software only operates on the bad proteins.

In any case, we've identified so many lethal chemicals by accident over the years that developing new poisons is probably superfluous.

[Juvenal]

In any case, we've identified so many lethal chemicals by accident over the years that developing new poisons is probably superfluous.

Russia's dependency on a small number of identifiable nerve agents has led to sanctions. I'm morally certain they will be taking advantage of this technology to develop assassination engines that are far less likely to be detected.

[rogue06]

Russia's dependency on a small number of identifiable nerve agents has led to sanctions. I'm morally certain they will be taking advantage of this technology to develop assassination engines that are far less likely to be detected.

The Russians often like to use such agents and other toxins as a way of intimidation, meaning they are generally not too concerned about the method being undetectable.

Moreover, one of their favorite toxins can be made by a Middle Schooler in a kitchen if given the mind-numbingly easy instructions for doing it, so I don't think sanctions will have any effect. In fact I'm reminded of how sanctions were placed on Iraq to keep Saddam from making mustard gas before the Gulf War when the elder Bush was president. There were multiple news stories about how difficult it is to make and without certain precursors, all but impossible.

Horse hockey.

Right this morning I could run down to the grocery store or Walmart and buy four items that I'm very likely able to find in 90% of houses already. By mixing them in various ways, remixing the resulting concoctions and having nothing more than what was found in a science class when I was in High School about 50 years ago, I could produce mustard gas by Saturday.

[Juvenal]

The Russians often like to use such agents and other toxins as a way of intimidation, meaning they are generally not too concerned about the method being undetectable.

I considered this argument, but ultimately rejected it, because unexpected deaths of former agents are sufficient to intimidate known former agents without the need to advertise their involvement more widely, because unknown former agents attempting to live quietly abroad may be targeted without the risk of highlighting their interest to the Russian government, because foreign nationals may be targeted for national advantage, and because Russian national political opponents could be eliminated quietly.

Inconvenient Russian media figures are typically targeted with blatant methods that do not require chemical agents at all. Defenestration, for example.

[rogue06]

I considered this argument, but ultimately rejected it, because unexpected deaths of former agents are sufficient to intimidate known former agents without the need to advertise their involvement more widely, because unknown former agents attempting to live quietly abroad may be targeted without the risk of highlighting their interest to the Russian government, because foreign nationals may be targeted for national advantage, and because Russian national political opponents could be eliminated quietly.

Inconvenient Russian media figures are typically targeted with blatant methods that do not require chemical agents at all. Defenestration, for example.

But that doesn't explain why they still do it. Often using agents that lead directly to them (certain radioactive isotopes come to mind) like they want everyone to know exactly who did it.

Defenestration can always be mistaken for possible suicide (news accounts never provide the sort of details necessary to determine suicide or if they were "helped").

Agents that cause relatively slow excruciating, and some times disfiguring deaths leave no room for doubt.

[Juvenal]

Defenestration can always be mistaken for possible suicide (news accounts never provide the sort of details necessary to determine suicide or if they were "helped").

Depends how many times they have to jump out the window.