91ɫ��Ƭ

Earlier this month, artificial intelligence (AI) researcher Jacob Coxon resigned from Anthropic after just four months. In an announcement on X, :

The people building AI earnestly believe that it could kill us all by the end of the decade.

A senior member of Anthropic’s staff, Evan Hubinger, actually agreed with Coxon, adding he personally thinks the chance of this happening in the next decade .

Understandably, . There’s now lots of talk about and increasing “” over the technology.

But how exactly might AI kill us all? There’s no shortage of fantastical scenarios, and most of them involve the concept of “superintelligent” AI – that is, AI that’s more capable than humans.

I’ve distilled these scenarios down to the top five, ordering them roughly from most vague to most precise. And I’d argue the list is also ordered from least probable to most probable.

1. We’ll never know

AI doomers often justify their concerns by means of an annoying catch-22 paradox: how can we possibly imagine what a superintelligence might do to take out less intelligent beings like us?

We’d have to be superintelligent to predict what a superintelligence would be able to do. It’s like asking your family dog to imagine thermonuclear war.

The good news here is that superintelligence is still perhaps some distance away. Current AI models are really good at solving particular problems, but that’s not the same as .

However, most challenging maths problems known. It’s apparently closing in on others, which might leave you feeling less optimistic here.

2. Paperclips

A superintelligent AI would likely be extraordinarily competent at achieving its goals. But it might be indifferent to human survival.

A classic example of such indifference comes from imagined superintelligent AI that’s been designed to optimise paperclip production. To produce its preferred form of office supplies, it quickly converts all available matter – including humans, planets and stars – into paperclips.

What we have here is the perfect execution of improperly specified objectives. The AI doesn’t hate humanity; it simply recognises we’re composed of atoms that could be better utilised for paperclips. It’s not personal.

The good news here is that this scenario confuses intelligence with power. A superintelligent AI doesn’t necessarily have the power to achieve its goals. Turning the planet into paperclip factories would require planning permissions.

Even if it got the permissions, building too many paperclip factories would lead to inevitable public outcry. Interest groups would block the proceedings in the courts. Environmental activists would block the bulldozers.

There’s a lot of friction in the world that prevents even the very intelligent from imposing their will on the rest of us. In fact, you could think of data centres as a current embodiment of the theoretical paperclip scenario. And humans are increasingly pushing back against .

3. Bioweapons

Humanity could be killed by a superintelligent AI making and releasing some dangerous new bioweapon into the atmosphere. This is, in fact, one outcome of by the AI Futures Project, a non-profit dedicated to forecasting the impacts of advanced AI.

This risk was made more concrete last month, when they’d used a genetic language AI model to synthesise 16 new viruses.

Worryingly, they just sent the genetic sequences off to a mail-order lab and it sent the viruses back in test tubes. The whole experiment cost a couple of hundred thousand dollars at most.

The good news here is that it’s . To do that, you need a virus that’s very transmissible, so it spreads far and wide. But it’s a rule of biology – viruses that spread easily are typically less fatal. By contrast, if a virus is very fatal, transmissibility tends to go down, as most people infected die before there’s time to spread the infection.

COVID killed . The deadliest pandemic in recorded history was the Black Death, when of Europe’s population in the 13th century. However, even the plague would likely be much less deadly today due to our increased medical knowledge and better sanitation.

4. Nuclear war

What if AI got into the nuclear command and control chain and started a nuclear war? We’ve come several times in the past 50 years.

We’re told that nuclear command and control is completely disconnected from the internet. But, as we saw in 2010, Iran’s nuclear centrifuges , thought to have been brought in on a USB stick. AI can also , which could lead to irreparable actions.

The good news here is that nuclear stockpiles are down. . And it wouldn’t be by the nuclear blast itself, but the famine in the nuclear winter that would follow.

5. Other humans

Perhaps the most likely risk is that we take ourselves out. And AI might precipitate this.

Imagine – and it doesn’t take a lot of imagination – that AI causes massive , pollutes the information space , fractures our politics, and destroys human relationships with .

Society might easily break. Slowly but surely, we’d stop being able to support human life at any scale.

What then to take away from all these scenarios? There are some things to be worried about for sure. But not to be too worried, I hope.

Toby Walsh is the author of , published by La Trobe University Press.

, Professor of AI, Research Group Leader,

This article is republished from under a Creative Commons license. Read the .