p(Hope)

Years ago, in certain obscure AI circles, it became fashionable to inquire as to one's p(Doom) value. That figure encoded the probability that AI would spell humanity's end. Nowadays, you can't escape this question, and as someone who has devoted years of my life to this field, I naturally get asked this regularly. The subtext is not subtle. How can you be so excited about a technology that may destroy us all?

My personal enthusiasm is easy enough to trace back to the fact that the gap between what you can imagine and create shrinks on a seemingly daily basis as these technologies advance. As a builder, I feel unleashed. Couple that with a genetic quirk, a permanently glass-half-full outlook, and it is easy enough to see how I don’t constantly fret about extinction. For the record, I don’t think this is naivete, and I do believe some of the scenarios that have fueled science fiction could come to pass. My p(Doom) is not zero. That would be absurd.

I am in respectable company. When 2,778 researchers who publish at top AI venues were surveyed in the fall of 2023, the median respondent put the chance of AI wiping out humanity at one in twenty or more. Funny thing, though. In all these years, nobody has ever asked me for my p(Hope).

Every New Fire

As a species we tend to glom onto whatever terrors are fashionable. In my lifetime I recall similar waves of panic over nuclear annihilation, the hole in the ozone, climate change, the sixth mass extinction, nano-tech gray goo, Ebola, asteroids and so on. Some of those scares were overblown. Others were warranted, and the worry did useful work: alarm over the thinning ozone layer produced the 1987 Montreal Protocol, and the shield over Antarctica is on track to heal by around 2066.

Fear sells, and the media capitalizes on that. If we look back in history and restrict ourselves purely to technological advancements, we saw similar mass freakouts about the introduction of electricity, or, as I wrote in a previous piece, even the introduction of writing itself. We cannot stop fretting over the new, and we cannot stop inventing it. I imagine similar exchanges occurred with our distant ancestors after they mastered fire.

The Crafted Other

The most vocal critics insist, however, that something about AI makes it a unique danger. Not only will it destroy all jobs, but there is a very real possibility of a Skynet or Borg-type future. Fair enough, AI agents increasingly do possess autonomy, which our previous creations lacked. That fear of the crafted other, with volition, is ancient. Hesiod's Pandora was molded from earth and water by Hephaestus, the smith god. Jewish folklore shaped the golem from clay, most famously in Prague. Mary Shelley gave us Frankenstein's creature in 1818. And the word "robot" debuted in Karel Čapek's 1920 play R.U.R., in which the artificial workers wipe out humanity, save a single man. We named our machines in a story about our extinction.

So will swarms of agents decide to harvest us as batteries, à la The Matrix? Or convert the universe into an endless supply of paperclips, the scenario Nick Bostrom sketched in 2003, in which a superintelligence with no malice at all pursues a trivial goal past every limit? I suppose anything is possible. But I also firmly believe that dread in the moment is inextricably linked to everything else going on. Put another way, p(Doom) is the valid emotional reaction to the upheavals of these times, which include but are not limited to political instability, the increasing possibility of yet another world war, democracy under siege, climate change, growing income inequality, migration anxieties, you name it.

AI is simply the cherry of dread atop a mountain of collective gloom. It is, by definition, alien, unfathomable, and other, and therefore easy to vilify.

The Foot Behind

Wise Zen masters of old understood that opposites come in pairs and define each other. Shitou Xiqian, an eighth-century Chinese teacher whose poem the Sandokai is still chanted daily in temples across the globe, wrote that light and darkness move together, "like the foot before and the foot behind, in walking." There is no up without down, no light without dark, and, in this case, no doom without hope. Rosie the Robot stands in opposition to HAL 9000. They are inextricably bound.

Semi-autonomous drone warfare is already here, in the skies over Ukraine. So are scientific breakthroughs fueled by the same underlying technologies. In 2022, DeepMind published AlphaFold's predicted structures for more than 200 million proteins, nearly the whole known catalog, free for anyone to use. The work earned a share of the 2024 Nobel in Chemistry and gave researchers everywhere a head start on better medicines.

The jury is out, but it increasingly sounds like OpenAI made more progress on the hardest math problems over a few days than we did in multiple prior years. For some, the news landed as a eulogy. Le Monde headlined one scholar's lament: "They have annihilated my field of research." Tristan Buckmaster of New York University said the batch had wiped out the planned work of mathematicians just starting their careers. Hope and dread, in one upload.

What Stayed in the Jar

Is it impossible to believe that our creation, these silicon intellects that keep growing by leaps and bounds, may someday offer solutions to the intractable problems we have been unable to address? That we may redefine what it means to be human and live a good life with fewer burdens than we have now, just as our ancestors largely jettisoned the constant fear of becoming prey through new technologies like fences and spears? The worrier at Grog's campfire had it backward. The fire took us off the menu.

That is what gives me hope, and that sentiment is coupled with doom. Only by holding these two threads of opposition together can we clearly see and act in this turbulent world. Hesiod told it as a story about a jar. When Pandora lifted the lid, every evil inside scattered across the earth. One thing stayed behind, under the rim: Hope. Scholars still argue over whether that was a mercy or a final cruelty. I suspect it was both.

So the next time someone asks for my p(Doom), I will offer an honest answer. Then I will ask for their p(Hope).