My AI Model Is A Bigger Existential Threat Than Yours
It seems the large model makers are currently engaged in a tit-for-tat battle over whose AI bot poses the largest existential threat to society, in some kind of morbid marketing strategy.
Early this week the BBC reported that OpenAI's chief scientist, Jakub Pachocki, was calling for "extreme caution", and warning that more intervention may be needed to ensure humans "remain in control of the future". Not to be outdone, just a few days later the BBC reported an Anthropic researcher going one better. Evan Hubinger, who leads the company's alignment science work, reckons there is a greater than 10% chance AI could kill all humans within the next decade.
This, however, is not a new trend. It has been going on for years.
In February 2019, OpenAI announced GPT-2, but refused to release it in full, as it was just too dangerous. Today's models, openly available to anyone with a browser, surpass GPT-2 by orders of magnitude.
Writing in The Economist last month, the man who built Britain's cyber defence agency took a rather different view on a related scare. Ciaran Martin, founding chief executive of the National Cyber Security Centre, argued that fears of AI-induced armageddon are overdone. The recent shenanigans in which a bot was reported to have escaped its sandbox and gone rogue? Down to a poorly controlled test environment.
And that model Anthropic was holding back because it was finding earth-shattering bugs in operating systems and browsers? That was overblown as well. According to Martin's account, the standout claims didn't survive contact with follow-up analysis, and the forecast tsunami became a slightly stormy spell.
Martin is quite generous about all this. If hyperbole about omnipotent bots is finally what gets the west to close its security holes, then so be it.
I struggle to be quite as charitable.
Forecasting the end of the world is an excellent distraction from the societal changes that AI bots are bringing in the here and now, and from how little has changed in terms of safeguarding and regulation between 2019 and today.
By accepting the extinction narrative, we hand the model makers the authority to police themselves. That is a far more plausible route to harm in the next decade than anything Hubinger suggests.
