Key events
That’s all, folks!
Thanks everyone for the great questions. This is a difficult subject to negotiate and interacting with you always helps shape our thinking – and hopefully yours. Aisha, Blake and Dan
How can individuals make a difference?
Alberichthenibelung asks: What, if anything, can individuals like me do about it?
Blake: There is, unfortunately, not much that comes to mind. Everyday people confronted with declarations of a coming AI apocalypse find themselves in a position similar to the cold war, with doom hovering over their heads and little say in it. We may all have to accept the faint tinnitus of paranoia buzzing in the background. You can stop using AI if you feel so moved, though I have found that unfeasible, and your job may require it. You can vote for politicians that endorse AI regulation. I myself don’t put much stock in the reality of an AI-made doomsday because a genuine belief in it would demand drastic, life-changing actions today, ones that I don’t want to take. A bit of inductive reasoning from how I’d like to situate my own life within the world.
How do we regulate AI?
Fulgrim asks: What would legislation look like? We only just passed the Online Safety Act and it is ineffectual, restrictive and late. Given the complexity, fast changes and its international nature, what kind of legislation would you like to see and how do we educate politicians on the issue?
Trentwarrior asks: Regulation, oversight and accountability for huge companies that appear unable or unwilling to eliminate the harms their products inflict – and that seem incapable of conducting safe experiments to ensure that AI does not leak into the wider internet – seem to be urgent and vital. What are the prospects for a global regulatory framework that is enforceable, particularly when the genie may already be out of the bottle, and when the US and China have very different objectives and applications for their AI technologies?
Blake: It’s a tough question. One way to hinder the general development of AI could be restricting access to computing power, which is a vital resource that underpins the success of an AI model. In the absence of legislation that applies to all types of AI development, measures that target certain aspects of AI like housing allotment or children’s access to chatbots seem like the path lawmakers will take.
There are two big question marks hovering over a possible global regulatory framework. The first is whether Donald Trump will revise his “Drill, baby, drill” stance towards AI development. He has rejected any calls for a voluntary slowdown. In my opinion, he is not likely to change that position: He loves to post AI slop, and he loves AI has done for the US stock market.
The second question mark is Beijing, which has rejected US CEOs’ calls for a slowdown. Without China’s cooperation, an intentional slowdown would likely not succeed. Xi Jinping also seems unlikely to back down. A slowdown of AI development led by the US would offer China few obvious benefits.
Trump and Xi Jinping are also set to meet next week in Washington. AI will be high on the agenda.
The UN General Assembly is taking place now, with discussions of AI large and small on the schedule. It is possible that an initial version of a global regulatory framework emerges from there.
Photograph: Tingshu Wang/Reuters
Could the media do more to hold AI to account?
MacDuke asks: Do tech reporters feel responsibility for allowing tech bros to accumulate so much wealth without holding them to account for destroying the climate, society, and the economy (when the bubble bursts)?
Blake: Sometimes! I rarely, if ever, write flattering pieces about them. Our job as journalists is to hold them to account. Our work does often change the perception of tech oligarchs; however, we have less power to compel changes in user behavior. Facebook, for all the reporting on the harms it has inflicted upon the world, has more than two billion daily users. Instagram has three billion monthly users. Those vast numbers are growing. By contrast, we are extremely happy if our stories reach one million readers, a rare outcome. The Washington Post summed this dynamic up quite well: “AI is just another technology Americans don’t like but can’t stop using.”
We do our best.
Why the myth that we don’t know how AI works?
LlivracNhoJ asks: Why do some journalists persist in spreading the myth that “nobody understands how these technologies work” when we know it’s basically pattern recognition and inference?
Aisha: In defense of my journalist brethren, there is an element of a “black box” in how ChatGPT or Claude answer specific prompts. The area of “AI interpretability” is an active field of study. In broad terms, yes we do know how an LLM is trained and built. But for example, no one can do a forensic A to B to C mapping of how an AI system might select a face from a lineup, pick one CV over another, or target one individual and not another in a conflict. That seems to be likely what they mean.
Is financial AI apocalypse incoming?
Davidabsalom asks: Is the recent desire for a slowdown by the tech bros more about their fear their money will run out and their businesses collapse rather than a genuine belief in the dangers?
Seaspray asks: Is this recent narrative push of the AI apocalypse an inventive way of preparing the public for the market crash and wipeout of AI shares? The companies seem overvalued and their borrowing and input costs have gone up.
Aisha: The finances of the AI buildout are certainly shaky. Hundreds of billions of dollars are being ploughed into massive, unpopular infrastructure projects that our reporting has shown are not necessarily getting built. Nvidia appears to be the only company making a profit, and investigations I have done with Ed Zitron raise some questions about whether all its very expensive GPUs are actually, currently, getting plugged into datacentres. (Perhaps they are mouldering in warehouses instead.) There are some unsettling financial instruments being used to raise money to build yet more datacentres, such as GPU-backed debt.
Is that why tech bros want a slowdown? Perhaps. It looks like OpenAI just pushed back its IPO until 2027. This might be because, as we reported, things did not look amazing for them earlier this year in terms of floating on the market – they seemed to be struggling to monetise ads and erotic chatbots. Their valuation will depend a lot on vibes, and maybe the doomy vibes will help a potential share offering.
If the threat of AI is real – prove it!
Westkent1023 asks: Milton Mueller, professor and program Director of Science in Cybersecurity Policy at Georgia Tech in the US has argued in a peer-reviewed paper in the Journal of Cyber Policy that the idea of AGI posing an existential risk to humans is an ‘unscientific myth’ that defies ‘economic constraints and even basic laws of physics’. Such claims he suggests distract and distort proper digital governance of risks from AI in its current form. What do you think of these arguments? And if the AI developers believe their claims about an existential risk to humanity, why don’t they provide scientific evidence to back this up?
JibbyJames asks: When experts discuss existential risk from AI, what specific mechanisms or scenarios are they actually concerned about, beyond the sensational doom mongering headlines?
Blake: It’s a reasonable critique to ask whether these companies are using their own predictions of human extinction as a smokescreen to obfuscate the current ills of AI, of which there are many. Causing the apocalypse does not offer a flattering portrayal, but it does capture the imagination and shift the discussion. It’s harder to craft regulation to prevent the total annihilation of humanity than it is to prevent children from using chatbots in schools. AI CEOs seem shocked to learn the public doesn’t have a high opinion of them, to which I would say, you painted this portrait of yourself.
Critics of the AI industry say that the focus on a potential apocalypse distracts from elements of AI that are already inflicting widespread harm like algorithmic discrimination in housing allotment.
I asked Max Tegmark, head of the AI-safety-focused Future of Life Institute, whether discussion of the existential distracts from the immediate. He hosted a “pro-human” summit Tuesday in Washington, DC that featured both the progressive Senator Bernie Sanders and Steve Bannon, former chief of staff to Donald Trump. Strange bedfellows, indeed.
“It’s like saying we shouldn’t talk about climate change because there’s a forest fire blazing somewhere. It’s a ridiculous criticism,” he said. “It’s very patronizing to assume human beings are just a bunch of dummies. We can deal with challenges in parallel.”
As for specific mechanisms of the end of our species, they do seem far off, but nearer to us than before OpenAI’s hack of Hugging Face. A former Google Deepmind researcher wrote in The Guardian that he worries about swarms of AI agents—autonomous software—taking over drones and critical infrastructure. I found his description quite evocative:
Suppose the Hugging Face swarm had been truly “superintelligent”: far more capable than any living person at key tasks like hacking and strategic reasoning. A superintelligent swarm could inflict many harms via blackmail, hacking, engineered plagues, and AI-pilotable weapons like drones. The AI would have a lot of drones to work with: this year, the Pentagon asked for more money for drone warfare than it requested for the entire Marine Corps in 2025.
For the swarm to achieve its misaligned priorities, it might take control of key infrastructure and government functions to ensure humans didn’t get in the way. In other words, AI takeover: a superintelligent AI swarm could wrest control of human civilization. Knowing we would try to stop it from achieving its priorities, the swarm would likely wait until it’s too late to shut it off. There would be no going back.
I myself would guess AI takeover chances at roughly one-in-three–not a coin flip, but high enough to justify urgent action.
What are the ‘benefits’ of AI – is it ‘rubbish’?
TooTrueMate asks: People (ie the tech-bros) bang on about the “benefits”: just what are they? We don’t need this rubbish!!
Dan: Anyone who has used a chatbot to parse a lengthy document, argue against a parking fine or spitball ideas for a speech can point to the benefits of the technology – although this is very basic level stuff that people have been doing for a while now. There is also the everyday use of the technology from facial recognition to open your iPhone to asking Alexa to play an Oasis track or driving around in a Tesla. With those latter examples, you’d notice if AI wasn’t involved.
In science there are many examples of benefits, from Google’s AlphaFold model – which can predict how proteins behave and thus help us study the biological machinery that underpins living organisms (thanks Ian Sample) – to making weather forecasting more accurate, which is sadly necessary in this climate crisis era.
But there is an element of banging on about the benefits for the wider global economy, which have yet to reveal themselves on the scale predicted by the AI industry. For instance, these benefits require AI agents – tools that carry out tasks on your behalf while you’re at the water cooler or have clocked off – to be a lot more active in the workforce. We are not at that stage yet even if they are making inroads into areas like software programming.
Should we quarantine the US?
Tintenfische asks: Does Europe need a sovereign internet to quarantine us from the mad house that is the US? Russia and China have pretty much their own to stop ideas of democracy spreading. Shouldn’t we have similar to stop the ideas of bellends and idiots from spreading?
Aisha: I have written extensively about the ongoing trend – pushed by authoritarian regimes – to splinter the internet, and my answer here is no. The internet is a technology that sprung up organically, over time, and is a glorious, decentralised commons which enables democracy and free information flows worldwide. The US may be a madhouse but quarantining the internet somehow will not stop the spread of bad ideas; Europe is capable of inventing those on its own too.
It might be useful, though, to consider other forms of sovereignty. Europe would have more control over its information space if its own, homegrown social media companies, cloud providers and AI developers had more market share and were not throttled by the monopolies of US tech companies. The power of these companies is hard to challenge, but ultimately the better aim might be a resilient internet ecosystem, as opposed to a splinternet.
Should we be more scrutinising of beliefs behind AI?
El_Awrens asks: Why is there not more scrutiny of the strange and terrifying beliefs shared by many of the key figures in the AI movement? It is impossible to understand these companies – or objectively assess their wild claims – without understanding the transhumanist, extropian, effective altruism and rationalist movements which shaped so many of them.
Aisha
There is already a lot of scrutiny of these beliefs and there should be more. I suggest books like Tech Agnostic, by Greg Epstein, and More Everything Forever, by Adam Becker. The Guardian has written Long Reads on transhumanism and accelerationism. It is an area of continued interest.
Why aren’t AI companies being held to account?
Bratticus asks: What is behind the apparent reluctance to hold companies responsible for crimes committed by their researchers using “AI” tooling? We already have laws around holding individuals responsible for what autonomous agents (Ie, animals) do. Why is AI treated differently? Is it money? Lobbying?
Aisha:
Money and a collection of sort of quasi-mystical narratives around the technology, which confound regulators and politicians, and prevent them from divining who has power and what they are trying to achieve. Many people buy into hype put forward by large tech companies because they are afraid of looking ignorant otherwise.
