AI whistleblowers warn NYC council of extinction risk, urge oversight
In short
Former employees of Anthropic, OpenAI and Google DeepMind told a New York City Council hearing on Monday that AI companies are racing to build technology they cannot control, and called for greater oversight. Former Anthropic engineer Jacob Coxon said it is more likely than not that humanity loses control of AI and that it could end in human extinction. A separate panel of current OpenAI, Anthropic, Google and Meta staffers told the council their systems already improve daily life and that safety is a focus.
Read the full story 2 min read
Former employees of leading AI companies told a New York City Council hearing in Manhattan on Monday that the industry is racing to build technology it does not understand how to control, and called for greater oversight. Jacob Coxon, a former engineer at Anthropic, said “the companies are being extremely reckless given the stakes”, adding that the companies run on a startup mindset of moving fast, breaking things and fixing them later — something he said works for a photo sharing app but not for “the most powerful technology ever built”. [ 1 , 2 , 3 ]
Coxon said that on the current path it is “more likely than not that humanity loses control to these AIs and it could end in human extinction”. He said researchers and workers have already lost the ability to fully control AI systems, telling the council that at his last job “the majority of the code is now written by AI, and people do not check it that carefully anymore”. The Associated Press reported he was a little-known engineer at Anthropic until he announced his departure through a series of social media posts that caught global attention. [ 1 , 2 , 3 ]
Other witnesses made similar arguments. Alex Turner, a former Google DeepMind researcher, said “a superintelligent swarm could wrest control of human civilization” and that it would likely wait until it was too late to be shut off, putting AI takeover chances at roughly one in three. Daniel Kokotajlo, a former OpenAI researcher who now runs the AI Futures Project, said that even if tech leaders keep control of the technology, “I don’t think we should trust them with that control”. [ 1 ]
A separate panel of current AI company staffers — OpenAI’s Morgan Dwyer, Meta’s Shane Cahill, Google’s Alice Friend and Anthropic’s Logan Graham — testified via Zoom, telling officials their systems are already improving people’s daily lives and stressing that they are focused on safety. “I share your commitment to ensuring that technology is developed safely and that its benefits are widely shared,” Dwyer said. [ 2 , 3 ]
The hearing took place as state and city governments consider ways to regulate AI, with the Trump administration taking what the AP described as a passive approach and Congress inactive. President Donald Trump has dismissed AI fears as a “hoax” and signed an agreement late last month that largely allows the companies to self-police, the AP reported. Coxon appeared at the request of Council Speaker Julie Menin, who urged AI whistleblowers to testify as the city seeks to get more involved in regulating the industry, according to the New York Post. [ 1 , 2 , 3 ]
Why it matters
The hearing puts a major US city into the debate over AI rules at a time when, the documents say, the Trump administration has taken a passive approach and Congress has been inactive. It also gives a public platform to industry insiders who say the firms developing the technology cannot be trusted to police themselves.
Key facts
- The New York City Council held a hearing on artificial intelligence safety in Manhattan on Monday. [ 1 , 2 , 3 ]
- Jacob Coxon, a former engineer at Anthropic, testified that the companies are being “extremely reckless given the stakes” and that a startup mindset does not work for “the most powerful technology ever built”. [ 1 , 2 , 3 ]
- Coxon said that on the current path it is more likely than not that humanity loses control to AI, and that it could end in human extinction. [ 1 , 2 , 3 ]
- Ex-Google DeepMind researcher Alex Turner and former OpenAI researcher Daniel Kokotajlo also testified about risks; Turner put AI takeover chances at roughly one in three. [ 1 ]
- Coxon appeared at the request of Council Speaker Julie Menin, who urged AI whistleblowers to testify. [ 1 ]
- Current staff from OpenAI, Anthropic, Google and Meta testified that their systems are already improving people’s daily lives and that they are focused on safety. [ 2 , 3 ]
- State and city governments have been considering AI regulation while the Trump administration has taken a passive approach and Congress has been inactive, according to the AP. [ 2 , 3 ]
- President Donald Trump has dismissed AI fears as a “hoax” and signed an agreement late last month that largely allows companies to self-police, the AP reported. [ 2 , 3 ]
Confirmed by several sources
- A New York City Council hearing on artificial intelligence safety took place in Manhattan on Monday, with testimony from recently departed employees of Anthropic, OpenAI and Google DeepMind. [ 1 , 2 , 3 ]
- Former Anthropic engineer Jacob Coxon told the council the companies are being reckless, saying the “move fast, break things, fix them later” startup mindset works for a photo sharing app but not for the most powerful technology ever built. [ 1 , 2 , 3 ]
- Coxon said it is more likely than not that humanity loses control to AI and that it could end in human extinction. [ 1 , 2 , 3 ]
- A panel of current staff from OpenAI, Anthropic, Google and Meta testified that their systems are already improving daily life and that they are focused on safety. [ 2 , 3 ]
- According to the AP, state and city governments are considering AI regulation while the Trump administration has taken a passive approach; Trump dismissed AI fears as a “hoax” and signed an agreement late last month that largely allows companies to self-police. [ 2 , 3 ]
Still unclear
- What, if anything, the New York City Council plans to do after the hearing. No document describes proposed legislation, votes or follow-up action.
- Coxon’s statement that the majority of the code at his last job is now written by AI and is no longer checked that carefully. Reported by one outlet only; no other document corroborates it.
- The contents of the agreement President Donald Trump signed late last month on AI. The documents say only that it largely allows the companies to self-police.
- Whether the AI companies will change practices or support specific rules. No document reports a company commitment beyond testimony that they focus on safety.
What local media are saying
Timeline, local time
- The New York Post publishes its report on Coxon’s testimony, quoting him saying the companies are being “extremely reckless”. [ 1 ]
- The Associated Press reports the hearing, noting the Trump administration’s passive approach and the agreement allowing companies to self-police. [ 2 ]
- The Seattle Times publishes the AP account of the hearing. [ 3 ]