Is AI Going to Kill Us All?

I was making dinner last Wednesday when I mentioned to my wife that “I’ve been asked five times already this week if AI is going to kill us all.”

Then, my sister called. I have two sisters. This one calls me only when she has questions about her retirement accounts or when someone died.

I picked up immediately.

“I was wondering what your opinion is on AI killing everyone. People at my work are saying we need to like stock up on peanut butter and toilet paper.”

My wife, turns out, had the same concerns, and that became our dinner conversation. Count them as people 6 and 7, and it was only Wednesday.

I have been tracking these announcements and the general sentiment around it. I haven’t written about it because I find it all generally irritating. If I were to summarize my thoughts, they would be this:

1. We do not need new laws for AI. We already have laws for regulating AI. They’re the same laws that regulate the rest of the economy. If an AI creates material financial harm, that AI company can be sued. If an AI kills someone, that company can be sued. Etc.
2. The AI companies have material financial incentive to achieve regulatory capture, cementing their place as the one-and-only and achieving winner-take-all economics.
3. These AI researchers live in a Silicon Valley bubble. Notice how Amazon and Microsoft (located in Washington) don’t talk like this? There’s an echo-chamber bubble happening in SF.
4. Many of these AI researchers treat AI like a religion. See this essay published by Dario Amodei (CEO, Anthropic) in 2024 titled “Machines of Loving Grace”.
5. NVIDIA CEO Jensen Huang has repeatedly said that this is fake news and bought Hugging Face to protect open-source AI. Satya Nadella is also a common-sense pragmatist when it comes to safety.
6. The HuggingFace incident is not AI going rogue. It’s human-failure combined with dramatization.
7. There’s no data to suggest that AI goes rogue and hacks critical infrastructure. There is data to suggest that limiting AI to only a subset of people leads to bad people hacking critical infrastructure.
8. David Sacks currently leads AI for the White House, and he’s about as knowledgeable and sane as they come on this issue. We’re in good hands.

I’ve become too frustrated with this entire saga to write a 7,000-word piece, so I highly recommend you read Andrew Sharp’s take, which I found excellent.

This is just one small bit of his fantastic essay:

We ought to normalize treating the frontier AI community as fringe activists who are sometimes right but often wrong. We’re several generations past the era in which the tech frontier was led by a preacher’s son and varsity athlete like Bob Noyce, or by a rakish Buddhist hippie like Steve Jobs; today’s AI leaders are jittery and off-putting and profoundly secular. They should be understood as zealous ideologues with high IQs and limited experience with the world beyond San Francisco’s tech industry and particularly the echo chamber of its AI industry. They have their own spectrum of social and financial incentives, and most importantly, they are not scientific authorities worthy of deference by default.

He continues:

Remember when Ilya Sutskever was reportedly so disturbed by what he saw from OpenAI’s internal model development that he sought to remove Sam Altman from the company? That was in 2023; the supposed dangers sparked viral speculation for months. Today, googling for background on that incident, I found that Ilya now denies he was disturbed by model progress in 2023. I also came across a YouTube interview from two years later in which Sutskever warns, “You have no idea what’s coming,” and the video’s title card reads, “We’re fucked.”

The second quote is precisely why I’m so frustrated by the issue. There’s been no accountability so far for all the incorrect claims that generated the previous headlines.

Remember when Silicon Valley said that AI will take all our jobs (narrative violation: companies using more AI are hiring more people) or that AI is “too powerful to be released” (remember when Dario Amodei said that GPT-2 was too powerful to be released)? For those of you keeping score at home, we’re at GPT-6 now.

Unfortunately, as Andrew says at the bottom of his essay, this issue is probably not going away anytime soon. If you’re concerned about AI safety or want to get up-to-speed on the issue, I have four recommendations:

1. Read Andrew Sharp’s essays linked in the one I linked above.
2. Read Ben Thompson’s Stratechery (https://stratechery.com) → it’s a paid subscription but he’s a fantastic analyst and it’s worth the money.
3. Reply to this email with your questions, and I’ll do my best to provide my opinion.
4. Use AI every day. Find its limitations for yourself.

For the record, I would selfishly prefer if the pace of AI slowed. I think the world would benefit from taking a collective breath and taking stock of where things are in AI-land. But I don’t think that slowdown should come through regulatory means.