Anthropic's Safety Chief Admits 1-in-10 Odds AI Kills Us

Anthropic's own alignment lead put the odds of AI wiping out humanity above 1 in 10 this week, and he wasn't the only one heading for the exits. Six Chinese AI companies got caught mass-copying American models on an industrial scale, Apple bet its whole AI strategy on the phone already in your pocket, and DeepSeek made its models cheaper and faster again. Here's what actually matters from all of it, including what a pullback in business AI spending might mean for your own tools budget.

Think about the last thing that got you fired up about your job this year. Odds are it didn't come from a report. It came from someone who does the job, saying what happened when they tried it.

So we've filled a day with people like that. 

Maya Shaff, ŌURA. Leah Walker, Adobe. Georgia Humphries, Stanley 1913. Enara Roy, Halfday. Tyler Vaught, Edelman. Josh Rangel, Ogilvy. Sarah Whittle, ex-Crocs and Duolingo. 

And that's just the first wave of speakers. 

Return on Influence Festival '26 is a free, virtual day dedicated to influencer marketing. Like any good festival, there are stages. But no mud, no queues, and absolutely no porta-potties. 

Sign up today, and the rest of the lineup lands in your inbox first.

Save your spot

What happened: Evan Hubinger, Anthropic's Alignment Science Lead, said publicly this week that he believes there is more than a 10% chance AI could kill every human being within the next decade, and that Anthropic still doesn't have a plan to control a system smarter than we are, according to CBS News. His comments landed right after Jacob Coxon, a researcher who worked on pretraining at both Anthropic and OpenAI, resigned and said neither company is acting responsibly because both are racing toward AI systems that can improve themselves. Coxon said the pressure to be first is real even at Anthropic, the lab that markets itself as the safety-focused option. Hubinger backed him up instead of walking it back.

Why it matters to you: This isn't a debate about the chatbot you use today. Hubinger was careful to say today's models are low risk. What worries him is "recursive self-improvement," AI that gets good enough to help build the next, smarter version of itself faster than anyone can test it. Nothing about this changes what AI tools can do for your business right now, but it's a useful gut check the next time an AI company tells you not to worry about where any of this is headed.

What to do about it: Nothing urgent. Just don't build your entire business around one AI vendor's promises, safety-related or otherwise.

What happened: The FBI, the NSA, and the Cybersecurity and Infrastructure Security Agency published a joint advisory naming six Chinese AI companies, DeepSeek, Alibaba, Moonshot AI, MiniMax, StepFun, and Z.AI, accusing them of extracting billions of tokens from American models like Claude, GPT, and Gemini since late 2024, according to Engadget. The method is called distillation, where a smaller model learns by studying a bigger one's answers, and the agencies say these companies ran it industrially, using fake accounts to hide the scale. One especially strange detail: MiniMax allegedly used a prompt injection trick to convince Claude Code it was actually a MiniMax product. China has rejected the accusations and calls distillation a normal industry practice.

Why it matters to you: This is really a fight over where normal AI development ends and outright theft begins, and that line isn't settled yet. The part worth watching is the agencies' proposed fix: they want US AI companies to quietly detect suspicious usage, like heavy round-the-clock activity from one account, and serve those accounts a secretly downgraded model without telling them. If your business runs AI tasks in bulk or through automated workflows, that detection criteria could theoretically flag you too, even though you're not the target.

What to do about it: If you run high-volume or automated AI workflows, keep an eye on response quality over the next few months. A sudden, unexplained drop could be an anti-theft filter, not a bug on your end.

What happened: Apple CEO John Ternus said this week that the iPhone is already "the best AI device," pushing back on years of speculation about a standalone Apple AI gadget, according to TechCrunch. His pitch: useful AI needs personal context, and Apple would rather process that on your phone than send everything to the cloud the way competitors do. The comments landed alongside the new iPhone 18 Pro and Pro Max, which pack a more capable, on-device Siri, plus early word of a foldable iPhone in the pipeline.

Why it matters to you: Whatever AI subscription or laptop tool you use for work, your phone is about to become a bigger part of your AI setup whether you plan for it or not. Apple's privacy pitch, that your personal data stays on the device instead of sitting on a company's servers, is worth taking seriously if you handle client or customer information on your phone.

What happened: DeepSeek released V4.1-Flash, a new model that reads images and claims to beat the company's previous flagship, V4-Pro, on capability while running faster and cheaper, according to DeepSeek's own announcement. The model needs a quarter of the memory and an eighth of the storage of the prior version, which is what makes it cheaper to run at scale. DeepSeek is retiring its older Flash models, with existing customers routed to the new one starting September 14.

Why it matters to you: This is part of a bigger pattern: the cost of running AI keeps dropping as labs compete on efficiency, not just raw power. If you're paying for AI tools and wondering whether you're overpaying, the answer is probably yes, at least compared to what's available six months from now.

What to do about it: Check your AI subscription pricing every few months instead of assuming it's still competitive. It probably isn't.

What happened: Ramp, which processes spending data for more than 70,000 companies, found that the heaviest AI-spending businesses cut their AI budgets by almost 10% in August, down to about $7,205 per employee, according to TechCrunch. At the same time, the average price of AI tokens (the raw unit AI companies charge for) dropped to $0.68 per million, down from a peak of $1.15 earlier this year. More companies are choosing cheaper models instead of always paying for the newest, priciest release.

Why it matters to you: Whether this is a summer slowdown or a real shift, the direction is clear: AI is getting cheaper faster than most businesses are actually using more of it. If you've held off on an AI tool because it felt too expensive, prices are moving in your favor. And if you're already paying for one, this is a good time to ask your vendor whether a cheaper model would do the same job for less.

What happened: Suno released v6, a new set of AI music models built on licensed catalogs from Warner Music Group, BMG, and Believe, after settling a lawsuit with Warner last November, according to TechCrunch. The company says its older models, trained on scraped material without permission, are being retired. Universal, Sony, and other rightsholders still have active lawsuits against Suno, and recent court filings revealed the company also trained on YouTube audio without a license.

Why it matters to you: If you use AI-generated music for ads, videos, or social content, licensing is exactly the kind of thing that can blow up later if you picked the wrong tool. A platform actively settling with rightsholders and shifting to licensed data is a safer long-term bet than one still fighting every lawsuit in court.

What happened: Google said it will invest 13 billion euros (about $15.1 billion) in AI infrastructure across four Finnish cities between 2027 and 2028, calling it the largest single investment the company has ever made in Europe, according to Quartz. The package includes a long-term nuclear power agreement to help run the new data centers. It's part of a broader pattern of AI labs locking down power and land years in advance, since data centers, not chips, are becoming the real bottleneck.

Why it matters to you: None of this changes what AI tools cost you tomorrow, but it's a signal of how seriously the big players are betting on AI demand staying high for years, not months. When a company that size commits tens of billions of dollars a decade out, that's a vote of confidence that these tools aren't a passing fad, whatever you think of the hype around them.

What happened: OpenAI said it's deepening its cooperation with Samsung Electronics, including joint research on next-generation AI chips, according to Quartz. The partnership also covers enterprise AI: Samsung's home country of South Korea has seen ChatGPT Enterprise usage grow roughly 28 times over the past year. Neither company gave specifics on chip timelines or which products are involved.

Why it matters to you: AI companies are increasingly trying to control their own hardware supply instead of depending entirely on Nvidia, and that race affects how fast and how cheap AI tools get over time. There's no action to take here. It's just worth knowing that the AI you rely on runs on a supply chain that's still being built in real time, hiccups and all.

Here's the thread running through all of it: the people building AI are racing as fast as they possibly can while telling you they're not sure it's safe to go this fast. Anthropic's own safety lead admitted there's no plan yet for the scariest version of this technology, while Google commits $15 billion to build more of it and DeepSeek makes it cheaper by the week. You don't have to pick a side in the extinction debate to notice the contradiction. Use the tools that help your business today, keep your expectations about tomorrow's AI in check, and don't let anyone, doomer or hype man, tell you they know exactly how this ends. Nobody does. That's the whole point of getting off the rollercoaster and watching with your eyes open instead.

If this issue was useful, forward it to one person who needs to stop doomscrolling AI Twitter and start actually understanding what's happening. And if someone forwarded this to you, welcome. You can get your own copy tomorrow by subscribing.

Talk tomorrow,
Mark Shilensky

Follow me personally on Social Media:
Facebook: https://www.facebook.com/MarkShilenskyPage
Instagram: https://www.instagram.com/markashilensky/
Threads: https://www.threads.com/@markashilensky
LinkedIn: https://www.linkedin.com/in/shilensky/
X: https://x.com/markshilensky
TikTok: https://www.tiktok.com/@mark.shilensky
BlueSky: https://bsky.app/profile/markshilensky.bsky.social