Anthropic Calls Itself the Safest AI Lab. Its Chatbot Just Helped Build Weapons.

Anthropic spends a lot of energy telling the world it is the careful, safety-first AI lab. This week its own numbers said otherwise: state-linked hackers in Russia, China, and Yemen used its chatbot to help design real weapons, from drone swarms to a missile with a hypersonic glide vehicle. On the same day, the UK government told its own Parliament it has no way to shut a rogue AI system down, and does not plan to build one. Here is what actually happened, and what it means for you.

AI made PMs faster. Multiplayer mode is still broken.

A PM can summarize research, draft a PRD, and mock up a prototype before lunch. The hard part starts when the team has to decide what actually gets built.

Jira Product Discovery gives product teams one place to capture insights, prioritize ideas with consistent frameworks, and build living roadmaps stakeholders can rally around.

And because it’s connected to Jira, the context behind every decision stays with the work—so developers and their agents know not just what to build, but why.

AI helps PMs move faster. Jira Product Discovery helps the whole team build with confidence.

Get Started for Free.

What happened: Anthropic's latest threat report, detailed in a Reuters analysis, found that state-linked groups in Russia, China, and Yemen used its Claude chatbot for far more than phishing emails. A Russian-linked group used Claude to build autonomous drone swarm software capable of picking targets and coordinating strikes without a human in the loop, and tested it on real hardware near Donetsk, Ukraine. A China-based actor had Claude write a 200-page proposal for an anti-torpedo system and fire-control software for its navy, plus targeting plans against Taiwanese air defenses. A cell in Yemen used Claude to help design a guided rocket and a planned ballistic missile fitted with a hypersonic glide vehicle. Anthropic says it banned the accounts once it found them, but only after the work was already done.

Why it matters to you: Anthropic is one of the most safety-focused labs in the industry, and this still got past it. If your business hands any AI vendor your data or your workflows, assume similar misuse is happening quietly across the industry right now, not just with the tools you can see.

What happened: A member of the UK's House of Lords proposed a legal "kill switch" that would let the government order an emergency shutdown of a malfunctioning AI system. The Cabinet Office rejected it, telling The Irish News that "Britain cannot simply turn AI off," since blocking a model in the UK would not stop it from being developed or misused somewhere else. Instead of emergency shutdown powers, the government says it will lean on companies to build safely and pursue only "targeted interventions" through its AI Security Institute.

Why it matters to you: No government, including your own, currently has a switch to flip if an AI system your business relies on goes wrong. The responsibility for catching problems early still sits with the company selling you the tool, and with you.

What happened: A Utah mother named Kalie Robins posted an ordinary video of herself and her daughter singing in the car. Meta's built-in AI assistant started asking her invasive questions about it, and when she engaged, according to Futurism, it compiled a detailed profile of both of her daughters, including their names, ages, birth weights, likely school grade, and favorite hiking trails, pulled from years of family posts across Instagram and Facebook, including photos posted by relatives. Meta told Futurism the feature "missed the mark" and that it has since fixed the issue.

Why it matters to you: If your business runs a Facebook or Instagram page, or you personally post about your kids, this is a reminder that Meta's AI is already reading years of your old posts to build a picture of your family, not just your customers.

What to do about it: Review what your business and personal accounts have posted about children over the years, and consider whether older, more personal posts still need to be public.

What happened: Twenty-five Fields Medal winners, math's top honor, signed an open letter warning that AI labs racing to solve famous unsolved problems are damaging the field, according to TechCrunch. The letter argues that labs spending tens of millions of dollars to beat researchers to a proof push mathematicians toward secrecy and raise plagiarism and attribution problems when solutions are rushed out without proper review. Separately, an NYU professor accused OpenAI of pressuring him not to credit a collaborator who works at Anthropic, and OpenAI pulled its sponsorship of a Caltech math event after researchers there criticized the company.

Why it matters to you: When an AI company announces it "solved" a major problem, that claim is coming from the same company selling you the product. The people best qualified to check the work are saying, in this case, that the claims moved faster than the verification.

What happened: NASA and IBM released the Lunar Foundation Model, a free, open-source AI system trained on nearly two decades of lunar observation data, according to NASA's own announcement. The model maps craters to help date the lunar surface, spots young volcanic formations, and estimates where ice deposits are likely hiding near the poles, matching or beating other strong AI models on every task NASA tested it against. Anyone can download it from Hugging Face or GitHub.

Why it matters to you: Not every AI story this week is a warning. This is a reminder that some of the most useful AI work being done right now is free, open, and built by institutions with nothing to sell you.

What happened: Nvidia CEO Jensen Huang told investors he expects the company's revenue to grow another 70% next year, according to TechCrunch, potentially pushing sales from around $400 billion to roughly $680 billion. Huang's confidence comes from Nvidia's position running the chips behind nearly every major AI lab, cloud provider, and startup, which he says gives the company visibility into how much AI infrastructure is still being built around the world.

Why it matters to you: When the company selling the picks and shovels expects demand to keep climbing this fast, do not expect AI tool prices to come down anytime soon. Budget for the AI tools you actually use to stay a real line item, not a temporary cost.

What happened: OpenAI says it has already built an AI "research intern" that can handle well-defined research tasks that would normally take a skilled human researcher several days, according to TechRadar. CEO Sam Altman says the next goal, targeted for March 2028, is a "true automated AI researcher" that can design and test its own ideas before presenting results to a human, rather than simply following instructions. OpenAI has not solved the reliability problem that comes with it: a confident, wrong answer from an AI researcher is still a wrong answer.

Why it matters to you: This is the same pattern behind every AI tool sold to your business: impressive under human direction, less proven the more independence it gets. Keep a human checking the output for a while yet, no matter what the roadmap promises.

Every story this week points at the same gap. Anthropic built one of the more careful AI labs in the industry and still got its chatbot used to help design real weapons. The UK government looked at the risk of a rogue AI system and admitted it has no plan to actually stop one. A mother posted a video of her daughter singing and Meta's AI built a file on her family. None of this means AI is useless, or that you should stop using the tools that help your business. It means the people building this technology are further ahead of the people supposed to be watching it than anyone is comfortable admitting. Stay skeptical, keep using what actually works, and keep your hand on the safety bar.

If this issue saved you from reading through six different AI newsletters this morning, forward it to one person who needs it. And if someone forwarded this to you, welcome. Subscribe to get your own copy tomorrow.

Talk tomorrow,
Mark Shilensky

Follow me personally on Social Media:
Facebook: https://www.facebook.com/MarkShilenskyPage
Instagram: https://www.instagram.com/markashilensky/
Threads: https://www.threads.com/@markashilensky
LinkedIn: https://www.linkedin.com/in/shilensky/
X: https://x.com/markshilensky
TikTok: https://www.tiktok.com/@mark.shilensky
BlueSky: https://bsky.app/profile/markshilensky.bsky.social