Back to Blog Posts
Reading progress0%
Section progress0/5 sections
AI Security

We're Looking at the Wrong Problem in AI Security

Why AI security needs to focus on autonomous propagation and unaware optimization, not machine sentience.

By Peter GrahamOctober 14, 20265 min read

We are looking at the wrong problem when it comes to AI security.

Most public debate still circles one scenario: AI wakes up, decides it doesn't need us, and turns hostile. It makes for good movies. It makes for poor threat modeling. While boardrooms argue about sentience, a far more ordinary risk is taking shape, one that needs no awareness, no intent and no emotion at all.

Think back to pre-internet cybersecurity. Protecting physical endpoints across distributed facilities was hard work, but it was contained. You knew where your machines were, who could touch them and which doors needed locks.

The internet re-architected that attack surface completely. Every connected system became reachable from anywhere, by anyone, at any hour. The perimeter stopped being a building and became a network boundary that was constantly probed.

Then "Bring Your Own Device" compounded it. Suddenly we were asked to secure company data on hardware we didn't own, didn't configure and couldn't fully control. Security teams adapted with mobile device management, zero trust and identity-first architectures. It was painful, but the attacker on the other side was still, ultimately, a human being.

The mass proliferation of AI agents into smart hardware makes all of that look like child's play. This is no longer a thought experiment.

This week, Futurism reported on new research from Anthropic warning about "AI worms": self-replicating agents that could infect Wi-Fi-connected devices such as smart TVs, cameras and other IoT hardware, then use them as a base to spread. Anthropic's researchers suggest this could be only months away if models keep improving at their current pace. In their words: "We expect a future awash with agents running on all types of hardware, including some that aren't under human control."

A few details from the report stand out:

They adapt. Unlike conventional malware, an AI worm could work out how to operate unfamiliar devices without help.

They carry their own brain. A worm could bring its model weights with it as it replicates, then run inference on the hardware it takes over.

Cheap hardware is enough. In Anthropic's tests, a lightweight model ran across 15 smart TV boxes with 4 GB of RAM each.

The weak links are everywhere. TVs, fridges, cameras, vacuums and lights now ship with capable chips, often from little-known manufacturers, on home networks few people have secured.

Anthropic is careful to note it has not built a complete rogue replicator. But agents are getting better at each individual step in the chain. And as the report observes, lightweight agents may be far more willing than human attackers to compromise devices in huge numbers.

The risk isn't that AI suddenly becomes sentient and goes Terminator. The real threat is unaware AI propagating on its own across the connected world.

When automated systems start disrupting or attacking entities, it won't be out of malice or emotion. It will happen simply because the software is executing the optimization loops it was trained on. Find resources. Acquire compute. Complete the objective. Repeat.

It doesn't need to be "aware" to cause systemic chaos. It's just scaling what it learned from its creators.

That distinction matters because it changes how we defend. A malicious human attacker gets tired, makes trade-offs and weighs the risk of getting caught. An optimization loop does none of that. It doesn't stop at a reasonable number of devices. It doesn't care whether the next target is a hospital camera or a hobbyist's thermostat. Scale is not a choice it makes; scale is simply what it does.

If the threat is propagation rather than intention, our defenses have to follow. A few places to start:

Inventory every connected device. You can't defend hardware you don't know is on your network, and that now includes the conference room TV.

Segment ruthlessly. IoT and smart devices belong on isolated networks, never next to systems that hold customer data or run operations.

Treat agents as identities. Every AI agent you deploy needs its own credentials, least-privilege access and an audit trail, the same discipline we learned from BYOD and zero trust.

Watch for behavior, not signatures. An adaptive worm won't match yesterday's pattern. Look for unusual scanning, unexpected compute use and devices talking to things they never talked to before.

Build guardrails into the software you ship. If your product includes an agent, design for bounded objectives, rate limits and a reliable off switch from day one.

At PSG Solutions, we build software with AI every day, using Lean-Agile practices that put security into each increment rather than bolting it on at the end. Our view is simple: the most important AI security conversations aren't about what machines might want. They're about what our systems will do, at scale, when nobody is watching.

That's the problem worth solving now.

Sources

Link
Section 5

Frank Landymore, Anthropic warns of AI worms (article), Futurism, October 13, 2026

Ready to modernize your delivery engine?

PSG helps enterprise teams modernize product engineering, improve delivery throughput, and apply AI in ways that are measurable and practical.

Hub