Twenty-two researchers put their names to a policy paper in Science warning that coordinated AI agents can now manufacture public opinion at scale. Two of them came on CXOTalk, and their finding is not the one most people expect.

The threat is not better fake content. It is that the fakes now coordinate with one another, increasingly with no one at the controls. Daniel Thilo Schroeder is a research scientist at SINTEF. Jonas R. Kunst is a professor of communication at BI Norwegian Business School. Here is how I opened episode 915:

AI swarms are the most dangerous influence weapons ever built. Daniel Thilo Schroeder and Jonas Kunst co-led a 22-author study published in Science to map this threat. Daniel, what are AI swarms, and how do they fabricate reality?

Michael Krigsman, CXOTalk episode 915

The command center is disappearing

there is, decreasingly needs for human oversight. So we’re kind of moving from, a central command that usually, was at play in traditional bot networks to kind of emergent hive behavior.

Jonas R. Kunst, Professor of Communication, BI Norwegian Business School

An old bot network had an operator you could find, subpoena, sanction. What Kunst describes has no operator. The agents run their own message experiments, learn what lands, and adapt among themselves. Schroeder’s explanation of why it works is not about intelligence.

large language models, for example, are super convincing even though they lack capabilities to reason. They are very good with talking you into something or talking you out of something.

Daniel Thilo Schroeder, Research Scientist, SINTEF

People are generally conformist, even though we don’t want to say this about ourselves, but if people believe something, we assign some credibility to it. This social, heuristic is hijacked by these swarms.

Jonas R. Kunst

Persuasion does not require reasoning. It requires volume, consistency, and the appearance of consensus, and the swarm manufactures all three.

Every individual message now passes as human

This large language model and this is in particular true for online social networks, would pass Turing test.

Daniel Thilo Schroeder

I said it back to be sure: each message is now indistinguishable from one a real person would write, which makes the message itself a dead end for detection. Then I described something I had run into myself.

I was looking at, TikTok the other day, and I came upon a TikTok post. Had something to do with Elon Musk and saying. Describing all of his properties in Texas. And there were, 600 or more at that time messages, most of which were supporting Elon Musk. And I was astonished because it was obvious that these are fake messages because just the volume and the consistency of the message.

Michael Krigsman

Notice what gave it away. Not any single post. The volume and the consistency, which is the argument of their paper, arrived at by accident on a phone.

Detection has to move to the group level

That’s the only way to do it because the individual accounts are so believable, the messages won’t reveal an AI agent, and by far not an AI swarm anymore. We need to look at the group level, the coordination level.

Jonas R. Kunst

This reframes an industry. The money going into AI content detection is spent on the message, the one place a swarm is already indistinguishable. The signal lives in the coordination: timing, overlap, the shape of the network. Kunst says platforms guard that data or price it out of reach.

The platforms are being paid for the pollution

because the system cannot, distinguish, or not with a high accuracy distinguish between a genuine human outrage and algorithmic mimicry, these platforms are, effectively paying malicious, actors to pollute their own information ecosystem.

Jonas R. Kunst

Engagement is engagement. A system optimizing for outrage cannot tell whether the outrage is real, and swarm traffic monetizes exactly like human traffic. So I put the uncomfortable version to them.

Especially since the AI swarms are so effective at creating social media posts that look indistinguishable from real, so they can therefore turn a blind eye. They have, plausible deniability.

Michael Krigsman

Nothing in the business model asks anyone to look.

The audience pushed on the part that lasts

Two audience questions shaped the hour. Anthony Scriffignano, a data scientist and repeat CXOTalk guest, asked about disinformation that spreads by accident, as content is distorted through reproduction, like a copy of a copy of a copy. Chris Petersen asked on X whether swarms are already coordinating across platforms and media types. Kunst described where the distortion ends up.

And when, large language models are trained on this data, these, synthetic narratives, calcify within, their model weights during retraining. And that poisons the internet’s epistemic substrate

Jonas R. Kunst

The fabricated consensus does not evaporate. It becomes training data, and the next generation of models learns it as fact.

An arms race we have not joined

Late in the hour I put it all together.

We have potentially millions of coordinated accounts making agentic decisions, autonomous decisions among themselves as they go Each particular message that they’re sending appears indistinguishable from that of a real person, and we do not yet have effective methods for identifying in a, i-in a, in a repeatable way, identifying these swarm accounts. And the social media c-platforms do not easily give up that data, so it’s therefore hard to simulate. Seems we’re screwed.

Michael Krigsman

Kunst’s framing of where we stand is the line I keep coming back to.

It’s an arms race, but we have not joined the war. We need to be better than the adversaries that use them. But to do that, we need to join this arms race, whether we like it or not.

Jonas R. Kunst

The asymmetry is what makes it a race we are losing by default, and Schroeder was blunt about how little it takes to enter.

I would even argue that you do not need these millions of agents, Maybe it’s enough to have a few that are very convincing

Daniel Thilo Schroeder

Building is cheap. Detecting requires data a handful of companies hold and will not release. Their answer is a distributed AI influence observatory, mandated researcher access to platform data, and defensive simulation to red-team networks before adversaries do.

The same gap I have written about for twenty years

My subject has always been the distance between what technology promises and what organizations actually get. Usually that shows up as a stalled ERP program or an AI pilot that never reaches production. Here it is a defense that does not exist yet, against an attack that already does. Capability arrives first, deployed by whoever moves fastest. Governance arrives years later, if it arrives. Nothing Kunst and Schroeder ask for is a technology problem. It is an institutional one.

What this means if you sit on a board

Executives who file this under politics are making a mistake. The same tactics work against a company: fabricated safety claims, synthetic boycotts, coordinated harassment of executives and board members. Kunst put it plainly.

People need to wake up to this reality. They need to place this threat at the top of their, let’s say, boardroom agenda and, really acknowledge that traditional digital engagement metrics are, fundamentally compromised.

Jonas R. Kunst

Read the last clause again. If your brand health dashboard reads sentiment and volume off social platforms, you are measuring a channel that can be manufactured cheaply, and you cannot separate the manufactured share from the real one.

So this week: stop treating social sentiment as ground truth in any decision that matters, and ask your communications and security teams who would notice a coordinated campaign against your company, and how. Most organizations cannot tell a real backlash from a synthetic one, which is the reason to start now rather than during the incident.