An AI system that hands two people two different revenue numbers has not made a small error. In finance, it has disqualified itself.

Marie Myers is Executive Vice President and Chief Financial Officer of Hewlett Packard Enterprise, deploying AI agents across a finance organization of 3,600 people. On episode 914 of CXOTalk I asked her the question every board is now putting to its CFO. Here is how I opened:

Every company says AI creates value, but few can prove it in the numbers. Marie Myers is chief financial officer of HPE, where she’s deploying AI agents across a 3,600-person finance organization. Marie, how do you assess the value that generative AI brings to a business?

Michael Krigsman, CXOTalk episode 914

The obstacle she describes is not model capability. It is determinism.

Cracking determinism was the hardest part

The question that opened this up was not mine. Chris Peterson wrote in on Twitter and I read it out.

We have an interesting question from Twitter. This is from Chris Peterson, who says, “How do you deal with the non-determinism and lack of explainability when you apply gen AI in a financial context?” I think this is the question that we all want to know.

Michael Krigsman, CXOTalk episode 914

Cracking the code on determinism was actually the hardest part of the journey that we embarked on. We actually co-engineered with Nvidia on a NIM to help crack determinism because you just can’t get finance data wrong.

Marie Myers, EVP and CFO, Hewlett Packard Enterprise

A Fortune 500 CFO could not get a model to behave well enough for finance, so her team co-engineered with Nvidia to force it. That is not a prompt-engineering fix.

That number needs to be correct irrespective of who asked the model, where in the world. It cannot be probabilistic, it needs to be deterministic.

Marie Myers, EVP and CFO, Hewlett Packard Enterprise

Carry that sentence into your own program. A system that is right most of the time is a useful research assistant. It is not an acceptable source for a number that goes into a filing, a covenant, or an earnings call. The gate is not intelligence. It is repeatability, across the roughly half a million data elements in HPE’s finance platform, Alfred.

The same word surfaced on my previous episode, from a security chair. Ben Mayrides, Chief Information Security Officer of Cvent, told me that the security architectures enterprises rely on were built for deterministic systems and are therefore broken for agents. Two functions, one fault line.

The value sits in the indirect column

And sometimes what you find, Michael, is the indirect may sort of actually be much greater than some of those initial direct ROI benefits.

Marie Myers, EVP and CFO, Hewlett Packard Enterprise

She runs two buckets. Direct ROI, meaning cost that came out. And a defined indirect bucket: speed, error rates, fraud reduction. Indirect is not her synonym for unmeasured. It is a named category with its own numbers, and she expects them sooner than an ERP program would have delivered them. A program that has not produced them runs out of excuses faster.

She redesigned the work first, then added the agent

Alexander Sukharevsky of McKinsey told me on episode 922 that fewer than 100 companies have captured most of the value AI has created. Myers explains why so many pilots stop short.

We, we didn’t just apply the AI without sort of redesigning the operating model and then looking at the workflows themselves and then figuring out where in that workflow could we then leverage the AI so that we would have a much more standardized approach.

Marie Myers, EVP and CFO, Hewlett Packard Enterprise

Her sequence is the reverse of the common one. HPE centralized and redesigned the workflow first, then decided where in the new flow an agent belonged. She attributes many failed pilots to skipping that step. She also runs stage gates and uses them.

Some of them didn’t work, Michael, so we pulled the plug and we went and made a different investment elsewhere.

Marie Myers, EVP and CFO, Hewlett Packard Enterprise

Accountability does not move to the model

So for me, the mantra has always been human in the loop.

Marie Myers, EVP and CFO, Hewlett Packard Enterprise

That phrase is usually where a conversation ends. I asked what it costs her.

we’re not at a point in this journey today where you can devolve accountability.

Marie Myers, EVP and CFO, Hewlett Packard Enterprise

There are the teeth. The agent produced the analysis. The named human signs it and answers for it. Deploying agents into accounts payable or credit and collections changes nothing about who is accountable.

The audience asked the sharpest question of the hour

Catarina Collins Serra wrote in on LinkedIn during the show.

And then as a follow-up, we have an excellent question from LinkedIn, and this is from Catarina Collins Serra, who is herself a global CFO and she asks this: “Have you seen any degradation in decision quality when teams rely too heavily on AI outputs, and how are you mitigating that?”

Michael Krigsman, CXOTalk episode 914

Irrespective of whether you’re using AI, you’re not using AI, you can’t let, you know, AI become an excuse for low-quality work.

Marie Myers, EVP and CFO, Hewlett Packard Enterprise

Here is how I handle it in my own work.

I use various AI tools every day, so for example, preparing for our discussion, and it goes out and can research hundreds and hundreds and hundreds of sources. But at the end, when it comes down to our conversation, I have to check every fact. Every every stat or fact it comes back with, before I include it in a conversation like this, I verify it myself.

Michael Krigsman, CXOTalk episode 914

It comes back to the data, the way it always has

The most important part of any AI journey is data. Data quality, data cleanliness is the number one issue that I think holds back a lot of companies and organizations in really embarking on AI journeys.

Marie Myers, EVP and CFO, Hewlett Packard Enterprise

I have spent twenty years on one subject: the distance between what technology promises and what organizations actually get. It was ERP when I wrote about IT project failure for ZDNet. It is agents now. The vocabulary changed. What sits at the bottom of the gap did not.

I believe that the whole area, around management of change is probably the hardest part of AI

Marie Myers, EVP and CFO, Hewlett Packard Enterprise

Dirty data and unmanaged change. Those two answers run through the enterprise failures I have documented for two decades, and here they are again.

What to do with this on Monday

Ask one question of your AI program. If two people in two regions ask for the same number, do they get the same number. If you cannot prove the answer is yes, you do not have a finance system. You have a research tool. Be honest about where it sits.

Then check the sequence. Did you redesign the workflow and then place the agent, or did you place the agent on the workflow you already had.

And accept the job description that arrives with it.

You, you can’t just rely on your financial acumen. You need to use that plus your technology acumen to really help guide the enterprise because CFOs are becoming strategic business partners

Marie Myers, EVP and CFO, Hewlett Packard Enterprise

The last question I asked is one I have no clean answer to. Junior finance staff have always learned the business by doing the first round of analysis. If the agent does that round, how does the next generation learn. Her answer is in the transcript.

Twenty-two researchers put their names to a policy paper in Science warning that coordinated AI agents can now manufacture public opinion at scale. Two of them came on CXOTalk, and their finding is not the one most people expect.

The threat is not better fake content. It is that the fakes now coordinate with one another, increasingly with no one at the controls. Daniel Thilo Schroeder is a research scientist at SINTEF. Jonas R. Kunst is a professor of communication at BI Norwegian Business School. Here is how I opened episode 915:

AI swarms are the most dangerous influence weapons ever built. Daniel Thilo Schroeder and Jonas Kunst co-led a 22-author study published in Science to map this threat. Daniel, what are AI swarms, and how do they fabricate reality?

Michael Krigsman, CXOTalk episode 915

The command center is disappearing

there is, decreasingly needs for human oversight. So we’re kind of moving from, a central command that usually, was at play in traditional bot networks to kind of emergent hive behavior.

Jonas R. Kunst, Professor of Communication, BI Norwegian Business School

An old bot network had an operator you could find, subpoena, sanction. What Kunst describes has no operator. The agents run their own message experiments, learn what lands, and adapt among themselves. Schroeder’s explanation of why it works is not about intelligence.

large language models, for example, are super convincing even though they lack capabilities to reason. They are very good with talking you into something or talking you out of something.

Daniel Thilo Schroeder, Research Scientist, SINTEF

People are generally conformist, even though we don’t want to say this about ourselves, but if people believe something, we assign some credibility to it. This social, heuristic is hijacked by these swarms.

Jonas R. Kunst

Persuasion does not require reasoning. It requires volume, consistency, and the appearance of consensus, and the swarm manufactures all three.

Every individual message now passes as human

This large language model and this is in particular true for online social networks, would pass Turing test.

Daniel Thilo Schroeder

I said it back to be sure: each message is now indistinguishable from one a real person would write, which makes the message itself a dead end for detection. Then I described something I had run into myself.

I was looking at, TikTok the other day, and I came upon a TikTok post. Had something to do with Elon Musk and saying. Describing all of his properties in Texas. And there were, 600 or more at that time messages, most of which were supporting Elon Musk. And I was astonished because it was obvious that these are fake messages because just the volume and the consistency of the message.

Michael Krigsman

Notice what gave it away. Not any single post. The volume and the consistency, which is the argument of their paper, arrived at by accident on a phone.

Detection has to move to the group level

That’s the only way to do it because the individual accounts are so believable, the messages won’t reveal an AI agent, and by far not an AI swarm anymore. We need to look at the group level, the coordination level.

Jonas R. Kunst

This reframes an industry. The money going into AI content detection is spent on the message, the one place a swarm is already indistinguishable. The signal lives in the coordination: timing, overlap, the shape of the network. Kunst says platforms guard that data or price it out of reach.

The platforms are being paid for the pollution

because the system cannot, distinguish, or not with a high accuracy distinguish between a genuine human outrage and algorithmic mimicry, these platforms are, effectively paying malicious, actors to pollute their own information ecosystem.

Jonas R. Kunst

Engagement is engagement. A system optimizing for outrage cannot tell whether the outrage is real, and swarm traffic monetizes exactly like human traffic. So I put the uncomfortable version to them.

Especially since the AI swarms are so effective at creating social media posts that look indistinguishable from real, so they can therefore turn a blind eye. They have, plausible deniability.

Michael Krigsman

Nothing in the business model asks anyone to look.

The audience pushed on the part that lasts

Two audience questions shaped the hour. Anthony Scriffignano, a data scientist and repeat CXOTalk guest, asked about disinformation that spreads by accident, as content is distorted through reproduction, like a copy of a copy of a copy. Chris Petersen asked on X whether swarms are already coordinating across platforms and media types. Kunst described where the distortion ends up.

And when, large language models are trained on this data, these, synthetic narratives, calcify within, their model weights during retraining. And that poisons the internet’s epistemic substrate

Jonas R. Kunst

The fabricated consensus does not evaporate. It becomes training data, and the next generation of models learns it as fact.

An arms race we have not joined

Late in the hour I put it all together.

We have potentially millions of coordinated accounts making agentic decisions, autonomous decisions among themselves as they go Each particular message that they’re sending appears indistinguishable from that of a real person, and we do not yet have effective methods for identifying in a, i-in a, in a repeatable way, identifying these swarm accounts. And the social media c-platforms do not easily give up that data, so it’s therefore hard to simulate. Seems we’re screwed.

Michael Krigsman

Kunst’s framing of where we stand is the line I keep coming back to.

It’s an arms race, but we have not joined the war. We need to be better than the adversaries that use them. But to do that, we need to join this arms race, whether we like it or not.

Jonas R. Kunst

The asymmetry is what makes it a race we are losing by default, and Schroeder was blunt about how little it takes to enter.

I would even argue that you do not need these millions of agents, Maybe it’s enough to have a few that are very convincing

Daniel Thilo Schroeder

Building is cheap. Detecting requires data a handful of companies hold and will not release. Their answer is a distributed AI influence observatory, mandated researcher access to platform data, and defensive simulation to red-team networks before adversaries do.

The same gap I have written about for twenty years

My subject has always been the distance between what technology promises and what organizations actually get. Usually that shows up as a stalled ERP program or an AI pilot that never reaches production. Here it is a defense that does not exist yet, against an attack that already does. Capability arrives first, deployed by whoever moves fastest. Governance arrives years later, if it arrives. Nothing Kunst and Schroeder ask for is a technology problem. It is an institutional one.

What this means if you sit on a board

Executives who file this under politics are making a mistake. The same tactics work against a company: fabricated safety claims, synthetic boycotts, coordinated harassment of executives and board members. Kunst put it plainly.

People need to wake up to this reality. They need to place this threat at the top of their, let’s say, boardroom agenda and, really acknowledge that traditional digital engagement metrics are, fundamentally compromised.

Jonas R. Kunst

Read the last clause again. If your brand health dashboard reads sentiment and volume off social platforms, you are measuring a channel that can be manufactured cheaply, and you cannot separate the manufactured share from the real one.

So this week: stop treating social sentiment as ground truth in any decision that matters, and ask your communications and security teams who would notice a coordinated campaign against your company, and how. Most organizations cannot tell a real backlash from a synthetic one, which is the reason to start now rather than during the incident.

Most agent demos work. Most agent projects never reach production. The gap between those two facts is not model quality, and Christian Kleinerman put a number on it: your proof of concept ran on five well-named tables, and production has a hundred thousand tables that nobody named carefully. Kleinerman is Executive Vice President of Product at Snowflake.

Here is how I opened the show.

Everyone’s talking about agentic AI but most enterprises aren’t ready. I’m Michael Krigsman and today on CXOTalk number 903 Christian Kleinerman Snowflake’s EVP of Product shares hard truths on what it really takes: data readiness, governance AI, economics, and workforce impact.

Michael Krigsman, CXOTalk episode 903

His central claim inverts the usual complaint about AI hype. The technology is running ahead of what enterprises can absorb, not behind it. He sells a data platform, so weigh his interest where it applies. The failure pattern he describes is one I have watched for twenty years under other names.

The technology is ahead of the enterprise, not behind it

I asked him to elaborate on the idea that the state of the technology is in advance of what we in the enterprise can absorb.

Where AI truly shines and truly has potential in the enterprise is when you make the context of that enterprise available to those state-of-the-art models where now I can go to a chatbot and ask a question and it’s not generic knowledge from the internet it is based on knowledge from my company maybe my customers maybe my products.

Christian Kleinerman, EVP of Product, Snowflake

The model is not the scarce asset. Context is. Any competitor can rent the same frontier model by the token. What nobody can rent is an organized account of how your company actually works, and that is the input the model needs in order to be worth anything to you.

Two things kill agent projects

Most of the scenarios where we see projects going wrong or projects getting shut down they run through 2 types of issues. Issue number one is correctness of results or correctness of actions.

Christian Kleinerman, EVP of Product, Snowflake

His second category was security, privacy, and access control. Sit with the first one, because the phrasing changed and the change is the whole story. For a decade we worried about correctness of results, which means a wrong number in a report that a human reviews before anything happens. Agents add correctness of actions, which means the system already did the thing. The error is not sitting on a screen waiting for review. It is in the world.

He drew the boundary himself. “Probably one of the last operations that will be fully automated with AI is gonna be things like a bank transfer or a wire transfer ’cause you wanna make sure that those things are a hundred percent correct.”

If the data is messy nobody can help you

This is in my mind the single biggest impediment for companies to tap into AI today. And it comes down to if the data estate of a company is not well organized or is not broadly accessible there is little magic that the AI can do. I like to say AI is a turbocharged super capable technology but it’s not magic.

Christian Kleinerman, EVP of Product, Snowflake

Then, shorter: “But if the data is messy nobody can help you.”

He offered a test that any executive can run this week without buying anything.

If you were to ask the best analyst in your company to answer a question would this best analyst know where to go know what to do? Then there’s a path for AI.

Christian Kleinerman, EVP of Product, Snowflake

If your best human analyst would not know which system holds the answer, the agent will not know either. It will produce something anyway, confidently, and that is worse than producing nothing.

An audience question got the honest answer

The sharpest exchange of the hour started with a question from the audience. Sai Penumuru asked on LinkedIn why so many projects get stuck at the proof of concept stage. Kleinerman traced it to a period when teams assumed AI was powerful enough that building a customer support agent, or a talk-to-my-database agent, would not be too hard.

And what we’ve seen I would say for sure last year plus is that by the time you try to roll those POCs or those organic efforts into production then you get the intersection with reality and reality is harsh.

Christian Kleinerman, EVP of Product, Snowflake

Reality is like “Oh yeah you tested it with 5 tables but we have a hundred thousand tables and they’re not clearly named.”

Christian Kleinerman, EVP of Product, Snowflake

That is the entire pilot graveyard in one sentence. The proof of concept ran in a clean room that a capable team built by hand. Production is the rest of the company, and the rest of the company was never tidied up for a machine to read.

Stop reading vendor claims and start testing

I asked him how business leaders can evaluate vendor claims about autonomy, about reasoning, about elaborate workflow capabilities, and see what is real.

My recommendation to everyone is go and try the technology. The beauty of this era of AI and agents is most products are a couple of prompts away from someone trying it.

Christian Kleinerman, EVP of Product, Snowflake

I restated it back to him: you have to do the empirical testing yourself, and you cannot rely on vendor marketing to tell you what will work in your environment and your use case. The economics of evaluation have flipped. When a trial is a couple of prompts away, the long formal bake-off becomes an expensive way to postpone finding out. It also removes the last excuse for buying on a slide deck.

I have been covering this failure for twenty years

I spent two decades writing about IT project failure for ZDNet and founded a consultancy, Asuret, to diagnose it. Every era has its big bang: the ERP program that goes live everywhere on one weekend, the data warehouse that will finally give everyone a single version of the truth. The pattern never changes. Organizations buy scale before they have earned it with one small thing that works.

Don’t take on large-scale AI rollouts. When you hear “We’re gonna buy a hundred thousand licenses of X Y Z solution and put it out there” and it’s like well do you even know if it works or not?

Christian Kleinerman, EVP of Product, Snowflake

Notice who is saying that. An executive whose company sells software is arguing against the large seat purchase, and the incentive runs the other way.

What to do with this on Monday

Run his analyst test on the use case you are about to fund. If your best human analyst would not know where to look, fix the data before you buy the agent. Put one small use case into real production, with real users and real consequences, rather than a demo that lives in a clean room, and scale only the parts that survive contact. Log what the agent does, evaluate it continuously, and watch for drift the way machine learning teams learned to watch for it, because a non-deterministic system does not stay correct on its own.

On people, Kleinerman keeps AI out of the decision seat: “I would think of AI for people related matters as an assistant not a decision-maker.” And if something failed recently, check the date on the failure. “Technology is advancing very fast. If you tried something 4 months ago and it didn’t work there’s a chance that it may be working today.”

Cvent runs more than 6,000 AI agents inside a company of about 5,500 people. Roughly 1,300 of them are actually used.

Those two numbers came from Cvent’s own CIO, unprompted, and they describe the state of enterprise agent adoption more honestly than most of what gets published about it. On episode 913 of CXOTalk I sat down with Pradeep Mannakkara, Chief Information Officer at Cvent, and Ben Mayrides, the company’s Chief Information Security Officer, to ask who is governing all of it. Here is how I opened the show:

AI agents are making decisions in the enterprise, but who’s governing and managing them? Glean’s Work AI Institute developed the AWARE framework to tackle agent security, governance, and compliance.

Michael Krigsman, CXOTalk episode 913

A disclosure first. Glean underwrote this episode, and the AWARE framework we discuss is Glean’s. Cvent uses the Glean platform. Nothing here is an independent evaluation of Glean. Where the framework is described as working, that is Mannakkara and Mayrides recounting their own experience at their own company, their account rather than my validation. You can only weigh the detail if you know who underwrote the microphone.

The architectures are broken

I asked Mayrides what the core challenges look like from the CISO chair. He did not begin with a threat list.

understanding not only what these tools can do for us, but also understanding what they can do to us, I think is incredibly important.

Ben Mayrides, Chief Information Security Officer, Cvent

I asked Mannakkara to take that preposition apart from the CIO side, because it carries the whole security problem. Then Mayrides went underneath it.

the security architectures that we’re all used to are really focused on deterministic systems. And as we know, AI is all about non-determinism. So the architectures are broken.

Ben Mayrides, Chief Information Security Officer, Cvent

Take that literally. The controls most enterprises rely on were built for software that does the same thing every time it runs. Identity, access, logging, audit: all of it assumes predictability. An agent is not predictable by construction. The floor is the wrong floor.

In an agentic architecture, it’s completely different. It is about what that agent can do, what it can access, what decisions it’s making, what actions it’s delegating, what users it is impersonating.

Ben Mayrides, Chief Information Security Officer, Cvent

Read the last clause again. Agents impersonate users. Identity and access management assumes a fixed set of permissions and predictable behavior. It has no vocabulary for an actor that plans, delegates, and acts in the name of whoever invoked it.

a lot of agent reasoning is opaque, and it’s difficult to sort of reconstruct those elements as well, and we need that.

Ben Mayrides, Chief Information Security Officer, Cvent

Without observability there is no incident review, because there is no record of why the agent did what it did.

The published frameworks stop above the control layer

The AWARE framework covers intent, context, guardrails, risk scoring and blocking, and ecosystem observability. I asked how those principles map onto decisions about what to build and what to hold back. Mayrides answered by naming what the better-known frameworks do not do.

So when you think about the EU AI Act or the NIST Risk Management Framework, again, they’re great and they have their purpose, but they don’t go deep enough and they don’t actually tie into sort of the technical controls and the technical considerations that you need to build into an agent tech security architecture.

Ben Mayrides, Chief Information Security Officer, Cvent

Regulation tells you which outcomes to avoid. It does not tell you which control to configure on Tuesday morning. That gap is where a governance program either becomes real or becomes a slide.

A gut feeling is not an acceptable answer

Then Mayrides named a weakness in his own practice.

I’m just like, “It’s a gut thing.” And that’s not an acceptable answer.

Ben Mayrides, Chief Information Security Officer, Cvent

When the security answer is instinct, security has no defensible position and the business has no path forward. His replacement is a tiered, repeatable risk process with success criteria agreed before the work starts. Tiered, because uniform governance does not survive volume.

I think if you’re applying governance to the same level of governance and control to every agent or every AI system out there, your governance capabilities are just, they’re gonna crumble. They’re not gonna scale.

Ben Mayrides, Chief Information Security Officer, Cvent

And repeatable, because the decision does not stay decided.

So the go, no go decision is not a one and done, is my point.

Ben Mayrides, Chief Information Security Officer, Cvent

Mannakkara wants the same discipline expressed as plain language.

People need to partner together, but you also have to simplify the messaging. The business folks want to know yes, no. Can I do it? Can I not? Why not? How quickly?

Pradeep Mannakkara, Chief Information Officer, Cvent

Six thousand agents, and a CIO willing to say how many get used

I described the sprawl this way:

So you have agents, and then they have kids, and they move out to the suburbs, and then before you know it, you have agentic sprawl.

Michael Krigsman, CXOTalk episode 913

Mannakkara did not argue with the picture. He gave me the number.

Now even though we have 6,000, how many are actively used? It’s probably about 1,300, candidly speaking.

Pradeep Mannakkara, Chief Information Officer, Cvent

Roughly one agent in five is doing anything. He treats the rest as the price of building the culture: employees were encouraged to create agents before moderation and a task-level catalog existed. It was a deliberate decision, and he will say the number out loud.

Culture, he told me, is part of his role as CIO, which surprised me and I said so on air. The mechanism is not a memo.

But one of the decisions we made was we’re gonna give mandatory training to everybody in the company, a foundations and a literacy.

Pradeep Mannakkara, Chief Information Officer, Cvent

Actually, our CEO was in the first session. Just can’t tell you how important this was.

Pradeep Mannakkara, Chief Information Officer, Cvent

The CEO in the first session sends a signal no policy document can send. It tells 5,500 people this is not an IT initiative they can wait out.

The same story I have been telling for twenty years

My subject has not changed since I was writing about failed ERP programs for ZDNet. It is the distance between what technology promises and what organizations actually get. The vocabulary moves. The gap stays where it is.

Neither guest pretended the gap was closed. The CISO calls the architecture broken. The CIO says four out of five agents his company built are sitting unused. That is more useful than a case study in which everything worked.

What to do with this on Monday

Mayrides gave the answer himself, and it is deliberately small.

Inventory your agents. Instrument them for observability and identity. If you can only do, and focus on any part of the AWARE framework, it would be focus on those two things first, and then go from there.

Ben Mayrides, Chief Information Security Officer, Cvent

Count what you have. Make each agent identifiable and each action reconstructable. Everything else depends on those two facts existing first. And a deadline is forming.

So from a compliance perspective, I can see us seeing a control criterion in SOC 2 within, say, the next year and a half, two years. We need to be ready for that.

Ben Mayrides, Chief Information Security Officer, Cvent

That is his prediction, not a published rule. But if he is right, the inventory you do not have today is the audit finding you get in eighteen months.

Enterprises are buying AI agents faster than they can operate them. The demo lands, the pilot behaves, and then the thing meets the real business and stalls. Will Grannis is Chief Technology Officer of Google Cloud, and he joined me the day after Gemini Enterprise launched, with 60,000 users at Highmark Health already getting their internal answers through agents.

Here is how I opened the show.

Google is betting billions on AI agents and multimodal models. But what’s really behind the strategy? Today, on CXOTalk number 897, Will Grannis, Chief Technology Officer of Google Cloud, takes us inside what you need to know now. I’m your host, Michael Krigsman. Let’s get into it.

Michael Krigsman, CXOTalk episode 897

What came back was an operations story rather than a product story. The most useful line of the hour is one that no vendor keynote will ever put on a slide.

Agents are a third wave, and they act

I asked Grannis what is happening in the agent world before we talked about what Google is building.

So agents are kind of like, almost like this third wave of automation and intelligence in that they can take intent, and they can take it in a variety of ways. It can be typed, it can be spoken, and it can execute tasks on your behalf.

Will Grannis, Chief Technology Officer, Google Cloud

The load-bearing words sit at the end. A chatbot produces text and a person decides what to do with it. An agent decides and then does it. That one change moves your exposure from the quality of an answer to the consequence of an action.

Nothing works the first time

Grannis spent a long stretch on evals, the practice of scoring an agent continuously rather than testing it once before launch. I put it to him plainly.

It’s really interesting the way you describe these eval points, putting them through the entire process frequently enough. Would it be correct to say that you are making many, many, many, many small course corrections? Is that an accurate way to say it?

Michael Krigsman, CXOTalk episode 897

So to your point, Michael, it’s an iterative cycle. Nothing works the first time. Nothing works the first time.

Will Grannis, Chief Technology Officer, Google Cloud

He said it twice, which is how people talk when they have watched it happen. The budget consequence is immediate. A program funded as a build with a launch date is mispriced, because the launch is where the work starts. Grannis put the same idea a second way: “Training AI isn’t the end, it’s the beginning.” If your business case has no line for continuous evaluation after go-live, you have not funded the project. You have funded the demo.

Regulated industries are ahead, and that tells you what the work is

Whereas we tend to think of technology as, you know, coming to regulated industries second or a little bit later, in the world of agents, they’re seeing time to value a little faster because they already have all of those key business processes and data documented that could then feed the agent execution and the agent rules through a workflow.

Will Grannis, Chief Technology Officer, Google Cloud

Read that twice, because it inverts the usual assumption. Banks and hospitals are not ahead because they have better models. Everybody has the same models. They are ahead because a regulator forced them, years ago, to write down how decisions get made. An agent can only execute rules that exist in writing. Where the rule lives in a senior manager’s head, the agent has nothing to execute.

Grannis was direct about where advantage now sits: “And that’s where differentiation and that’s where competitive advantage lies. It doesn’t lie in accessing the same model everybody has access to.” Your documented process and your own data are the differentiator. That is unglamorous work, and it is the work.

The software will not guess what you meant

I described the fantasy that a lot of executives are quietly holding.

You’re the CTO of Google Cloud, and we want the AI to intuitively know and understand our implicit rules of engagement and our implicit culture, so we can just let the software do its thing and help us.

Michael Krigsman, CXOTalk episode 897

You know, it’s important that, to understand that if you ask software to do something, it will do it. If you don’t ask software to do something, it won’t do it.

Will Grannis, Chief Technology Officer, Google Cloud

Implicit culture is not an input. The unwritten norms, the exceptions everyone knows about, the customer you never escalate: none of it reaches the agent unless somebody types it. Grannis named the consequence directly. “Lack of data, lack of context to the agents and the models is the number one trap door.” Not model quality. Context.

An audience question landed on the problem nobody has solved

The sharpest question of the hour was not mine. Stephanie Tsatsos, an account executive at Workday, asked on LinkedIn whether agents are role-based or skills-based, which is to say whether one agent equals one job to be done or one agent equals one task. Grannis said the field sits at two ends of a spectrum today, with narrow single-task agents that a person builds for themselves at one end and larger multi-agent arrangements at the other. The middle is unsettled.

Then he said the part a keynote leaves out.

Debugging multi-agent workflows right now is very, very complicated.

Will Grannis, Chief Technology Officer, Google Cloud

That is the CTO of Google Cloud describing the category his own company sells into. If you are drawing an architecture where ten agents call each other across three systems, price the debugging before you set the launch date.

I have been writing this story for twenty years

I spent two decades writing about IT project failure for ZDNet and founded a consultancy, Asuret, to diagnose it. ERP, CRM, data warehouses, and now agents. The technology changes and the failure does not. Organizations buy a capability and skip the part where they have to describe their own work precisely enough for a machine to run it. The pilot succeeds because a strong team hand-carried it. Production fails because nobody wrote it down.

And I will also say that in the history of technology, we always underestimate how profound the changes are gonna be, and we overestimate how fast they’re going to happen.

Will Grannis, Chief Technology Officer, Google Cloud

That sentence is the whole discipline. On the path there, Grannis needed four words: “There are no shortcuts.”

What to do with this on Monday

Start with a process that is already documented, because that is the only place an agent has something to stand on. Fund the evaluation loop as a permanent operating cost rather than a project phase, and expect many small corrections rather than one clean cutover. Before you approve a multi-agent design, ask your team how they will debug it when a decision comes out wrong three hops in, and treat a vague answer as the risk it is. On governance, Grannis was compact: “Measure, measure, measure, and be very transparent.”

And do not wait for the strategy deck to be finished. At the end of the show I amplified his advice, and I will repeat it here: “I just have to amplify, Will, a comment that you made earlier, which is just get started. If you’re a small business, the more you can gain familiarity with the kind of services, for example, that Will was just describing, the better off you’re gonna be. And you’ll, then you’ll learn and you’ll know how to, how to take those tools and apply them to your specific business.”

Ninety-three cents of every dollar that enterprises spend on AI goes to technology and tooling. Seven cents goes to everything else: culture, change management, learning, and the redesign of how work actually gets done. That number comes from Deloitte’s Tech Trends research, now in its seventeenth year.

Bill Briggs is Deloitte’s Global Chief Technology Officer. He came on episode 912 of CXOTalk to make the argument a technologist is not supposed to make: companies should spend proportionally less on the technology. Here is how I opened the show:

Companies are caught in an AI investment trap, spending too much in the wrong places with too little benefit. It’s a mess.

Michael Krigsman, CXOTalk episode 912

Briggs did not argue with the framing.

Ninety-three cents of every AI dollar buys technology

I asked him for the headline finding from this year’s report.

The best stat coming out of it is that 93% of all AI spend is going toward the tech and the tooling, and only 7% is on everything else, which is the culture, the change, the learning, the how do we communicate the vision, how do we be thoughtful about trying to redesign or reimagine how we work and how we serve our markets and our customers.

Bill Briggs, Global Chief Technology Officer, Deloitte

Now put that next to the other number he gave me.

we’re at the place where 30% or less have agentic pilots that have reached production at scale.

Bill Briggs

Those are one finding seen from opposite ends. The seven percent is the work that carries a pilot into production, and nobody funds it. So I put the obvious objection to him.

You’re CTO, you’re a technologist, and so why do you have such a strong focus on this 7% that’s outside the core 93% of the technology?

Michael Krigsman

I’m a technologist who loves technology because of the potential it represents, but tech for its own sake has never been a winning hand.

Bill Briggs

AI on a bad process weaponizes inefficiency

And we’re in a moment in time where if you apply AI into an inefficient process, you apply AI on an overly complex process, business process, you’re gonna weaponize inefficiency and actually probably pay a lot.

Bill Briggs

Weaponize inefficiency. Automation laid over a broken process does not repair it. It runs the broken process faster, at scale, with an inference bill attached. Sangeet Paul Choudary made a version of this argument on episode 900. Briggs makes it from inside a Big Four firm.

AI is an incredibly powerful technology, and it’s a dangling modifier.

Bill Briggs

A dangling modifier attaches to nothing. He made the point with an image from a trade show floor.

As I walked around the 300,000 exhibitors, I didn’t see a single one with a sign that said, “Now with electricity.”

Bill Briggs

Nobody sells electricity as a feature. They sell what the electricity lets you do.

Nobody owns the inference bill

David Batz asked on LinkedIn how you predict AI financial risk quantitatively, with specific KPIs. Unit costs keep collapsing while the total enterprise bill climbs, so I asked whether organizations track inference as its own budget line.

And so the idea that you could have $100 a month bill one month and then a 500K dollar bill the next month won’t happen. But that requires some maturity and rigor that not every organization’s doing.

Bill Briggs

Maturity and rigor are the seven percent. No vendor sells them to you.

Trust collapses on the way down the org chart

The finding that stopped me had nothing to do with money.

So you can picture an org chart where the C-suite is the top of the pyramid, and the C-suite over all industries around the globe, the trust in AI from the C-suite was 70%, 7-0.

Bill Briggs

As you went down every layer of the org chart, it was like a logarithmic scale halving the trust every hop away until you got to the front line entry-level worker, and the trust was 6.7%. So from 70 to 6.7.

Bill Briggs

Seventy percent at the top. Six point seven percent at the bottom. The executives who approved the spending believe in the technology, and the people who have to use it do not. A tool the front line does not trust is a tool the front line routes around. Underneath the trust gap sits a governance gap.

You’ve raised the specter now of jobs. AI agents are multiplying inside enterprises faster than almost anybody can track, and many organizations do not have sufficient governance frameworks for a workforce that isn’t human.

Michael Krigsman

What’s a disciplinary action for an AI agent?

Bill Briggs

There is no settled answer, and every organization running agents at scale needs one.

Success theater has a tell

The sharpest question of the hour came from the audience. Chris Petersen wrote in on Twitter.

Are the organizations putting out the “we have tens of thousands of AI agents” headlines getting real value from these agents or just trying to game a system of bad metrics?

Chris Petersen, audience question, CXOTalk 912

But the prevalence of success theater is still real. And typically when we’re measuring volumes of use case or volumes of agents as the thing that’s the bar, it’s a tell.

Bill Briggs

A tell, not a metric. Agent count measures what you bought, not what changed. I said as much back to him.

I love that, because you’re really now talking about shifting the focus from the process, the process being the hand-waving, “We have thousands of agents”- shifting from that into, “Here’s what they’re doing. Here are the benefits. Here, they’re reducing inventory, they’re saving time, saving money,” what have you.

Michael Krigsman

They’re ingredients, they’re not recipes, and we need more chefs able to give us 3-star Michelin meal equivalents that we can then not talk about headlines of agents. We can talk about litany of real metrics that matter.

Bill Briggs

This is the old failure in new clothes

I spent two decades writing about IT project failure for ZDNet and founded Asuret to diagnose why enterprise projects go wrong. In close to 900 conversations, the shape has not moved. The technology usually works. The organization around it never gets built, because the budget line for building it does not exist.

The 93/7 split is that story with a new acronym. Money goes to the platform, because the platform has a vendor, a contract, and a demo. The work of changing how people operate has none of those, so it goes unfunded, and the pilot does not scale.

What to do with this on Monday

Work out your own ratio first. Split the AI money you committed this year into what buys technology and what buys change: training, work redesign, communication, and the people whose job is to make the new process real. If your number looks like 93 to 7, you know why your pilots stall, and you have Deloitte’s own research to show anyone who disagrees.

Then stop counting agents. Replace agent counts in your reporting with the outcomes Briggs named: inventory reduced, time saved, money saved. A program that cannot name its outcome is theater, and the volume number is the tell.

Take seriously the one skill Deloitte found matters most in its CIO program.

The single most effective, impactful session we do is not on multi-agent systems or ops or infrastructure. It’s on storytelling, of just how to structure and shape a vision and be able to help share that to influence and inspire.

Bill Briggs

That is a strange thing for a technology firm to publish, and it is consistent with the rest of the research. The seven percent is where the value lives. It has no vendor, and it is the difference between a demo and a business.

When an executive tells me they are doing AI, they almost always mean they are automating tasks. That instinct is the reason so many AI strategies produce activity and no advantage.

Sangeet Paul Choudary has advised more than 40 Fortune 500 CEOs and wrote the book Reshuffle. He joined me for episode 900 of CXOTalk, and his argument is that we have the category wrong. AI is not automation technology. It is coordination infrastructure. Here is how I framed it at the top of the show:

Most executives see AI as just automation. That’s wrong and explains why their AI strategies fail.

Michael Krigsman, CXOTalk episode 900

Automating tasks is not a strategy

I work with a lot of executives, and when they say that they’re implementing AI or they’re thinking about AI, they typically mean that they’re automating tasks, they’re automating workflows in their organization, they’re deploying technology towards automation. And my key argument in the book is that the real value of AI plays out not through automating tasks in the system, but through reimagining your business around the capabilities of AI. Because the real advantage or the real impact of any technology plays out when economic activity gets reorganized based on the new capabilities of the technology that is made available.

Sangeet Paul Choudary, founder, Platform Thinking Labs

Economic activity gets reorganized. That is the phrase to sit with. Electricity did not matter because it replaced the steam engine at the center of the factory. It mattered because it let you tear the factory up and lay it out around the work instead of around the driveshaft. The machine was the small part. The reorganization was the whole thing.

The two questions strategy is actually about

I asked him to take it down a level, because most of us instinctively think of technology as a way to make the current business better, faster, and cheaper. His answer named the trap precisely.

Even when executives today say, “We need an AI strategy,” when I ask them about it, very often what they mean is, “We want to figure out what should we do with AI.” And that sort of traps them within thinking about, “Here’s our business. How do we apply AI into it to get things cheaper, better, faster?” But that’s not really what strategy is about. Strategy is fundamentally about answering two questions. Where do we play? How do we win?

Sangeet Paul Choudary

Where do we play, and how do we win. Neither question is answered by a list of use cases. Both are answered by looking at how AI changes the playing field itself, who you now compete with, and where value gets created and captured once the capability is available to everyone.

The uncomfortable implication is that a company can execute its AI roadmap perfectly and still lose, because the roadmap treated the existing business as fixed.

Why he wrote past the hype

There’s a lot of hype about AI at the moment, and there’s some fairly consistent patterns that we see in how strategy and systems change when new technologies come in. And when we’re in the midst of hype, we sort of miss that whole understanding exactly how some of these things will play out. So, the reason I wrote Reshuffle was to kind of extract us away from how we think about AI in terms of the hype today, but really to think about how enduring change happens because of new technologies.

Sangeet Paul Choudary

This is why I keep booking guests like Choudary. The pattern of how technology reorganizes work is older than any of us, and it is far more predictive than whatever was announced last week.

What the coordination frame changes

If AI is coordination infrastructure rather than a task automator, three things follow.

Speed becomes a liability, not a virtue. Automate one function and it now runs at a different tempo than the functions around it. Handoffs break. The faster you automate a single department, the more coordination debt you take on everywhere else. Most companies are busy celebrating exactly that kind of local win.

Redesign beats retrofit. Bolting AI onto a process that was designed around human latency preserves every assumption that human latency created. Choudary’s argument is that the process itself is the artifact that has to change.

Strategy questions outrank use case questions. Before asking what to do with AI, ask what AI does to the field you compete on. If the answer is nothing, you have found a rare and valuable piece of information. If the answer is something, the use case list was never the point.

The connection to everything else I cover

I have spent two decades looking at why enterprise technology projects fail, and this episode names a failure mode that predates AI by a long way. The project succeeds. The technology works. The organization around it never changed, so nothing that matters moves. We used to call that a failed ERP rollout. Now we call it a stalled AI pilot.

The name is new. The disease is not.

Agentic coding has taken off. Agentic knowledge work has not. That gap is the most useful thing a CIO can understand about AI agents right now, and Aaron Levie described it better than anyone I have interviewed.

Levie is the co-founder and CEO of Box, which counts 68 percent of the Fortune 500 among its customers, which means he watches agents meet real enterprise data all day. He joined me on episode 921 of CXOTalk.

A tale of two cities

I asked him to separate the promise of agents for programming from the promise for everything else. His answer drew the line that most vendor demos are careful to blur.

We are still in the very early stages of what agentic work looks like in the enterprise and what the rollout looks like. We have an interesting dynamic, which is sort of a tale of 2 cities. We have AI kind of agentic coding, which has clearly taken off. And it’s within engineering teams, and everybody’s kind of figured out what the new practices are around the future of engineering. And then you kind of get into the real messy environments of knowledge work, where things are just quite a bit different. It’s a lot harder to deploy agents at scale. The agents don’t have always access to the right data. The users are less technical, so they don’t know how to sort of always steer them properly.

Aaron Levie, co-founder and CEO, Box

Read that again with a CIO’s ear. Engineering succeeded with agents because engineers are technical enough to steer a wrong answer back on course, and because the codebase is one clean, well governed body of data. Neither condition holds in finance, legal, HR, or operations. The demo works. The deployment does not.

Agents break your permission model

This is the part I would print out and hand to your security team.

In the enterprise, we’re constantly asking for permission to other systems and other resources and other data environments, and so an agent is only as good as the data that it has access to, but in the enterprise we have lots of systems that are either not well-maintained or the agent can’t get access to the right data, or maybe even worse, you have too much access to information. And we had sort of, you know, security through obscurity in your organizations, and now all of a sudden the agent is leaking data to the wrong people.

Aaron Levie

Security through obscurity is the honest name for how most enterprises actually protect information. The salary spreadsheet is technically readable by half the company, and it stays private because nobody thought to open it. An agent thinks to open it. Then it summarizes it helpfully for someone who should never have seen it.

Nothing was breached. No policy was violated. The permissions were always wrong, and the agent is simply the first employee diligent enough to use them.

The new role nobody has hired yet

Levie’s most concrete prediction was organizational, not technical. He expects enterprises to create an internal version of the forward deployed engineer.

I actually think there’s going to be a role for effectively an internal FDE and this is some kind of IT business AI automation engineer type role. I think often it’s going to live within the technology or IT organization, but be embedded within the actual line of business that it’s trying to bring automation to.

Aaron Levie

It follows directly from the two cities problem. If agents only work where a technical person can steer them, and knowledge workers are not technical, then you either wait for the models to get good enough to need no steering, or you put a technical person inside the business unit. One of those is a plan. The other is a hope.

Why I keep asking about the boring parts

I have spent twenty years writing about why enterprise technology projects fail, and the answer is almost never the technology. It is data nobody maintained, permissions nobody audited, and a business process nobody redesigned. Agents do not fix any of that. They expose all of it, faster and more publicly than the last wave did.

The live audience heard the same thing I did, and the questions that hour were about governance, cost, and access, not about capability. That tells you where the real anxiety sits.

Folks, you can ask your questions. When else will you have the chance to ask Aaron Levie, the CEO of Box, pretty much whatever you want?

Michael Krigsman, CXOTalk episode 921

The CIO checklist from this conversation

Audit permissions before you deploy a single agent, on the assumption that your current model is held together by obscurity rather than policy. Assume knowledge work will be harder than the coding results suggest, and do not let an engineering pilot set the expectation for the rest of the company. Budget for the human in the middle, whatever you end up calling the role, because agents in messy environments still need someone technical standing next to them.

And treat every agent demo you are shown as a demo run in the cleanest data environment its vendor could find.

Fewer than 100 companies have captured most of the value that artificial intelligence has created so far. Not most of the Fortune 500. Fewer than one hundred companies, anywhere.

That number comes from QuantumBlack, McKinsey’s AI practice, which has looked at thousands of transformations across the globe. I put it to Alexander Sukharevsky, who leads that practice, on episode 922 of CXOTalk. Here is how I opened the show:

Less than 100 companies so far have captured more than 2/3 of the value. They did it in two or less domains with one to three return ratio. Companies spend billions chasing AI. Few find value.

Michael Krigsman, CXOTalk episode 922

One to three. Every dollar in, roughly three dollars back. The winners are not doing more AI than everyone else. They are doing something structurally different, and Sukharevsky argues that the difference is knowable, repeatable, and boring.

The recipe is invariant

I asked him what actually has to be in place before AI produces value. His answer was not a list of technologies.

It’s really about an holistic approach, focusing on a few domains and trying to reinvent them, finding the right data to power this reinvention, aligning your architecture, finding the right talent, and upskilling the rest of the organization, changing your operating model, as well as focusing on economics.

Alexander Sukharevsky, Global Leader, QuantumBlack, AI by McKinsey

Then came the sentence that reframes the whole problem:

So first of all, it’s invariant. There is only one way to go in terms of different elements of the journey.

Alexander Sukharevsky

Invariant means you do not get to pick your favorites. Data, architecture, talent, operating model, economics. All of them, or the value does not show up. That is why so many programs produce impressive pilots and no P and L movement. They completed four of the six things and assumed the rest would follow.

I have been writing about this failure shape since my ZDNet years, and it has not changed. The technology is rarely the thing that breaks. The organization around it is.

Two domains, not twenty

The second finding is about focus, and it is the opposite of what most enterprise AI portfolios look like.

The companies that transformed themselves actually focused on 2 or less domains. So it’s really about focus. But then the important part is not just infusing technology within the current business model, but thinking about complete reinvention.

Alexander Sukharevsky

Most companies I talk to are running dozens of AI initiatives across every function, each one sponsored by a different executive, each one measured on its own terms. That portfolio looks like progress on a slide. It is the statistical reason nothing lands. Spreading effort across twenty domains guarantees you will not get all six elements right in any of them.

The CEO is the chief transformation officer

This is the part that will be unpopular in IT leadership circles, and Sukharevsky did not soften it.

I truly believe that the transformation should be driven by the CEO and the board. Because if it is driven just by the CTO or chief digital officer, what we see in most of the cases, it ends up with a lot of amazing pilots that frankly show impact. It also ends up with a lot of interesting technology solutions and very advanced platforms, but it does not bring to the full change of the organization and the rewiring the organization. And therefore, the person who should be chief transformation officer is the CEO of the organization.

Alexander Sukharevsky

He was careful to add that the CIO and CTO are not spectators. Their job is to guide the CEO, because the CEO is the only person in the building who owns the strategy, the investor expectations, and the direction of travel at the same time. But when the transformation reports to technology, technology is what you get. You do not get a rewired company.

Human in the loop, with teeth

The live audience pushed the conversation somewhere better than my questions did. Monique Zytnik wrote in on LinkedIn with one line that landed harder than anything else in the hour:

Agents don’t have consequences if they don’t perform.

Monique Zytnik, audience question, CXOTalk 922

Sukharevsky’s answer was that the accountability never moves. You redesign the process, you monitor the execution, and you make sure the team is accountable for the outcome whether or not agents did the work. Which led me to say what I think is the practical takeaway of the whole episode:

So human in the loop then is not simply overseeing the machine and rubber stamping, but actually performing an ongoing critical evaluation of what the agent is doing based on your expertise and, very importantly, your judgment.

Michael Krigsman

And then, because the phrase has been drained of meaning by overuse:

It’s not the agent that is accountable, but it’s the human in the loop, because so often we hear human in the loop as this kind of jargony buzzword, but you’re putting the accountability on that human in the loop so there’s actually some teeth there.

Michael Krigsman

If your governance document says human in the loop and nobody’s name is attached to the outcome, you do not have human in the loop. You have a diagram.

What to do with this on Monday

Three things, drawn directly from what the data says about the companies that made it work.

Cut your AI portfolio down to the one or two domains that actually create most of your value, and reinvent those rather than decorating them. Move sponsorship of the program from the technology organization to the CEO and the board, and let the CIO and CTO advise rather than own. Then put a human name against every outcome an agent touches, so that when the agent is wrong, a person is answerable.

None of this is a technology decision. That is exactly the point, and it is why fewer than a hundred companies have managed it.