On 1 August 2012, a US trading firm called Knight Capital opened for business with some badly deployed new code. In the first 45 minutes of trading, its system fired more than 4 million orders into the market while trying to fill just 212 customer orders. The loss was more than $460 million.
No rogue AI. No hackers. One botched deployment. John Willis, who has spent five decades inside technology transformations, says that is what AI's real disaster will look like, only faster and at far bigger scale.
🎧 Short on time? Hit play: Watch the full episode on YouTube (47 min)
John co-wrote The DevOps Handbook, wrote Rebels of Reason, a people-first history of AI, and advises banks, insurers and other high-consequence organisations on how to adopt it. Yoshi and I asked him what actually keeps him up at night.
I worry way less about this evil monster model just waking up and deciding to destroy the humankind on Wednesday.
His worry is the boring one: an AI agent set up slightly wrong, copied across thousands of tasks, triggering a chain of failures nobody can unpick. Knight Capital, at machine speed.
How this lands on Monday morning
Whether you run projects, sit in a cost team, work for a contractor or are rolling AI out across a department, five checks to run this week:
Name the value before the tool. Write down what a new AI tool should improve (faster tender returns, fewer reworked RFIs) and measure that. Licences issued and prompts sent don't count.
Aim for ten days, not two. If a process takes three months today, target ten days. It's still a huge leap, and it leaves room to spot what's going wrong.
Decide what never goes into a public model. Client spreadsheets, cost data and anything commercially sensitive. Write it down and tell people.
Build the stop button first. Who can switch an agent off, and what limits what it can touch? The SEC found Knight's own systems sent 97 warning emails before the market opened. Nobody acted on them.
Keep a register of what people build. Every home-made tool and vibe-coded app is something someone will have to own, fix and secure later.
Go fast to learn, not just to ship
John isn't anti-speed. DevOps was all about delivering faster. But one of his workshop students told him everyone at work was telling him to speed up, while John was telling him to slow down. His answer:
So all I'm asking you to try to propose, instead of going from three months to two days, go three months to ten days.
Ten days still gives you a fighting chance to define the value and the risks. The point of speed is the feedback loop. In his words, "The faster you can do something, the faster you can learn." The organisations that treat early AI setbacks as learning, not failure, are the ones he expects to win.
I'm guilty of the opposite myself. We've all sat up until three in the morning vibe coding something, then looked at it the next day and wondered what we'd done.
Measure value, not tokens
John has little time for token leaderboards, or for the pitch that giving 40,000 staff an AI agent makes everyone ten times more productive. When a big tech boss suggested well-paid engineers should be spending heavily on tokens, his reaction was blunt:
No, they should be creating half a million in value. Like the question is value. It isn't delivery.
Or more simply: "just giving people toys doesn't equal value."
The bigger risk is your reputation
I put it to John that AI doesn't just create technical debt. It exposes the decades of it we already have. He agreed, then went further.
The technical [debt] tsunami that you're gonna have in three to four years is the simple problem. The decimation of your brand is the more scarier problem.
He gave a real example. In one of his "ideation hackathons", where nobody writes code, a group of executive assistants to senior IT leaders at a large corporation casually planned to load a spreadsheet of sensitive data into a public AI model. "It's happening like you couldn't believe," he said. Now picture that habit across every firm in a project's supply chain.
Also in the episode
Why the argument over whether today's AI is "intelligent" is, in John's words, "a waste of oxygen". What Rebels of Reason taught him about the people behind AI, and why he doesn't think large language models are the last word (he's watching world models from Fei-Fei Li and Yann LeCun). Using AI to build Wardley maps for strategy. The Jevons paradox that has AI enthusiasts working longer hours than ever. And his career advice, which I loved: "If somebody doesn't find you interesting, move on."
Why it matters: Most AI failures on projects won't look like science fiction. They'll look like an unchecked setting, an over-trusted tool or a spreadsheet in the wrong place, repeated at scale. The teams that get the most from AI will go fast to learn, measure value rather than activity, and build the stop button before they need it.
What's one AI tool or agent running in your team that nobody formally signed off? Hit reply and tell me.
Links and Stuff
John Willis on LinkedIn: linkedin.com/in/johnwillisatlanta
Rebels of Reason: The Long Road from Aristotle to ChatGPT: amazon.co.uk
Listen on
The voice-to-text app that built its own speech model
Most dictation apps are a wrapper around someone else's speech model. Wispr Flow trained its own.
It's called Canto, and it was built for the way people actually talk to AI: thinking out loud, changing direction mid-sentence, half-whispering in an open office.
Wispr tested it on 10 hours of real dictations from more than 2,300 speakers. Canto had the lowest word error rate of every model tested, including models from Google, OpenAI, AssemblyAI and Deepgram. That means your prompts to ChatGPT, Claude and Cursor come through the way you said them.
Talk naturally in any app and Wispr Flow types clean, formatted text, 4x faster than typing.
Not subscribed yet? Subscribe to the Project Flux newsletter for the week's AI and tech news for project delivery, every week.
Project Flux is independently owned, funded, and operated by its founder. Any external references or features are for community benefit only and do not imply endorsement, control, or ownership.




