Day: May 25, 2026

Where AI Belongs in a Business Application and Where It Doesn’t

Most failed AI deployments share one root cause. AI was embedded inside an operational system that needed reliability, when it should have been placed as an intelligence layer above a system that needed insight. That single misplacement explains more of the AI failure stories we are watching across enterprise programmes than any model choice, vendor decision, or training-data debate. It is the architectural mistake that quietly costs more than any of the visible ones. Operations excellence has never been about what is new. It has been about what survives – what runs, day after day, with consistency, traceability, and bounded accountability. Generations of engineering, manufacturing, and service organizations have built that discipline brick by brick. AI’s arrival does not replace it. It tests it. The question every CIO and operations leader now faces is no longer “where can we add AI?” It is “where should AI sit – and where, deliberately, should it not?” This piece offers a method for answering that question: three zones, a choosing principle, five assignment criteria, and an honest view of where the trajectory leads. The Clash – Operations and AI have different requirements Operations excellence has four requirements that AI does not naturally provide. Determinism: An operational system must produce the same output from the same input. That is not a feature; it is the definition of reliability. AI is probabilistic by design. The same prompt can return different completions; the same data, run through the same model under different conditions, can yield different outputs. For knowledge work, this variability is acceptable, even useful. For transaction processing, it is unacceptable. Auditability: Every operational decision must be traceable. Who decided, on what basis, with what data, and at what time. AI reasoning, especially in foundation models can be opaque, or post-rationalized after the fact. “Chain-of-thought” outputs are summaries of behaviour, not records of it. Regulators, auditors, and courts have not yet developed a settled standard for treating model reasoning as evidence. Until they do, AI-led decisions live with an auditability gap that operational systems cannot tolerate. Bounded accountability: Operational responsibility must sit with a named human or system. When AI “acts on behalf of” a process, accountability diffuses unless the architecture explicitly prevents that diffusion. Who is responsible when the agent calls the wrong API? When the model retrieves stale data and a customer commitment is made based on it? Operations excellence has clear answers to these questions before AI enters the picture. AI introduces ambiguity that must be designed away. Continuous, reversible improvement; Operations get better through incremental, measured, reversible change – the discipline often called kaizen. AI updates rarely behave this way. A model upgrade can be a step-change. A prompt revision ripples through every downstream consumer. The relationship between the change made and the behaviour that emerges is non-linear, sometimes by design. Figure 1 · Four operations-excellence requirements vs four AI defaults. The honest reading is not that AI is unsafe for operations. It is that AI was never designed to satisfy operations-excellence requirements directly. When AI is bolted onto an operational system as if it were just another feature, those four mismatches surface as production incidents. When AI is integrated under operations excellence – bounded, zoned, governed at the architecture level, it amplifies the discipline already there. The integration model determines the outcome, not the model itself. The three zones: AI-driven, AI-assisted, AI-free If the mismatches are predictable, the response can be deliberate. Three zones, three different relationships between AI and the operational system around it. AI-driven: The system is fundamentally an intelligence layer. Its job is to find patterns, surface anomalies, suggest actions, and generate insights from data that would be inaccessible to a human at the speed required. Without AI, there is no system. The work AI does here is the work, not the assistant. Where AI does not cross: the boundary into systems of record. The maintenance work order, the parts release, the invoice, the regulated filing – those return to operational systems that own them. AI surfaces the need. Operations execute the response. AI-assisted: The system has a deterministic core, the system of record that runs the operation. AI sits adjacent to it, accelerating retrieval, drafting responses, surfacing context. The system would run without AI; AI makes it faster, more usable, or more accessible to non-specialists. Where AI does not cross: the diagnostic decision, the customer commitment, the contract execution, the safety-critical sign-off. The human stays the principal. AI is the apprentice. AI-free: The system is the operation. Transaction processing, financial settlement, compliance approvals, safety-critical outputs. AI may inform the operator with insight, context, or warning. AI does not execute the operation itself. Why this zone exists: not because AI is technically incapable, but because the cost of probabilistic behaviour in these systems is asymmetric and severe. Figure 2 · Three zones, three relationships between AI and the operational system around it. The choosing principle The single test that determines zone assignment: Does this decision require determinism, or can it benefit from intelligence? If the answer is determinism, the system stays AI-free, or at most AI-assisted. If the answer is intelligence and the surrounding governance is ready to support probabilistic behaviour with confidence, AI-driven is on the table. A useful corollary, with deep roots in Japanese engineering culture under the name 枯れた技術の水平思考 (Gunpei Yokoi’s principle of lateral thinking with mature technology, developed at Nintendo): operational systems that have been refined over years should not be rewritten just because AI is available. If a system meets its requirements reliably, the burden of proof falls on the AI-augmented version, not on the working incumbent. Don’t disturb what is working. Five criteria for assigning any application to a zone The principle becomes practical through five questions. Run any application through them. The zone reveals itself. Figure 3 · The five-question test for zone assignment. The three zones in production These zones are not a thought experiment. They are how the most carefully designed AI programmes in Japan automotive are