The Route That Outsourced Judgment to the Tool
How an AI recommendation became a decision without anyone deciding (Route Rebuilder | Episode 5)
👋 Welcome to this week’s edition of Empathy Engine. Every Wednesday, I publish a new article for paid subscribers first, then unlock the full piece for everyone late Thursday morning. Each week, I turn product leadership friction into practical tools, sharper language, and more defensible decisions.
Research Binder: the receipts (citations + source notes) are compiled in a PDF at the bottom of this post.
If You’re Skimming
An AI tool can recommend, suggest a fix, score its own confidence, even auto-merge. A recommendation is still not a decision.
The broken route here: human-in-the-loop on paper becomes human-near-the-loop in practice, then absent under pressure.
The fix is not less AI. It is explicit routing: a named judgment owner, real authority, an escalation trigger, and an audit trail.
Run the Five-AI-Decision Field Test at the bottom to inspect your own workflows before the next high-confidence suggestion ships.
Who Owns the Judgment When the Tool Says Yes?
Before this article had a clean thesis, it gave me a live demonstration of the problem. I had the first draft reviewed by several AI systems, and one came back with exactly the feedback a tired writer wants to see: polished, confident, encouraging, and close enough to my own language that it felt safe. It told me the piece was strong. It said the series callbacks were present and praised the recurring structure. The review looked clean. The signal looked strong. The grade was flattering enough to make my inner editor briefly consider taking the afternoon off and calling that discernment.
Then I checked it against the actual draft. Some of the feedback was genuinely useful. But part of the review praised structural elements that were not there. It had turned plausibility into confidence, and confidence into something that almost felt like approval. The failure would not have been the AI making a wrong call. The failure would have been me letting a confident recommendation become editorial judgment without owning the yes myself. That is the route failure this piece is about.
I caught it only because I went and checked. The same trap waits in an engineer’s review queue, where the confident recommendation is a code change, the deadline is real, and the merge button does not care whether anyone actually decided.
Episode 1 Parts A and B:
This series has been tracing one failure through different rooms. In Episode 1, a warning died in a busy channel because receiving is not routing. In Episode 2, a handoff became a gap because a closed ticket is not a completed handoff. In Episode 3, a review gate lived on the diagram but stopped operating in the work. In Episode 4, a green dashboard counted closure while rework hid in the shadow. Episode 5 follows the same pattern one layer deeper, into the moment an AI recommendation becomes the decision because no one explicitly owned the judgment point.
Episode 2:
Episode 3:
Episode 4:
This is not an anti-AI argument. AI-assisted tools can help teams move faster, spot patterns, and reduce routine friction. The problem begins when speed quietly absorbs judgment. A dangerous workflow is not necessarily the one with no human anywhere near it. Sometimes it has a human nearby, a confidence score on the screen, a review step in the policy, and a merge that still behaves as if the decision was already made. The first route to inspect is not the code path. It is the decision path.
When the Recommendation Became the Route
When did the recommendation become a decision?
That is the question most teams do not ask early enough. They ask whether the tool was accurate, whether the test suite passed, whether the code looked clean. The deeper question is whether the recommendation passed through a real decision point before action happened. A recommendation is an input. A decision is an owned commitment.
Picture a composite team, drawn from patterns I have seen across many organizations rather than any single client. Their AI code review tool flags an issue, suggests a fix, attaches a confidence score, and routes the change into the review path. A reviewer recognizes the shape of it and approves quickly. At first nothing seems broken. The process is faster, the pull request looks cleaner, the queue moves. Then a vulnerability surfaces, or a logic assumption fails under load, and the postmortem can reconstruct the recommendation, the score, the timestamp, and the merge. What it cannot reconstruct is the human judgment that owned the yes.
A decision needs an accountable route: someone who could approve, someone who could reject, someone who could escalate, and the evidence that belonged in that choice. Without that route, the system has motion but not ownership. The practical question is not whether AI belongs in code review. It is whether the workflow still knows when an AI recommendation is only a recommendation.
The Score That Sounded Like a Yes
What made the score feel like authority?
In Episode 1, we named the Ack Trap: acknowledgment creating the appearance of control. At a confidence score, the same trap runs at machine speed. A high number feels like proof. It starts to sound like “already checked,” especially when the reviewer is busy, the change looks clean, and the tool has been useful before.
Research on automation bias gives this failure mode a name. One review of thirty-five quantitative studies on human-AI collaboration, together with two major systematic reviews of automation bias and verification complexity, points to the same pattern: people can over-rely on automated recommendations, particularly when the system appears reliable, when verification is complex, and when workload is high. Those three amplifiers, workload, task complexity, and perceived reliability, describe a normal code review almost exactly. That does not mean humans always over-trust AI. Trust can be too high, too low, or badly calibrated. Code review is simply a near-perfect environment for the calibration to drift.
I have had green signals quiet my better judgment, and not only from AI. Sometimes the signal was a test suite. Sometimes it was a work item moving across a board in Azure DevOps with just enough completion language to make everyone feel better than the evidence deserved. The part of me that hesitated usually noticed something small: a teammate who answered too quickly, a dependency mentioned once and then gone from the conversation, a “done” column that had become more important than the conditions that made the work done. The signal won because the signal was easier to defend than the instinct. Nobody wants to be the person slowing the room because a status update feels off. A green check has institutional charisma. It looks objective, calm, and finished, which is a very convenient costume for unfinished judgment. The problem was not that I made the wrong call. It was that I let the visible signal stand in for the uncomfortable question: do we actually know enough to say yes?
The trap is not the score itself. The trap is the workflow treating the score as if it carries authority. A confidence score can prioritize attention and suggest where to inspect. It cannot own the yes. Confidence is not authority, and a high score should focus judgment, not replace it.
The Pull Request Looked Fine
What can polished code hide?
The risky AI suggestion may not look sloppy. That is the uncomfortable part. It may be clean, idiomatic, neatly formatted, and surrounded by green checks. When teams imagine bad AI code, they imagine something obvious: broken syntax, strange names, a function that looks like it wandered out of a forum answer from 2008 wearing someone else’s hoodie. Real risk is quieter.
Studies of AI-generated code make the boundary clear. One multi-language study of two hundred coding tasks across four languages and five models found nontrivial security and quality issues that vary by language, model, and task. Secure-generation research reduced observed vulnerabilities by more than sixty-five percent in tested scenarios, which is encouraging and also a reminder that the baseline risk was real. None of this proves AI code is broadly worse than human code. The more precise claim is that AI-generated code can carry a distinct, nontrivial risk profile, and some of that risk is visible only to a reviewer who understands the system context.
A pull request can pass surface checks and still miss the point. Tests confirm the behavior the suite already imagined. Linters confirm the code looks orderly. Static checks catch some categories and miss others. Clean style is not system understanding. The fluency is the trap: code in the house style, with a reasonable-sounding explanation, can read as comprehension when it is only polish. Three things often hide behind that polish: a security gap, a logic assumption, and an outdated pattern. The fix is not paranoia. It is context-aware review, routed by risk, so that a payment path or an authentication change never travels the same road as a typo fix.
The Human Was Present, but the Judgment Was Optional
Was the human in the route, or just nearby?
Human-in-the-loop sounds reassuring until you ask what the human can actually do. A human who can see the workflow is not the same as a human who must own the decision before action. Awareness is not authority.
No one builds this gap on purpose. The same practical lens from earlier episodes applies, and it is a lens, not a validated taxonomy: the Designers set the confidence thresholds and the auto-merge rules, the Inspectors assume the tool covers their scope, and the Enforcers meet the gap after deployment, as incident responders, customers, or regulators. Each role acts reasonably. The route was simply never built to force explicit judgment where the stakes are high.
I have mistaken presence for oversight. In Agile coaching and transformation work, I have been in plenty of rooms where a decision moved forward because the right people were technically present. The product person was there. The delivery lead was there. The board was open. The status was visible enough for everyone to feel informed. I had visibility. I had awareness. I had a seat in the room, which is a dangerously flattering substitute for authority.
What I did not always have was the power to stop the route. I could ask whether we were sure this was ready, or point out that a dependency was fuzzy or a risk acknowledged but not owned. That is not the same as owning the decision. The person who owns the stop has explicit authority to say, this does not move yet. The person near the loop can raise a concern and hope the system respects it. This is where organizations fool themselves. They confuse invited with empowered, putting people near decisions and counting proximity as governance. The question I now carry into AI-assisted workflows is whether the human has timing, authority, and ownership before the action happens.
The evidence here is suggestive rather than settled, especially outside IT-support and similar high-volume workflows. A review of ninety-eight studies of human-AI collaboration in IT support found that automation often shifts humans toward exception-handling roles, present but no longer in the decision. That is the quiet drift this episode is about. A meaningful loop has a named decision point, authority to stop, and a rationale that can be reconstructed later. Human oversight has to be a decision point, not a decoration. Anything less is human-near-the-loop theater.
The Decision Receipt Had a Blank Line
Who owned the yes before action happened?
Every postmortem reaches a moment where the room wants a receipt. A healthy workflow can answer without hunting for a culprit. An unhealthy one has artifacts of action but not artifacts of judgment. That is the Unowned Yes: the moment an AI output is treated as sufficient to proceed without any named human owning, authorizing, and standing behind the call. The yes happened. Ownership did not.
I have been in rooms where the action was perfectly traceable and the decision was nowhere to be found. The ticket moved. The status changed. The update was recorded. The work passed through enough systems that everyone could reconstruct the motion with forensic confidence. What the room could not reconstruct was the yes. That silence has a particular texture. People stop looking at the dashboard and start looking at each other. Someone scrolls the comments again, as if the missing owner might appear between two timestamps like a rabbit in a hat. Someone says, I thought that was covered in the review, and nobody wants to ask the next question, because the next question makes the room smaller. Who owned the decision? Not who touched the ticket. Not who attended the meeting. Who owned the yes? More often than not, the answer was worse than dramatic. The answer was blank.
The route had captured activity, not judgment. That is how unowned decisions survive in professional systems. They hide inside motion. They borrow legitimacy from meetings, tools, status updates, and dashboards, so everyone assumes the decision must have been made somewhere. A traceable action says, here is what moved. An owned decision says, here is who accepted the risk before it moved. Those are not the same thing.
This is also why audit trails are easy to misunderstand. An audit trail does not make a decision good. It makes a decision reconstruct-able. Logs are not morality. Logs are memory, and without memory teams learn slowly.
The cost of an Unowned Yes is local, and easiest to see when you trace it backward. Bystander Burn is the time several people spend half-reviewing an AI output nobody accepted. Investigation Tax is the duplicated effort to reconstruct why a high-confidence change failed. Compounding Rework is the cleanup when downstream work was built on a foundation no human ever confirmed. None of these arrive as one dramatic failure. They arrive as friction, context switching, and a sprint that always runs optimistic. You do not need a global benchmark. Multiplying the hours by your own loaded rate is enough to make the cost local and hard to ignore. Named ownership is not scapegoating. Ownership means the decision was real. Blame means the system wants a sacrifice. Those are different games.
Rebuilding the Route Without Turning It into Molasses
Which AI-assisted decisions can move, which need review, and which need escalation?
The answer cannot be “review everything by hand.” That sounds responsible until the workflow collapses under its own ceremony and people route around it. The answer also cannot be “let AI handle anything above a confidence threshold,” because a threshold is a signal, not a governance model. A better route is tiered judgment, and it is the route Maya Johnson rebuilt after her team’s incident. Maya is a composite, an engineering lead assembled from real patterns rather than one person, but the rebuild she ran is ordinary and repeatable.
Tier 1 is low-risk, reversible, routine work. The AI may execute with logging, and the team samples outcomes to check for drift. The route stays fast because the action can be unwound. Tier 2 is medium-risk work: customer-facing changes, cross-team impact, novel patterns. It needs a named human owner before execution and a captured rationale. A simple trigger helps here. If a change touches authentication, payment logic, or an external interface, it defaults to at least Tier 2, and if you are arguing whether something is Tier 1 or Tier 3, it probably belongs in Tier 2. Tier 3 is high-risk work: security, privacy, payments, irreversible changes, high blast radius. If customer impact is not reversible, it belongs here. Two-person verification may be appropriate, but as a risk-management choice, not a magic charm.
That tiering needs a record to land, and the record can be light. A minimum decision receipt is enough: who owned the yes, the risk tier, a one-line rationale, the escalation trigger, the timestamp, and which tool or model version produced the recommendation. It is a lightweight memory, not surveillance and not a compliance artifact. NIST, ISO, and EU-style AI governance frameworks support risk-based oversight, accountability, and traceability, but they do not mandate this exact three-tier design. This is a practical operating pattern inspired by those principles, not a compliance guarantee.
The smallest version of this is not a system at all. I have used it in Agile rooms for years: a confidence check. Not a governance platform, not a ceremony that requires laminated cards, three consultants, and a naming convention no one can remember by Thursday. Fist of Five is the classic. Five fingers means I am fully in. Three means I can support this but I have concerns. One or two means something important is unresolved. The magic is not the hand signal. The magic is that the room no longer depends on one person being brave enough to interrupt the momentum.
That matters because hesitation usually dies in professional rooms. People do not want to be accused of overthinking, blocking, or bike-shedding, or any of the other little workplace crimes we invent when speed wants to outrank judgment. A lightweight check changes the shape of the concern. Instead of requiring someone to object, the route asks everyone where they are, and then asks who owns the next yes. I have watched that move change the temperature of a room. Someone who would never make a speech holds up three fingers, someone asks what would make it a four, and the real issue appears: a fuzzy dependency, thin acceptance criteria, a risk nobody owns. The decision moves out of personal courage and into the route.
Fist of Five is the human-room version of the same instinct the tiers encode. The Tiered AI Judgment Protocol is that idea translated into AI-assisted work: make confidence and ownership visible before the route moves. Not more bravery. A better route for judgment. Different risks deserve different routes. Low-risk work should not wait behind a fake committee, and high-risk work should not slide through because a score looked impressive.
The Smallest Useful Friction
What should happen before the recommendation becomes action?
The cheapest route repair is often a pause. Not a meeting, not a governance epic, not a thirty-tab spreadsheet maintained by someone whose job title contains the word “transformation.” A pause. Cognitive forcing functions are structured interventions that make people engage their own judgment before accepting automated advice. Evidence from human-AI decision research suggests they can reduce over-reliance in some settings. The effect is not universal, and the tradeoff is real, because added friction slows people down. That is exactly why the pause has to be small.
Before accepting an AI recommendation, ask three questions. What context might the AI have missed? What would change my decision? Who owns the downstream risk? Those questions do not guarantee correctness. They do something more modest and more useful: they make judgment visible before the yes. A pause is not delay. It is decision hygiene. A question after action is a postmortem prompt. A question before action is a control. Pause, check, own the yes. Risk-tiering decides where the pause belongs, so a typo fix never carries the same ritual as an authentication change.
Audit the Next Five Decisions
How can you find your first route repair this week?
Do not begin by auditing your entire AI strategy. That is how useful work gets trapped in an enterprise fog machine, where someone schedules a steering committee, someone proposes a taxonomy, and six weeks later the same broken route is still moving recommendations into action. Start smaller. Inspect the route, not the people.
Look at the next five AI-assisted decisions your team makes. Code review is the cleanest place to start, but triage calls, prioritization, and summaries turned into actions all count. For each one, ask five questions. Who owned the yes? What risk tier was this? Was the human step required or optional? Was the rationale logged before action? What would have triggered escalation? This is not a statistically representative audit. Five decisions will not diagnose an organization, and the number five is practical, not scientific. That is the point. Five is small enough to do this week, and enough to expose whether the same missing field keeps appearing. If you cannot name who owned the yes, you found your first route repair.
The Route Still Needs to Know Who Owns the Yes
The future of software work will include more AI, not less. Teams will use it to generate code, review code, triage alerts, and recommend changes. Pretending otherwise is not prudence. It is nostalgia wearing a security badge. The serious question is not whether AI belongs in the workflow. It is whether the workflow keeps judgment visible as AI gets faster. Speed without ownership is not maturity.
A route can be fast and accountable. It can use AI without surrendering judgment, letting low-risk work move while requiring stronger human ownership where the stakes rise. That is the rebuild. The recommendation did not become risky when the tool produced it. The workflow became risky when the recommendation turned into action without a named owner, a risk tier, a rationale, an escalation trigger, and an audit trail.
This is also the conviction behind my book, Collaborate Better. Better collaboration is not softer language or more cheerful meetings. It is the discipline of making work, ownership, decisions, and judgment visible enough for people to act on them together, especially when the tools get faster. You can learn more at CollaborateBetter.us.
The next time an AI-assisted workflow looks efficient, do not start by asking whether the tool is smart enough. Ask who owns the yes before the tool acts. The AI did not decide. The workflow let it.
Next in Route Rebuilder: Part 6, The Route That Survived Pressure. The final episode moves from failure to resilience, and looks at the few routes that hold because someone built them to make ownership, truth, and escalation easier when the stakes rise.
P.S. Pull one recent AI-assisted change or decision. Ask who owned the judgment, whether they exercised it, and what would have escalated if they had not. Reply and tell me what you found. I read every one.
Regards,
Mark 👋
Evidence note
The research families behind this article are automation bias and trust calibration in human-AI collaboration, the security and quality of AI-generated code, risk-based AI governance (NIST, ISO, and EU-style frameworks), accountability and traceability, and cognitive forcing functions. Claims are kept conditional on purpose. AI-generated code is treated as carrying a distinct risk profile, not as broadly worse than human code. The tiered protocol is a practical pattern inspired by governance principles, not a mandated standard or a compliance guarantee. All costs in this article are local; compute your own.
Previous:
What Happens When Velocity Looks Healthy but the Work Does Not Stay Done?
👋 Welcome to this week’s edition of Empathy Engine. Every Wednesday, I publish a new article for paid subscribers first, then unlock the full piece for everyone late Thursday morning. Each week, I turn product leadership friction into practical tools, sharper language, and more defensible decisions.
More Content to Discover:
Half Your Team Is Using AI. You Don't Know Which Half.
Intro If you read the piece we published last week on Leadership in Change, you saw the structural diagnosis: mandate pressure, decision collapse, intake collapse. Three failures that sit underneath almost every AI pilot that goes sideways. That piece mapped the architecture. What it did not cover is what happens on the human side of the same problem, wh…
The dangerous AI workflow is not always the one where humans are removed.
Often, the riskier pattern is sneakier: humans are present, dashboards are green, confidence scores look strong, and the workflow still moves before anyone owns the yes.
That is the gap I am naming here, because “human-in-the-loop” only matters when the human has timing, authority, and ownership before the tool acts.