Pattern Drop: The “Never Let AI Do This” List
Because some moments really do need a human to make the call
I’ve been thinking about how AI behaves when it tries to be helpful. Most of the time, the mistakes aren’t dramatic. They’re small things that make you pause and say, “I don’t think it should be deciding that on its own.” It reminds me of a new hire who means well but occasionally steps into territory where you’d prefer they ask first. The difference is that a person learns from the correction. AI repeats the same move unless you build a place for it to stop and ask.
So instead of treating these as hard prohibitions, I’ve started thinking of them as moments where AI should pause and hand the decision back to a human. Almost like a little tap on the shoulder saying, “Here are a few options. Which one do you want?” And the important part is that the workflow shouldn’t forget what happened. The pause, the options, the human’s choice, and the final action should all be stored. That history becomes training material. It’s how the system gets better instead of guessing.
And this is where observability comes in. I’m starting to think that observability in AI workflows might be one of the most underrated pieces of the whole puzzle. If you can’t see what the model tried, what it suggested, what the human picked, and how the workflow moved forward, you’re basically flying blind. You end up with a system that behaves in ways you can’t explain. I suspect that’s where most of the fear comes from. People aren’t scared of AI. They’re scared of AI they can’t observe.
Here are some of the situations where that pause matters.
When AI tries to apologize for something the company didn’t do
It apologizes for anything. Weather. Delays that never happened. Feelings it invented. I suspect it thinks apologies are a safe move, but they’re usually the worst move. This is a moment where AI should stop and ask the human whether an apology is even appropriate, and if so, what kind. And whatever the human decides should be saved so the model learns the difference between a real apology and a made‑up one. Observability helps you see how often this happens and whether the model is improving.
When AI tries to offer compensation
It hands out refunds like it’s giving away stickers. I could be wrong, but I don’t think it understands money or policy. It just knows the word “refund” tends to calm people down. This is a perfect place for AI to pause and present options: no compensation, a small gesture, or escalate to someone who actually knows the rules. And again, the choice should be stored. Over time, the model starts to understand the pattern instead of improvising. With observability, you can see whether the model is still guessing or actually learning.
When AI tries to move timelines
It treats dates the way people treat leftovers. Just shift them around and hope nobody notices. From what I can tell, time is more of a suggestion to it. Better to have AI flag the timeline issue and ask the human whether anything should change. And the human’s answer becomes part of the workflow history so the model learns how real teams treat deadlines. Observability lets you spot drift before it becomes a mess.
When AI tries to invent future plans
It loves announcing things. New features. New initiatives. New directions. Half the time I’m reading something and thinking, “Where did that come from?” This is another checkpoint moment. AI should surface the idea and let the human decide whether it’s real or fantasy. And the decision should be saved so the model learns what counts as an actual plan. Observability helps you see how often the model wanders into fiction.
When AI tries to soften job requirements
It wants everyone to feel welcome, which is nice until it rewrites a senior role into something that sounds better suited for someone still figuring out what Git is. This is where AI should ask the human whether the requirement should stay firm, be adjusted, or be clarified. And the answer becomes part of the training record. Observability helps you catch these shifts before a hiring manager wonders why the job description suddenly looks like a summer internship.
When AI tries to diagnose severity
It will call something “critical” because it saw the word once in a similar sentence. I suspect it’s guessing. Badly. Better for AI to present a few severity options and let the human choose. And the choice should be stored so the model learns what “critical” actually means in your environment. Observability helps you see whether the model is still guessing or starting to understand your thresholds.
When AI tries to make commitments
It will promise delivery dates, resources, and contractual obligations with the confidence of someone who has never had to explain themselves in a meeting. This is a moment where AI should stop and ask the human whether any commitment should be made at all. And the human’s decision becomes part of the workflow memory. Observability helps you track how often the model tries to make promises it shouldn’t.
These moments aren’t failures. They’re signals. They tell you where AI needs supervision and where human judgment still matters. Capability grows when you know exactly where AI should pause and hand the decision back. And the real improvement comes from storing the entire exchange. The pause, the options, the human’s choice, and the final action become the training data that actually reflects how your team works.
Observability is what ties it all together. Without it, you’re guessing. With it, you can see the workflow as it actually behaves. You can see where the model drifts. You can see where humans step in. You can see whether the system is improving or just repeating the same mistakes. I suspect that observability might be the best way to turn AI from something unpredictable into something you can trust.
Since we’re talking about guardrails, here’s a small protocol you can use right away. I’ll use the compensation example because it’s one of the easiest places for AI to go off the rails.
Intent
Add a checkpoint to stop unauthorized compensation in customer messages and store the human’s decision for future training.
Inputs
The draft message, the compensation rule, and the escalation path.
Steps
Check the message for compensation language.
Confirm whether the language is allowed.
If it isn’t, present options to the human.
Log the human’s choice and the final action.
Output
A workflow step that can’t approve compensation on its own and learns from each decision.
I’ll wrap this up with the thought that keeps coming back to me. If AI doesn’t know when to pause, it eventually makes choices you never meant it to make. Better to teach it where the handoff belongs, save the decisions, and give yourself enough observability to see how the system is actually behaving.