Agentic AI Systems in Production: What Nobody Warns You About

Multi-step AI agents that call tools, browse the web, write code, and make decisions sound powerful — and they are. They're also deeply unreliable in ways that aren't obvious from the demos. Here's what production deployment of agentic systems actually taught me. Why agent demos succeed and agent deployments fail A demo runs one happy path once, with a human watching. Production runs thousands of paths, unattended, against systems that time out, rate-limit, and return shapes nobody documented. The gap is not model quality. The gap is that an agent is a distributed system where one of the nodes is non-deterministic, and most teams build it as if it were a function call. The failure I see most often is compounding error. If each step in a chain is 95 percent reliable, a five-step agent is about 77 percent reliable end to end, and a ten-step agent is around 60 percent. Those are not acceptable numbers for anything touching money or customer data. The first architectural decision is therefore not which model to use — it is how few steps you can get away with. Bound the loop before you build anything else Every agent needs a hard stop: maximum iterations, maximum wall-clock time, and maximum spend. I set all three, and I set them low. An agent that has taken twelve turns to answer a question is not about to succeed on turn thirteen; it is stuck in a loop re-reading the same tool output. Failing fast and escalating to a human is cheaper than letting it burn tokens until a timeout fires somewhere upstream. I also cap the conversation the agent carries. Unbounded history is the most common cause of runaway cost, because every turn re-sends everything before it, so token spend grows quadratically with turn count. Summarising older turns into a compact state object keeps the context flat and, in my experience, actually improves decisions — the model stops getting distracted by its own earlier reasoning. Treat every tool call as an untrusted boundary The model decides which tool to call and with what arguments. That means the arguments are user-influenced input, and they must be validated exactly as you would validate a request body from the public internet. I schema-validate every tool argument, and I never let a model-produced string reach a shell, a SQL query, or a file path without going through the same allowlist a form submission would. The subtler risk is prompt injection through tool results. If your agent reads a web page, a support ticket, or a PDF, that content can contain instructions, and the model has no reliable way to distinguish data from directive. My rule is that content retrieved by a tool is never permitted to expand the agent's permissions. Whatever it says, the agent can still only call the tools it was already allowed to call, on the records it was already scoped to. Make every action idempotent and reversible Agents retry. They retry because a tool timed out, because a validation failed, or because the orchestrator restarted. If your create-invoice tool is not idempotent, retries mean duplicate invoices. I pass an idempotency key derived from the agent's run ID and step index into every write operation, so a replayed step is a no-op rather than a second side effect. For anything genuinely destructive — sending email, charging a card, deleting records — I do not let the agent execute directly. It writes a proposed action to a queue, and either a deterministic rule or a human approves it. This sounds like it defeats the purpose of automation. In practice the agent still does 95 percent of the work; you are just refusing to let a probabilistic system be the last checkpoint before an irreversible outcome. Observability is the difference between a bug and a mystery When a traditional service misbehaves you read a stack trace. When an agent misbehaves you need the full transcript: the system prompt, every tool call with arguments and results, token counts, latency per step, and the final answer. I log all of it, keyed by run ID, from day one. Without it you cannot answer the only question that matters after an incident, which is why the model chose what it chose. I also track a small set of aggregate metrics that predict trouble before users report it: average steps per run, percentage of runs hitting the iteration cap, tool error rate by tool, and cost per successful outcome. A rising steps-per-run number almost always means an upstream tool started returning something the model finds confusing, and it shows up in that metric days before it shows up in complaints. Evaluate on outcomes, not on vibes The hardest discipline is building an evaluation set before shipping. Twenty to fifty real cases with known-correct outcomes, run on every prompt change, scored automatically where possible. Without it, prompt tuning is superstition: you change a sentence, the demo looks better, and you have no idea whether you improved the system or overfitted to the one example you were staring at. Score the outcome, not the wording. For a data-extraction agent the question is whether the extracted fields match the source document, not whether the prose around them reads nicely. Once outcome scoring exists, model upgrades become a measurement rather than an argument — you run the set against the new model and see the number move. Reliability compounds downward: five steps at 95 percent each is about 77 percent end to end, so minimise steps before optimising prompts Cap iterations, wall-clock time and spend on every agent; a stuck agent never recovers on the next turn Validate model-produced tool arguments like untrusted input, and never let retrieved content expand the agent's permissions Make every write idempotent with a run-scoped key, and route irreversible actions through an approval queue Log full transcripts by run ID and track steps-per-run — it degrades days before users complain Build a 20-50 case evaluation set that scores outcomes, or prompt tuning is guesswork Agentic systems are worth building, but they are systems engineering problems wearing an AI costume. The teams that succeed treat the model as one unreliable component inside an architecture designed for retries, bounded blast radius, and full observability. The teams that struggle treat it as a magic function and discover the hard way that probability does not respect deadlines.

Agentic AI Systems in Production: What Nobody Warns You About

Integrating Claude and GPT with Zoho CRM: An Architecture

Agentic AI in the Sales Pipeline: What Works Today

Building AI-Assisted Support with Zoho Desk and LLMs

Related articles

Integrating Claude and GPT with Zoho CRM: An Architecture

Agentic AI in the Sales Pipeline: What Works Today

Building AI-Assisted Support with Zoho Desk and LLMs