Agentic AI Systems in Production: What Nobody Warns You About

A demo runs one happy path once, with a human watching. Production runs thousands of paths, unattended, against systems that time out, rate-limit, and return shapes nobody documented. The gap is not model quality. The gap is that an agent is a distributed system where one of the nodes is non-deterministic, and most teams build it as if it were a function call.

The failure I see most often is compounding error. If each step in a chain is 95 percent reliable, a five-step agent is about 77 percent reliable end to end, and a ten-step agent is around 60 percent. Those are not acceptable numbers for anything touching money or customer data. The first architectural decision is therefore not which model to use — it is how few steps you can get away with.

Every agent needs a hard stop: maximum iterations, maximum wall-clock time, and maximum spend. I set all three, and I set them low. An agent that has taken twelve turns to answer a question is not about to succeed on turn thirteen; it is stuck in a loop re-reading the same tool output. Failing fast and escalating to a human is cheaper than letting it burn tokens until a timeout fires somewhere upstream.

I also cap the conversation the agent carries. Unbounded history is the most common cause of runaway cost, because every turn re-sends everything before it, so token spend grows quadratically with turn count. Summarising older turns into a compact state object keeps the context flat and, in my experience, actually improves decisions — the model stops getting distracted by its own earlier reasoning.

The model decides which tool to call and with what arguments. That means the arguments are user-influenced input, and they must be validated exactly as you would validate a request body from the public internet. I schema-validate every tool argument, and I never let a model-produced string reach a shell, a SQL query, or a file path without going through the same allowlist a form submission would.

The subtler risk is prompt injection through tool results. If your agent reads a web page, a support ticket, or a PDF, that content can contain instructions, and the model has no reliable way to distinguish data from directive. My rule is that content retrieved by a tool is never permitted to expand the agent's permissions. Whatever it says, the agent can still only call the tools it was already allowed to call, on the records it was already scoped to.

Agents retry. They retry because a tool timed out, because a validation failed, or because the orchestrator restarted. If your create-invoice tool is not idempotent, retries mean duplicate invoices. I pass an idempotency key derived from the agent's run ID and step index into every write operation, so a replayed step is a no-op rather than a second side effect.

For anything genuinely destructive — sending email, charging a card, deleting records — I do not let the agent execute directly. It writes a proposed action to a queue, and either a deterministic rule or a human approves it. This sounds like it defeats the purpose of automation. In practice the agent still does 95 percent of the work; you are just refusing to let a probabilistic system be the last checkpoint before an irreversible outcome.

When a traditional service misbehaves you read a stack trace. When an agent misbehaves you need the full transcript: the system prompt, every tool call with arguments and results, token counts, latency per step, and the final answer. I log all of it, keyed by run ID, from day one. Without it you cannot answer the only question that matters after an incident, which is why the model chose what it chose.

I also track a small set of aggregate metrics that predict trouble before users report it: average steps per run, percentage of runs hitting the iteration cap, tool error rate by tool, and cost per successful outcome. A rising steps-per-run number almost always means an upstream tool started returning something the model finds confusing, and it shows up in that metric days before it shows up in complaints.

The hardest discipline is building an evaluation set before shipping. Twenty to fifty real cases with known-correct outcomes, run on every prompt change, scored automatically where possible. Without it, prompt tuning is superstition: you change a sentence, the demo looks better, and you have no idea whether you improved the system or overfitted to the one example you were staring at.

Score the outcome, not the wording. For a data-extraction agent the question is whether the extracted fields match the source document, not whether the prose around them reads nicely. Once outcome scoring exists, model upgrades become a measurement rather than an argument — you run the set against the new model and see the number move.