Generative AI Enterprise Integration: Patterns That Work at Scale

The default enterprise AI project is a chat window bolted onto a knowledge base. It demos well and gets used for a fortnight. The reason adoption fades is that it asks people to change where they work: they must leave the CRM they live in, go somewhere else, phrase a question, and carry the answer back by hand.

The pattern that sticks is AI embedded in the existing workflow. A summary that appears on the record. A draft reply pre-filled in the ticket. A risk score in the column where the sales manager already looks. Nobody has to remember to use it, because it arrives where the work already happens, and that difference in placement matters more than any difference in model quality.

The simplest high-value pattern takes a record that already exists, sends the relevant fields to a model, and writes a derived field back. Summarise a long case history into three lines. Classify an inbound lead. Extract commitments from a meeting note. The unit of work is one record, which makes it cheap, bounded, and trivial to retry.

It is also the easiest to govern, because the input and output are both stored next to the record. When someone asks why a lead was scored the way it was, you can show them the exact input and the exact output with a timestamp. Start here. Teams that begin with an ambitious autonomous agent and skip this pattern usually end up building it later anyway, after the agent proves ungovernable.

Calling a model API directly from inside a CRM's scripting environment works until it does not. Those environments have short execution limits, awkward retry semantics, and poor logging. Once you pass a few hundred calls a day, a thin external service between the business system and the model is worth building — it owns queuing, retries, schema validation, caching, and the audit log.

That middleware becomes the single place where policy lives. Which models are allowed, what gets redacted before leaving the network, how long prompts are retained, what the per-team spend cap is. Scatter those decisions across a dozen integration scripts and you cannot answer a compliance question; centralise them and you can answer it in one place.

General-purpose models know nothing about your pricing rules, your policies, or last quarter's incidents. Retrieval over your own systems is what converts a plausible assistant into a useful one. The integration question is which sources are authoritative, how often they change, and who is allowed to see what.

Permissions are the hard part and they must be designed in from the start. A retrieval layer that can read every document is a data-leak channel dressed as a productivity tool. Filter by the requesting user's entitlements before ranking, and treat that filter as a security control subject to the same review as any other access-control code.

The governance that actually works is narrow and mechanical: log every prompt and response with a user and timestamp, redact identifiers that do not need to leave the network, cap spend per team, and keep a documented list of approved use cases. That is a short list, and it is enough to pass most internal reviews.

The failure mode is governance by committee, where every use case needs a bespoke risk assessment and nothing ships for eight months. Codify the rules in the middleware so compliance is the default path rather than a review gate. If the only way to call a model is through a service that already redacts, logs, and caps, then approval becomes a question about the use case rather than about the plumbing.

Enterprise AI costs escalate quietly. A summarisation feature that runs on every record update instead of on demand can multiply spend without anyone noticing until the invoice. I attach a cost estimate to each pattern before building it: expected calls per day, tokens per call, and the resulting monthly figure. If that number is uncomfortable on paper it will be worse in production.

The biggest practical savings come from caching identical requests, using the cheapest model that passes your evaluation set rather than the best available, and triggering on meaningful events rather than on every write. Model choice in particular should be an evidence-based decision — in my own testing the newest model was both slower and less reliable than the previous one, which is exactly the sort of thing you only learn by measuring.