The data foundation AI agents actually need

A common reflex before the first AI project is to build a comprehensive data infrastructure first: a central data warehouse, clean pipelines from every system, a unified data platform. That sounds like solid preparation. In practice it delays the first production use case by months, often longer than the entire later implementation takes. What's usually left at the end of that preparation is a platform for which nobody has yet defined a concrete use case.

An agent for a specific use case doesn't need all of a company's data. It needs targeted access to the few sources relevant to exactly that use case. An invoice-checking agent needs access to invoices, purchase orders and payment terms, not the entire customer history or the HR system.

This narrowing isn't a compromise, it's an advantage. Fewer systems mean less integration work, fewer permission questions, and a smaller risk that the agent gets access to data it doesn't need for its task. A tightly scoped agent is easier to audit than one with access to everything.

Three building blocks cover most first use cases. First: relevant documents in searchable form (policies, product information, prior comparable cases), kept current and without contradictory duplicates. Second: structured access to operational systems through an interface, not a manual export that's already stale the next day. Third: a permission model that matches existing access rules in the company, so that an agent can't see more than the person on whose behalf it acts.

Miss any one of these three building blocks, and no language model, however capable, compensates for the gap. An agent without current documents guesses. An agent without structured system access works off stale data. An agent without a clear permission model is a compliance risk the moment it goes live.

These three building blocks can be tackled in order of urgency. Documents first, because they're usually fastest to clean up. Then system access, because it typically needs a technical connection that takes some lead time. The permission model last, but never truly last. It should be settled before the agent works with real data for the first time, not only after an audit asks about it.

First: outdated or duplicate documentation, where nobody knows anymore which version is current. Second: knowledge that only exists in unstructured form (scanned PDFs, email threads, handwritten notes) that no system can access programmatically. Third: inconsistent identifiers between systems, for example when the same order carries different numbers in the CRM and the ERP and the two records can't be matched automatically. Fourth: the absence of a person who is actually responsible for a data domain and can grant access without the request disappearing into a queue between departments.

Every one of these four blockers can be fixed, but none of them by a better language model. They're organizational problems, not technical ones, even though they feel like technical problems when an agent project stalls on them. Checking these four points before a project starts saves you the most common cause of failed first use cases.

A language model compensates remarkably well for linguistic imprecision: vague phrasing, inconsistent terminology, informal notes. It does not compensate for wrong or missing facts. If the stored return policy is outdated, the agent confidently gives the outdated answer, indistinguishable from a correct one.

That makes ignoring data cleanup more dangerous than with classic software. A conventional system with no data obviously returns no answer. An agent built on a language model returns an answer that sounds convincing and is still wrong. These errors surface later, usually with a customer or in an audit, not during development.

That's why an honest spot check before any project start pays off: take ten typical requests from the planned use case and check by hand whether the existing sources would answer each one correctly. If the spot check already fails on two or three cases, that's not a reason to delay the project. It's the list of things that still need cleaning up before launch.

For a first use case, a deliberately small setup is enough: a well-maintained source of truth for the chosen process, a way for the agent to retrieve it programmatically, a clear decision on who grants access, and a defined fallback for cases where the agent is unsure. These four elements can be built for a single, well-chosen use case in weeks, not quarters.

The mistake is rarely too little infrastructure. It's choosing the wrong order: the big platform first, the first use case second, instead of the other way around. The small setup isn't a stopgap that gets fully replaced later. It's the first, correctly sized version of an infrastructure that grows deliberately with each additional use case.

The fallback deserves particular attention, because it's the piece most often forgotten. It should specify the context an uncertain case arrives with: which question the agent couldn't answer, which data it drew on, and what caused the uncertainty. Without that context, the person taking over starts from zero.

Once an agent makes decisions relevant to a customer or an audit, you'll need to be able to trace which data it saw at the time of the decision. Building that logging in from the start is considerably cheaper than retrofitting it later, once the system is already live and proof is needed.

That same logging is also the foundation for quality control: when an agent decides wrong, the log shows whether the cause was missing, outdated, or misinterpreted information. Without it, every root-cause search stays a guess. For companies that fall, or will foreseeably fall, under the EU AI Act, the same logging is also the basis for the required traceability of automated decisions.

Before a company launches its first official AI project, individual employees are usually already using public AI tools on their own initiative, for drafting text, summarizing and sometimes for analyzing internal documents. That often puts customer data, contract content or internal figures into systems outside the company's control, with no bad intent, simply because it was the fastest available solution.

This shadow usage isn't a side issue. It's the actual starting point for many data projects. A look at what employees are already typing into which tools shows the real need more reliably than any workshop, and it reveals where an official, controlled solution with clear data access is urgently needed. An open conversation about it, without blaming employees, usually surfaces this practice faster than a formal policy does.

The right moment for a larger investment in shared data infrastructure isn't the start. It's the moment several agents or use cases begin competing for the same data sources, for example when a third agent needs the same customer dataset that the first two already connected to independently. At that point, a shared layer that all agents use is worth it, instead of building a separate connection for every use case.

Before that point, a big data investment is a bet on future demand you don't yet know. After that point, it's a response to demand that has already become visible. The second bet is the far safer one and usually the cheaper one too, because it covers exactly the requirements that actually emerged, rather than ones someone imagined in advance.