Where Does Your AI's Context Actually Come From?

An employee in finance asks the company assistant a simple question: what did we agree with our logistics provider about late-delivery penalties, and has anyone raised it recently?

The answer arrives in seconds, and it is good. Behind that one question, the assistant assembled its context from five places. Her prompt. The contract draft that happened to be open on her laptop. A search index built over the document libraries on the intranet. The ticket system, reached through a tool connection, where a colleague had logged a dispute last month. And a web search for the provider’s published terms.

Five sources, one request. Ask around the company who can name all five, and you will usually get silence. The permission model, the thing we point to whenever someone asks about data governance, covers exactly one of them.

Why the model we have does not fit

Enterprise governance was built for data at rest. Data sits in a system. Someone has access to it. That access is documented, reviewed, and occasionally revoked. Integrations between systems are designed once, reviewed once, and then run the same way for years. That model has served us well. It also rests on three assumptions that AI assistants quietly break.

Assembly is decided per request. A classic integration is fixed: system A sends field X to system B every night. An assistant decides at runtime which sources to consult, and part of that decision is made by the model itself. There is no integration you can review once and then trust.

Aggregation creates new sensitivity. Each of the five sources above is harmless on its own. A contract draft, a dispute ticket and a public price list are three ordinary documents. Combined into one answer, they describe the company’s negotiating position. Classification, however, is done per source. Nobody classifies the combination, because until recently the combination only existed in someone’s head.

The boundary moves. We decide where data may be stored, and we have become good at it. But assembled context leaves the organization at inference time, not at storage time. The question is no longer only where the data lives. It is where the context goes when the model is called.

None of this is primarily a security problem or a tooling problem. It is a missing architecture view. We maintain landscapes of applications, interfaces and data objects. We have almost nothing that shows what flows into an AI assistant and where it goes afterwards.

Four source classes

When I draw that view, the sources fall into four classes. For each one, three questions matter: what flows, which part of it is private, and who decides.

From the user. The prompt and anything uploaded with it. This looks like the easy case, because the user chose to share it. In practice, the gap between what someone believes they are sharing and what is actually transmitted is large. People paste a whole spreadsheet to ask about one row. They upload a document and forget the appendix. The user decides, but with poor visibility.

From the user’s device. Open files, screen content, a local search index, the clipboard. Assistants built into the operating system or the office suite read from here by design, because that is what makes them useful. Yet the device appears in no application landscape as a data source. We model servers and cloud applications. The laptop is treated as an endpoint to secure, not as a system that feeds data into another system.

From internal systems. Retrieval indexes and tool calls into business applications. This is where the most common real failure lives, and it is an unspectacular one: the index flattens the permission model of the source system. A document library has fine-grained permissions. The search index built over it for the assistant often has exactly one, that of the service account that crawled it. Someone who could never open a document in the source gets its content back as an answer, paraphrased and without a link. Nobody breached anything. The architecture simply did not carry the permissions across.

From external servers. Web search, third-party tool servers, public APIs. Tool connections are increasingly standardized, the Model Context Protocol (MCP) being the best-known example, and that makes adding one very easy. Here the flow runs in two directions. What goes out: the query, and often a good part of the assembled context, sent to a server you do not operate. What comes in: content you did not write, which the model reads as input. The second direction deserves more attention than it gets. A web page or a tool response can contain text that the model treats as an instruction rather than as information, and then the assistant acts on behalf of whoever wrote that page. It is the same lesson I described in Security by Design for Analytics Platforms: the moment you bring many sources together in one place, you change your own threat model.

Five places you can actually intervene

Once the flows are visible, the control points become visible too. There are five, and each one has a price.

1. At the source. Classify documents and exclude some of them from indexing altogether. This is the most robust control, because what never enters the flow cannot leak from it. The price: someone has to do the classifying, and every exclusion makes the assistant less useful.
2. At the index. Build retrieval that respects the permissions of the source system, per user and per query. This closes the most common failure described above. The price: it is harder to build and slower to query, and every source system models permissions differently.
3. At the connector layer. Decide which tool servers an assistant may use, with which scopes, from an allowlist. The price: a review process that slows down every new connection, which is exactly what tempts people to build around it.
4. At the boundary. Control what leaves the organization at inference time: where the model is hosted, under which jurisdiction, and whether inputs may be used for training. I have written about the jurisdiction side in Digital Sovereignty for CIOs. The price: fewer models to choose from, and sometimes a noticeably weaker one.
5. On the device. Policies for what assistants may read locally, and which assistants may be installed at all. Its price is the lowest of the five, which is why it is so popular.

Most organizations only do the last one and call it governance. A device policy is useful. But it covers one of four source classes at one of five control points, and it says nothing about the index that flattens permissions or the tool server nobody reviewed.

What this means for enterprise architecture

The good news is that none of this needs a new discipline. Enterprise architecture already has the instruments: landscapes, data-flow views, classification schemes and review gates. They just have not been pointed at context yet. Three artefacts close most of the gap.

An inventory of context flows per assistant. For each assistant in use: which sources it can reach, through which index or connector, and where the model runs. It belongs in the same repository as the application landscape and should be maintained the same way. An assistant is an application whose interfaces are decided at runtime. That is a reason to document it more carefully, not less.

Classification of the composite. Classify what an assistant can combine, not only the individual sources. If an assistant can join contract data with ticket history and the public web, the composite needs its own classification, and it will usually be higher than that of any single source.

A review gate for new connectors. Every new tool connection is an integration and deserves an integration review: what flows out, what comes in, and whose permissions apply. Keep it light enough that people actually use it. This is organizational work as much as architectural work. It belongs at the same shared table where security, legal and architecture already meet for data questions, as I argued in Meet Once, Not Twice. A connector review that runs separately from data governance will drift away from it within a year.

Conclusion

There is one question every CIO should be able to answer in a single sentence, for every assistant in the organization: which sources feed it, whose permissions govern what it retrieves, and at which point its data crosses the boundary of the company.

If you can answer that, you have the basis for governing AI. If you cannot, you do not have an AI governance problem yet. You have a missing architecture view, and that is exactly the part enterprise architecture already knows how to fix.