Enterprise AI Context: Make AI Know Your Business

IN SHORT

Enterprise AI fails quietly when the model lacks company context: internal policies, product rules, customer history and decisions that never appeared in public training data.
Copying documents into a chat window or uploading a folder does not fix it. Context needs to be current, permission-aware and assembled for each request.
A context layer connects to systems of record, keeps an up-to-date index, applies each user's permissions, and supplies the right facts and rules to every model call.
Context is a data engineering problem first. The model is the easy part.

This is part 2 of Enterprise AI. Part 1 covered cost. This part covers the most common reason enterprise AI answers are wrong: the model does not know the business it is working for.

01What does “out of context” look like?

A sales assistant quotes standard pricing to a customer who has a negotiated contract.
A support bot explains a returns policy that changed two months ago.
An internal assistant summarises a project using the public meaning of an acronym your company uses differently.
A finance copilot calculates a margin without the cost allocation rules every analyst knows by heart.

In every case the answer is fluent and confident. That is what makes it dangerous: it reads like a correct answer from someone who does not work there.

02Why does it happen?

A model knows what was in its training data, which is public text up to a cutoff date (see how LLMs are trained). Your company's knowledge was never in it:

Policies and rules live in internal wikis, PDFs and people's heads.
Operational facts (orders, contracts, accounts, inventory) live in systems of record that change daily.
Decisions and history live in tickets, email threads and meeting notes.
Vocabulary is local: product names, team names, acronyms, internal codes.

Without that context the model fills the gap with what is typical. Typical is not what your company does. This is the enterprise version of why LLMs hallucinate.

03Why doesn't uploading documents fix it?

Many teams try the quick route: paste documents into a chat, connect a shared drive, or upload a folder to an enterprise plan's knowledge feature. It helps for one person and one question. At company scale it breaks down:

It goes stale. The upload is a snapshot. The policy changes; the copy does not.
It ignores permissions. Either everyone sees everything in the folder, or the feature cannot reflect the access rules of the source systems (see filter by permission before you rank).
It misses live data. Order status, contract terms and account health are in databases and SaaS tools, not documents.
It cannot be tested. Nobody can say which context was used for which answer, so nobody can measure or improve it.

04What is an enterprise AI context layer?

A context layer is the part of an AI system responsible for giving each model call the right information, and only the information this user is allowed to see.

It has four jobs:

Connect to the systems of record: document stores, the CRM, the ticketing system, the data warehouse.
Keep current: sync on change, track the last update of every source, and re-index affected chunks (see chunk by structure).
Assemble per request: retrieve documents with hybrid search, fetch live facts through tools, and add the business rules that apply.
Enforce access: every piece of context carries its permissions, and the user's scope is applied before ranking.

context/assemble.py

def build_context(user: User, task: Task) -> Context:
    scope = permissions.for_user(user)
    return Context(
        rules=policy_store.applicable(task.kind, region=user.region),        # versioned business rules
        facts=live_facts(task, scope),                                       # CRM/order/contract via tools
        passages=retrieve(task.query, scope=scope, k=6),                     # current, permission-filtered
        glossary=glossary.terms_in(task.query),                              # company vocabulary
        provenance=True,                                                     # every item keeps its source id
    )

The output is not just text for a prompt. It is a record of exactly which facts, rules and passages were used, which is what makes answers checkable and auditable (see no audit trail).

05Business rules are context too

Some of the most important context is not a document at all: “never offer more than 15% discount without approval”, “EU customers get a 30-day return window”. Keep rules like these as versioned, structured data the system applies, and pass the relevant ones to the model, instead of hoping it infers them from a policy PDF.

A model without your context is a confident outsider. A context layer is how it gets the briefing your employees already have.

06Where does this fit?

Context spans L1 data and knowledge (sources, indexes, permissions) and L3 intelligence (what to assemble for each decision). It is mostly data engineering, and it is what turns a generic model into one that answers like it works for you. It is the second core piece of the owned intelligence layer.

Previous: Paying frontier prices for routine work. Next: Unowned intelligence.