21 C
New York
Friday, August 28, 2026

Making Your Knowledge Prepared for Agentic AI


There’s a variety of pleasure proper now about agent frameworks,
orchestration patterns, and protocols. All of it issues, however nearly none of
it delivers worth in case you skip the information layer. Earlier than any agent framework can
produce helpful outcomes, your information must be in a form {that a} machine can
eat, belief, and act on. On this article, we focus on what your
information must seem like for agentic AI to derive worth from it.

We have spent fairly a little bit of time constructing information architectures for the human
client. We’re about at hand these architectures to a really completely different type
of client, and most of them aren’t prepared for it.

The customers of your information are altering

For over three many years, we have been constructing information techniques for people.
Dashboards, studies, analyst queries, all of it designed round an individual
sitting in entrance of a display. And it labored, as a result of people deliver an
monumental quantity of implicit context, together with the curiosity to trace
down no matter they’re lacking from the folks round them.

A human analyst is aware of what “income” means in your explicit
group. They know which tables to question and which of them to keep away from.
They discover when a quantity appears to be like off, when a complete is suspiciously spherical,
when a date falls on a public vacation, or when a worth appears too low. That
intuition is doing a considerable amount of invisible context and information
work.

A human hesitates at information that appears unsuitable; an agent acts on it
anyway

Brokers have none of it. They can not lean on the tribal information and
sample recognition folks accumulate over years, so that they want context
made express, entry in actual time, and high quality they’ll depend on. And the
distinction that issues most is that this: when the information feels unsuitable, a human
double-checks; an agent confidently acts on it. That behavioral hole is
what the remainder of this dialogue is constructed round.

What “AI-ready” has to imply now

For a human client, the information solely needed to be ok; the
analyst did the remaining. The which means, the sanity examine, and the judgment
about whether or not a quantity might be trusted all lived in an individual’s head.
When the identical information is handed to an agent, each little bit of that implicit
labor has to maneuver into the information itself. That reveals up as 5
attributes, every the flip facet of one thing a human used to do for
free.

  • Trusted: an individual pauses at a quantity that feels unsuitable; an agent
    acts on it. The arrogance a human used to provide must be inbuilt, so the
    information should be correct, contemporary, and validated earlier than the agent ever sees it.
  • Contextual: an individual is aware of your “income” determine already has returns
    taken out, and that your fiscal yr begins in February; an agent must be instructed
    each. That means that used to reside in somebody’s head must be made express within the
    information.
  • Traceable: when an individual decides, they’ll clarify why afterward; when
    an agent decides in 30 seconds, that reasoning is gone except you seize it as
    it occurs. You’ve gotten to have the ability to reconstruct what the agent did and why.
  • Ruled: an individual’s entry is bounded by their function and their judgment;
    an agent’s must be bounded by design. Entry should be scoped, managed, and
    auditable.
  • Operational: an individual reads a dashboard after which goes and does
    one thing; an agent must be ready to do the one thing. The information cannot simply be
    readable, it must be actionable.

All 5 come all the way down to the identical concept. Every is a job people used to do
with out considering, now pushed into the information itself. Miss one, and the
agent will not degrade gracefully the way in which an individual would. It fails
confidently.

None of those attributes builds itself. The remainder of the article works
by means of 4 subjects that do, roughly within the order you need to deal with
them.

  1. Knowledge Contracts and High quality makes information Trusted. We begin right here,
    as a result of a single unsuitable truth poisons each layer constructed on high of it.
  2. Traceability and Governance information why an agent acted and bounds
    what it may possibly attain, making information Traceable and Ruled.
  3. The context layer encodes what your metrics and entities imply,
    making information Contextual.
  4. From Searchable to Actionable lets brokers question reside techniques and
    write again, making information Operational.

We’ll take them one subject at a time, and present what it takes to construct
every attribute in. Work by means of all 4, and the 5 attributes cease
being summary objectives. They grow to be one thing you may engineer,
turning abnormal information into AI-ready information.

Knowledge Contracts and High quality: Brokers Cannot Scent Dangerous Knowledge

People have a odor take a look at for unhealthy information. They discover when a quantity
appears to be like off, when a date is not sensible, or when a worth appears unsuitable.
Brokers haven’t any such intuition. As Simon Willison places it, language fashions are gullible, they consider
no matter they’re handed and act on it. Feed an AI agent a unsuitable
worth, and it will not pause to marvel, it’ll use the quantity and
produce a assured, unsuitable reply. With out trusted information, nothing else
in agentic AI works, so that is the place we start.

Brokers deal with each worth as fact

Contemplate a concrete state of affairs. A pricing agent is requested for the
present worth of Product X. Yesterday, the value was up to date from
$49.99 to $59.99. However the agent’s information supply hasn’t refreshed, it
nonetheless reveals the outdated quantity.

The agent does not hesitate, it retrieves $49.99, quotes the
buyer, the shopper buys, and the corporate loses $10 on each unit
offered. Each step the agent took was technically right. It adopted
its workflow completely. The information it accessed was the
drawback.

The leaders most assured their information is AI-ready additionally title information
readiness their greatest barrier

A human gross sales rep would have paused: “Wait, did not we replace this
final week?” They’d double-check. They’ve institutional reminiscence and a
really feel for when one thing’s off. The agent has neither. Errors do not
set off warnings; they cascade silently by means of the workflow. And
this is not a uncommon edge case. Within the 2026 State
of Knowledge Integrity and AI Readiness
report, Exactly and Drexel
College’s LeBow School of Enterprise surveyed 505 information and analytics
leaders, of whom 87% believed their information was prepared for AI, but 43% named
information readiness as the one greatest barrier to getting worth from it. That
hole between confidence and readiness is the organization-level model
of the pricing agent, certain of itself and unsuitable. A separate KPMG International AI Pulse survey of two,145 leaders factors the
identical manner, with almost half of executives now seeing AI’s prices exceed its
advantages. Most enterprises are one stale discipline away from the state of affairs
above.

Schema is regulation: information contracts as code

So how do you stop AI brokers from accessing unhealthy or stale
information? The reply is information contracts, treating schema as regulation,
not a well mannered suggestion.

This reverses a decade of “schemaless is versatile” considering,
for human customers, unfastened schemas are merely inconvenient, whereas
for AI brokers, they’re harmful. A knowledge contract, written within the
Open Knowledge Contract
Commonplace
,
the format the Knowledge Contract CLI
makes use of (and really useful in Thoughtworks tech radar
33
),
defines the principles explicitly. A product_pricing contract would possibly
specify:

  • Properties with strict logical sorts.
  • A high quality rule that worth should be better than zero.
  • A high quality examine on forex that rejects something outdoors USD, EUR, or
    GBP.
  • Critically, a freshness SLA, pricing information should have been refreshed
    inside the final 24 hours.

Within the Open Knowledge Contract Commonplace, that contract is proven
beneath.

apiVersion: v3.1.0
type: DataContract
id: product-pricing
title: Product Pricing
model: 1.0.0
standing: lively
schema:
  - title: product_pricing
    physicalType: desk
    properties:
      - title: product_id
        logicalType: string
        physicalType: varchar(64)
        required: true
        distinctive: true
        primaryKey: true
        primaryKeyPosition: 1
      - title: worth
        logicalType: quantity
        physicalType: decimal
        required: true
        high quality:
          - kind: sql
            description: Each worth should be better than zero
            question: SELECT min({property}) FROM {object}
            mustBeGreaterThan: 0
      - title: forex
        logicalType: string
        physicalType: varchar(3)
        required: true
        high quality:
          - kind: sql
            description: Forex should be a supported ISO code
            question: SELECT depend(*) FROM {object} WHERE {property} NOT IN ('USD', 'EUR', 'GBP')
            mustBe: 0
      - title: ingested_at
        logicalType: timestamp
        physicalType: timestamp
        required: true
slaProperties:
  # the rule that may have caught the stale-price state of affairs
  - property: latency
    worth: 24
    unit: h
    ingredient: product_pricing.ingested_at

Enforcement occurs alongside three dimensions.

  • Schema enforcement ensures sorts and constraints are revered and made
    express by the contract.
  • Freshness SLAs outline the utmost acceptable staleness per dataset,
    nightly batch updates aren’t sufficient when an agent solutions in actual time.
    Key the SLA to when the information was final efficiently loaded, not when a worth final
    modified, in order that regular information is not flagged as stale and a stalled pipeline cannot
    masquerade as contemporary.
  • High quality gates validate contracts in CI/CD, blocking deployments when
    they fail.

Discover how this adjustments the sooner pricing state of affairs, it
prevents it by design. If the pricing information hasn’t been refreshed
in 24 hours, the contract is violated earlier than the agent ever sees
the information.

The quarantine sample

Defining a contract is one factor. What occurs when information
violates it? You want a circuit breaker and that is the
quarantine sample.

The movement works like this. Uncooked information arrives from supply
techniques, APIs, databases, streams. Earlier than it enters the agent
accessible information retailer, it passes by means of a contract validation
gate that checks three issues, does it match the schema, is it
inside the freshness SLA, and does it cross the standard
guidelines?

If it passes all three, it flows into the licensed, agent
prepared tier. If it fails any one among them, it is quarantined,
routed to a useless letter queue for human evaluate, with alerts
fired.

Dangerous information lands in a dead-letter queue, by no means in entrance of the
agent

The purpose is that the agent by no means sees the unhealthy information. It
does not get poisoned by stale costs or corrupted embeddings. In
the pricing state of affairs, if the ingested_at timestamp is older
than 24 hours the contract is violated and the file is
quarantined, so when requested in regards to the worth the agent says, “I
do not have present pricing information” reasonably than confidently quoting
the unsuitable quantity. That could be a much better failure mode. And it is a job
for the information structure, not the mannequin. A greater mannequin will not rescue
you from unhealthy information.

Medallion structure for brokers

A medallion structure is an analytical information design sample for
organizing information in a lakehouse,
popularized by Databricks.

Dangerous information will get quarantined, however the place does the good information go?
That is what the medallion structure organizes, and its first three
tiers are nicely established:

  • Bronze: uncooked, immutable ingestion. You retain every thing for audit path
    and lineage.
  • Silver: validated and deduplicated. Schema is utilized, information contracts
    are enforced, and that is the place the quarantine sample lives.
  • Gold: licensed. That is what the semantic mannequin compiles towards,
    entry is ruled, and metrics are trusted.

For agentic architectures, there is a helpful fourth tier value
including: Adaptive Gold the place brokers grow to be lively
individuals in information curation reasonably than passive customers (proven
within the determine beneath). They monitor their very own question patterns,
determine steadily accessed mixtures, and materialize optimized
datasets, successfully constructing their very own warehouse views based mostly on
actual utilization. The concept brokers can actively curate information, reasonably
than solely learn it, is already in manufacturing, at DataHub’s CONTEXT 2025 summit, Apple
described brokers performing as “digital stewards” of its information catalog,
repeatedly scanning metadata, flagging gaps, and proposing
updates, turning passive documentation into an lively governance
associate. Apple’s brokers curate the catalog; Adaptive Gold factors
that very same active-curation sample on the datasets themselves. That
final step is an extrapolation, however a modest one from one thing
already operating.

Determine 1: Medallion tiers for brokers:
information flows from uncooked Bronze by means of validated Silver to licensed
Gold and agent curated Adaptive Gold, whereas brokers are restricted to
Gold and above.

Bronze and Silver are for people; brokers see solely Gold and
above

The important thing architectural precept is that brokers ought to solely
entry Gold tier or above. Bronze and Silver exist for
lineage, debugging, and human investigation. Exposing uncooked or
partially validated information to brokers invitations the
pricing drawback again in.

The identical guidelines for unstructured information

Every thing to this point has seemed like a desk, costs, currencies,
timestamps, however most of what brokers eat is not tabular. It is
paperwork, wikis, PDFs, and assist tickets, chunked and embedded
right into a vector retailer for retrieval. In case your brokers do RAG, that is
the information they run on, and it wants the identical belief ensures, even
although you may’t write worth > 0 on a paragraph. The patterns
carry over, solely the standard dimensions change.

The stale-price state of affairs has a twin right here. A coverage doc will get
up to date, however the vector index is not re-embedded, so the agent
retrieves the outdated model and solutions confidently from it, the identical
failure because the stale worth, solely now it is an embedding reasonably than a
row. The freshness SLA carries over, however be exact about what
the clock measures, the purpose is not when the content material final modified,
it is when the index was final efficiently rebuilt towards its
sources. A 24-hour SLA means the re-indexing job should have accomplished
inside the final 24 hours, if it hasn’t, the index is stale and
quarantined even when nothing seems to have modified, as a result of a
silently failed indexer is strictly when you may’t inform whether or not
one thing did. That one heartbeat catches each the up to date however
unindexed
doc and the pipeline that quietly stopped.

Contracts transfer from the content material to the encircling metadata.
You may’t constrain the prose, however you may require that each chunk
carry a supply, a model, a timestamp, and an entry scope, and
reject something that does not. That metadata can be what makes
retrieval traceable and governable later.

High quality gates get checks suited to textual content, reject empty or
truncated chunks, catch near-duplicate paperwork that skew
retrieval, flag failed extractions and OCR rubbish, and look ahead to
embedding drift. A malformed or empty embedding warps similarity
search, so it by no means reaches the shop, for a similar motive a foul
worth by no means reaches the agent, a warped index makes the agent
retrieve confidently unsuitable content material.

Whether or not the information is a priced row or an embedded paragraph, the
job is equivalent. The structure has to odor what’s unhealthy earlier than
the agent does.

Confidence-threshold routing

Contracts, quarantine, and the medallion structure deal with the
clear circumstances. However there is a grey space, information that is not clearly unhealthy,
however is not absolutely reliable both. That is the place
confidence-threshold routing is available in, bridging full autonomy
and full human management.

The agent processes a request and assesses information high quality alerts,
and checks not simply mannequin confidence, however data-level alerts
like freshness, completeness, and consistency. If confidence is at
or above the brink (say 85%), the agent proceeds autonomously.
Beneath it, the agent defers to a human. The brink is configurable
per use case, for instance, pricing would possibly demand 90%, whereas an
inner FAQ is okay at 70%.

Let’s return to the pricing state of affairs one final time. The value
information is three days stale; the freshness SLA says 24 hours. The SLA
violation routinely drives the boldness rating beneath the
threshold, no matter how assured the mannequin itself feels about
its reply. The agent ought to reply by pulling a human in:

“I am not assured this worth is present. Routing to a human for
verification.”

Knowledge high quality alerts ought to drive the
threshold, not simply the mannequin’s personal confidence

In different phrases, information high quality alerts ought to drive the
threshold, not simply the mannequin’s personal confidence. A mannequin may be
certain of a stale reply, and the freshness SLA overrides that
misplaced certainty.

The exhausting half is popping these high quality alerts right into a single
rating and weighing it towards the mannequin’s personal confidence. That is an
open design drawback, not a solved one. Begin with a tough gate reasonably
than a easy composite. Any contract or SLA breach forces a human,
no matter how the opposite alerts look. Add weighted scoring later,
and solely as soon as you may present it beats that easy rule.

The place to begin

You do not have to construct all of this directly, and most groups
cannot. Contracts, quarantine gates, a medallion structure, and
confidence-threshold routing are quite a bit to face up in a single go. The
excellent news is that they are additive, every one lowers danger by itself,
and you’ll layer in the remaining over time. Start with the very best
leverage strikes and increase from there.

  1. Outline freshness SLAs for each dataset brokers contact. The identical
    dataset can have completely different freshness necessities per client, akin to a
    pricing desk that is nice on nightly batches for a dashboard may have close to actual
    time updates when a quoting agent is determined by it.
  2. Implement quarantine gates. Validate towards contracts earlier than information
    enters agent accessible storage. Begin along with your highest danger datasets akin to
    pricing, stock, buyer information.
  3. Begin with the Knowledge Contract CLI. Deliver contract governance into
    CI/CD, outline contracts as YAML, validate routinely, block deployments on
    failure. Deal with information contracts with the identical rigor you’d give an API
    contract.
  4. Add confidence threshold routing. When high quality alerts drop beneath a
    threshold, defer to a human. Begin excessive (round 90%) and regulate downward as you
    construct belief and observe accuracy.

We have made information reliable. However when brokers act autonomously on that information,
who’s watching?

Traceability and Governance: Auditing Autonomous Brokers

Even with excellent information, autonomous motion raises a more durable query, when a
regulator asks why the agent did what it did, are you able to reply? Conventional
techniques file what occurred. Agentic ones have to elucidate why. That shift, from
what to why, is the place governance will get exhausting.

The audit hole

Image a financial institution operating agentic AI for commerce finance, the place the governance
structure is the true innovation.

An agent processes a letter of credit score. It checks KYC information, verifies the
buyer is not on a sanctions record, evaluates the credit score phrases, and approves a
$2.4 million transaction, all in about 30 seconds. Six months later, a regulator
asks a easy query, “Why was this permitted?”

Conventional audit logs can let you know what occurred,
however they cannot let you know why.

Conventional audit logs can let you know what occurred, which tables have been
queried, at what time, by which service account. What they cannot let you know is
why. Why did the agent examine the sanctions record earlier than the credit score phrases? Why
did it approve regardless of a minor documentation discrepancy? What options did
it think about and reject? The hole between “what” and “why” is the place regulatory
danger arises, and the EU AI
Act
‘s Article
12
requires high-risk techniques
to maintain computerized logs for precisely this motive, so the “why” may be
reconstructed after the actual fact. Closing that hole is what agentic lineage
is for.

Agentic lineage

The way in which to shut this audit hole is agentic lineage, an extension of
conventional information lineage. The place conventional lineage tracks which sources have been
accessed, agentic lineage tracks why the agent determined to entry X, as a result of it
discovered Y in supply Z.

Concretely, for the commerce finance case, a single hint represents the
end-to-end workflow of processing letter of credit score LC-4892. Inside that hint,
every span is a person step:

  • Span 1: retrieved buyer KYC information from the compliance database, end result:
    verified.
  • Span 2: checked the sanctions record by way of the OFAC API, end result: clear.
  • Span 3: evaluated credit score phrases towards the coverage engine, end result: inside
    limits.
  • Last span: the choice, APPROVE, with a 94% confidence rating and the complete
    reasoning chain connected.

That is precisely what a regulator wants. Not “the
agent accessed the compliance database at 14:32:07 UTC” however “the
agent checked KYC first, then sanctions, then credit score phrases, and
permitted as a result of all three handed.” The traces and spans
mannequin
is borrowed instantly from distributed techniques
observability, so engineers already perceive the psychological mannequin from
instruments like Jaeger and Zipkin. For the agentic equal, Langfuse, Arize
Phoenix
, and OpenTelemetry
for AI are the rising decisions. All three function on the
Thoughtworks Expertise Radar, OpenTelemetry at Undertake, Langfuse at
Trial, and Arize Phoenix at Assess.

The regulatory enamel are actual

This is not a theoretical train. The EU AI Act is essentially the most
particular regulation on the books. Article 12 requires
high-risk AI techniques to routinely log occasions over their lifetime
so their operation may be traced, and Article
19
requires
suppliers to maintain these logs for a minimum of six months. Breaching
these record-keeping obligations falls within the Act’s center penalty
tier, as much as €15 million or 3% of worldwide annual turnover,
whichever is greater. For a big firm, even 3% of worldwide turnover
runs into the a whole bunch of hundreds of thousands.

Collectively, Articles 12 and 19 translate into three obligations on your
structure:

  • Mechanically log occasions throughout the system’s lifetime, sufficient to hint
    the way it operated, not simply remoted timestamps.
  • Retain these logs for a minimum of six months, which implies your
    observability infrastructure has to deal with long-term storage.
  • Be capable of reconstruct the “why” after the actual fact. The regulation mandates the
    logs; making them reply a regulator’s query is on you. Meaning
    capturing the complete reasoning chain, which sources have been consulted, what logic was
    utilized, and which options the agent weighed and rejected.

The EU is furthest forward, and for now no different jurisdiction has a regulation fairly
prefer it. However you do not have to guess on the place regulation lands to see the purpose.
Eventually one thing will drive the query of why an agent did what it
did, whether or not that is a regulator, an auditor, a buyer disputing a call, or
simply your individual crew attempting to debug one. The secure assumption is not {that a}
explicit regulation is coming, it is that you’re going to need to reply that query
regardless. A system you may’t clarify is one you may’t absolutely belief,
defend, or repair.

Staged autonomy

Figuring out you want audit trails is one factor; rolling this out safely is
one other. You do not deploy an agent with full autonomy on day one, any extra
than you’d hand a model new worker unrestricted entry. Autonomy is earned
in phases:

StageAgentHumanMonitoring
Shadow ModeRecommends actionsEvaluations advice and executes if acceptableAll suggestions are logged to trace accuracy over
time
SupervisedPrepares motion and waits for approvalEvaluations motion and approves or deniesAll proposed actions and human selections are logged
Autonomous with guardrailsAgent acts inside outlined boundaries (greatest drawn by
reversibility, not transaction measurement)
Defines guardrailsAll actions logged, alerts fired on exceptions
Full autonomyAgent carries out all actionsSpot checksSteady, by different brokers and people

You would not give a brand new rent the company bank card on day one. They begin
with buy requests, graduate to supervised spending, and finally earn a
card with limits. Brokers ought to earn belief the identical manner.

Promotion up this ladder ought to activate proof, not a hunch. Meaning
testing an agent earlier than every step, not solely watching it in manufacturing. Brokers are
exhausting to check. They’re nondeterministic, pricey to name, and act by means of instruments with
actual unintended effects. So groups mock or replay the instrument and mannequin interactions so assessments
run deterministically in CI. They rating the agent’s selections with evals reasonably
than calling reside providers on each run. Constructing that harness is a self-discipline of
its personal, and past the scope of this text.

Delegated entry and just-in-time credentials

As brokers earn autonomy, the query turns into, what permissions ought to they maintain?
Three safety patterns matter most right here.

  • Delegated Entry: When Alice asks the agent to examine her account, the
    agent ought to act with Alice’s permissions, not by means of a broad service account
    that may see each buyer’s information. Shared service accounts destroy attribution.
    When a regulator asks “who accessed this buyer’s information?”, “the service
    account” tells you nearly nothing. With delegated entry, the reply is “Alice’s
    agent, performing on Alice’s behalf, with Alice’s permissions.”
  • Simply-in-time Credentials: As a substitute of a persistent API key that by no means
    expires, subject a short-lived token for every particular process. The agent must
    examine the sanctions record? Challenge a token scoped to OFAC API learn entry for
    that particular buyer, legitimate for 5 minutes. When the duty completes, the
    token expires. No standing credentials sitting round ready to be
    compromised.
  • Least Privilege: The agent will get the minimal entry the duty requires.
    Processing a letter of credit score does not want attain into HR techniques or advertising
    information.

Collectively, these three patterns deal with the attribution and scope challenges
that undermine many present agentic deployments.

Additionally they defend towards the sharpest safety danger in agentic techniques. Simon
Willison calls it the deadly
trifecta
, an agent
turns harmful the second it holds all three of entry to non-public information,
publicity to untrusted content material, and a approach to talk externally. Put these
collectively and a single poisoned doc or net web page can hijack the agent by means of
immediate injection and quietly exfiltrate no matter it may possibly attain. Delegated entry,
just-in-time credentials, and least privilege shrink how a lot a hijacked agent
can attain, breaking the trifecta. Later we add a second lower on the identical drawback,
maintaining retrieved textual content out of the authorisation path fully, so {that a} poisoned
doc can not grant a permission within the first place.

The place to begin

Of the 4 subjects, that is the one the place going slowly is the best
intuition. However separate two issues which can be straightforward to conflate. Autonomy is
earned in phases, so no one expects you to grant it suddenly.
Observability shouldn’t be staged in any respect. It goes in from day one, at full
power, regardless of the autonomy degree, as a result of retrofitting it onto a
operating system is painful. What you construct on high can keep intentionally
conservative; the instrumentation beneath can not.

  • Instrument from day one. Of every thing right here, that is the one to do
    first, including observability after deployment is much more durable. Each agent workflow
    ought to emit traces with spans for every step, together with reasoning and sources
    consulted. The tracing sample right here is nicely established, so lean on a confirmed
    instrument (like
    OpenTelemetry)
    reasonably than constructing your individual.
  • Begin in shadow mode. Lowest danger, highest studying. Brokers
    suggest, people resolve. You construct the audit path earlier than you want it for
    compliance and measure accuracy earlier than granting autonomy.
  • Implement delegated entry. Brokers inherit the invoking person’s
    permissions and use just-in-time credentials with brief expiry home windows. No
    persistent tokens.
  • Construct to be explainable. Whether or not or not a regulator ever asks, an
    audit path that solutions “why” is what helps you to debug a foul resolution, defend a
    good one, and belief the system sufficient to widen its autonomy. Wire it in now,
    it’s miles more durable so as to add later.

Semantic layers bridge the institutional information hole between brokers and
human analysts, constructing on trusted information and auditable actions supplied by the
earlier subjects.

The Context Layer: Instructing Brokers What Your Knowledge Means

Semantic layers present the specific context AI brokers want after they grow to be
the first customers of information, context that human analysts carry implicitly,
based mostly on years of expertise.

Your agent does not know what “income” means

Ask an agent, “What was Q3 income for Product X?” A human analyst is aware of
exactly what to do, which desk to question, whether or not income means gross or web,
what Q3 maps to in your fiscal calendar. They absorbed all of it over years of
institutional information, tribal docs, and Slack threads.

The agent has none of it. It does not know which joins join merchandise to
orders to income, or that your fiscal calendar begins in February. With that
context lacking, it both hallucinates a solution or provides up. The semantic
layer fills that hole, supplying the business-domain context.

What the context layer is

A semantic layer is a set of declarative definitions of your metrics, how
income is calculated, what an lively buyer is, what the numbers imply. Each
client goes by means of the identical definitions, so all of them derive constant,
correct outcomes. However an agent that acts wants greater than definitions of numbers.
It must know what the issues are, and what it could do to them. These are
three separate our bodies of definition, and an agent wants all three.

The area mannequin says what exists. Entities, their relationships, and
the which means guidelines of the enterprise: an order belongs to a buyer, an lively
buyer is one who bought within the final ninety days. It provides the agent the
vocabulary to interpret a request and plan towards it. It’s consulted, by no means
executed; no question path to information runs by means of it.

The semantic mannequin says how the numbers are computed. Metrics and
dimensions, one versioned components every, compiled to the identical SQL each time and
run towards the analytical retailer. That is the semantic layer beneath a extra precise
title, and the job is to place correctness within the compiler reasonably than within the
mannequin’s guess.

The functionality mannequin says what the agent might do. A curated set of
operations towards reside techniques, some that learn (examine cost standing, retrieve
a troubleshooting information) and a few that write (subject a refund). Every carries
permissions and an proprietor, and the performing ones carry preconditions and a
reversibility class as nicely.

Nouns, numbers, and verbs. Collectively they’re the context layer, and
what unites them shouldn’t be that they’re all about which means, as a result of the aptitude
mannequin plainly shouldn’t be. It’s that every one is a spot the place a assure is
declared as soon as, in model management, as an alternative of being labored out afresh by the
mannequin on each request. The definitions are the layer; the interface, MCP immediately,
is simply the door.

A reader who works with dbt will object that its
semantic fashions already declare entities, so why
separate the area mannequin out. As a result of entities declared contained in the metrics layer
are scoped to metrics, and the aptitude mannequin must be written in the identical
vocabulary because the semantic one or the 2 drift aside. A refund acts on the identical
buyer the income determine counts. One vocabulary beneath, otherwise you get two.

Determine 2: The context layer: a site mannequin of entities and
relationships, a semantic mannequin of metrics compiled to SQL towards the analytical
retailer, and a functionality mannequin of guarded reads and actions towards reside techniques,
with provenance alerts throughout all three. The area mannequin has no arrow out
as a result of it’s consulted reasonably than executed; the opposite two are written in its
vocabulary. Dashboards and analysts attain the semantic mannequin; brokers are the primary
client to want all three, which is the shift this text is about.

All three fashions are code in supply management. They undergo code evaluations, get
examined in CI, and progress by means of environments earlier than reaching manufacturing. When
the definition of “income” or the rule on refunds adjustments, you alter it in a single
place and it propagates in all places. Brokers by no means attain the underlying information
instantly; they undergo the context layer, which constrains and governs each
what they’ll ask for and what they’ll do.

Metrics as code

In observe, the enterprise logic lives proper within the definition, income =
order_amount - discount_amount
, not buried in a BI instrument or an advert hoc SQL view.
The agent receives a pure language query, and the semantic mannequin resolves
it to right, constrained SQL. The agent does not guess desk names or be a part of
paths; it makes use of the definition.

The examples right here use dbt
MetricFlow
syntax (dbt is
mid-migration from measures to a metrics-first spec; the widely-used type is
proven right here, and the idea holds both manner). Dice.js, Snowflake, and
Databricks all observe related patterns. The instrument issues lower than the
self-discipline of getting enterprise logic into model managed code.

semantic_models:
  - title: orders
    mannequin: ref('orders')
    defaults:
      agg_time_dimension: order_date
    entities:
      - title: order_id
        kind: major
      - title: customer_id
        kind: international
    dimensions:
      - title: order_date
        kind: time
        type_params:
          time_granularity: day
    measures:
      - title: income
        agg: sum
        expr: order_amount - discount_amount
        create_metric: true

Identical query, very completely different SQL

Let’s think about an instance. Ask “What was Q3 income for Product X?” of an agent
with out a semantic mannequin, and it guesses at desk names, makes use of the unsuitable
column, has no fiscal-calendar mapping, and misses the be a part of.

-- Earlier than metric definition
SELECT SUM(quantity)
  FROM sales_data
 WHERE product = 'Product X'
   AND quarter = 'Q3'

Ask the identical query with a semantic mannequin, and the agent is constrained
to the proper desk, the net-revenue components from the YAML definition, the
proper fiscal-calendar dates, and the legitimate be a part of path.

-- Constrained by metric definition
SELECT SUM(order_amount - discount_amount)
  FROM orders o
  JOIN merchandise p
    ON o.product_id = p.id
 WHERE p.title = 'Product X'
   AND o.order_date
       BETWEEN '2025-07-01'
           AND '2025-09-30'

The semantic mannequin does not make the agent smarter. It stops it from
guessing. For an agent that acts on the reply unchecked, that is what
issues.

How brokers use it

Take the semantic mannequin by itself, the trail a quantitative query travels.
Finish to finish, the movement appears to be like like this. The agent sends a natural-language
query (step 1). The semantic mannequin appears to be like up metric definitions, legitimate
dimensions, be a part of paths, and entry guidelines, by way of MCP (step 2), then generates
constrained SQL (step 3), each inside the identical element. The information warehouse
executes the question (step 4). The end result flows again to the agent with full
lineage metadata (step 5).

Determine 3: One of many three paths: a quantitative query
answered by means of the semantic mannequin. Questions on what issues are go to the
area mannequin, and reads or actions towards reside techniques undergo the aptitude
mannequin.

Brokers decide from ruled metrics, by no means uncooked tables they’ll
misinterpret

The semantic mannequin constrains what the agent can ask for. dbt’s, for
occasion, dynamically surfaces solely the size relevant to the chosen
metrics, which prevents the agent from producing believable sounding however
incorrect queries. And that lineage metadata in step 5 is the inspiration for the
traceability we coated earlier. Context and traceability reinforce one
one other.

The place to begin

The temptation with a context layer is to mannequin the entire enterprise earlier than
you ship something. Resist it. Begin with the semantic mannequin, as a result of the worth is
concentrated in a handful of metrics, the contested ones that imply completely different
issues to completely different groups. Let your first agent use case set the scope, and develop
the area mannequin and the capabilities it truly wants reasonably than those you
can think about. A slim, right context layer beats a sprawling, half-agreed
one.

  • 1. Discover your conflicting metric definitions. Most organizations have
    a number of definitions for his or her most essential metrics, income being the basic,
    with its gross vs web, with or with out returns variations. These conflicts are
    your greatest agent danger and your quickest win.
  • 2. Decide a instrument, however concentrate on the self-discipline. Any mainstream semantic
    layer instrument will do; what issues is the self-discipline behind it, metric definitions
    in model management, one agreed definition per metric, and brokers querying
    by means of the layer, not the uncooked schema.
  • 3. Route brokers by means of the context layer, by no means the uncooked schema. The
    agent ought to see ruled metrics and dimensions, not uncooked tables and joins. MCP
    is the frequent approach to expose the layer immediately, and dbt, Dice, and AtScale all ship
    MCP servers, however the precept holds nevertheless you join, the purpose is the
    abstraction, not the protocol.
  • 4. Check adversarially. The easiest way to seek out gaps is adversarial testing,
    each hallucination factors to a lacking definition. Repair the definition, not the
    immediate. And do not boil the ocean, begin with the metrics your first agent use
    case wants.

Traversing the area mannequin: information graphs

The semantic mannequin shines for structured metric queries akin to “what was
income by area.” However some agent duties demand richer relationship reasoning
throughout entities, occasions, and time. Contemplate a buyer who purchased Product X, then
churned after a pricing change. A hard and fast variety of hops like that’s an abnormal
be a part of. What flat tables deal with badly is traversal whose depth you do not know when
you write the question, following a sequence of relationships till you discover what you
are searching for. That’s the area mannequin’s territory, the entities and the way they
join.

The frequent approach to retailer and traverse that map is a information graph, which is a
storage selection for the area mannequin reasonably than a fourth factor to construct.
GraphRAG from Microsoft makes use of group detection to
deal with summary queries that conventional RAG cannot, and
Graphiti builds temporally conscious information graphs
for evolving details. (Each sat at Trial on the Thoughtworks Radar as of
2026.) The semantic mannequin nonetheless defines the metrics; the graph carries the
connections between prospects, merchandise, occasions, and selections over time.
Collectively they provide brokers one thing near institutional reminiscence, the sort of
information that may take a brand new rent months to soak up.

Now brokers have trusted information, governance, and context. However can they
truly act?

From Searchable to Actionable: Agent-Prepared Knowledge Entry

As soon as brokers perceive your information and governance is in place, the query
shifts to entry. How do brokers attain the information and act on it? The
reply is greater than “RAG”. It is a full spectrum, from retrieval, to real-time
queries, to managed write-back actions. That entire spectrum is the
functionality mannequin, the third of the three, and the write-back finish is the place its
guardrails earn their maintain.

Your agent can learn, however it may possibly’t act

Let’s take an instance. An worker studies a PO (buy order) subject. An
very best agent would do three issues, retrieve the related troubleshooting information,
examine whether or not the PO cost service is down proper now, and create a
assist desk ticket if wanted.

Conventional RAG, the sample most organizations have deployed, solely does step
one. It searches paperwork and retrieves content material. It will possibly’t question a reside
monitoring system to examine service standing, and it actually cannot create a
ticket in ServiceNow or Jira. That hole between searchable and actionable is
the topic of this ultimate subject, and we are going to use the PO state of affairs to
elaborate.

The information entry spectrum

This framing comes from Microsoft’s Cloud
Adoption Framework for AI
, which formalizes it as RAG + MCP-Learn +
MCP-Write.

  1. Retrieval. RAG, vector search, doc lookup. The agent finds
    related content material. Most organizations reside right here immediately.
  2. Actual-Time Question. The agent queries reside techniques,
    checks service standing, reads from databases in actual time.
  3. Write-Again. Probably the most highly effective and most harmful tier. The agent
    creates tickets, updates information, triggers workflows.

Every step up the spectrum provides functionality, and danger. The PO state of affairs maps
cleanly throughout all three.

  • Retrieve the information (Retrieval)
  • Test cost standing (Actual-Time Question)
  • Create the ticket (Write-Again)

The shift to agentic AI requires all three, not simply the retrieval
most groups have constructed.

MCP has rapidly grow to be the default approach to wire these tiers up, and its rise
has been remarkably quick. However the mechanism issues lower than the demarcation.
What counts is maintaining retrieval, real-time reads, and write-back as separate,
intentionally ruled ranges of entry, whether or not you expose them by means of MCP or
your individual native APIs.

Three primitives, one protocol

Brokers attain all of this by means of MCP, the Mannequin Context Protocol. Its
primitives sit on a danger gradient, Sources (read-only) are secure, Prompts form
conduct, and Instruments change state. That gradient maps straight onto the tiers,
Sources to retrieval and Instruments to write-back, which is why the secure path is to
expose Sources first and graduate to Instruments solely beneath governance. Within the PO
state of affairs, Sources serve the troubleshooting docs, a Immediate guides triage, and
Instruments run check_service_status() and create_support_ticket().

Antipattern: naive API-to-MCP conversion

The way you design these Instruments issues as a lot as while you attain for them. The
frequent, pricey mistake is to take current REST APIs and wrap them one-to-one,
so each endpoint turns into a instrument. The result’s instrument sprawl, 50 instruments with
names like get_po_payment_status, create_ticket_po_payment,
create_ticket_po_payment_network. The agent then has to decide on amongst 50
barely-distinguished instruments with little context, and LLMs are unhealthy at that;
accuracy drops sharply because the instrument depend climbs. The Thoughtworks Tech Radar put
“naive API-to-MCP
conversion”

on HOLD for precisely this motive.

The higher method exposes the identical performance as a handful of
well-designed capabilities with wealthy descriptions and parameterized inputs.
check_service_status takes a service title and site, one instrument for all
providers and all areas. create_support_ticket is parameterized with
class, precedence, and outline. The descriptions are detailed sufficient for
the LLM to know when to achieve for every one.

5 to 10 nicely
described enterprise capabilities will outperform 50 skinny API wrappers nearly
each time

The precept is to design capabilities, not endpoints. 5 to 10 nicely
described enterprise capabilities will outperform 50 skinny API wrappers nearly
each time. And this precept is protocol-agnostic, whether or not an agent reaches
your information by means of MCP, by means of one other agent, or by means of no matter normal comes
subsequent, the properties that make it agent-ready are the identical, wealthy descriptions,
parameterized entry, clear schemas.

What a functionality declares

A wealthy description tells the agent when to achieve for a functionality. It says
nothing about whether or not the agent is allowed to, or what occurs whether it is unsuitable.
That’s the remainder of the declaration. Each functionality carries
permissions, who might invoke it and performing as whom, and an proprietor,
the particular person accountable when it misbehaves. Those that act carry two extra.

Preconditions are the situations that should maintain earlier than the motion might
proceed, checked towards reside state in the mean time of performing reasonably than towards
regardless of the agent learn earlier in its plan. A refund wants an unique cost,
not but refunded, inside the quantity the invoking person might authorise.

Reversibility is the category of injury the motion can do: cleanly
reversible, reversible at a value by means of some compensating transaction, or
irreversible. That is the extra helpful predictor of secure autonomy than the cash
concerned. A $50,000 inner ledger correction you may again out is a safer factor
to automate than a $200 cost to an exterior account you can not claw again. The place
the staged autonomy ladder earlier keys its guardrails to transaction measurement, choose
keying them to reversibility, and let irreversible actions require human approval
no matter stage the agent has reached.

Reversibility predicts secure autonomy higher than the scale of the
transaction

Which raises the query of the place the principles in these preconditions come from,
as a result of most of them are written down in prose someplace, in a refund coverage, a
contract, a compliance handbook.

Retrieved textual content informs, it by no means gates

Enterprise paperwork stay the place the enterprise writes its guidelines down. However a rule
that gates an motion should not be learn and interpreted in the mean time of performing.
Guidelines are extracted from these paperwork forward of time, curated by a human, and
saved as declared preconditions within the functionality mannequin, every with a hyperlink again
to the passage it got here from.

At motion time the agent should still learn unstructured content material, a grievance
ticket, a contract clause, to work out what to suggest. Solely the declared guidelines
resolve what’s permitted, and they’re checked deterministically towards reside
state. The boundary is between informing and gating. Retrieved textual content
can form what the agent suggests and function proof for a human approver, however
it by no means carries the authority to authorise the motion itself.

That boundary can be a safety property. Eradicating retrieved textual content from the
authorisation path means a poisoned doc can not grant an agent a permission
it didn’t have already got, which is a stronger declare than merely shrinking what a
hijacked agent can attain. It isn’t an entire defence, as a result of injected textual content can
nonetheless affect what the agent proposes, and a human approver proven fabricated
proof might wave it by means of. What it removes is the trail the place the doc
authorises the motion instantly, with no one in between.

The provenance hyperlink is what retains the declarations sincere because the paperwork
transfer beneath them. Watch out what you promise right here. Detecting {that a} doc
modified is simple; figuring out that the change invalidated a precondition derived from
it’s a judgement, not a diff. What the hyperlink buys you is a evaluate queue, the
derived guidelines flagged for a human to re-check when their supply strikes, within the
identical spirit as keying a freshness SLA to when the index was final rebuilt reasonably
than to when the content material final appeared to vary.

The place no declaration covers the scenario, the agent doesn’t improvise from
its personal studying of coverage. It escalates. That is the exhausting gate from earlier in a
completely different setting, the identical intuition that claims any contract or SLA breach forces
a human reasonably than a decrease rating. An undeclared case degrades the agent to
supervised, to not autonomous.

Extraction and curation is a pipeline like some other, and it wants an proprietor,
a cadence, and any individual who clears the evaluate queue. Which is the topic of a
later part, as a result of none of this maintains itself.

Finish to finish: the PO cost state of affairs

With all three tiers in place, the PO subject we opened the part with runs
finish to finish, the agent retrieves the troubleshooting information (a read-only
Useful resource), checks the reside cost standing (a Device that reads), and recordsdata a
ticket (a Device that writes), all in a single workflow.

Determine 4:
One agent, three tiers: retrieval, real-time question, then write-back,
mixed right into a single response.

Executed manually, the worker would wait in a queue, clarify the problem, have a
assist agent examine the monitoring dashboard, and get a ticket created. The
agent is now capable of do all this in a single cross.

The place to begin

The secure manner in is to climb the tiers, not leap to write-back. Most groups
already reside in retrieval, the read-only tier the place danger is lowest. Write-back
is the place the true hazard sits. So earn your manner up. Map what every use case
wants, expose read-only entry first, and add write-back final, solely
as soon as you may log each motion. Do not let the fun of an agent that may act
rush you previous the steps that make performing secure.

  • 1. Map your information entry tiers. Take your high three agent use circumstances and
    classify what every wants, retrieval, real-time question, or
    write-back. Most gaps reside in real-time question and write-back.
  • 2. Design capabilities, not endpoints. Group current APIs into 5–10
    well-described enterprise capabilities. Wealthy descriptions matter, they’re what the
    LLM makes use of to resolve which instrument to name.
  • 3. Begin with MCP Sources. Learn-only entry is the bottom danger entry
    level. Expose information bases, config information, and documentation as Sources.
    Graduate to Instruments solely as soon as governance is in place.
  • 4. Instrument from day one. Earlier than deploying any agent with write
    entry, log each instrument invocation, who triggered it, what was known as, when, and
    critically, on whose behalf. This feeds the audit path from the Traceability
    and Governance part.

The AI-ready information stack

We have now walked by means of all 4 subjects, contracts that make information trusted,
a context layer that makes it significant and actionable, entry patterns that allow brokers act
on it, and observability that makes these actions auditable. Handled individually,
they seem like 4 work streams you possibly can employees independently. However they don’t seem to be
impartial. They construct on each other, and the order during which they’re constructed
issues.

Determine 5: The AI-ready information stack: three dependent
layers constructed bottom-up, with observability slicing throughout all of them
from day one.

The dependencies run bottom-up. You may’t connect which means to information you may’t
belief, so context sits on the inspiration. You may’t safely let brokers act
with out that which means to constrain them, so entry sits on context. Skip both
of these and every thing above it collapses. That is precisely why so many agentic
AI applications stall. They bounce straight to agent entry with out constructing the
basis beneath. Observability is completely different. Reasonably than a fourth tier
stacked on high, it runs alongside all three. Each layer must be traceable and
auditable from the second it handles actual work. The belief checks, the semantic queries, the agent’s
actions, all of it must be explainable in manufacturing, not everytime you get
round to instrumenting it. It’s also a lot more durable to retrofit onto a operating
system than to construct in from the beginning. Both manner, you wire it in from day
one.

Who owns all this?

The stack has yet one more dependency the diagram cannot draw. Each layer in it
produces an artifact that must be saved true, a knowledge contract, a metric
definition, an entry scope, an observability hint. Artifacts do not keep
themselves. A contract with no proprietor drifts out of sync with the supply it
describes. A definition of “income” with no proprietor forks again into the three
conflicting variations you simply consolidated. An entry scope with no proprietor
quietly widens till it is a standing service account once more. The expertise is
vital, however it’s the working mannequin that retains it sincere.

The self-discipline that makes this work is treating information as a product. Every
dataset, contract, and metric has a named proprietor, a printed contract and SLA,
and a versioned lifecycle, the identical manner an API does. You will not all the time know each
client, and for public or broadly shared information you may’t, which is exactly
why the contract issues, it is the secure promise unknown customers construct on,
and a deprecation coverage is how you alter it with out breaking them. When the
product_pricing contract blocks a deployment at 2 a.m., somebody is accountable
for it. When finance and gross sales disagree on “income,” somebody owns the choice.
When a brand new agent asks for entry, somebody owns the scope and evaluations it. These
aren’t infrastructure questions; they’re possession questions, and no instrument
solutions them for you.

A human client of an unowned, drifting dataset notices and works round it.
An agent consumes it at machine velocity and scale, and propagates the error simply
as quick. The sooner and extra autonomous your customers, the much less you may afford
information with out an proprietor.

The place do you stand?

Earlier than deciding what to construct, it helps to find your self. Rating every
attribute towards the alerts beneath, all drawn from the subjects above.

AttributeHuman-eraIn TransitionAgent-ready
TrustedUnfastened schemas, no freshness SLAs; high quality rests on an
analyst noticing when a quantity appears to be like off
Contracts on just a few essential datasets; high quality checked
however not enforced in CI/CD.
Contracts enforced as code, freshness SLAs per client,
quarantine earlier than agent storage, brokers learn Gold solely (tables and
embeddings)
ContextualMetric definitions reside in BI instruments, SQL, and other people’s
heads; people provide the context
Some metrics outlined as code, however definitions nonetheless
battle and brokers should still hit the uncooked schema
A context layer in Git: entities and relationships in a site
mannequin, one semantic definition per metric, and a curated set of capabilities;
brokers route by means of it, by no means the uncooked schema
TraceableLogs present what an individual queried and when; the why lives in
the analyst’s head
Traces on some agent workflows; reasoning captured
inconsistently
Each agent workflow emits traces with spans, reasoning,
and sources; any resolution’s “why” is reconstructable
RuledFolks entry information by means of their very own roles; techniques share
broad service accounts
Brokers run on scoped however long-lived, coarse credentialsDelegated per-user entry, just-in-time credentials, least
privilege; lethal-trifecta paths closed
OperationalNo agent acts on the information; folks learn dashboards and take
actions by hand
Brokers retrieve by way of RAG; real-time reads rising;
write-back experimental or ungoverned
All three tiers by way of well-designed capabilities; write-back
gated by staged autonomy and instrumentation

Do not common the rows, as a result of the stack is dependency ordered,
your readiness is capped by your weakest foundational layer, a
flawless context layer sitting on untrusted information continues to be not agent
prepared. Discover your weakest row, and that is the place the following funding
goes.

4 issues to begin on

Every subject got here with its personal beginning factors. Deal with these as tactical
checklists for the work itself. The 4 beneath are the place to begin. The primary,
instrumenting from day one, is not a build-order step. It runs alongside
every thing else, which is why it comes first and by no means stops. The opposite
three construct from the underside of the stack up, since you’re solely as prepared as your
weakest foundational layer. The best-leverage single transfer amongst them is the
context layer, since context strikes accuracy additional than an even bigger mannequin does,
however it solely pays off as soon as the information beneath it may be trusted. Construct as much as
it.

  1. Instrument from day one. This is not a step within the sequence a lot as a
    fixed that runs beneath all of them. Put traces and spans in each workflow
    from the beginning, as a result of observability is much more durable to retrofit than to construct
    in, and you will need audit trails that reply “why” for debugging immediately and
    regulators tomorrow.
  2. Contract every thing. Freshness SLAs, strict schema enforcement,
    quarantine for unhealthy information. That is the ground the remaining stands on, brokers cannot
    odor unhealthy information, so the information structure has to odor it for them.
  3. Context over fashions. As soon as the information may be trusted, a context layer
    is the highest-return factor you may construct on high of it. Its semantic mannequin
    alone carries the purpose: in AtScale’s
    text-to-SQL
    benchmark
    ,
    accuracy jumped from beneath 20% on the uncooked schema to over 92.5% with a semantic
    layer, on the identical mannequin.
  4. Learn earlier than write. Begin with MCP Sources (read-only) and graduate
    to Instruments (write) solely with governance in place. Earn autonomy in phases, shadow
    mode, then supervised, then autonomous with guardrails.

When brokers grow to be the first customers of your information, your information structure
turns into your AI structure.

We go a lot deeper on all of this, and on the broader operational and
analytical information structure selections round it, in our forthcoming O’Reilly
e book, Knowledge Structure for Software program
Architects
.


Related Articles

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Latest Articles