Think about two startups constructing related merchandise. Each select Claude Sonnet 5 and ask it to automate the processing of incoming buyer requests. One month later, the primary agent classifies inquiries, retrieves the suitable knowledge, and completes duties constantly. The second hallucinates lacking particulars, loses monitor of the unique request, and falls right into a loop after two steps. The mannequin is identical. The outcomes should not.
This hole helps clarify why AI tasks fail even when groups begin with a succesful LLM. Gartner predicts that greater than 40% of agentic AI tasks will probably be canceled by the tip of 2027 due to escalating prices, unclear enterprise worth, or insufficient danger controls. For firms exploring agentic AI for enterprise, mannequin choice is barely a part of the choice.
So what creates the distinction? It’s not merely the intelligence of the mannequin, however the system during which it operates, the harness. At SCAND, we now have designed and shipped agentic programs throughout industries, and we see this sample constantly: the LLM isn’t the issue. The design, structure round it nearly all the time is.
Why AI Brokers That Work within the Demo Break in Manufacturing
A demo exhibits that an AI agent can full a predefined activity underneath managed circumstances. The system normally receives a transparent request, works with a restricted set of knowledge, and makes use of instruments whose conduct is already recognized. Just a few profitable situations can create the impression that the agent is prepared for a full-scale launch, however real-world operation rapidly exposes its weaknesses.
In manufacturing, consumer requests could also be incomplete, ambiguous, or contradictory. Knowledge turns into outdated, context home windows are overflown, APIs return sudden responses, entry permissions change, and exterior companies turn into briefly unavailable. The extra steps an agent should full, the better the prospect that an error at one stage will have an effect on each motion that follows. On account of Context Rot or Context Fatigue.
The issue is amplified by the truth that AI brokers don’t function like deterministic applications. With the identical enter, conventional code normally follows a predefined sequence of actions. An LLM selects the following step probabilistically, which implies that the identical mannequin with the identical immediate might use completely different instruments, interpret retrieved knowledge otherwise, and produce completely different outcomes.
In observe, three varieties of failure happen significantly usually. Dumb RAG supplies the agent with incomplete, irrelevant, or outdated context. Brittle Connectors disrupt execution when APIs, parameter schemas, permissions, or response codecs change. Compounding Error causes errors to build up: an incorrect choice enters the context, influences the following step, and progressively strikes the agent away from the unique activity.
These points might stay hidden in a demo. In manufacturing, they recur throughout a a lot wider vary of situations and switch remoted deviations into systemic failures. This is the reason LLM agent reliability relies upon not solely on the capabilities of the LLM but in addition on how context, instruments, validation, and constraints are designed round it.
What Truly Determines How Your AI Agent Performs
An AI agent’s efficiency is dependent upon greater than the LLM it makes use of. Even a extremely succesful mannequin won’t deal with multi-step duties reliably if it receives irrelevant context, selects the improper instruments, loses essential data between actions, or fails to acknowledge when the method ought to cease.
These facets are managed by way of harness design — the engineering layer across the LLM. It determines what knowledge, context the mannequin receives at every stage, how the agent interacts with APIs and exterior programs, the place it shops the duty state, the way it handles errors, and what constraints it should comply with. In observe, the harness turns a standalone mannequin right into a manageable agentic system that may function inside actual enterprise processes.
Harness design is broader than immediate engineering. A immediate defines the agent’s position, objective, and behavioral guidelines, however it doesn’t management the complete execution circulate. It doesn’t determine which knowledge must be retrieved, whether or not a instrument response could be trusted, what number of occasions a failed motion must be retried, if circulate is safe, when compaction begins, or when human approval is required. All of this belongs to agentic system design.
That is particularly essential in multi-step situations. For instance, the ReAct agent structure for LLM programs follows a cycle of reasoning, motion, and statement. Nonetheless, the sample alone doesn’t assure reliability. With out management over context, reminiscence, instrument calls, and stopping circumstances, the agent might select the improper motion, enter a loop, or progressively drift away from the unique objective. The standard of the system subsequently relies upon not on a single immediate, however on how rigorously the mannequin’s work is managed at each stage.
The Elements of Agentic System Design That Truly Matter
The reliability of an AI agent doesn’t rely on a single expertise. It is dependent upon how properly all of the parts surrounding the mannequin work collectively. The weather under type efficient AI agent design patterns and distinguish a steady manufacturing system from a formidable however fragile prototype.
Reasoning Structure
That is the route an agent follows to achieve a consequence. ReAct alternates between reasoning, motion, and statement, whereas PlanReAct introduces an preliminary strategy planning stage for extra complicated duties. With out this construction, an agent might instantly take the primary obtainable motion, lose sight of the unique goal, and start making disconnected selections.
Reminiscence Layer
Reminiscence serves because the agent’s working pocket book. It shops the present activity state, essential outcomes from earlier steps, and data from earlier classes. RAG, information, databases, and devoted state shops might all assist this layer. With out managed reminiscence, each new request feels nearly like the primary one, and important context progressively disappears.
Expertise administration
Expertise administration defines which capabilities an agent can use and the way they need to be utilized. Every talent might embody directions, instruments, and validation guidelines for a particular activity. With out clear talent administration, the agent might choose unsuitable instruments, repeat actions, or full the identical activity inconsistently.
Subagent Administration and Handoffs
In multi-agent system design, specialised brokers resemble departments throughout the similar firm, with every one liable for a selected space of labor. AI agent orchestration determines which agent receives a activity, when it ought to move the consequence to a different agent, and the way separate outputs are mixed. With out clear coordination, brokers duplicate work, use conflicting variations of knowledge, or ignore each other’s findings.
Loop and Execution Finances Limits
These limits act as an emergency cease that forestalls the system from repeating an unsuccessful motion indefinitely. They might cowl the variety of steps, retries, instrument calls, execution time, and value. With out them, an agent can turn into trapped in a loop involving a paid API or proceed working after it’s not making progress.
Guardrails and Entry Permissions
Guardrails and permissions perform as security insurance policies and entry controls. They outline which knowledge the agent might view, which instruments it could use, and which actions it’s approved to carry out. With out these restrictions, a mannequin error may end in a deleted file, an e mail despatched to the improper recipient, or an operation the agent ought to by no means have been allowed to execute.
Approval Administration
Approval administration introduces checkpoints earlier than vital actions. An agent might acquire data and put together a choice independently, however it ought to request human affirmation earlier than making a cost, deleting knowledge, or sending an official doc. With out a human within the loop, the system receives an excessive amount of autonomy in conditions the place the price of an error is particularly excessive.
Context and Instrument Schema Administration
The context window is just like a workspace: when it turns into overcrowded, essential data is more durable to determine. Context compaction removes repetition and retains probably the most related particulars, whereas clear instrument schemas clarify how every instrument must be used. With out them, the agent turns into confused by collected context and generates incorrect API calls.
Logging and Tracing
Logging data the agent’s whole execution path, together with the context it obtained, the actions it chosen, instrument responses, and the rationale the method ended. With out this historical past, the group sees solely the wrong final result and can’t determine the place the failure started. Diagnosing the agentic system then turns into guesswork, whereas recurring issues stay unresolved.
What Good Agentic System Design Seems Like in Apply
In observe, a strong agentic AI implementation doesn’t start with connecting an LLM. It begins with defining how the agent will obtain context, retain activity state, choose instruments, and reply to sudden conditions.
This may be illustrated by an AI agent constructed by SCAND for an actual property platform. As an AI agent improvement firm, SCAND needed to handle not solely the mannequin layer but in addition reminiscence (context layer), instrument orchestration, entry controls, and monitoring.
One of many fundamental challenges in such a system is reminiscence. Passing the complete dialog historical past to the mannequin with each request rapidly fills the context window and will increase processing prices. Storing too little data creates the alternative downside: the agent forgets beforehand outlined circumstances and begins requesting the identical knowledge once more. The AI agent reminiscence structure subsequently must protect activity state, compress lengthy histories, and supply the mannequin with solely the knowledge related to the present step.
Instrument administration is equally essential. When an agent has entry to dozens of capabilities, it should decide which instrument to make use of, in what order to carry out actions, and tips on how to interpret the response. Orchestration and expertise administration constructions this course of and helps forestall the agent from choosing an unsuitable instrument, repeating a accomplished step, or counting on conflicting knowledge.
AI agent guardrails on this state of affairs transcend filtering outputs. They limit entry to the system, outline permitted actions, and make the agent’s work extra clear. Customers and directors ought to have the ability to perceive which instruments are being known as and the place further oversight is required.
A separate monitoring layer data the agent’s actions, instrument responses, and adjustments in context. This permits the group to research not solely the ultimate reply but in addition the complete path that led to it. That’s particularly essential in lengthy workflows, the place an error might happen a number of steps earlier than the wrong consequence turns into seen. Monitoring permits alternatives for self-evolving characteristic assist.
This instance exhibits that agent reliability comes from the mix of reminiscence, orchestration, constraints, and observability. The LLM stays a central element, however system stability is dependent upon how constantly its work is managed across the mannequin.
Why Harness Design Is a Enterprise Determination, Not Simply an Engineering One
The structure round an AI agent determines not solely the standard of its outputs but in addition the extent of operational danger. If the system doesn’t limit the mannequin’s actions, a single mistake might end in an e mail being despatched to the improper recipient, deleted knowledge, repeated calls to a paid API, or a choice primarily based on unverified data. Guardrails, entry permissions, and obligatory approval for vital operations should subsequently replicate the enterprise course of and the potential value of failure.
For many firms, predictability issues greater than most execution pace. An agent can’t turn into a part of a daily workflow if, underneath the identical circumstances, it sometimes selects a special instrument, skips a validation step, or fails to finish the duty. Harness design doesn’t make an LLM totally deterministic, however it narrows the vary of acceptable conduct by defining execution sequences, limits, validation standards, escalation guidelines, and stopping circumstances.
The return on funding additionally is dependent upon structure. Enterprise worth doesn’t come merely from utilizing a extra succesful mannequin, however from making certain that the agent interacts reliably with knowledge, individuals, and enterprise programs. Trade analysis identifies orchestration, centralized governance, role-based entry, tracing, and human-in-the-loop controls as important for scaling AI brokers. In a single enterprise use case, agentic workflows lowered improvement cycles by as much as 60% and reduce manufacturing errors by half.
Harness design ought to subsequently not be handled as a one-time configuration earlier than launch. It’s an ongoing engineering self-discipline, just like DevOps and safety — constructed into the system from the beginning and repeatedly revised as instruments, knowledge, dangers, and enterprise necessities evolve.
The place to Begin Earlier than Constructing an AI Agent
Earlier than improvement begins, step one is to decide on a particular enterprise course of. The group ought to outline the duty the agent will carry out, the anticipated final result, and the conditions during which the agent shouldn’t be used. A broad objective comparable to “automate buyer assist” just isn’t sufficient; the preliminary scope have to be clear and measurable.
The subsequent step is to evaluate the price of failure. This determines the place the agent might act independently and the place its output have to be reviewed, permitted, or escalated to a human. The upper the potential impression of an error, the narrower the agent’s autonomy must be.
High quality standards also needs to be established earlier than implementation. These might embody response accuracy, execution time, value per activity, acceptable error charges, and circumstances underneath which the method should cease. With out predefined metrics, it’s tough to differentiate a profitable agent from a convincing demo.
Lastly, the system must be launched by way of a restricted pilot primarily based on actual situations. Testing ought to embody incomplete knowledge, ambiguous requests, unavailable APIs, and sudden instrument responses. Solely after the agent behaves constantly underneath these circumstances ought to its obligations and entry be expanded.
The Actual Variable in AI Agent Efficiency
Selecting the best LLM issues, however it doesn’t assure that an AI agent will work reliably in manufacturing. Variations between GPT-5, Claude, and different fashions can have an effect on reasoning high quality, pace, and value, but the ultimate consequence relies upon largely on the system constructed across the mannequin.
Reliability is formed by the standard of context, reminiscence structure, tool-use guidelines, intermediate validation, autonomy limits, and stopping circumstances. A highly effective LLM can’t compensate for a poorly structured course of, whereas considerate system design makes agent conduct extra constant, predictable, and controllable.
When constructing an agentic system, firms ought to consider not solely the mannequin’s capabilities but in addition the way it will function inside actual enterprise processes. SCAND supplies AI agent improvement companies for firms that must design, combine, and launch dependable agentic programs.
Regularly Requested Questions (FAQs)
What Is Harness Design in AI Brokers?
Harness design is the engineering layer round an LLM that manages context, reminiscence, instruments, entry permissions, error dealing with, and stopping circumstances. It determines how the mannequin interacts with exterior programs and the way predictably the agent operates in real-world processes.
Why Do AI Brokers Fail in Manufacturing If They Work within the Demo?
A demo normally covers a restricted set of ready situations. In manufacturing, an agent encounters incomplete requests, outdated knowledge, API failures, sudden instrument responses, and lengthy execution chains. These circumstances expose weaknesses in reminiscence, orchestration, validation, and system constraints.
What Is the Distinction Between ReAct and PlanReAct Structure?
ReAct alternates between reasoning, motion, and statement at every stage. PlanReAct provides an preliminary planning step earlier than execution begins. ReAct is appropriate for comparatively easy sequential processes, whereas PlanReAct works higher for multi-step situations that contain uncertainty and require the plan to be adjusted.
How Can I Make My AI Agent Extra Dependable?
Outline clear autonomy limits, construction reminiscence and context, management instrument utilization, and validate intermediate outcomes. Reliability additionally is dependent upon execution and value limits, human approval for vital actions, and logging and tracing for each run.
Do I Want a Totally different LLM to Get Higher Outcomes From My AI Agent?
Not all the time. A extra succesful mannequin might enhance reasoning high quality, however it won’t repair issues with context, reminiscence, instruments, or stopping circumstances. Your entire system across the LLM must be assessed first. In lots of circumstances, enhancing the structure has a better impact than switching fashions.