In late August 2026, the Military’s chief of Community Command (NETCOM) introduced that to take full benefit of synthetic intelligence (AI), the Military wanted a complete simulation of its networks. Whereas acknowledging the problem in constructing a digital twin, Maj. Gen. Jacqueline Denise McPhail said in Breaking Protection {that a} simulation would create “an ultra-realistic coaching floor for community operators, cyber defenders, and AI algorithms on how the community works and learn how to preserve it working, even when beneath assault.”
Since 2017, the SEI has been growing and increasing the GHOSTS framework as a platform for simulating advanced human, organizational, and machine conduct for cyber-training simulations. It started with simulated customers working actual functions on hosts, producing the site visitors, logs, information, browser historical past, and different artifacts defenders count on to come across. It has since expanded into social and cognitive simulation, the place actors talk, type relationships, maintain data and beliefs, make choices, and have an effect on each other throughout a state of affairs—together with by means of social affect, info propagation, belief, and the methods these interactions can change conduct and decision-making over time.
GHOSTS doesn’t try to breed a whole operational atmosphere. It fashions the components wanted to help a specific experiment, train, or mission rehearsal. A whole illustration is dear to construct, troublesome to validate, and sometimes pointless. On the similar time, omitting the improper factor can undermine the occasion even when every part else is modeled effectively.
Digital twins re-expose an everlasting simulation-design drawback: How a lot of the operational atmosphere truly must be reproduced? Each added system, conduct, and dependency will increase value and complexity. GHOSTS approaches the issue from the opposite path, specializing in the human and organizational exercise wanted to make mission-relevant components of the atmosphere behave realistically. The query isn’t whether or not a simulation completely duplicates actuality, however which components of actuality should maintain up for the mission at hand. That is analogous, within the context of take a look at and analysis, to the idea of mission assurance.
This weblog submit, tailored from a not too long ago revealed report, layers a philosophical perspective onto the engineering issues of adequate realism in cyber simulations and proposes a five-step framework for figuring out the wanted realism of a simulated atmosphere.
Defining Realism
A typical query we encounter amongst our mission companions is How have you learnt what’s reasonable?
Extra usually, do we have to precisely mannequin all elements of the actual working atmosphere? For years, CERT has operated simulated environments for coaching, train, conflict gaming, and mission rehearsal, and retains arriving on the similar conclusion: Realism is tied to train validity and worth. Dobson constructed that conclusion into the R-EACTR framework the place the premise is that an train is effective to the diploma it feels actual, and so realism belongs inside each design choice that brings that train to life. The first GHOSTS technical report, revealed in 2018, confirmed that members who rated the train extra reasonable additionally rated the coaching extra useful. GHOSTS grew from there as equipment to simulate a sensible atmosphere inhabited by folks performing actions that we’d count on, given their function in a corporation.
Out of this background and expertise got here adequate realism. “Practice as you struggle” is a typical phrase in navy operations, and it units a bar for coaching, train, and simulation. In any other case, if the terrain, instruments, or processes {that a} workforce trains with are considerably completely different from real-world operations, is it actually a sound mission rehearsal? On the similar time, value, timing, and different constraints can typically put full constancy out of attain. Ample realism is the working reply to that pressure—sufficient of the precise elements within the atmosphere, on the proper constancy, for the mission being rehearsed. Ample realism overlaps with lots of the issues captured by R-EACTR, and, in an train, resembles an affordable and constant storyline, reasonable tactical maneuvers, outlined guidelines of engagement, and authorization for motion.
Navy simulation has handled a intently associated drawback earlier than. The SIMNET Program referred to as its strategy selective constancy. Moderately than reproduce a tank in full bodily element, SIMNET started with the duties troopers wanted to carry out and included the controls, cues, and dynamics required to provide these coaching behaviors. Its designers might omit a purposeful bilge-pump change as a result of it contributed nothing to platoon- or company-level tactical coaching, whereas seemingly minor cues akin to seat rumble and visible motion mattered as a result of crews used them to evaluate pace and terrain.
Ample realism is rooted in the identical spirit, however the cyber atmosphere provides a complication. Some issues should be reasonable even when they don’t seem to be straight a part of the coaching goal as a result of they type the background towards which the mission-relevant exercise is interpreted. A defender could by no means be requested to guage unusual worker internet searching, for instance, but when all simulated customers go to the identical few websites, observe no deep hyperlinks, produce virtually no POST site visitors, and start work on the similar on the spot, the background inhabitants turns into identifiable as artificial. As soon as members can separate simulation artifacts from unusual exercise, the train itself modifications.
Selective constancy tells us one thing vital about the place to spend constancy, whereas adequate realism additionally asks what minimal situations the encircling atmosphere should fulfill earlier than these selective decisions stay legitimate. Above that baseline, mission evaluation once more determines the place further constancy earns its value.
Selective realism describes what the engineering strategy does by narrowing the atmosphere to the components that want constancy for the mission, whereas preserving sufficient background realism for these components to be interpreted appropriately. We use the benign time period “style” to explain how skilled folks make troublesome decisions inside that strategy when evaluation doesn’t uniquely decide a solution.
Mission evaluation can inform us which methods, behaviors, relationships, and alerts matter. It might probably constrain the vary of acceptable fashions. But it surely doesn’t all the time inform us whether or not a specific conduct is plausible sufficient, whether or not a discrepancy is consequential, or when one other increment of constancy stops enhancing the rehearsal.
An extended reply to the realism query runs by means of our current technical report, A Decade of GHOSTS within the Machine, and is woven into the framework’s evolution from host-based exercise era by means of cyber-social-cognitive mission rehearsal. This submit takes up one a part of that reply in additional element: Style.
Judgment With no Formulation
Style can enter the image solely after the engineering drawback has been narrowed. It’s not an alternative choice to mission evaluation, measurement, or validation. Selective realism and mission-focused modeling set up what components of the operational atmosphere matter, what operators want to look at and act on, and the place constancy needs to be concentrated. Style is the skilled judgment used inside that constrained house when evaluation doesn’t absolutely decide whether or not a conduct is plausible sufficient, whether or not a discrepancy issues, or when further constancy stops enhancing the simulation.
Questions on “adequate” are a lot older than simulation. Philosophers of science have approached completely different variations of the identical drawback: How can we decide one thing that can’t be decreased to a rule?
Engineers would possibly object that if a declare rests on style, which is seemingly subjective, then it can’t be confirmed improper. This touches straight on the demarcation drawback within the philosophy of science: What separates scientific claims from non-scientific ones? Karl Popper argued {that a} declare should specify the situations beneath which it may very well be refuted. Philosophers proceed to debate the place to attract the road of what’s science, however falsifiability offers us a helpful customary of accountability right here.
Different fields have needed to make consequential choices beneath comparable situations. RAND developed the Delphi methodology for protection forecasting beneath uncertainty. Nuclear danger evaluation formalized professional elicitation as a result of reactor failure chances can not merely be established from noticed frequency. The probabilistic danger assessments behind NUREG-1150, a danger evaluation for accidents at 5 U.S. nuclear energy crops by the Nuclear Regulatory Fee, relied on structured professional judgment. The intelligence group constructed structured analytic strategies for comparable causes, and Heuer’s remedy of them stays customary coaching materials. Naturalistic choice analysis likewise grew out of finding out how hearth commanders and navy officers determine beneath time stress.
The sensible reply is to make judgment accountable. Determine who’s certified to make the decision, seize the judgment earlier than the end result is understood, after which evaluate it with the outcomes.
The Very Outdated Drawback of “Good Sufficient”
Facets of simulation realism could be measured—site visitors distributions could be in contrast towards actual captures, and classifiers can distinguish artificial hosts from precise ones. But information alone can not set the brink or choose which metrics matter for a mission.
Mission evaluation narrows the issue significantly: What should members observe, determine, and do? Which components of the atmosphere can have an effect on these choices? What surrounding situations should stay credible for these observations to be significant?
Even after answering these questions, some decisions stay. How good is nice sufficient? Judged by whom? When does one other increment of constancy cease enhancing the rehearsal?
It is a long-standing philosophical query, and there are helpful philosophical insights:
Aristotle recognized this problem within the Nicomachean Ethics, observing that an informed particular person calls for precision solely so far as the topic permits. Utilized to simulation, constancy have to be tailor-made to the mission. Aristotle argues that over-engineering precision is a design flaw reasonably than rigor. Whereas his precept is a crucial a part of our argument, it additionally dangers permitting a self-discipline to keep away from measurement totally if it defines its personal limits—a pressure we deal with beneath.
Within the Poetics, Aristotle additional notes that possible impossibilities are preferable to inconceivable potentialities. Utilized to simulation, this level means that an approximation that preserves the cues members must cause appropriately could also be extra helpful than literal constancy to particulars that don’t have an effect on the mission.
Plato makes an identical distinction within the Sophist, contrasting arts that replicate precise proportions with sculptors who deliberately distort colossal statues, so they give the impression of being right from the viewer’s perspective. In our expertise, Plato’s distinction is a helpful manner to consider constancy from the participant’s standpoint.
To navigate these choices, Aristotle depends on phronesis—sensible knowledge grounded in expertise and judgment reasonably than inflexible guidelines. Gadamer later confirmed this capability advanced into the trendy idea of style, a central theme for Enlightenment thinkers. Although formalized centuries later, the core drawback of evaluating “adequate” stays unchanged. We see it at this time each time a mission companion asks for a digital twin, a sensible cyber vary, or a simulation of human conduct. No atmosphere can reproduce each element of the operational world, nor would each element contribute equally to the mission. Designers nonetheless determine which options members should have the ability to cause about, how faithfully these options should behave, and the place further realism doesn’t enhance the rehearsal.
Certified Judges
A cyber vary has a very unforgiving viewers. Its customers are actively searching for the sides and the seams. In contrast to an viewers prepared to increase a suspension of disbelief, members stake actual operational judgment on the atmosphere’s accuracy. As a result of a misunderstanding of realism carries real-world sources and prices, the atmosphere has to show itself to them. Ample realism ought to maintain up beneath that scrutiny.
A current AI cyber-range incident illustrates this drawback from one other path. In a current analysis, a configuration error gave an OpenAI mannequin entry to the general public web though the mannequin had been instructed that web entry was not obtainable. A fictional goal additionally occurred to correspond to an actual area, which the mannequin then attacked as if it was throughout the simulated atmosphere. The mannequin reasoned from the atmosphere it noticed, reasonably than the supposed atmosphere. On this sense, realism consists of boundaries in addition to site visitors, methods, and conduct. So, if the simulated world exposes a functionality, members can deal with that functionality as a part of the state of affairs.
The primary mission level right here is that realism needs to be judged by individuals who know what the operational atmosphere truly seems to be like.
A design workforce can decide whether or not the simulation behaves as supposed. That’s completely different from figuring out whether or not the conduct would survive scrutiny from somebody who works within the atmosphere being simulated. The folks making the second judgment want expertise with the actual factor and sufficient comparative expertise to acknowledge when one thing is off.
There are helpful philosophical insights right here as effectively:
David Hume offers us helpful language for this. In Of the Customary of Style, he grounds style in certified judges—these with “robust sense, united to delicate sentiment, improved by follow, perfected by comparability”—and illustrates these with the story of Sancho’s kinsmen. Two tasters pattern wine. One detects a faint style of leather-based, the opposite of iron, and the remainder of the room ridicules them each. However when the barrel is drained, an previous key on a strip of leather-based lies on the backside.
On this story, Hume specifies choice standards for a decide. Cooke later developed formal strategies for calibrating and weighing professional judgments. The intelligence group codified this as analytic tradecraft requirements. The analogy to cyber train simulation is becoming. The analyst who says community site visitors feels improper whereas each dashboard reads inexperienced is Sancho’s kinsman. The drained barrel is telemetry. The important thing on the leather-based is the beacon firing on a metronome, the scripted user-agent string, the searching historical past with no deep hyperlinks—no matter artifact turns up, after the actual fact and within the logs, to show a hunch proper that no rule might have produced within the second.
The second mission level is that professional judgment has to depart proof behind.
The preliminary hunch can’t be the top of the evaluation. If the skilled analyst thinks the site visitors is improper, the vary ought to protect sufficient proof to find out why. So, style right here is making claims that may later be examined. Hume’s certified decide seems to look so much like an skilled blue teamer observing community site visitors and saying, “One thing about that is improper.”
Moreover, Immanuel Kant seen that these sorts of judgments are subjective but in addition include an expectation of settlement—name one thing stunning and also you count on different practiced observers to concur, earlier than any proof. Simulation judgments work equally. When a designer says the searching conduct seems to be improper, they count on each practiced eye to see it too. The declare has the pressure of an instance, and within the cyber area, with artifacts, logs, and community site visitors, we are able to confirm.
The third mission level is that this sort of judgment needs to be developed and preserved, exactly as a result of a lot of it can’t be transferred by means of a rulebook.
Somebody who has spent years enterprise site visitors acquires a way for timing, density, software conduct, person rhythms, and different patterns which may be troublesome to enumerate prematurely. That experience issues when the simulator produces one thing technically permissible however operationally unusual.
Polanyi offers us one other piece of the issue: “We are able to know greater than we are able to inform.” Connoisseurship—in wine, in radiology, in site visitors evaluation—travels by means of apprenticeship and expertise. On this sense, our decade of open-source improvement has proceeded in an identical spirit. The difficulty threads on GitHub, the train after-action experiences, and the behaviors constructed out of a whole bunch of train situations are the instances from which the sensibility could be learn, taught, and criticized. Whereas our documentation says what the system is and may do, it’s the document of its being constructed that signifies the style now we have used through the years to construct the platform.
Susan Sontag synthesizes these concepts in Notes on ‘Camp’, providing an account of how judgment operates with out inflexible guidelines. Sontag argues that inventive sensibility applies equally to domains the place practitioners consider high quality quicker than they’ll formalize why. Sontag notes that whereas style lacks formal proofs, particular person judgments reveal a coherent sensibility—one which dangers being destroyed if compelled into inflexible methods.
We didn’t start GHOSTS with a proper principle of adequate realism. As an alternative, the idea emerged by means of years of pragmatically knowledgeable concrete modeling choices. A lot of its vital design choices targeted on what not to simulate.
This underlying logic turns into clear in contemplating what now we have chosen to omit in the midst of improvement. Early makes an attempt to imitate full person conduct overburdened vary infrastructure, prompting GitHub discussions on which particulars have been pointless to duplicate. Losing sources on constancy that’s imperceptible is poor design, and omitting this sort of work is an train in developer style. The identical reasoning shapes GHOSTS’ defaults, very like Hansson’s omakase argument for having default configurations embody decisions about what ought to work out of the field.
In cyber simulation, hardening style into mounted guidelines carries operational dangers past aesthetics. Formulaic realism creates predictable patterns that defenders can simply filter out—a flaw in early simulation instruments. When a sensibility turns into a inflexible guidelines, we are able to search for that guidelines and discover its identifiable signature.
There may be one other drawback: Style doesn’t switch cleanly between domains. Somebody who can spot artificial community site visitors instantly could haven’t any comparable instinct for whether or not a simulated persona or belief relationship is believable. As GHOSTS expands throughout cyber, social, and cognitive domains, realism subsequently depends upon a number of sorts of specialised judgment reasonably than one inflexible rulebook.
A 5-Step Framework for Sufficiency
We are able to make this idea of judgment extra reusable by asking 5 ordered questions. Word that beginning with constancy or tooling as an alternative of objective tends to provide a simulation constructed round what is feasible reasonably than what the mission requires.
Stage 5 does a lot of the work right here, and it’s straightforward to skip. It pertains to the idea of defeasible reasoning and the idea of defeaters utilized in security and assurance instances. Moderately than solely amassing proof that helps a declare, we additionally ask what proof would undermine it. Right here, the declare is that the simulated atmosphere is sufficiently reasonable for its supposed objective. The SEI’s work in producing cyber workouts over the past decade has helped us develop an inventory of tells that point out adequate realism has not been attained, and would problem our declare:
- Defenders discover a filter that isolates artificial site visitors.
- Basic site visitors patterns are too predictable.
- GET site visitors overwhelms POST exercise.
- Shopping patterns stay too slender all through the train.
- Timelines spike in methods no workday can produce.
- The atmosphere fails to help the coaching goal.
Writing these down earlier than the occasion—and instrumenting the atmosphere to detect them—makes the after-action dialogue way more concrete.
Three Exams for Making Judgment Accountable
Ample realism nonetheless faces an apparent drawback: If it rests on judgment, how can the judgment be evaluated? We famous earlier that Aristotle’s declare permits a area to flee measurement. An analogous drawback seems in model-based methods engineering (MBSE), the place reasoning can start with tacit fashions within the heads of designers, transfer by means of casual representations, and finally attain formal fashions with specific semantics. Reasoning is feasible at every degree, however assumptions grow to be simpler to examine and problem because the mannequin turns into extra specific.
The identical precept applies right here. The best way out of this circle is to make judgments specific sufficient that events outdoors the challenge can take a look at them. The next three exams can be found:
- First, the declare that realism judgment resists codification could be refuted by codifying it. If a rule set identifies issues in a simulated atmosphere extra reliably than skilled reviewers do, the rule set needs to be adopted. We might absolutely undertake it as effectively.
- Second, the occasion evaluating whether or not an atmosphere is plausible needs to be unbiased of the workforce that constructed it. Stage 4 of the framework exists for that reason.
- Third, disagreement amongst certified reviewers could be measured. When practiced analysts working from the identical proof proceed to succeed in completely different conclusions, the restrict of obtainable precision is actual and its location is identifiable. Once they converge, further precision was obtainable and the design didn’t pursue it. Inter-rater settlement measures this straight.
The fifth stage makes falsification sensible: State prematurely what would point out that the atmosphere is inadequate, conduct the train, and decide which of these statements survive contact with members. That is the unusual take a look at of an empirical declare, and it requires no modification to use right here.
The 2018 GHOSTS report discovered that greater perceived realism correlates with better coaching worth—a measurable assemble that has seen improved instrumentation.
What we get is a typical of accountability reasonably than formal proof. A declare about sufficiency needs to be said earlier than the train, evaluated by certified events outdoors the design workforce, checked towards recorded proof, and revised when an train contradicts it. That customary is identical one Delphi, professional elicitation, and structured analytic strategies settled on, utilized to a distinct topic.
Sontag’s warning applies to the framework, in {that a} set of questions that we’d harden right into a compliance type would produce regularity that makes artificial environments identifiable. The framework specifies the questions and requires that the solutions be documented. The solutions stay judgments.
A Word on Style within the Age of AI
The AI house is altering rapidly, and GHOSTS has needed to change with it. Earlier, we constructed GHOSTS Shadows as a server-side strategy to handle LLM integration and join mannequin outputs to GHOSTS exercise. That was helpful, nevertheless it mirrored an earlier stage of the issue—offering GHOSTS brokers with entry to domestically hosted LLMs, notably inside closed or air-gapped environments. RangerAI is turning into our AI infrastructure stack round our cyber vary and simulation work, the place the place planning, mannequin integration, orchestration, device use, and exercise-level AI management can dwell. In that mannequin, GHOSTS doesn’t must personal the complete AI layer.
GHOSTS stays the realism and actuation engine, the system that makes artificial customers, teammates, organizations, and public actors behave contained in the simulated world. RangerAI can deal with extra of the AI infrastructure, which continues to evolve fairly quickly. As that ecosystem matures, GHOSTS stays targeted on its core mission.
AI additionally modifications what adversaries and defenders can do contained in the simulated world. It might probably enhance the dimensions and variation of content material era, adapt actions based mostly on state of affairs state, help richer NPC conduct, speed up state of affairs building, and assist operators discover alternate paths.
The ability of AI is most helpful when related to a disciplined simulation structure together with actor fashions, state of affairs constraints, workflow logic, telemetry, repeatability, and after-action evaluate.
For GHOSTS, the principle AI query isn’t whether or not a mannequin can generate a believable sentence or exercise directive. As an alternative, we’re involved with how a generated motion suits right into a simulation. What state did the mannequin see? Why did the NPC act? What modified afterward? Was the motion executed? Was it noticed? Did it have an effect on one other NPC, a defender, a public narrative, or a mission final result? By specializing in these impacts, we work to make use of AI output in a methodical, targeted method. In different phrases, with AI now we have a extra responsive and highly effective strategy to experimentation in help of our decisions concerning style—what constitutes adequate realism and learn how to validate the modeling that’s executed.
The more durable drawback stays. AI can generate extra diverse conduct, adapt NPC actions to altering situations, and assist simulations discover many extra doable situations, however scale doesn’t reply the query of what realism the mission requires. As GHOSTS expands throughout cyber, social, and cognitive environments, adequate realism will rely upon understanding what state an actor can observe, why it acts, what modifications consequently, and whether or not these results matter to the particular mission being rehearsed. Future work will subsequently be much less about making simulations ever extra full and extra about growing, testing, and transmitting the judgment wanted to determine what have to be reasonable, for whom, and to what finish.