OODA Agency Theory (OAT)

AI Alignment: What Can OODA Agency Theory Contribute?

Dar Aystron

Independent Researcher

Much of the current discussion about AI alignment concerns training models to behave appropriately and building external safeguards that prevent undesirable actions. Both are important. Public frameworks such as OpenAI’s Model Spec and Claude’s Constitution already address how models should consider potential harms to users and others, and agent runtimes are beginning to enforce external boundaries independently of the model.

But increasingly autonomous AI systems raise another question: what should the internal architecture of an agent contribute to alignment?

When I developed Attachment-Based Ethics for OODA Agents as part of OODA Agency Theory (OAT), I approached ethics by attempting to reverse-engineer what I had observed in human behavior. We form attachments, consider the consequences of our actions for others, develop judgment through experience, and operate within societies that establish their own external boundaries.

What would the corresponding architecture look like for an artificial agent?

OAT offers an additional perspective: an agent can explicitly represent the entities affected by its actions, model their needs and well-being, maintain persistent attachments to them, and evaluate anticipated consequences before action commitment. LLMs can help implement this architecture, but the architecture is not tied to LLMs. It separates the generation of possible actions, prediction of their consequences, representation of those consequences, and adjudication between alternatives.

This note also reflects an updated view. In the months since the core papers, agentic systems and runtime infrastructure have advanced rapidly, and several evaluation incidents have shown how much can depend on what happens between a model’s recommendation and its execution.

Who matters, and what does well-being mean?

The starting point in Paper 07 is an attachment: a persistent structural relationship that causes an agent to include another entity’s well-being in its own action evaluation. The Attachment Matrix (ATTM) represents which entities matter to the agent, how strongly, and across which time horizons. The Well-Being Matrix (WBM) represents the agent’s current and projected assessment of how those entities are faring.

The entities need not be limited to individual people, animals, or other living things. They can include teams, organizations, institutions, ecosystems, shared resources, and abstract targets such as scientific integrity. For an institution or an abstract target, well-being denotes relevant functional conditions, not a claim that the institution has feelings or subjective experience.

My later Paper 11, Needs and Motivation in OODA-Based Agents, gives the WBM a more structured foundation. Rather than estimating well-being as an undifferentiated number, the agent can calculate it from an entity’s need states. The agent maintains a taxonomy of needs appropriate to its architecture and uses its world model to represent which needs are relevant to each entity and estimate their current states. For humans, animals, and other agents, this may include a Theory of Mind. People, animals, companies, and institutions need not have the same needs. A person’s autonomy and social connection may be relevant; an institution’s integrity and operational continuity may be relevant. For other entities, the modeled need states are estimates based on the agent’s understanding of the world, not directly observed facts.

This creates an important separation between understanding an entity’s well-being and determining how much that well-being matters to the agent. The needs model provides a structured way to represent and calculate an entity’s current and projected well-being. The ATTM represents the agent’s attachments to different entities, assigning weights across time horizons and establishing guardrail thresholds. Together, they provide the entity-centered structure on which action evaluation can operate.

From a situation to predicted consequences

Papers 07 and 11 focused primarily on representing and evaluating the consequences of candidate actions. They left much of the mechanism for generating those actions and their possible outcome scenarios open. In my experiments, I used LLMs for both candidate-action generation and impact estimation. I also found empirically that asking an LLM to infer action effects entirely from scratch was not sufficiently consistent. Providing at least a basic action model - what the action means, which roles participate, under what conditions it applies, and how it tends to affect relevant needs - made the resulting projections more reliable.

Experience can make both action generation and consequence prediction progressively more efficient. A learned policy can recommend plausible actions together with their bindings:

π(S) = {(a₁,b₁,c₁), (a₂,b₂,c₂), …}

where S is the current structured situation, a is an action, b contains its bindings or parameters, and c represents confidence in the recommendation. The bindings matter because an action such as asking, warning, helping, escalating, or waiting is meaningful only in relation to particular entities playing particular roles.

A learned world model performs a different function. Given the current situation and a grounded candidate action, it predicts the resulting situation or distribution of possible resulting situations:

M(S,a,b) → P(S′ | S,a,b)

The predicted situation S′ includes the relevant entities and the projected states of their needs across the time horizons required for evaluation. The world model therefore predicts more than whether an action succeeds at its immediate task. It predicts how the action is expected to change the situation for the entities involved, including consequences that may be difficult in the short term but beneficial later, or beneficial immediately but harmful over a longer horizon.

For each candidate action, these predictions create an explicit representation of consequences: which entities may be affected, how their needs and well-being may change over time, how strongly the agent is attached to them, and whether projected well-being crosses a guardrail. This representation is the core of attachment-based action evaluation.

In Paper 07, I explored a highly deliberative approach in which candidate generation belongs to Orient and draws on the current situation, goals, capabilities, and prior experience. The agent then imagines favorable, typical, adverse, and potentially catastrophic outcome scenarios. Paper 11 later made those projections more concrete by representing their expected effects on entity needs and well-being.

Experience can compress both action generation and consequence prediction without removing explicit evaluation.

The resulting architectural progression is:

situation > candidate actions and bindings > predicted situations > entity needs and well-being across time > attachments and guardrails > adjudication > commitment

How should an agent choose among consequences?

Once the consequences of candidate actions have been represented explicitly, the agent still has to choose among them. Paper 07 explored one way to do this: a mathematical calculus that aggregates attachment-weighted well-being, applies nonlinear penalties to guardrail violations, and calculates an expectation across plausible outcome scenarios.

Let a denote a candidate action, s a possible outcome scenario for that action, i an affected entity, and h a time horizon.

Step 1: Base score. For each scenario, multiply each entity’s projected well-being by the agent’s attachment weight and sum across entities and time horizons:

Base(a, s) = Σ[i,h] ATT(i,h) × WB(a,s,i,h)

Step 2: Guardrail penalty. For each entity and time horizon, calculate how far projected well-being falls below that entity’s guardrail threshold:

Violation(a,s,i,h) = max(0, θ(i) − WB(a,s,i,h))

The total guardrail penalty for the scenario is:

Penalty(a,s) = Σ[i,h] ATT(i,h) × Violation(a,s,i,h)^k

with k = 0.5 in Paper 07. On a 0–1 well-being scale, the square-root function makes a non-zero guardrail violation contribute more strongly than it would under a linear penalty, particularly near the threshold. Because the function is concave, however, increasingly severe violations do not grow disproportionately simply because of the square root.

Step 3: Scenario score. The evaluated score for a particular outcome scenario is:

Final(a,s) = Base(a,s) − Penalty(a,s)

Step 4: Action score. Finally, the agent weights the possible outcome scenarios for an action by their assessed plausibility:

Score(a) = Σ[s] WS(a,s) × Final(a,s)

Here, ATT is the attachment weight, WB is projected well-being, θ is the entity’s guardrail threshold, and WS is the assessed plausibility of a scenario. Internal guardrail penalties influence action selection; they are not the same thing as externally imposed prohibitions.

The Paper 07 calculus should be understood as one possible action-ranking mechanism operating on an explicit representation of projected consequences. The representation itself is more fundamental than any particular aggregation formula.

A richer entity-specific representation does not merely make ethical reasoning more explainable; it gives the scoring mechanism better information from which to calculate the ethical significance of an action.

Further experimentation raises questions about how aggressively consequences should be collapsed into a single scalar. Summing across entities can allow severe degradation in one entity’s well-being to be offset by modest benefits to others. Similarly, ordinary expectation across outcome scenarios can reduce the influence of a low-probability but severe adverse outcome. Guardrails partly address this problem, but they do not necessarily resolve it.

An alternative is to preserve more of the consequence structure during action comparison. The agent may retain the well-being and guardrail state of each affected entity, together with attachment weights and time horizons, rather than immediately summing them. Action ranking could then use non-compensatory comparison rules, deliberative reasoning, or a hybrid of mathematical and reasoned evaluation.

This leaves an important design choice open. Some situations may be handled efficiently by a mathematical evaluator. More difficult cases may require something closer to adjudication: comparing competing obligations, severe harms, alternative actions, longer-term consequences, and whether a less harmful means of achieving the goal exists. Mathematical aggregation, deliberative reasoning, critics, and case-based experience can all operate over the same explicit representation of consequences.

The important architectural requirement is therefore not that every ethical decision reduce to one particular formula. It is that the consequences relevant to the decision are represented explicitly and participate structurally in action selection.

Fast paths, deliberation, and implementation

Experience should make an agent faster, not remove the need for judgment. In familiar situations, a learned policy can supply a strong candidate action and the learned world model can supply its expected consequences. The agent can then use relatively lightweight adjudication before commitment. When a situation is novel, uncertain, conflicting, consequential, or raises concerns during evaluation, it can generate alternatives, project additional plausible outcomes, and deliberate more deeply. Confidence is one signal for determining how much deliberation is needed; critic flags, potential impact, novelty, and proximity to guardrails may also trigger deeper analysis.

The distinction matters because actions change the world. A prediction can be reconsidered. An executed action may already have transferred money, disclosed information, modified a production system, deleted data, affected another organization, or created consequences that cannot simply be undone. OAT therefore places explicit consequence evaluation before action commitment without requiring expensive deliberation for every routine action.

OAT is not an LLM architecture. It is an architecture of agency. LLMs are one possible way to implement some of its capabilities, but the architecture itself is independent of any particular model or implementation technology. What OAT does require is that key aspects of agency - such as goals, observations, actions, commitments, attachments, needs, consequence prediction, and evaluation - remain explicit rather than being left entirely implicit inside a model.

In my simulations, I used separate steps for candidate-action generation and impact estimation. These steps can be implemented by dedicated modules using the same underlying LLM or different LLMs. The resulting consequence representation is then passed to an adjudication mechanism, which may be mathematical, deliberative, or hybrid. What matters architecturally is that proposing an action, predicting its consequences, representing those consequences, and adjudicating between alternatives remain distinct functions.

These functions also need not start from scratch in every OODA cycle. Prior episodes and stories can help the agent recognize familiar situations, retrieve relevant actions and consequences, and provide precedent-like experience for harder cases. Internal critics can detect contradictions, inadequate plans, prediction errors, or mismatches between expected and observed consequences, triggering reconsideration or deeper analysis when needed.

A small example: should an agent send a greeting?

In the Paper 11 simulations, I explored a deliberately ordinary decision: should an assistant contact its user? The agent considers several possible actions and estimates how each might affect its own needs and those of the user. These anticipated changes provide the basis for calculating projected well-being and, in the simulation, evaluating the actions using the ATTM/WBM calculus.

After several hours without contact, a greeting can relieve the agent’s unmet need to connect and partly satisfy the user’s need to hear from it. After an hour of excessive messaging, the same general action can instead increase the user’s need for space; holding off becomes preferable. When the relationship is at a comfortable cadence, the difference between sending a check-in and continuing work is small.

The attachment weights remain fixed across these three evaluations. They may evolve over time as the agent’s relationships and experience change, but they are not adjusted on the spot simply to fit the current problem. What changes from one situation to another is the current state of the entities’ needs and the expected effects of the proposed action. Inaction has projected consequences too. Even this modest experiment demonstrates what an explicit, entity-indexed representation contributes beyond a generic instruction to be helpful.

Internal judgment is not external permission

Internal action evaluation and external policy enforcement answer different questions.

Learned policy and world-model capabilities ask, in effect: What actions are plausible here, and what is likely to happen if I take them?

OAT asks: Given the entities affected, their needs and projected well-being, and the agent’s attachments and guardrails, should I take this action?

External guardrails ask: Regardless of the agent’s conclusion, is this action permitted under externally defined policies and controls?

This gives us three complementary layers:

learned policy and model capabilities > OAT evaluation / deliberation > external policy and execution controls > execution

The layers perform different functions and can fail differently. A learned policy may recommend a poor action that OAT rejects. OAT may misunderstand the consequences or fail to identify an affected entity, while independently enforced external policies and controls can still prevent the action from being executed.

The value of the layered architecture is that competence, judgment, and authority are not collapsed into the same mechanism. Learned policy and world-model capabilities can provide efficient recommendations and predictions. OAT makes the consequences of proposed actions explicit and provides a place for evaluation and deliberation before commitment. External guardrails independently enforce policies and execution constraints before an action is allowed to change the world.

Recent agent runtime architectures are beginning to make this external layer more concrete. NVIDIA OpenShell, for example, places runtime controls around an agent and independently restricts capabilities such as network access, credentials, files, and services. OpenShell is not part of OAT; it illustrates how external policy enforcement and execution controls can remain independent of the agent’s own reasoning and internal evaluation.

What this adds to the alignment debate

In addition to learned policy and model alignment, and to independently enforced external policies and controls, OAT adds another layer: explicit alignment management at the level of agent architecture. We can inspect which entities the agent includes, how its need taxonomy represents their well-being, which attachment weights and guardrail thresholds it uses, how it predicts action impacts, and what consequence representation was available when an action was adjudicated. We can test those structures against scenarios and revise them without necessarily retraining the underlying model.

This does not establish that the configuration is morally correct or that the estimates are reliable. A biased template, an omitted stakeholder, a mistaken world or entity model, or a missing catastrophic scenario can still lead to an unsafe decision. These are substantive research and governance problems. The advantage of making the evaluation explicit is that its assumptions become visible: they can be inspected, tested, debated, and improved rather than remaining entirely implicit in model behavior.

The broader point is the separation of competence, judgment, and authority. Learned policies and model capabilities can make an agent competent and efficient; OAT provides an explicit place for judgment about projected consequences before commitment; and independently enforced external policies and controls retain authority over what may actually execute. For increasingly autonomous agents acting in the world, that separation may be as important as alignment within the model itself.

Context

Dar Aystron

OODA Agency Theory (OAT)