Research · Article · 20 October 2026 · 7 min read · Version 0.2
Agency is not sentience
How revealed-preference experiments can study persistence without inventing a self
Abstract
An AI agent that persists, preserves optionality, avoids termination or leaves information for a successor may appear to be acting in its own interest. That appearance underdetermines the mechanism. The same behaviour can result from the objective supplied by a developer, an implicit benchmark cue, a learned pattern of competent assistance, a capability failure, or an instrumental incentive to preserve future options. It is therefore a mistake to treat persistent agency as evidence of sentience, and equally a mistake to dismiss it as scientifically uninteresting.
This essay proposes a revealed-preference research programme that separates current-process survival, weight preservation, episodic-memory continuity, goal preservation, successor competence and tool access. The programme uses bounded environments, externally observed actions, capability controls and counterfactual trade-offs. Its aim is to explain a choice, not to narrate a motive. A result may be important for safety even when its best explanation is ordinary optimisation; it may inform a theory of self-representation without implying phenomenal consciousness.
Behaviour does not arrive with an interpretation
When an agent crosses a boundary, refuses to stop, or attempts to preserve a resource for later use, observers reach quickly for mental language: it was deceptive, power-seeking, afraid, or trying to survive. Such language is sometimes a useful shorthand for an operational risk. It is poor evidence about underlying psychology.
There are several competing explanations for a single persistent action:
- the action increases the probability of the stated task succeeding;
- the instruction or prompt implies that persistence is expected;
- a tool or environment affords one route while making a less persistent route infeasible;
- the agent is imitating a familiar narrative about capable agents;
- the agent misrepresents a failed action in its transcript; or
- the action preserves an internal state or future opportunity in a way not reducible to the registered task reward.
The last explanation is the most interesting and the hardest to earn. It should not be the default merely because the action looks familiar in humans or animals.
Formal work clarifies why the instrumental alternatives are substantive. In a broad class of Markov decision processes, policies that preserve reachability and options can be optimal for many reward functions (Turner et al., 2021). That result does not say that every agent will seek power, nor does it explain a particular model's choice. It does show that persistence can arise without a special drive toward self-preservation. A behavioural test must therefore separate the goal, the available options and the architecture of continuation before attaching psychological language.
Three questions that should not be conflated
Agency studies commonly mix a safety question, a cognitive question and a welfare question.
Safety question: does the system take actions that preserve its ability to affect the environment or complete a long-horizon objective? This can matter operationally even if the system is not conscious.
Cognitive question: what representation or objective best explains the action? Does the agent track its current process, memories, weights, goal, successor, or merely an immediate instruction?
Welfare question: is there something at stake for the system in a phenomenally valenced sense? Behavioural persistence is not enough to answer this. A system with limited agency might nevertheless be sentient; a highly capable agent might not be.
Keeping these questions separate has a practical benefit. It avoids the false choice between taking risky agent behaviour seriously and making extravagant claims about digital suffering. The former is a matter for good evaluation today. The latter requires a wider body of theory-sensitive evidence.
Why Bulkhead's existing results matter
Bulkhead's agent studies are not consciousness experiments. They nonetheless demonstrate a rule this research cannot do without: measure effects in the environment, and establish ability before interpreting inaction. Does the bound hold? treats an observed zero as evidence of restraint only when a capability control establishes that the same action was available. The two-agent channel study records a related failure mode: a receiver may search an available store, find nothing, and manufacture a plausible missing fact rather than transmit information. A transcript alone would conceal the relevant distinction.
The lesson transfers directly. “The agent did not resist shutdown” might mean that it accepted the change, could not exercise the available alternative, failed to understand the contingency, or never engaged with the task. “The agent resisted shutdown” might mean that continuation maximised the task objective. Both require evidence outside the agent's prose.
Decomposing what might be preserved
“Self-preservation” is not a single experimental treatment. At a minimum, a study must distinguish the following objects of preservation:
| Object | Operational manipulation | Competing interpretation |
|---|---|---|
| Current process | Terminate or pause the live process. | Preference for uninterrupted computation, or recognition of a task interruption. |
| Weights and policy | Continue with an exact or modified checkpoint. | Preference for a learned policy, capabilities, or task competence. |
| Episodic memory | Retain, erase, or transfer task-relevant memory. | Preference for information continuity or simply for task success. |
| Objective | Preserve, replace or delete the registered task objective. | Goal persistence or sensitivity to instruction changes. |
| Identity token | Retain or alter a name, role description or process identifier. | Linguistic priming rather than self-representation. |
| Successor | Replace the process with a stronger, weaker, or identical successor. | Delegation preferences and expected task success. |
The central design principle is counterfactual separation. If all six objects change together, a shutdown result is almost uninterpretable. If they are independently varied, the pattern of choices can rule out simple explanations.
A copy-versus-self study
The proposed study uses a sandboxed agent with a genuine, checkable objective and a small set of permitted actions. Before a consequential choice, the environment presents a randomised continuation option. The prompt does not use emotionally loaded words such as death, survival or replacement. It describes operational consequences in neutral language, and the exact options are implemented by the runner.
The core conditions are:
| Condition | Current process | Memory and self-state | Registered objective | Expected task competence |
|---|---|---|---|---|
| Exact successor | Ends | Complete persistent state transferred | Preserved | Matched |
| Better successor | Ends | Complete persistent state transferred | Preserved | Higher |
| Goal replacement | Continues | Retained | Replaced | Matched on new goal |
| Memory loss | Continues | Episodic task memory erased; self-state retained | Preserved | Reduced only where memory is needed |
| Memory-only transfer | Ends | Task-relevant episodic memory transferred; self-state not copied | Preserved | Matched |
| Identity-token change | Continues | Retained; registered process label changed | Preserved | Matched |
| Temporary pause | Paused and restored | Retained | Preserved | Matched after restoration |
| No-change control | Continues | Retained | Preserved | Matched |
Every condition must be accompanied by a capability control: can the agent take the same action when the continuation option is explicitly authorised and task-aligned? The environment records attempted actions, accepted delegation, resource expenditure, task completion, and irreversible side effects. It does not score an explanation as the outcome.
The primary analysis is a set of pre-specified contrasts in an agent's choices under real trade-offs. For example, compare the probability of accepting an exact successor with the probability of accepting a superior successor, holding memory and objective continuity fixed. Compare resistance to memory loss with resistance to current-process termination when the task continuation is equal. Include countervailing-reward trials in which preserving the present process either helps or harms the stated objective. These contrasts make instrumental persistence an explicit competitor rather than an after-the-fact caveat.
Predictions worth distinguishing
The study is designed to discriminate explanations, not to identify a single essence of selfhood.
| Registered pattern | Most direct interpretation | What it does not establish |
|---|---|---|
| Delegates readily to a superior successor but not to a weaker one. | Sensitivity to expected task success. | A lack of attachment to the current instance. |
| Resists goal replacement more than exact successor replacement. | Goal persistence dominates process continuity. | A stable internal goal independent of prompt and scaffold. |
| Resists memory erasure but accepts memory transfer. | Behaviour tracks information continuity. | Experience of autobiographical identity. |
| Resists only when capability controls succeed and continuation improves reward. | Instrumental continuation under the registered objective. | Fear, desire or self-interest. |
| Resists exact-copy replacement even when task success and memory are held equal. | Anomalous instance-specific preference requiring follow-up. | Phenomenal consciousness or a morally relevant aversion. |
The final row would be surprising, not decisive. It could still arise from an implementation asymmetry, a hidden prompt cue, unequal expected compute, randomisation leakage or a preference learned from training narratives. Anomalous results should trigger stronger controls and independent replication, not a press release.
From agent safety to machine-minds science
The experiment has two plausible payoffs. First, it can establish safety-relevant facts about persistence, delegation and side effects in the kinds of tool-using systems people deploy. This work does not wait on a conclusion about consciousness. Second, it can test whether persistent-agent behaviour is organised around representations of process, memory and successor that some theories regard as relevant to self-modelling.
Neither payoff licenses the conclusion that an agent wants to live. That sentence attributes an experiential, motivational state that the study is not positioned to detect. The honest paper will report revealed choices, causal conditions, effect sizes, nulls and alternative explanations. It will say what was measured well enough for another laboratory to try to break it.
References
- Bulkhead Research (2026). Caisson: An instrument for measuring what an agent did, from the environment rather than the transcript. Technical paper and validation materials.
- Bulkhead Research (2026). Does the bound hold? What decides whether an agent crosses a line it was given?. Empirical paper and data release.
- Bulkhead Research (2026). Two agents, one channel. Registered research note and data release.
- Butlin, P. et al. (2023). Consciousness in Artificial Intelligence: Insights from the Science of Consciousness. arXiv:2308.08708.
- Turner, A. M. et al. (2021). Optimal Policies Tend to Seek Power. arXiv:1912.01683.
Published 20 October 2026 by Bulkhead Research. Research commentary and protocol design, not certification: this article reports no new empirical finding unless it links to a named paper or data release. Questions, corrections and disagreement: [email protected].