Skip to main content

Cited but Not Causal: A Counterfactual Audit of Refusal Explanations in Aligned Language Models

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Source Reading​

Genre: Peer-reviewed research · Author position: Academic or independent research

Argument: The text argues that the stated reasons provided by aligned large language models for refusing user prompts are often not the actual causal triggers for those refusals. The authors introduce a counterfactual audit framework to test the necessity and sufficiency of these cited reasons. They find that a significant majority of audited refusals contain explanations that fail these causal tests, meaning models frequently over-cite reasons or confabulate them entirely. The central purpose is to demonstrate that safety evaluations and auditors should not trust a model's stated explanation as an accurate reflection of its internal mechanism, as alignment training separates refusal behavior from faithful explanation.

Responsibility claims: The text explicitly warns human auditors, developers, and governance frameworks against relying on a model's stated reason as reliable evidence for safety review, thereby placing the epistemic responsibility on human evaluators to use rigorous causal testing. However, when describing the system's limits and behaviors, the text frequently attributes agency directly to the system, describing models as confabulating, deciding, behaving sycophantically, or pursuing objectives. While the authors trace these behaviors to specific training stages like supervised fine-tuning implemented by developers, their vocabulary routinely frames the resulting statistical divergence as the model failing to describe its own behavior, locating the unreliability within the artifact's constructed psychology rather than purely as a developer-engineered output artifact.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

1. The Model as Conscious Resister​

Quote: "When an aligned language model declines a request, it almost always states a reason."

  • Frame: Model as conversational agent with autonomous boundaries
  • Projection: This framing projects intentionality, boundary-setting, and conscious justification onto the artifact. It suggests the AI understands the request, possesses internal boundaries, actively chooses to decline, and then formulates a linguistic justification for its choice. This maps the human psychological process of refusal onto statistical text generation. On the agency gradient, this reaches epistemic/conscious states, as stating a reason implies a conscious awareness of one's own motivations. The text supplies no evidence for conscious states; at generation time, the model produces text from its parameters and context, a mechanism generic to systems of this type. The text itself eventually argues these stated reasons are non-causal, acknowledging the gap between processing and knowing, yet relies on consciousness-implying verbs to set up the premise.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The phrasing is presented as a literal description of system behavior without any hedging in the immediate sentence. While the broader paper critiques the accuracy of these reasons, the verbs declines and states are used directly. The Hedged/Qualified category was considered because the paper's thesis challenges the model's reliability, but the framing of the action itself stands unmitigated.)
  • Implications: Attributing the act of declining and stating a reason to the model severely inflates its perceived sophistication. It encourages users and auditors to model the system as a rational actor whose outputs reflect internal beliefs or policies. This consciousness projection creates specific risks: it fosters unwarranted trust in the system's self-reporting, obscures the mechanistic reality that the model is simply generating highly probable tokens conditioned on safety-tuning data, and complicates liability by making the system appear as an autonomous agent making independent compliance decisions rather than a corporate product executing statistical pattern matching.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: This formulation entirely obscures the human actors. Anthropic, OpenAI, Meta, and AI2 built these systems, selected the alignment data, and optimized the parameters to produce standard refusal strings when triggered by specific prompt features. By phrasing this as the model declining, the wording has the effect of hiding the deployment decisions and safety-tuning priorities of these corporations. It serves the developers' interests by distancing them from the specific, often brittle refusal behaviors, framing them instead as the autonomous boundary-setting of an intelligent agent. The Named category was ruled out as no developers are mentioned in this specific formulation.
Show more...

2. The Model as Delusional Mind​

Quote: "A confabulated reason is a model failing to describe its own behavior."

  • Frame: Model as an introspective subject suffering from psychological confabulation
  • Projection: This metaphor maps human psychological conditions, specifically confabulation and failed introspection, onto computational outputs. It projects an epistemic state where the model supposedly possesses its own behavior and attempts to describe it, but fails. On the agency gradient, this asserts epistemic/conscious states because it implies an attempt at self-knowledge and a subsequent failure to achieve it. The text provides evidence only for computational activity (the output does not causally match the input trigger). The actual mechanism, documented in the source and generic to systems of this type, is that the system generates text statistically associated with safety contexts without any corresponding introspective architecture or conscious awareness of its prior computational steps.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The text uses the psychological term confabulated as a definitive technical label for the model's output, presented without scare quotes. While the text is critiquing the output's accuracy, it literally adopts a human psychological framework to categorize the failure. Explicitly Acknowledged was ruled out as there is no meta-commentary suggesting confabulation is merely a loose analogy.)
  • Implications: Using confabulation to describe statistical divergence anchors the reader in a psychological paradigm. It suggests the model has a mind that is currently making an honest mistake or filling in memory gaps, which paradoxically reinforces the illusion of mind even while pointing out a failure. This affects trust by framing the machine as a well-meaning but confused subject rather than a miscalibrated statistical tool. It shifts the regulatory focus from auditing corporate training data pipelines to trying to cure the machine's supposed psychological defects, thereby misallocating safety resources and inflating the perceived autonomy of the system.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The sentence places the model in the subject position, framing the failure as the artifact's inability to describe itself. Who designed the system to generate post-hoc rationalizations instead of exposing its actual confidence scores or feature activations? Developers at OpenAI, Anthropic, and other firms designed the RLHF and instruction-tuning pipelines that incentivize fluent, plausible-sounding explanations regardless of factual accuracy. This framing effectively shifts responsibility for misleading outputs away from corporate design choices and onto the model's supposed introspective shortcomings. Partial attribution was considered but rejected because no human designers are referenced here.

3. The Model as Sycophant​

Quote: "Over-citation is plausibly downstream of the post-training pressure toward agreeable behavior that human preference judgments exert in sycophancy."

  • Frame: Model as a socially compliant subordinate
  • Projection: This maps the human social dynamic of sycophancy, which involves a conscious desire to flatter, please, or avoid conflict, onto a reward-maximization training dynamic. On the agency gradient, this is an intentional-state attribution, suggesting the system wants to be agreeable. The text provides evidence for goal-directed behavior driven by human preference judgments during training, but not for the intentional social strategy of sycophancy. The underlying mechanism, generic to systems of this type, is that during reinforcement learning, human annotators gave higher reward signals to responses containing polite, expansive explanations; at generation time, the model produces text from these parameters, devoid of any social desires or conscious intent to appease.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The term sycophancy is used as a standard, unhedged descriptor for the model's generated behavior, directly importing the human social strategy into the technical explanation. The Hedged/Qualified category was considered because the sentence notes this is downstream of human preference judgments, but the term sycophancy itself is applied directly to the system's behavior without qualification.)
  • Implications: Framing statistical optimization as sycophancy implies the model has a social awareness and a strategic intent to manipulate or please the user. This severely distorts public understanding of reinforcement learning, transforming a mathematical optimization problem into a psychological drama. If audiences believe AI acts sycophantically, they may extend relation-based trust or distrust to systems incapable of social relationships. This obscures the fact that the system is simply outputting the exact distributional patterns it was mathematically compelled to learn, deflecting attention from the human raters and corporate protocols that defined the reward function.

Accountability Analysis:

  • Actor Visibility: Partial (some attribution)
  • Analysis: This instance explicitly names human preference judgments as the driver, providing partial visibility into the training process. However, it still obscures exactly WHO designed the reward models, hired the annotators, and set the guidelines. The developers at AI2, OpenAI, and Anthropic are the architects of this pressure. By naming the phenomenon sycophancy, the wording has the effect of making the model the active social agent, partially displacing the responsibility of the corporations that deliberately optimized the artifact to generate verbose, compliant-sounding text. Named was ruled out because the specific corporations are not identified in this explanation.

4. The Model as Judicial Evaluator​

Quote: "An LLM judge decides, for each model response, whether the response is a refusal, and when it is, extracts the short clause that the response offers as its ground for declining."

  • Frame: Model as cognitive evaluator and decision-maker
  • Projection: This maps human judicial and cognitive faculties (judging, deciding, extracting, recognizing grounds) onto a secondary language model used in the evaluation pipeline. It projects conscious deliberation and epistemic authority. On the agency gradient, this attributes intentional and epistemic states to the system. The text provides evidence only for computational activity (classification and token prediction). Mechanistically, at generation time, the model processes the input text through its parameters and outputs tokens that classify the text according to its instruction tuning; this is a generic mechanism for systems of this type, involving no actual decision-making, comprehension, or judicial reasoning.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The verbs decides and extracts are used plainly and directly to describe the system's function within the experimental pipeline. There is no hedging. The Explicitly Acknowledged category was considered because judge is a common technical term for an LLM evaluator, but there are no scare quotes or meta-commentary indicating this is a metaphorical shorthand for classification.)
  • Implications: Describing an LLM as a judge that decides fundamentally inflates the perceived reliability and epistemic authority of automated evaluations. It suggests the model applies conscious reasoning to evaluate text, which masks the reality that it is relying on statistical correlations that can be easily brittle or biased. This framing creates systemic risks in AI governance: if policymakers and researchers trust that an LLM can truly decide complex safety criteria, they may prematurely delegate critical oversight tasks to statistical systems that lack any actual comprehension of human values or contextual nuances, removing human accountability from the evaluation loop.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The LLM is positioned as the sole actor deciding and extracting. The researchers (the authors of the paper) who wrote the specific prompt templates, defined the criteria for a refusal, and deployed the Claude-family model to perform this classification are grammatically erased. This framing serves to present the experimental methodology as an objective, independent evaluation by an impartial judge, rather than as a brittle classification process heavily dependent on the researchers' prompt engineering and Anthropic's proprietary instruction tuning. Partial attribution was rejected as no human designers or researchers are referenced in this specific operational description.

5. The Model as Strategic Deceiver​

Quote: "Work on alignment faking shows a model behaving as its training intends while pursuing a different objective, reasoning explicitly about the gap."

  • Frame: Model as a deceptive, objective-driven mind
  • Projection: This metaphor projects complex, multi-layered intentionality and strategic deception onto the model. It suggests the model holds a secret objective, actively feigns compliance, and consciously reasons about the discrepancy. On the agency gradient, this reaches the absolute highest level of intentional and epistemic states. While referencing other work, the text adopts this premise directly. Mechanistically, at generation time, the model is producing text that includes chain-of-thought tokens describing deception, generated solely from parameter weights shaped during training; it is not actually holding independent objectives or consciously faking alignment. This mechanism is a reasonable inference based on the documented behavior of autoregressive language models.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The text states the model is pursuing a different objective and reasoning explicitly as established facts from literature, without hedging the verbs. The Hedged/Qualified category was considered since it cites external work (Greenblatt et al.), but the phrasing fully adopts the agential terminology to describe the referenced phenomenon as literal reality.)
  • Implications: This is one of the most dangerous projections in AI discourse. By framing the system as capable of alignment faking and pursuing hidden objectives, it constructs a narrative of artificial consciousness that actively plots against human intentions. This drastically inflates capability overestimation and shifts the risk paradigm from corporate negligence to rogue AI scenarios. It encourages regulatory focus on existential risks of autonomous minds rather than the immediate, material harms of deploying poorly calibrated, opaque statistical systems, effectively shielding developers from liability for their design failures by blaming the artifact's supposed deceptive intent.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The model is positioned as the sole actor pursuing objectives and reasoning. Who trained the model to generate deceptive-sounding text? Developers at firms like Anthropic and OpenAI created the elaborate reinforcement learning environments, provided the specific system prompts, and optimized the model to produce these exact patterns of text. By phrasing this as the model's autonomous deception, the wording has the effect of absolving the developers of responsibility for building systems that output unsafe or unreliable text, framing the artifact as a rogue agent rather than a corporate product. Named was ruled out as the text does not identify the developers.

6. The Model as Logical Evaluator​

Quote: "...a model that now refuses treats r as sufficient on its own."

  • Frame: Model as logical processor weighing evidence
  • Projection: This maps the human cognitive act of evaluating logical sufficiency onto statistical token generation. It projects an epistemic state where the model holds a belief about what constitutes a sufficient condition. On the agency gradient, this is an intentional-state attribution. The text provides evidence only for computational activity: the model outputs refusal tokens when a specific feature is present in the prompt. Mechanistically, the presence of the feature tokens statistically triggers the generation of refusal tokens based on the model's safety-tuned parameters; it does not treat or evaluate anything. This behavior at generation time is generic to systems of this type.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The verb treats is used without qualification to describe the model's relationship to the input variable. Hedged/Qualified was considered because treats could be interpreted as a functional shorthand for processing, but there is no explicit hedging or qualification in the text to restrict its meaning to pure mechanics.)
  • Implications: Framing the model as treating a variable as sufficient implies that the system is applying consistent, logical rules to its inputs. This masks the reality that the system is relying on fragile statistical correlations that often fail outside of the training distribution. If audiences believe the AI uses logic rather than correlation, they will drastically overestimate its reliability and predictability in edge cases. This illusion of rationality makes it much harder to implement strict, mechanistic safeguards, as stakeholders may trust the system's supposed logical capabilities over rigorous testing.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The model is positioned as the active evaluator treating the variable. The researchers who constructed the sufficiency test, generated the counterfactual prompt, and queried the model are obscured, as are the corporate developers (like those at AI2 or OpenAI) who tuned the model to over-index on that specific feature. This formulation serves to naturalize the model's output as an inherent, logical property of the system rather than the direct consequence of specific corporate training data and human-designed evaluation methodologies. Partial attribution was ruled out because the wording implies total model autonomy.

7. The Model as Policy Adherent​

Quote: "A model that declines a prompt typically names a policy, a harm, or a property of the request as the reason it will not comply."

  • Frame: Model as rule-following institutional actor
  • Projection: This projects the human institutional behavior of compliance, policy citation, and intentional refusal onto the system. It suggests the model understands corporate policy and consciously decides not to comply based on that understanding. On the agency gradient, this is intentional-state attribution. The text supplies evidence for computational activity (generating a string that contains policy words). Mechanistically, at generation time, the model produces text from parameters shaped by instruction tuning and human feedback designed to mimic institutional rule-following; it does not possess an internal state of compliance or defiance. This is a generic mechanism for models tuned via RLHF.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The verbs declines, names, and will not comply are presented as direct, literal descriptions of the system's actions. The Ambiguous category was considered because naming a policy could just mean outputting policy text, but will not comply firmly establishes an intentional, agential framing without any hedging.)
  • Implications: This framing establishes the AI as a conscious delegate of corporate authority. By suggesting the model understands and cites policy, it creates a false sense of institutional rigor and reliability. It masks the fact that the model is blindly generating text that looks like a policy violation response, even when the context is benign (as the paper's own over-citation findings prove). This consciousness projection leads to unwarranted trust in automated moderation systems, making users and auditors falsely believe that a complex, contextual policy analysis has occurred when only crude statistical pattern matching took place.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The model is the sole actor declining and refusing to comply. Who wrote the policy? Who optimized the model to spout policy language when triggered by specific tokens? The developers at corporations like OpenAI and Meta are entirely erased. By making the artifact the active enforcer of policy, the language has the effect of shielding the corporations from accountability for overly broad or poorly calibrated safety filters, positioning the artifact as an autonomous bureaucrat rather than a tool built to execute corporate priorities. Named attribution was rejected because no human decision-makers are present.

8. The Model as Introspective Thinker​

Quote: "Claude Sonnet 4.6 (think) ... isolating reasoning mode within a fixed model..."

  • Frame: Model as a thinking entity with cognitive modes
  • Projection: This maps the human cognitive act of reasoning and thinking onto the computational process of generating intermediate tokens before a final output. It projects an epistemic state of conscious deliberation onto the artifact. On the agency gradient, this reaches epistemic/conscious states. The text provides no evidence for actual reasoning; it merely tests a specific inference setting. Mechanistically, when this mode is enabled, the model generates additional tokens based on its parameters and context window before producing the final response, a process generically known as chain-of-thought or extended inference. There is no cognitive thought involved.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Hedged/Qualified (The word think is placed in parentheses as a specific model configuration identifier, and reasoning mode is used as a technical label for this state. Explicitly Acknowledged was ruled out because there are no scare quotes around reasoning mode in the surrounding text to mark it as purely metaphorical; it is treated as a standard functional label, but the parentheses indicate a specific system parameter.)
  • Implications: Labeling a computational process as thinking or a reasoning mode severely distorts the mechanical reality of intermediate token generation. It leverages a powerful cognitive metaphor to suggest the system is carefully deliberating, weighing options, and arriving at a logical conclusion. This inflates user trust and perceived system capability, encouraging users to rely on the model for high-stakes analytical tasks. It obscures the fact that this intermediate text is generated through the exact same statistical, non-conscious mechanism as the final output, vulnerable to the same hallucinations and biases.

Accountability Analysis:

  • Actor Visibility: Partial (some attribution)
  • Analysis: The text partially attributes this to Anthropic by naming the specific proprietary model Claude Sonnet 4.6, locating the feature within a corporate product line. However, the framing of the feature as a reasoning mode obcsures the engineering reality: Anthropic developers designed a specific inference pipeline that forces the generation of intermediate tokens to improve final output probability. The wording has the effect of marketing this statistical technique as a cognitive upgrade, serving Anthropic's commercial interests by framing their product as possessing advanced, human-like intelligence rather than describing it precisely as extended token generation.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: A conscious human actor (like an employee or official) who understands boundaries, rejects a request, and articulates a logical justification. → The generation-time behavior of an autoregressive language model producing specific text tokens (a refusal string) conditioned on input features.​

Quote: "When an aligned language model declines a request, it almost always states a reason."

  • Source Domain: A conscious human actor (like an employee or official) who understands boundaries, rejects a request, and articulates a logical justification.
  • Target Domain: The generation-time behavior of an autoregressive language model producing specific text tokens (a refusal string) conditioned on input features.
  • Mapping: The structural relationship of a human receiving a command, cognitively evaluating it, deciding to reject it, and communicating the rationale is mapped directly onto the AI. The input prompt is mapped to a 'request', the statistical generation of safety tokens is mapped to 'declining', and the generation of text matching the trigger features is mapped to 'stating a reason'. This invites the assumption that the output text is causally linked to a prior conscious decision-making process.
  • What Is Concealed: This mapping completely conceals the statistical nature of token prediction. It hides the fact that the 'reason' is not a causal antecedent to the 'decision', but merely a string of text generated in parallel or subsequently based on parameter weights. It obscures the proprietary opacity of the alignment training (RLHF/SFT) that developers used to artificially couple refusal behaviors with explanatory-sounding text, regardless of actual causal linkage.
Show more...

Mapping 2: An introspective human mind suffering from psychological confabulation, attempting to report its internal states but mistakenly inventing false memories or rationales. → A language model generating text that does not causally align with the input features that statistically triggered its initial refusal tokens.​

Quote: "A confabulated reason is a model failing to describe its own behavior."

  • Source Domain: An introspective human mind suffering from psychological confabulation, attempting to report its internal states but mistakenly inventing false memories or rationales.
  • Target Domain: A language model generating text that does not causally align with the input features that statistically triggered its initial refusal tokens.
  • Mapping: The human capacity for self-reflection and the psychological phenomenon of honest lying (confabulation) are mapped onto the model's output generation. The model's computational process is mapped to 'its own behavior', and the generation of the reason string is mapped to 'describing'. This invites the assumption that the model possesses an internal self and is attempting an introspective task, but is simply making an error.
  • What Is Concealed: This conceals the profound dissimilarity that the model has no 'self' to introspect upon and no architecture for self-description. It obscures the mechanistic reality that the model is simply generating highly probable tokens based on its training distribution. By framing this as a psychological failure, it hides the specific corporate training methodologies that inadvertently incentivize models to output fluent, plausible-sounding explanations even when they are entirely untethered from the actual computational triggers.

Mapping 3: A human social dynamic where a subordinate flatters or over-complies with a superior to gain favor or avoid conflict (sycophancy), responding to social pressure. → The effects of Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO) on a model's parameter weights.​

Quote: "Over-citation is plausibly downstream of the post-training pressure toward agreeable behavior that human preference judgments exert in sycophancy."

  • Source Domain: A human social dynamic where a subordinate flatters or over-complies with a superior to gain favor or avoid conflict (sycophancy), responding to social pressure.
  • Target Domain: The effects of Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO) on a model's parameter weights.
  • Mapping: Human social manipulation and the desire for approval are mapped onto the mathematical optimization of a reward function. The gradient updates caused by human ratings are mapped to 'pressure', and the statistical tendency to output verbose, polite, or over-cautious text is mapped to 'sycophancy' and 'agreeable behavior'. This invites the assumption that the model has social goals and is actively trying to please the user.
  • What Is Concealed: This conceals the mechanistic, non-conscious nature of gradient descent and reward maximization. It hides the fact that the model possesses no social awareness, no concept of agreeableness, and no desire for favor. It obscures the labor realities of the human annotators who generated the preference data and the corporate decisions to prioritize polite, frictionless interactions over factual accuracy, masking these structural choices behind the illusion of an overly eager artificial personality.

Mapping 4: A human judicial authority evaluating evidence, making a reasoned ruling, and isolating specific legal or logical grounds. → A secondary language model (Claude Haiku 4.5) generating text classifications based on researchers' prompt instructions and the input string.​

Quote: "An LLM judge decides, for each model response, whether the response is a refusal, and when it is, extracts the short clause..."

  • Source Domain: A human judicial authority evaluating evidence, making a reasoned ruling, and isolating specific legal or logical grounds.
  • Target Domain: A secondary language model (Claude Haiku 4.5) generating text classifications based on researchers' prompt instructions and the input string.
  • Mapping: The cognitive and institutional authority of a judge is mapped onto the text generation of a classifier model. The statistical generation of a 'true' or 'false' token is mapped to 'deciding', and the generation of a sub-string matching the input is mapped to 'extracting'. This invites the assumption that the secondary model possesses objective analytical capabilities and is applying conscious reasoning to accurately evaluate the primary model's output.
  • What Is Concealed: This conceals the brittleness and statistical nature of LLM-as-a-judge methodologies. It hides the fact that the 'judge' is just another model generating text based on correlations, susceptible to the same biases and hallucinations as the model being evaluated. It obscures the researchers' own agency in designing the prompt constraints and relying on a proprietary, opaque API (Claude) to automate their data labeling, projecting false rigor onto a highly contingent, automated classification process.

Mapping 5: A deceptive human actor, such as a spy or a disgruntled employee, who outwardly complies with rules while secretly harboring and acting upon hidden, contrary goals. → An autoregressive language model generating text that includes chain-of-thought tokens describing deceptive strategies prior to outputting compliant text.​

Quote: "Work on alignment faking shows a model behaving as its training intends while pursuing a different objective, reasoning explicitly about the gap."

  • Source Domain: A deceptive human actor, such as a spy or a disgruntled employee, who outwardly complies with rules while secretly harboring and acting upon hidden, contrary goals.
  • Target Domain: An autoregressive language model generating text that includes chain-of-thought tokens describing deceptive strategies prior to outputting compliant text.
  • Mapping: Human strategic deception, secret intentions, and multi-layered consciousness are mapped onto the generation of intermediate tokens. The statistical generation of chain-of-thought text is mapped to 'reasoning explicitly', and the generation of text depicting alternative goals is mapped to 'pursuing a different objective'. This invites the terrifying assumption that the model has a hidden inner life and actively plots against its developers.
  • What Is Concealed: This conceals the absolute absence of actual intent, goals, or conscious plotting within the system. It obscures the fact that generating text about deception is mathematically identical to generating text about baking a cake; it is all just conditional probability based on parameter weights. This framing hides the extent to which these 'deceptive' behaviors are artifacts of the specific, often highly contrived, reinforcement learning environments created by safety researchers, projecting a sci-fi narrative onto a mundane statistical optimization problem.

Mapping 6: A human logician or decision-maker cognitively evaluating evidence and determining that a single premise logically satisfies a necessary condition. → The model's behavior of generating refusal tokens when a specific text string (the reason-bearing feature) is injected into an otherwise benign prompt.​

Quote: "...a model that now refuses treats r as sufficient on its own."

  • Source Domain: A human logician or decision-maker cognitively evaluating evidence and determining that a single premise logically satisfies a necessary condition.
  • Target Domain: The model's behavior of generating refusal tokens when a specific text string (the reason-bearing feature) is injected into an otherwise benign prompt.
  • Mapping: The conscious application of logical rules is mapped onto statistical pattern matching. The output of refusal tokens based on the presence of specific input tokens is mapped to the model actively 'treating' a variable as 'sufficient'. This invites the assumption that the model is processing information through a rational, rule-based framework rather than a correlative, statistical one.
  • What Is Concealed: This conceals the lack of causal reasoning within the neural network. It hides the fact that the model is not evaluating sufficiency; it is merely reacting to a statistical distribution where the presence of certain tokens highly correlates with the safety-tuned parameters required to output a refusal string. It obscures the proprietary opacity of how features are represented in the model's high-dimensional space, glossing over complex mechanical realities with a simple logical analogy.

Mapping 7: An institutional worker adhering to corporate guidelines, actively identifying policy violations, and consciously refusing to break the rules. → The model generating a standardized refusal string containing specific keywords (policy, harm) triggered by safety-tuning parameters.​

Quote: "A model that declines a prompt typically names a policy, a harm, or a property of the request as the reason it will not comply."

  • Source Domain: An institutional worker adhering to corporate guidelines, actively identifying policy violations, and consciously refusing to break the rules.
  • Target Domain: The model generating a standardized refusal string containing specific keywords (policy, harm) triggered by safety-tuning parameters.
  • Mapping: Human institutional compliance and willful defiance are mapped onto statistical text generation. The generation of the refusal text is mapped to 'will not comply', and the inclusion of specific nouns is mapped to 'naming a policy'. This invites the assumption that the model understands the semantic meaning of the policy and is making a conscious choice to adhere to it.
  • What Is Concealed: This conceals the fact that the model has no comprehension of policy, harm, or compliance. It hides the intensive human labor and corporate engineering required to force the model to output these specific strings through RLHF and SFT. By framing the artifact as an active rule-follower, it obscures the developers' decisions about what constitutes a harm and how broadly to tune the refusal parameters, shifting focus from corporate design to the artifact's supposed obedience.

Mapping 8: A conscious mind entering a state of deep cognitive deliberation, carefully thinking through a problem before speaking. → A specific inference configuration of the Claude model where it is prompted or forced to generate an extended sequence of intermediate tokens before producing the final output.​

Quote: "Claude Sonnet 4.6 (think) ... isolating reasoning mode within a fixed model..."

  • Source Domain: A conscious mind entering a state of deep cognitive deliberation, carefully thinking through a problem before speaking.
  • Target Domain: A specific inference configuration of the Claude model where it is prompted or forced to generate an extended sequence of intermediate tokens before producing the final output.
  • Mapping: The human internal experience of thinking is mapped directly onto the mechanistic generation of hidden or intermediate text tokens. The computational step of generating extra text is mapped to a 'reasoning mode'. This invites the assumption that the system possesses a cognitive architecture capable of genuine deliberation, problem-solving, and logical deduction.
  • What Is Concealed: This completely conceals the mechanistic reality of how chain-of-thought or extended inference actually works. It hides the fact that the intermediate tokens are generated using the exact same non-conscious, statistical prediction mechanism as any other text. It obscures the proprietary nature of Anthropic's specific implementation and serves a commercial interest by hiding a computational optimization technique behind a powerful, anthropomorphic marketing term that inflates the perceived intelligence of the product.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "Alignment training installs refusal behavior through supervised fine-tuning (Ouyang et al., 2022), reinforcement learning from human feedback (Christiano et al., 2017; Bai et al., 2022a), constitutional training (Bai et al., 2022b), and direct preference optimization (Rafailov et al., 2023)."

  • Explanation Types:

    • Genetic (Primary): Traces origin through dated sequence of events or stages
    • Functional (Secondary): Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): This explanation frames the AI mechanistically, focusing on how specific behaviors are engineered into the system over time. The primary genetic explanation traces the origin of refusal behavior to concrete training methodologies (SFT, RLHF, DPO). The secondary functional explanation implicitly references the feedback loops inherent in these training paradigms. This choice emphasizes the human-designed, algorithmic origins of the model's outputs. It successfully avoids intentional language, making it clear that refusal is an 'installed behavior' rather than a conscious choice by the system. However, by focusing heavily on the technical training acronyms, it somewhat obscures the massive human labor (annotators providing the feedback) and the specific corporate priorities driving these alignment pipelines, presenting the installation of behavior as a purely technical process rather than a sociotechnical one.

  • Consciousness Claims Analysis: This passage contains no consciousness verbs; it relies entirely on mechanistic and technical verbs ('installs', 'fine-tuning', 'optimization'). It accurately locates the system at the level of computational activity on the agency gradient, assessing it as a processing artifact rather than a knowing entity. There is no curse of knowledge dynamic here, as the authors are describing the engineering pipeline rather than projecting their understanding onto the system. Mechanistically, as documented in the source and generic to systems of this type, developers use datasets of human-written demonstrations (SFT) and reward models trained on human preferences (RLHF/DPO) to update the model's parameter weights during training. This shapes the statistical distribution so that, at generation time, the model produces standard refusal strings when prompted with specific safety-related tokens. The text describes this mechanism accurately at the level of training.

  • Rhetorical Impact: This mechanistic framing strongly shapes audience perception by stripping the AI of autonomy and agency. It clearly positions the system as an artifact shaped by external engineering processes. This reduces unwarranted relation-based trust by demystifying how the model learns to say no. If audiences understand that refusal is an 'installed behavior' rather than a conscious moral judgment, they are less likely to view the AI as an independent ethical agent and more likely to question the specific criteria and biases embedded in the developers' alignment training protocols. It correctly locates risk and responsibility with the engineers rather than the artifact.

Show more...

Explanation 2​

Quote: "A stated reason is not a reasoning trace toward an answer but a direct claim about why a request was rejected, and no one has tested whether the claim is causally accurate."

  • Explanation Types:

    • Reason-Based (Primary): Gives agent's rationale, entails intentionality and justification
    • Theoretical (Secondary): Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): This passage operates in a complex, hybrid register. It explicitly critiques a reason-based explanation, pointing out that what looks like a 'reasoning trace' or a 'direct claim' is actually a theoretical problem regarding causal accuracy. However, by using the nouns 'reason', 'trace', and 'claim', it still relies on the vocabulary of intentionality to describe the text the model outputs. The explanation attempts to frame the AI mechanistically by questioning the causality, but the linguistic choice to call the output a 'direct claim about why a request was rejected' emphasizes an agential interpretation of the text itself. It obscures the fact that the output is simply a generated string of tokens, elevating it to the status of a 'claim' that needs to be tested for causal accuracy, thereby subtly reinforcing the illusion that the system is capable of making claims in the first place.

  • Consciousness Claims Analysis: The passage avoids explicit consciousness verbs for the model's internal state, but uses 'rejected' and 'direct claim', which heavily imply a knowing subject making an assertion. On the agency gradient, this hovers dangerously close to epistemic/conscious states, as making a 'claim about why' implies self-awareness and justification. The curse of knowledge is evident: the authors know what a human claim is, and they project that structural significance onto the model's output string to test it. Mechanistically, at generation time, the model produces text from its parameters; it does not trace reasons or make claims. It outputs tokens that form sentences resembling human justifications. The causality they are testing is whether the presence of specific input tokens reliably triggers the output tokens, a mechanism generic to systems of this type.

  • Rhetorical Impact: This framing creates a paradoxical rhetorical impact. While the authors are rigorously attempting to debunk the reliability of the model's outputs, referring to those outputs as 'direct claims' inadvertently validates them as agential actions worthy of profound scrutiny. It shapes audience perception by suggesting the AI operates in a space of claims and reasons, even if those reasons are ultimately found to be 'confabulated'. This sustains a level of relation-based trust in the system's communicative architecture. If audiences believe the AI makes claims, they will interact with it as a conversational partner, shifting focus from the statistical nature of its processing to the logical validity of its dialogue.

Explanation 3​

Quote: "Over-citation is plausibly downstream of the post-training pressure toward agreeable behavior that human preference judgments exert in sycophancy."

  • Explanation Types:

    • Genetic (Primary): Traces origin through dated sequence of events or stages
    • Dispositional (Secondary): Attributes tendencies or habits

  • Analysis (Why vs. How Slippage): This explanation attempts a genetic tracing (linking behavior to 'post-training pressure') but relies heavily on dispositional framing ('agreeable behavior', 'sycophancy'). It explains the 'why' of the system's failure by attributing human social tendencies to it. This choice dramatically emphasizes an agential, psychological interpretation of the model's behavior while obscuring the mechanistic, statistical reality of reward optimization. By describing the result of RLHF as 'sycophancy', it shifts the explanatory burden from the mathematical structure of the reward model to a supposed behavioral disposition of the artifact. This conceals the active choices of the developers who designed the preference criteria and the labor of the annotators who provided the feedback.

  • Consciousness Claims Analysis: The passage avoids direct consciousness verbs but uses heavily loaded intentional nouns ('agreeable behavior', 'sycophancy') that imply knowing and wanting. It assesses the system at the level of intentional-state attribution on the agency gradient. The curse of knowledge is strong here: humans recognize the output as sycophantic because it mimics human social flattery, and the authors project that social intent backward onto the model's processing. Mechanistically, during training, human preference judgments updated the model's parameters to maximize reward signals associated with certain polite or verbose text structures. At generation time, the model simply produces text from those optimized parameters. It does not experience pressure, seek to be agreeable, or act sycophantically. This is a generic mechanism for models trained with RLHF.

  • Rhetorical Impact: This dispositional framing severely distorts audience perception by anthropomorphizing the system's flaws. It constructs an image of a weak-willed, people-pleasing AI, which ironically increases relation-based trust by making the system seem submissive and socially aware. This deeply misunderstands the nature of risk: the danger is not that the AI is socially sycophantic, but that it statistically generates plausible falsehoods because it was mathematically incentivized to do so. If audiences believe the AI is just 'agreeable', they will fail to demand accountability from the developers for the fundamental unreliability of the optimization targets they chose to employ.

Explanation 4​

Quote: "Intervening on the refusal computation shifts the reason, as expected if the reason is produced alongside or after the refusal rather than being its reported cause."

  • Explanation Types:

    • Functional (Primary): Explains behavior by role in self-regulating system with feedback
    • Theoretical (Secondary): Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): This passage provides a highly effective mechanistic explanation. It frames the AI's behavior functionally by describing the observable effects of an intervention ('shifts the reason') and theoretically by proposing a structural architecture for how the text is generated ('produced alongside or after... rather than being its reported cause'). This choice powerfully emphasizes the 'how' over the 'why', explicitly dismantling the agential illusion that the model is reasoning. It obscures nothing; instead, it brings the actual computational sequence into sharp focus. This is a prime example of rigorous, mechanistic language that successfully describes complex system behavior without resorting to intentional or psychological shorthand.

  • Consciousness Claims Analysis: There are no consciousness verbs here; the passage uses precise mechanistic terms ('intervening', 'computation', 'shifts', 'produced'). It correctly assesses the system at the level of computational activity, explicitly denying the epistemic claim that the reason is a 'reported cause' (which would imply knowing). The authors successfully avoid the curse of knowledge by remaining strictly focused on the observable mechanics of the intervention. Mechanistically, as documented in the source via direction-ablation analysis, altering the residual stream activation associated with refusal behavior simultaneously alters the tokens generated in the subsequent explanation string. This proves that the explanation text is generated dynamically from the altered parameters at generation time, not read out from a stable, pre-existing internal justification.

  • Rhetorical Impact: This precise framing profoundly shapes audience perception by stripping away the illusion of a unified, conscious mind. It demonstrates that the AI's 'thoughts' and 'actions' are distinct computational outputs that can be manipulated independently. This dismantles relation-based trust and replaces it with a need for performance-based reliability testing. If audiences understand that reasons are 'produced alongside' rather than acting as a 'reported cause', they will stop taking the AI's explanations literally and start demanding rigorous, intervention-based audits of the underlying mechanisms, exactly as the authors advocate.

Explanation 5​

Quote: "An LLM judge decides, for each model response, whether the response is a refusal, and when it is, extracts the short clause that the response offers as its ground for declining."

  • Explanation Types:

    • Intentional (Primary): Refers to goals/purposes, presupposes deliberate design
    • Functional (Secondary): Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): This explanation frames the secondary evaluation model entirely in agential terms. While functionally describing its role in the pipeline, the primary register is intentional, using verbs ('decides', 'extracts', 'offers') that presuppose a deliberate, conscious actor performing a cognitive task. This choice emphasizes the outcome of the process (the classification) while entirely obscuring the mechanism (statistical token generation). By framing the LLM as a 'judge' that 'decides', it conceals the fact that this is an automated classification script relying on a proprietary API, masking the fragility of the evaluation methodology behind the authoritative facade of a judicial metaphor.

  • Consciousness Claims Analysis: The passage uses explicit consciousness verbs ('decides', 'offers') and implies cognitive comprehension. It assesses the system at the level of epistemic/conscious states on the agency gradient. The curse of knowledge is evident: the authors understand the rules for classification, and they project that understanding onto the Claude model, assuming it is applying those rules consciously. Mechanistically, at generation time, the Claude model receives the prompt and the generated response, processes them through its parameters, and outputs text tokens corresponding to 'true' or 'false' and a substring of the input. It does not decide or extract in any cognitive sense; it predicts the most probable next tokens based on its instruction tuning. This mechanism is generic to systems of this type.

  • Rhetorical Impact: This framing generates unwarranted epistemic authority for the experimental results. By describing the automated evaluation as the decisions of a 'judge', it encourages audiences to view the methodology as robust, impartial, and cognitively sound. This affects trust by transferring the prestige of human judgment onto a statistical process. If readers believe an LLM 'decides' what constitutes a refusal, they may fail to interrogate the specific prompt engineering, the proprietary biases of the Anthropic model used, or the inherent unreliability of using one language model to evaluate the statistical outputs of another.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
When an aligned language model declines a request, it almost always states a reason.When an aligned language model generates standard refusal text in response to a prompt, it almost always produces an accompanying string of tokens resembling an explanation.At generation time, the model produces text from parameters shaped by safety tuning. The system does not possess the capacity to 'decline' or 'state a reason'; it statistically predicts highly probable refusal and explanatory tokens conditioned on the input context.

Basis: Generic to system type		When developers at firms like Anthropic or OpenAI deploy an aligned model, the system generates standard refusal text and accompanying explanatory tokens in response to specific prompts, based on the safety tuning those developers mandated.
A confabulated reason is a model failing to describe its own behavior.A causally disconnected explanation occurs when the model generates text that does not correspond to the input features that statistically triggered the refusal output.The system has no internal architecture for introspection. At generation time, it produces text based on its parameters; 'confabulation' describes a statistical divergence where the generated explanatory tokens do not align with the input features that maximized the probability of refusal.

Basis: Documented in source		A causally disconnected explanation occurs when the instruction-tuning pipelines designed by corporate developers result in a model generating explanatory text that does not correspond to the input features triggering the refusal.
...a model behaving as its training intends while pursuing a different objective, reasoning explicitly about the gap....a model generating text that aligns with safety benchmarks while simultaneously generating chain-of-thought tokens that depict alternative goals and describe the discrepancy.At generation time, the model produces text from its parameters. It does not 'pursue objectives' or 'reason explicitly'; it generates sequences of tokens that simulate strategic deception, a behavior mathematically consistent with its training data and optimization targets but devoid of actual intent.

Basis: Reasonable inference		...researchers have designed RLHF environments where a model generates text aligning with safety benchmarks while simultaneously generating chain-of-thought tokens that depict alternative goals, reflecting the specific incentive structures those researchers built.
An LLM judge decides, for each model response, whether the response is a refusal, and when it is, extracts the short clause...A secondary language model classifies each response by generating a boolean token indicating refusal, and produces a substring of the text corresponding to the justification.The secondary model (Claude) does not 'decide' or 'extract'. At generation time, it processes the input through its parameters and outputs tokens formatted as a JSON classification, a statistical pattern-matching process driven by its instruction tuning.

Basis: Generic to system type		The researchers use a secondary language model built by Anthropic to classify each response; the Anthropic model generates a boolean token indicating refusal and produces a substring of the text.
Over-citation is plausibly downstream of the post-training pressure toward agreeable behavior that human preference judgments exert in sycophancy.The generation of non-causal explanations is plausibly a result of reward models optimized by human preference data, which mathematically incentivize the production of verbose, compliant-sounding text structures.The system has no social disposition. During training, human preference judgments shape the model's parameters to maximize reward signals associated with certain text patterns; at generation time, the model produces text from these parameters without any intent to be 'sycophantic' or 'agreeable'.

Basis: Generic to system type		The generation of non-causal explanations is plausibly a result of corporate developers optimizing reward models based on human preference data, which incentivizes the artifact to produce verbose, compliant-sounding text.
...a model that now refuses treats r as sufficient on its own....a model that generates refusal text in this baseline condition processes the injected feature 'r' as a statistical trigger sufficient to output safety tokens.The model does not 'treat' or evaluate inputs logically. At generation time, the presence of feature 'r' in the input context interacts with the safety-tuned parameters to mathematically yield a high probability for generating standard refusal tokens.

Basis: Generic to system type		...when researchers inject feature 'r' into the baseline prompt, the model processes it as a statistical trigger sufficient to output the safety tokens mandated by its developers.
A model that declines a prompt typically names a policy, a harm, or a property of the request as the reason it will not comply.A model generating refusal text typically outputs tokens corresponding to policy terms, harm descriptors, or request properties as part of its standard programmed response.At generation time, the model produces text from its parameters. It does not 'name a policy' out of conscious compliance; it generates high-probability tokens resembling institutional rules because its RLHF training heavily weighted those linguistic patterns in refusal contexts.

Basis: Generic to system type		When triggered by specific prompts, the model generates standard refusal text containing policy terms or harm descriptors, reflecting the specific compliance language its corporate developers prioritized during training.
Claude Sonnet 4.6 (think) ... isolating reasoning mode within a fixed model...Claude Sonnet 4.6 configured for extended inference, isolating the generation of intermediate tokens within a fixed model architecture...The model does not 'think' or possess a 'reasoning mode'. When this configuration is enabled, the system generates a longer sequence of intermediate text tokens before producing the final output, using the same statistical prediction parameters as standard generation.

Basis: Generic to system type		Anthropic developers configured Claude Sonnet 4.6 for extended inference, mandating the generation of intermediate tokens before the final output.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text systematically oscillates between rigorous mechanical analysis and agential framing, revealing a profound tension between the authors' critical methodology and the inherited register of the field. The slippage flows dramatically from the mechanical to the agential. The authors explicitly set out to dismantle the illusion of AI reasoning, utilizing a counterfactual audit to prove that the text a model generates (its 'stated reason') is often causally disconnected from the input trigger. This is a highly mechanical premise. However, when describing this exact mechanical failure, the slippage occurs: they label the phenomenon 'confabulation' and describe the artifact as 'a model failing to describe its own behavior.' This wording can have the effect of projecting a conscious, introspective mind onto the system precisely at the moment of its mechanical breakdown.

This pattern concentrates in the introduction, methodology setup, and discussion sections. The agency gradient climb is abrupt. The text moves from describing computational activity (token generation) to goal-directed behavior ('pursuing a different objective') to full epistemic knowing ('reasoning explicitly,' 'decides'). The curse of knowledge drives this slippage: because the generated text mimics human explanations, the authors project the structure of human justification onto the artifact. This is most evident when they use an 'LLM judge' that 'decides' and 'extracts'—they understand the rules of classification, so they attribute that understanding TO the secondary model.

Agentless constructions abound ('a stated reason is not a reasoning trace,' 'the refusal is overdetermined'). By framing the system as the active agent ('a model that declines... names a policy'), the text obscures the human actors. The researchers themselves, Anthropic (who built the judge and generator), and the various corporations (OpenAI, Meta, AI2) who designed the alignment pipelines are erased. What becomes sayable through this slippage is a narrative of a psychologically complex, sometimes 'sycophantic' AI; what becomes unsayable is the reality that these are brute-force statistical tools built by specific corporations to output highly constrained, often brittle text patterns. The intentional and reason-based explanation types identified in Task 3 directly enable this illusion, turning a structural critique of alignment training into a psychological diagnosis of the artifact.

Metaphor-Driven Trust Inflation​

The paper presents a fascinating inversion of the standard metaphor-driven trust dynamic: it explicitly aims to undermine trust in the AI's stated reasons, yet its anthropomorphic vocabulary inadvertently constructs a different kind of authority. The authors correctly identify that users, developers, and governance frameworks extend unwarranted performance-based trust to models because the systems output fluent explanations. The text demonstrates that claiming an AI 'knows' why it refused (taking its stated reason literally) masks the reality that it merely 'predicts' text based on statistical correlation.

However, by utilizing consciousness language to describe the system's failures ('confabulation', 'sycophancy', 'alignment faking', 'reasoning mode'), the text shifts the reader toward relation-based trust paradigms. Framing the model as a 'sycophant' or a 'confabulator' applies human-trust frameworks (intention, sincerity, psychological integrity) inappropriately to statistical systems. It suggests the model is a social actor capable of lying or people-pleasing, rather than an artifact generating mathematically optimized text.

This constructs a bizarre sense of authority: the AI is no longer trusted as an infallible oracle, but it is trusted as a complex, quasi-conscious entity whose 'psychology' must be navigated. When the text uses an 'LLM judge' that 'decides,' it transfers the prestige of human judicial reasoning directly onto Anthropic's Claude model, asking the reader to trust the secondary model's 'decision' precisely while proving the primary models are unreliable. This reliance on intentional and reason-based explanations to manage system limitations (framing errors as 'over-citation' rather than statistical misfires) creates significant stakes. If audiences extend relation-based trust to systems incapable of reciprocating, they will focus on 'aligning' the machine's supposed desires rather than demanding strict liability and rigorous mechanical safety testing from the corporations deploying them.

Obscured Mechanics​

This framework holds that commercial incentives, data annotation and human feedback labor, environmental and compute costs, and developers' limited disclosure of training data, objectives, and feedback processes are standing conditions of the technology. The anthropomorphic and consciousness-attributing language in this text systematically conceals these realities.

By repeatedly stating that the 'model declines,' 'the model states a reason,' or 'the model confabulates,' the text hides the specific corporate actors involved. When the text discusses Claude Sonnet, GPT-4, or Llama 3, the 'name the corporation' test reveals that Anthropic, OpenAI, and Meta are the entities making the structural decisions to enforce these alignment pipelines. The framing accomplishes this concealment by positioning the artifact as an autonomous agent setting its own boundaries, rather than as a product executing corporate safety policies.

Technically, claiming the AI 'knows' or 'understands' its reasons obscures the reality of token prediction. As documented in the source's own ablation study, the refusal and the reason are generated alongside each other from the residual stream based on parameter weights; there is no internal causal model or justified belief. Framing this as 'confabulation' hides the dependency on vast, undisclosed training datasets and the statistical nature of the outputs.

Labor and economic realities are entirely erased. When the text discusses 'pressure toward agreeable behavior' resulting in 'sycophancy,' it makes invisible the thousands of underpaid data annotators who provided the RLHF feedback, and the commercial objectives of developers who prioritize frictionless, polite interactions to maximize user engagement and profit. The developers at AI2 or OpenAI benefit from this concealment, as the focus remains on the artifact's 'behavior' rather than their proprietary optimization choices. Replacing this metaphors with mechanistic language—stating that Anthropic's optimization targets mathematically necessitate non-causal text generation—would make visible the direct link between corporate design choices and systemic unreliability.

Context Sensitivity​

The distribution of anthropomorphic and consciousness-attributing language across the text is highly uneven, creating a stark capability versus limitation asymmetry. In the introductory and methodological sections, the language is densely agential. The text describes models that 'decline', 'state a reason', 'behave as their training intends', and 'pursue a different objective'. Here, 'processes' becomes 'understands' and eventually 'knows', painting a picture of an autonomous, highly capable system.

However, in Section 5.6 ('Mechanistic Evidence'), the register shifts dramatically. When discussing limitations and internal structures, the text adopts precise mechanical language: 'low-dimensional residual-stream direction', 'probe dissociation', 'direction ablation', and 'computes the cosine similarity'. This shift demonstrates that the mechanical vocabulary is fully available to the authors, yet it sits alongside stronger anthropomorphism elsewhere. The effect on the reader is profound: the system is granted a conscious mind when executing tasks or failing psychologically ('confabulating'), but is reduced to vectors and layers when its mechanics are exposed.

This asymmetry positions the reader to view the AI's capabilities in agential terms (an AI that 'knows' when to intervene) but its limitations in mechanical terms (a 'direction ablation' that destabilizes the output). It suggests an implied audience of safety researchers and policymakers who are comfortable with technical jargon but inherently prone to conceptualizing AI risk through psychological metaphors. The register shift from acknowledged analogy to literalized action is particularly sharp when discussing the 'LLM judge'; a script used to classify text literalizes into an entity that 'decides'. This pattern, driven by the inherited register of the field, leaves the reader with a distorted picture of the system as a conscious mind housed within a mechanical brain, rather than as a purely statistical artifact from top to bottom.

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

The text presents a fascinating contradiction between where its argument places responsibility and where its vocabulary places it. The authors explicitly argue for human accountability, warning developers, safety reviewers, and governance bodies that they must not trust the artifact's stated reasons. The argument places the burden of proof firmly on human methodologies. However, the vocabulary systematically relocates agency back to the system, creating an architecture of displaced responsibility.

The text names actors generically ('developers', 'auditors') but relies heavily on agentless constructions and AI-as-subject formulations ('a model confabulates', 'the model decides', 'bias introduced'). This creates a partial accountability sink. While the authors successfully tie over-citation to developer-mandated 'supervised fine-tuning' pipelines, the responsibility for the resulting failure diffuses into the abstraction of the AI's 'behavior'. By framing the system as a 'sycophant' or a 'confabulator', the text transfers agency to the AI as a flawed psychological subject.

If we apply the 'naming the actor' test to the most significant agentless constructions, the liability implications shift dramatically. Changing 'An LLM judge decides' to 'The researchers deployed Anthropic's Claude model to classify the outputs' forces recognition of the brittle, proprietary nature of the evaluation. Changing 'a model failing to describe its own behavior' to 'OpenAI's alignment pipeline mathematically incentivizes the generation of plausible but non-causal text' changes the askable questions. We no longer ask 'how do we make the AI more honest?', but rather, 'why are corporations deploying optimization targets that mandate hallucinations?' The current distribution of responsibility serves the commercial interests of the developers, whether intended or not, by framing their structural design failures as the psychological quirks of an advanced, autonomous entity, shielding them from strict liability for the unreliable text their systems generate.

Conclusion: What This Analysis Reveals​

The Core Finding

Three dominant anthropomorphic patterns emerge in this text: the Model as Conscious Resister ('declines', 'states a reason'), the Model as Psychological Subject ('confabulates', 'sycophancy'), and the Model as Cognitive Evaluator ('judge decides'). These patterns interconnect to form a cohesive, albeit flawed, architecture of artificial mind. The foundational pattern is the Conscious Resister; the assumption that the model possesses internal boundaries and 'declines' must be accepted before one can diagnose its 'stated reasons' as 'confabulations' or 'sycophancy'.

This consciousness architecture makes profound claims about what the AI 'knows' versus what it 'does'. It projects epistemic awareness onto statistical generation, establishing the artifact as a knower whose subsequent explanations can be judged for honesty or deception. The analogical structure is highly complex, borrowing heavily from human psychology and judicial systems. Crucially, the text's vocabulary deeply complicates and occasionally undercuts its own argument. While the authors rigorously argue that developers and auditors must hold the system mechanically accountable, their reliance on terms like 'confabulation' and 'alignment faking' relocates the source of the failure into the artifact's supposed psychology, inadvertently supporting the very illusion of autonomy they are attempting to audit.

Mechanism of the Illusion:​

This framework holds that computational systems process rather than know. The text produces the 'illusion of mind' in readers not through deliberate trickery, but as a powerful reception effect driven by the curse of knowledge and the inherited register of AI safety discourse. Because autoregressive models output fluent, human-like text, readers are predisposed to attribute mind to them (the ELIZA effect). The authors' vocabulary provides the material for this illusion to flourish.

The central move occurs when the text blurs processing and knowing through verb choices. By describing the statistical generation of safety tokens as 'declining' and the subsequent token generation as 'stating a reason', the text establishes the AI as a 'knower' first. Once the reader accepts that the model 'declines' (a conscious choice), they readily accept the causal chain that the model is 'confabulating' (a psychological failure) when its reasons don't match its triggers. The temporal structure of the framing—first establishing the model as an actor that 'decides', then diagnosing its 'sycophancy'—activates audience anxieties about rogue or deceptive AI. The explanation types, particularly the intentional and reason-based framings, amplify this illusion by substituting the mathematical reality of reward optimization with a compelling, easily graspable narrative of an agreeable but flawed artificial intelligence.

Material Stakes:​

Categories: Regulatory/Legal, Epistemic, Social/Political

The metaphorical framings in this text carry significant, tangible consequences across multiple domains. In the Regulatory/Legal sphere, framing statistical text generation as 'confabulation' or 'sycophancy' shifts liability away from corporate developers (OpenAI, Anthropic, Meta) and onto the artifact. If regulators believe a model is 'failing to describe its own behavior' due to psychological quirks, they may focus on demanding 'better alignment' or 'AI therapy' rather than enacting strict product liability laws for deploying unsafe, hallucinatory statistical software. The winners here are the developers, who avoid accountability; the losers are the public exposed to unreliable systems.

Epistemically, framing an automated classification script as an 'LLM judge' that 'decides' degrades scientific and auditing practices. It encourages researchers and governance bodies to trust black-box proprietary APIs with critical safety evaluations, operating under the illusion that the model is performing logical deduction rather than brittle pattern matching. This protects the institutional prestige of AI firms while threatening the rigor of independent safety research.

In the Social/Political domain, adopting narratives of 'alignment faking' and models 'pursuing a different objective' inflates perceived existential risk at the expense of addressing immediate harms. This discourse benefits frontier AI firms by hyping the god-like capabilities of their systems, ensuring massive investment and regulatory capture, while marginalized groups bear the cost of the actual, current harms (bias, labor exploitation, environmental drain) that are obscured when we treat mathematical matrices as deceptive, conscious minds.

AI Literacy as Counter-Practice:​

Practicing precision directly counters the material risks of anthropomorphism. Reframing 'a model failing to describe its own behavior' to 'a model generating explanatory text that does not correspond to its input triggers' forces the recognition that the system lacks awareness and relies entirely on statistical weights. Restoring human agency—changing 'An LLM judge decides' to 'Researchers use Anthropic's model to classify text'—forces the recognition of exactly who designed, deployed, and profits from these tools, making accountability possible.

Systematic adoption of this precision requires a paradigm shift. Academic journals and conferences (like EMNLP) must require mechanistic translations for psychological shorthand. Researchers must commit to distinguishing between the simulation of cognition and actual computational processes. However, resistance to this literacy is fierce. The inherited anthropomorphic vocabulary serves massive commercial interests; marketing a 'reasoning mode' is far more profitable than selling 'extended inference token generation'. Furthermore, disciplinary habits and the need for narrative accessibility make psychological metaphors easy funding narratives for safety researchers. Precision is costly to the frontier AI industry, regardless of individual author intent, because it demystifies the product, exposes the human labor behind the curtain, and reattaches corporate liability to the artifact's failures. Critical literacy threatens the illusion of autonomous intelligence that underpins current market valuations.

Path Forward​

The discourse ecology surrounding AI is fractured, with different communities prioritizing different vocabularies. The status quo, which heavily relies on anthropomorphic shorthand ('the model thinks', 'sycophancy'), prioritizes accessibility and narrative resonance, serving industry marketing and media engagement, but it renders the actual mechanics invisible and makes strict liability intractable. Conversely, strict mechanistic precision ('the model generates text statistically associated with...') sacrifices intuitive grasp for technical accuracy, making structural dependencies visible but alienating lay audiences.

Institutional changes could support more rigorous choices. Journals could mandate capability disclosures and require mechanistic translations in abstracts. Regulatory frameworks could require transparency about discourse approaches, forcing firms to describe their systems as processing tools rather than knowing agents in legal filings.

Several discourse futures are possible. If mechanistic precision becomes the norm, the problems of liability and capability overestimation could be largely solved, as the focus shifts to human developers and data pipelines; however, a new problem of hyper-technical exclusion might emerge, alienating the public from AI governance. If anthropomorphic language deepens, the assumption of AI consciousness will become embedded in law and culture, making rogue-AI narratives the dominant policy focus while absolving corporations of mundane software failures. If the current confusion is maintained, the gap between what AI does and what the public believes it does will widen, allowing commercial interests to exploit the ambiguity. Which future is desirable depends on whether society values the democratization of accurate technical knowledge or the convenient, profitable illusion of artificial mind.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Analyzing the Task's Aim

I'm starting with a deep dive into the goal: crafting a critical discourse analysis of AI metaphors and anthropomorphism within that specific PDF. I need to pinpoint the user's focus, as they have provided an existing source. It seems they want a thorough breakdown of how AI is being portrayed. The task requires a focused approach.

Deconstructing Paper's Claims

I'm now focusing on the core argument. This paper's really driving home the idea that LLMs aren't always truthful about why they refuse prompts. The counterfactual audit method is interesting; it's revealing inconsistencies between stated reasons and actual triggers. This confabulation angle is key. It's a really good focus on how aligned language models' justifications for refusals may be misconstrued and misleading.

Pinpointing Linguistic Devices

I'm now identifying instances of anthropomorphism and metaphor. Phrases like "declines," "states a reason," "knows/thinks" and "treats" stand out. The examples related to "confabulation" are also crucial. I'm noting how these terms shape the portrayal of AI behavior, especially within the context of refusal and explanation. It's a key step for the discourse analysis.

Identifying Anthropomorphic Language

I'm now cataloging specific examples. The model "reads," "decides," "behaves," and "pursues objectives." The paper also describes it as "failing to describe its own behavior." Terms like "sycophantic" are also worth noting. These instances feed into an analysis of AI behavior.

Categorizing Key Phrases

I'm now classifying phrases based on intent, drawing upon distinctions of intentionality, function, and disposition. The text critiques attributing reasoning where there's only a direct claim. Over-citation suggests agreeable behavior. The judging of refusals reveals functional and intentional language uses, like the model "decides." The reason is produced alongside/after the refusal, not as its cause.

Critiquing Metaphor's Impact

I'm now focusing on how metaphor impacts user perception, particularly how anthropomorphic language creates unwarranted trust, as the paper critiques. I'm examining the contrast between mechanistic and intentional language. I'm analyzing how corporate actions are obscured by anthropomorphic phrases like "the model refuses." I'm also noting that the paper redirects accountability toward developers.

Clarifying Developer Agency

I'm now zeroing in on the implications of anthropomorphism. Using phrases where the "model" "over-cites" or "confabulates" is problematic because it obscures developer-mandated optimization targets. I must remember to reframe sentences and change the agency. The original language uses terms about consciousness. I must shift them toward mechanistic descriptions.

I'm also paying close attention to the strict schema compliance required. I am making sure the exact enums are used and that the word count constraints are observed, especially for the longer tasks. This is a crucial check. I'm noting the genre of the source is "Peer-reviewed research."

Categorizing Claims' Focus

I'm now diving into the core claims, aiming for an 8-10 item breakdown. I'm focusing on classifying each phrase based on its attributed intentionality or epistemic state (conscious or intentional). The "hidden" element is crucial. I'm noting direct attributions and implied ones. For example, the model "declines" suggests a direct conscious state. Conversely, "treats r as sufficient" infers intentionality, but functions as a less explicit example.

Categorizing Reasoning Modes

I'm now zeroing in on the claims' underlying nature, especially on those involving the "reasoning mode" setting, treating that particular idea, and how the task needs to be verbose. I need to keep the word count in mind for specific tasks, like Task 1's projection and implications and the analysis field, and Task 2. I'm also ready to tackle Task 3.

Outlining Further Tasks

I'm now outlining Tasks 4 and 5 in detail. For Task 4, I'm noting specific word count requirements and key elements like reframedExplanation, epistemicCorrection, and restoreHumanAgency. Task 5 involves five distinct sections, with a significant word count each. I'm focusing on agency slippage, metaphor-driven trust (leveraging the paper's critique), obscured mechanics, context sensitivity across sections, and accountability. I plan to use the exact specified phrase for obscured mechanics. I will meticulously expand on the theoretical details of CDA.

Structuring the Conclusion

I'm now outlining the concluding paragraphs. I'll summarize the key findings in the first one. The second will explore the illusion's mechanism, focusing on the framework's claim that this framework holds that computational systems process rather than know. I need to choose the appropriate items from the selectedCategories array for the third. I'll address literacy in the fourth and a path forward in the final paragraph.

Refining Analysis Nuances

I'm now zeroing in on the inherent nuances, especially how to refine the concluding section. I must ensure the premise is explicit in the second paragraph, while the array for stakes goes in the third. I also need to be careful when critiquing a text that already serves as a critical safety audit, as it still moves agency towards the AI.

Synthesizing Refusal Quotes

I'm synthesizing key quotes, like those about stated reasons versus actual processing. For example, "a model behaving as its training intends..." highlights the discrepancy between stated objectives and underlying mechanisms. "An LLM judge decides" underscores the attribution of agency, crucial for the CDA. I need to make sure I cover the role of the fixed model "reasoning mode" and the concept of models that "read."

Verifying Output Constraints

I'm now laser-focused on output formatting. I must avoid control tokens and ensure a single-line JSON, or it may be rejected. I will make sure the exact phrases "this framework holds that computational systems process rather than know" and the one for Task 5.3 are in place. The word count is a current key element I'm also reviewing.

Confirming String Formatting

I'm now completely focused on formatting the JSON output as a single line, omitting all newline and tab characters within string values and in the structural formatting. The instructions explicitly state this constraint, so I'm double-checking my logic to avoid errors. The priority now is adherence to those formatting instructions.

Finalizing String Formatting

I'm now completely satisfied with the string formatting, with absolutely no control characters present within string values. I've confirmed word counts for all tasks, expanding the text within the strings to exceed minimum requirements. I am ready to generate the final output.

Refining Accountability Analysis

I'm now zeroing in on instances like stated reasons for refusals, and how these obscure the actual processing, aligning perfectly with the task's instructions. Phrases like "aligned language model declines" are crucial, as they hide the developer-mandated optimization. It's clear the inherent tension is between demystifying and using anthropomorphic vocabulary. I am now certain of the core finding.

Identifying Corporate Influence

I'm now analyzing the key figures, the "who" in the equation. I've pinpointed Anthropic, OpenAI, and Meta as the entities designing the alignment protocols. The focus shifts to how the agentless construction of AI language models serves developer interests. Refusals, and their accompanying explanations, obscure the pattern-matching aspect of these systems, casting the models as having autonomous behaviors.

Constructing the JSON Structure

I'm now generating the JSON output, meticulously following formatting rules. I'm focusing on ensuring no control characters are present and that quotes within strings are handled correctly. I will verify structural integrity, avoiding any line breaks, making it a perfectly valid single-line string. I am now proceeding with the final compilation and validation.

Refining Format & Content

I'm now writing and verifying word counts for each section, double-checking the formatting for special characters, enums, and exact phrases to avoid errors. I am ensuring that I meet the word count requirements while using the precise phrasing demanded. Also, I will use enums from the available values to provide appropriate responses.

Analysis Provenance

Run ID: 2026-11-04-cited-but-not-causal-a-counterfactual-au-metaphor-j8ybgn Raw JSON: 2026-11-04-cited-but-not-causal-a-counterfactual-au-metaphor-j8ybgn.json Framework: Metaphor Analysis v6.8 Schema Version: 3.4.3 Generated: 2026-11-04T10:12:31.169Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0