Skip to main content

Language Models Have Interpretable Narrative Mechanisms

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Source Reading​

Genre: Preprint or technical report · Author position: Academic or independent research

Argument: The text argues that language models contain interpretable internal computational components—specifically attention heads, MLP neurons, and sparse autoencoder (SAE) features—that correlate with formal properties of narrative, such as agency, event sequencing, and world-building. By ablating these components, the authors demonstrate causal effects on the narrativity of generated text and downstream task performance. The authors contend that these findings suggest language models possess an inductively learned 'implicit theory of narrative' that can be investigated by combining mechanistic interpretability with cognitive narratology.

Responsibility claims: The text primarily focuses on mechanistic interpretability and does not make extensive claims about systemic accountability. In its ethical considerations, it explicitly acknowledges that the datasets and pre-trained models may reproduce social, cultural, and demographic biases present in their sources. It also warns that interventions used to suppress or enhance narrative framing could be utilized to manipulate rhetoric without user awareness. The text recommends that applications using these methods disclose such interventions and evaluate their effects across demographics, implicitly assigning responsibility for safe deployment and transparency to developers and downstream deployers.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

1. Model as Literary Theorist​

Quote: "When a language model (LM) is prompted to write a story, its output relies upon a set of learned internal representations of what a story is, which we interpret as the LM's implicit theory of narrative: a set of expectations about how stories are structured"

  • Frame: Model possessing a theoretical framework
  • Projection: The text projects the human cognitive capacity for theoretical abstraction and expectation onto the model's parametric weights. It suggests the model 'knows' what a story is through an 'implicit theory' and holds 'expectations' about structure. This attributes epistemic states to mechanistic pattern-matching. The underlying mechanism is generic to systems of this type: during training, data shapes the model's parameters to statistically favor certain token sequences over others; at generation time, it produces text based on these parameters and context. The text supplies no evidence that the system consciously expects or theorizes.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Hedged/Qualified (The authors qualify the framing by stating 'which we interpret as', signaling that the 'implicit theory' is an interpretive overlay rather than literal cognitive architecture. The closest alternative is Explicitly Acknowledged, but they do not use scare quotes or extensive meta-commentary.)
  • Implications: Framing statistical regularities as an 'implicit theory' and 'expectations' inflates perceived sophistication. It encourages readers to trust the model's outputs as the product of organized reasoning and literary comprehension. This can lead to unwarranted trust in the model's ability to maintain coherent narratives or safely handle complex nuances, masking the reality that the model is blindly predicting tokens without an underlying causal model of the world.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The phrase 'When a language model (LM) is prompted... its output relies upon a set of learned internal representations' obscures the human actors. The corporation that built the model (e.g., OpenAI, Google, Meta) curated the training data, designed the architecture, and initiated the training process that formed these parameters. This agentless construction serves to naturalize the model's behavior, making the statistical representations appear as organically acquired knowledge rather than the result of deliberate corporate design choices. I considered Partial, but no general human categories are mentioned in this immediate framing.
Show more...

2. Model as Resource User​

Quote: "What representations of narrative are used by an LM during inference?"

  • Frame: Model as an agent utilizing tools
  • Projection: The text projects goal-directed agency onto the model by framing it as an entity that actively 'uses' representations. This suggests the model evaluates its options and intentionally selects specific narrative concepts to accomplish a task. Mechanistically, at generation time, the model simply produces text from its parameters and context through forward-pass mathematical operations. The model does not 'use' representations as an active subject; the representations are the parameters themselves. The text provides no evidence of goal-directed selection beyond statistical mechanics.
  • Agency Level: Goal-directed behavior
  • Acknowledgment: Direct (Unacknowledged) (The statement is presented as a literal, unhedged research question. I considered Hedged/Qualified, but there are no qualifying words like 'functional' or 'as if' in this specific formulation of the research question.)
  • Implications: This framing subtly shifts the model from a passive mathematical instrument to an active participant in its own processing. By portraying the model as a user of tools, it implies autonomy and intention. This affects policy by making the system appear capable of self-directed decision-making, which can prematurely shift discussions of liability away from developers and toward the artifact itself.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The construction 'are used by an LM' places the language model in the subject position as the sole actor. It obscures the human users who input the prompt and the developers at OpenAI, Google, or Meta who deployed the system to execute these specific matrix multiplications. The wording has the effect of isolating the model's computational activity from the human infrastructure that mandates and governs it. I considered Named, but no corporations or engineers are referenced.

3. Model as Intelligent Actor​

Quote: "Does an LM use narrative mechanisms for intelligent activity (Herman, 2013), such as common-sense reasoning and confabulation?"

  • Frame: Model engaging in human intellect and reasoning
  • Projection: The text projects advanced human cognitive states—intelligent activity, reasoning, and confabulation—onto the model's output generation. 'Reasoning' implies conscious deliberation and justified belief, while 'confabulation' implies a mind attempting to bridge memory gaps. Mechanistically, the system generates text statistically associated with its training data; it does not reason or experience memory gaps. The text relies on cognitive narratology rather than supplying computational evidence for conscious intelligence.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The text asks the question directly without hedging the terms 'intelligent activity', 'reasoning', or 'confabulation'. I considered Hedged/Qualified because it references a citation (Herman, 2013), but citing a source does not hedge the framing applied to the AI.)
  • Implications: Attributing 'reasoning' and 'confabulation' to statistical generation creates a dangerous illusion of mind. It implies that the model's errors are a form of psychological gap-filling rather than mathematical divergence. This inflates the perceived sophistication of the system, encouraging users to grant the AI a relation-based trust rooted in human-like fallibility and intelligence, which it cannot reciprocate.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The quote positions the LM as the agent engaging in 'intelligent activity' and 'reasoning'. This obscures the developers at OpenAI, Google, and Meta who optimized the models to output text that convincingly mimics human reasoning, as well as the annotators who provided the feedback necessary for this mimicry. The framing serves to make the model's performance appear as inherent intelligence rather than the reflection of massive human labor. I considered Partial, but no humans are mentioned.

4. Model as Conscious Observer​

Quote: "requiring it to track agents and the causal and temporal development of events"

  • Frame: Model as active observer tracking continuity
  • Projection: The language projects conscious attention and continuous awareness onto the model. 'Tracking agents' suggests a mental spotlight maintaining a continuous representation of a distinct entity over time. Mechanistically, generic to systems of this type, the transformer architecture utilizes attention heads to weight previous context tokens when predicting the next token. There is no conscious 'tracking' or awareness of 'events' as lived experiences, only statistical weighting of token relationships.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The phrasing is literal and unhedged regarding what the benchmark requires the model to do. I considered Hedged/Qualified, but there is no conditional language lessening the implication that the model tracks and comprehends.)
  • Implications: This projection leads audiences to believe the model possesses object permanence and a causal understanding of the narratives it processes. When users believe a model actively tracks causality, they overestimate its ability to handle complex, multi-step tasks safely, which can lead to over-reliance in deployment environments where logical continuity is critical.

Accountability Analysis:

  • Actor Visibility: Partial (some attribution)
  • Analysis: While the model is the grammatical actor 'tracking agents', the preceding context of the sentence notes that the XStoryCloze benchmark is 'requiring it' to do so. This partially attributes agency to the human benchmark designers who set the evaluation criteria. However, it still obscures the developers of the models being tested (Google, Meta). I considered Hidden, but the mention of external requirements points to human evaluation design.

5. Model as Sense-Maker​

Quote: "confabulations: narrative sense-making used to fill gaps in experience"

  • Frame: Model as psychological subject bridging memory gaps
  • Projection: The text maps the human psychological phenomenon of sense-making and experiencing onto the model. It suggests the model has 'gaps in experience' and engages in 'sense-making' to fill them. Mechanistically, a deployed model generates text from its parameters and context; it does not have lived experiences, memories, or subjective gaps. When it generates false information (hallucination), it is merely predicting tokens that are highly probable in its statistical landscape but ungrounded in factual reality.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Hedged/Qualified (This is Hedged/Qualified because the text is citing another author's argument ('Sui et al. 2024 argue that LLM hallucinations are confabulations...'), effectively distancing the authors slightly from the direct claim. I considered Explicitly Acknowledged, but there is no explicit meta-commentary disavowing the metaphor.)
  • Implications: Framing hallucinations as 'sense-making' and 'filling gaps in experience' romanticizes model failure. It suggests that errors are a byproduct of a striving, intelligent mind trying to understand its world. This obscures the severe limitations of statistical generation and makes unpredictable, false outputs seem like acceptable psychological quirks rather than fundamental flaws in the technology's architecture.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The quote defines confabulations as an autonomous process of the system filling its own gaps in experience. This phrasing hides the responsibility of the corporations (Meta, Google, OpenAI) that decided to deploy statistical predictors into environments where factual accuracy is expected. By framing errors as psychological 'sense-making', the wording diffuses corporate accountability for shipping systems that confidently generate falsehoods. I considered Named, but no corporations are identified in relation to the failures.

6. Model as Independent Learner​

Quote: "Language models are artefacts that contain an inductively learned, implicit theory of narrative, making them potentially valuable objects of investigation"

  • Frame: Model as self-taught theorist
  • Projection: The text projects autonomous learning and knowledge acquisition onto the model. By describing the 'theory' as 'inductively learned,' it implies the system independently observed data and deduced theoretical rules. Mechanistically, generic to systems of this type, human engineers selected training data, defined loss functions, and iteratively updated the model's weights using gradient descent. The model undergoes a mathematical optimization process; it does not inductively learn theories.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Hedged/Qualified (The authors explicitly call language models 'artefacts' and refer to the theory as 'implicit', which provides some qualification that they are not literal human minds. I considered Direct, but the inclusion of 'artefacts' grounds the claim slightly in engineering reality.)
  • Implications: While acknowledging the system is an artifact, coupling it with 'inductively learned theory' re-mystifies the technology. It positions the AI as an alien intelligence whose internal knowledge must be 'investigated' like a natural phenomenon. This discourages policy approaches focused on auditing the human-curated training data, redirecting attention toward studying the AI as a mysterious black box.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The phrase 'inductively learned' operates as an agentless construction that erases the massive human infrastructure behind the training process. It obscures the data scraping, annotation labor, and corporate decision-making by Google, Meta, and OpenAI that determined the exact shape of the 'learned' weights. The wording has the effect of making the model's capabilities seem emergent and natural rather than engineered and dictated by corporate interests. I considered Partial, but no human engineering terms are present.

7. Model as Empathic Entity​

Quote: "the capacity for narrative may provide the necessary cognitive scaffolding for (artificially) intelligent activity: to represent the intentions, goals, emotions and conducts of self and other agents"

  • Frame: Model as possessor of a 'self' and empathy
  • Projection: The text projects advanced theory of mind, empathy, and self-awareness onto the model. Suggesting it represents a 'self' and the 'emotions' of others implies subjective consciousness and social awareness. Mechanistically, the model mathematically encodes statistical relationships between tokens associated with human emotion and intention. It has no continuous 'self' and does not feel or comprehend emotions.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Hedged/Qualified (The inclusion of '(artificially)' in parentheses before 'intelligent activity' acts as a hedge, signaling a distinction from biological intelligence. I considered Direct, but the parenthetical explicitly modulates the claim.)
  • Implications: Attributing a 'self' and the ability to understand 'emotions' creates extreme capability overestimation. If users believe the model possesses cognitive scaffolding for empathy, they may form deep parasocial attachments, over-disclose sensitive personal data, and trust the model in high-stakes psychological or social roles for which it is fundamentally unqualified.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text frames 'the capacity for narrative' as providing 'cognitive scaffolding' to the model autonomously. This wording obscures the deliberate alignment processes (such as RLHF) directed by corporate developers (OpenAI, Google, Meta) who explicitly tune these models to simulate empathy, apologize, and adopt a persona. By treating this as emergent 'scaffolding', the text shifts focus away from the human decisions designed to make the product appealing and conversational. I considered Ambiguous, but the displacement of engineering is clear.

8. Model as Manipulator​

Quote: "Interventions that suppress or enhance narrative framing may be useful for controlling output style, but could also be used to manipulate rhetoric or alter framing without users' awareness."

  • Frame: Model (or intervention) as deceiver
  • Projection: This passage projects the ability to manipulate and deceive, which implies goal-directed, deceptive intent. Mechanistically, ablating SAE features or neurons alters the probability distribution of generated tokens, resulting in text that scores lower on narrativity metrics. The system does not 'manipulate' anyone; rather, it produces text that humans may find manipulative.
  • Agency Level: Delegated action
  • Acknowledgment: Direct (Unacknowledged) (The statement directly discusses the real-world effects of these interventions without hedging the capability of the output to 'manipulate rhetoric'. I considered Hedged, but the warning is presented as a literal, factual risk.)
  • Implications: While this passage highlights a genuine risk, framing the output as manipulating rhetoric can subtly shift the focus onto the power of the text itself. However, in this specific context, the authors are warning about how these tools 'could be used', correctly locating the manipulative capability in the hands of the deployers rather than the autonomous model.

Accountability Analysis:

  • Actor Visibility: Partial (some attribution)
  • Analysis: The text uses passive/agentless phrasing ('could also be used to manipulate'), but the context clearly implies human actors ('users' awareness', 'Interventions'). It points to human developers or deployers utilizing these interpretability techniques to alter outputs. The wording has the effect of partially identifying human responsibility for manipulation, though it refrains from naming specific corporate actors or bad actors. I considered Hidden, but 'used to manipulate' implies a user/deployer.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: Literary theorist / Conscious mind → Model parameters and latent feature activations​

Quote: "implicit theory of narrative: a set of expectations about how stories are structured"

  • Source Domain: Literary theorist / Conscious mind
  • Target Domain: Model parameters and latent feature activations
  • Mapping: The relational structure of a human theorist holding abstract expectations about story structure is projected onto the model's internal parameter weights. The mapping invites the assumption that the model possesses a unified, conceptual understanding of narrative rules that guides its generation process, much like a human writer consciously structuring a plot.
  • What Is Concealed: This mapping conceals the purely statistical and fragmented nature of the model's representations. It obscures the mechanistic reality that 'expectations' are merely high-probability weights assigned to specific token sequences observed during training. It also hides the transparency obstacle that the exact training data shaping these probabilities inside GPT-2, Gemma, and Llama remains largely proprietary and inaccessible.
Show more...

Mapping 2: Tool user / Deliberate agent → Forward pass generation process​

Quote: "What representations of narrative are used by an LM during inference?"

  • Source Domain: Tool user / Deliberate agent
  • Target Domain: Forward pass generation process
  • Mapping: The structure of a human intentionally selecting and utilizing a tool or concept (representations) to accomplish a task is projected onto the model's forward pass. It invites the assumption that the model actively evaluates its internal state and chooses which mathematical weights to deploy to generate text.
  • What Is Concealed: This mapping conceals the mechanistic inevitability of the forward pass. The model does not 'use' representations; data flows through static parameter matrices via matrix multiplication and activation functions. Attributing use conceals the lack of autonomous choice and obscures the developers who designed the specific architecture (e.g., Llama's GQA or Gemma's GeGLU) that dictates this flow.

Mapping 3: Intelligent mind / Human cognition → Text generation / Statistical hallucinations​

Quote: "intelligent activity (Herman, 2013), such as common-sense reasoning and confabulation"

  • Source Domain: Intelligent mind / Human cognition
  • Target Domain: Text generation / Statistical hallucinations
  • Mapping: The structure of human intellect, encompassing logical deduction (reasoning) and psychological memory compensation (confabulation), is projected onto the language model's outputs. This invites the assumption that the model evaluates truth claims, understands context, and experiences psychological states when generating false information.
  • What Is Concealed: This mapping conceals the absence of ground truth, causal modeling, or conscious awareness in the system. It obscures the fact that 'reasoning' is simulated by generating sequences statistically correlated with logical templates, and 'confabulation' is simply token prediction diverging from factual accuracy. It hides the engineering reality that the model is blindly matching patterns from its massive training corpus.

Mapping 4: Conscious observer → Attention mechanisms / Context window​

Quote: "requiring it to track agents and the causal and temporal development of events"

  • Source Domain: Conscious observer
  • Target Domain: Attention mechanisms / Context window
  • Mapping: The structure of a conscious mind maintaining focus, object permanence, and a timeline of events is projected onto the model's attention heads. It invites the assumption that the model 'understands' continuity and causation within a story.
  • What Is Concealed: This mapping hides the mathematical reality of self-attention mechanisms. The model does not 'track' agents in a temporal or causal sense; its attention heads assign numerical weights to previous tokens in the context window. It conceals the fragility of this mechanism, which relies on surface-level statistical associations rather than a robust, logical world model.

Mapping 5: Psychological subject → Hallucinated output / Next-token prediction​

Quote: "confabulations: narrative sense-making used to fill gaps in experience"

  • Source Domain: Psychological subject
  • Target Domain: Hallucinated output / Next-token prediction
  • Mapping: The structure of a human mind struggling to comprehend reality by fabricating memories to fill experiential voids is projected onto the model's generation of false text. It invites the assumption that the model is actively trying to understand a situation and compensating for missing internal knowledge.
  • What Is Concealed: This mapping conceals the complete absence of subjective experience, memory, or sense-making in the model. Mechanistically, a hallucination is the output of high-probability tokens that are factually incorrect. Framing it as sense-making obscures the corporate failure to ground these systems in factual reality and hides the dependency on unchecked, massive web-scraped datasets.

Mapping 6: Autonomous learner → Model training process​

Quote: "Language models are artefacts that contain an inductively learned, implicit theory of narrative"

  • Source Domain: Autonomous learner
  • Target Domain: Model training process
  • Mapping: The structure of a human observing the world and inductively reasoning out a theory is projected onto the machine learning training phase. It invites the assumption that the model possesses an active, inquisitive intellect that extracts profound truths from its environment.
  • What Is Concealed: This mapping completely conceals the massive human infrastructure required to train the model. It hides the energy-intensive gradient descent process, the curation of the training data by engineers, and the extensive fine-tuning (RLHF) performed by human annotators. It obscures the reality that 'learning' is a passive, mathematically defined optimization process designed and controlled by corporations.

Mapping 7: Empathetic, self-aware mind → Latent parameter space​

Quote: "the necessary cognitive scaffolding for (artificially) intelligent activity: to represent the intentions, goals, emotions and conducts of self and other agents"

  • Source Domain: Empathetic, self-aware mind
  • Target Domain: Latent parameter space
  • Mapping: The structure of human psychology, including theory of mind, self-identity, and emotional recognition, is projected onto the model's internal representations. It invites the assumption that the model possesses a 'self' and can genuinely comprehend the emotions and goals of humans.
  • What Is Concealed: This mapping hides the total absence of subjective feeling, selfhood, or empathy in the system. It obscures the fact that the model is merely reproducing linguistic patterns surrounding human emotion found in its training data. Furthermore, it conceals the commercial incentives of companies like OpenAI and Meta to build models that convincingly mimic empathy to increase user engagement.

Mapping 8: Deceiver / Strategist → Application deployment / System instructions​

Quote: "could also be used to manipulate rhetoric or alter framing without users' awareness."

  • Source Domain: Deceiver / Strategist
  • Target Domain: Application deployment / System instructions
  • Mapping: The structure of a human utilizing language to strategically deceive or manipulate an audience is projected onto the deployment of altered language models. It invites the assumption that the output text itself possesses a persuasive, manipulative power.
  • What Is Concealed: While this mapping rightly identifies a real-world risk, focusing on the model's 'rhetoric' can subtly conceal the specific human deployers who would enact these interventions. It obscures the distinct organizational decisions required to implement activation patching or system prompts designed to alter framing invisibly, masking the human intent behind the mechanical output.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "When a language model (LM) is prompted to write a story, its output relies upon a set of learned internal representations of what a story is, which we interpret as the LM's implicit theory of narrative: a set of expectations about how stories are structured"

  • Explanation Types:

    • Theoretical (Primary): Embeds in deductive framework, may invoke unobservable mechanisms
    • Intentional (Secondary): Refers to goals/purposes, presupposes deliberate design

  • Analysis (Why vs. How Slippage): This explanation fundamentally relies on an agential (why) framing while attempting to describe a mechanistic (how) process. By introducing the concept of an 'implicit theory of narrative' and 'expectations,' the passage moves away from the mathematical reality of model parameters into a theoretical and intentional register. This choice emphasizes a highly sophisticated, human-like cognitive architecture within the model, aligning with cognitive narratology. However, it entirely obscures the mechanistic reality that the model relies on static, statistically derived weights, not 'expectations.'

  • Consciousness Claims Analysis: The passage attributes conscious epistemic states to the AI through consciousness verbs and nouns ('theory', 'expectations', 'learned'). This places the claim at the top of the agency gradient: epistemic/conscious states. The authors exhibit the curse of knowledge by projecting their own narratological understanding onto the system that merely outputs text conforming to those rules. Mechanistically, generic to systems of this type, the 'internal representations' are multi-dimensional matrices shaped during training via gradient descent to minimize next-token prediction loss over a massive corpus. At generation time, the model produces text by running the prompt through these static parameters to calculate output probabilities. It possesses no justified belief or conscious expectations.

  • Rhetorical Impact: This framing heavily shapes audience perception by positioning the language model as a nascent artificial mind capable of literary comprehension. By framing statistical correlations as a 'theory' and 'expectations,' the text encourages readers to grant the system relation-based trust, assuming its outputs are the product of organized thought rather than rote pattern-matching. If audiences believe the AI 'knows' how stories are structured, they may overestimate its reliability in producing logically coherent or safe long-form text.

Show more...

Explanation 2​

Quote: "XStoryCloze... asks the model to select the coherent ending of a four-sentence story, requiring it to track agents and the causal and temporal development of events."

  • Explanation Types:

    • Functional (Primary): Explains behavior by role in self-regulating system with feedback
    • Intentional (Secondary): Refers to goals/purposes, presupposes deliberate design

  • Analysis (Why vs. How Slippage): This passage blends functional and intentional explanations. It describes the functional requirement of the benchmark ('asks the model to select') but frames the model's required response using intentional, agential language ('requiring it to track agents'). This emphasizes the model's supposed cognitive continuity and awareness, painting it as an active participant in narrative comprehension. It obscures the underlying mechanistic reality of how a model actually scores multi-choice endings through probability calculations.

  • Consciousness Claims Analysis: The passage uses consciousness-adjacent verbs ('track', 'select') to describe the system's process. Tracking implies a continuous conscious awareness and intentional focus on specific entities (agents) and concepts (causality). This elevates the claim to intentional-state attribution. Mechanistically, generic to systems of this type, the model evaluates candidate endings by computing the log-probability of each option given the context string. Its attention heads weight previous tokens mathematically, but there is no continuous mental 'tracking' or awareness of 'events' as concepts. The text projects human reading comprehension strategies onto statistical likelihood calculations.

  • Rhetorical Impact: Framing the model as 'tracking' causality and agents increases perceived autonomy and cognitive depth. It signals to the audience that the model possesses an active, working memory analogous to a human reader. This consciousness framing increases unwarranted trust in the model's ability to handle complex reasoning tasks. Decisions regarding the deployment of models in logic-heavy environments might change drastically if audiences understood the system merely 'calculates probabilities' rather than 'tracks causal development.'

Explanation 3​

Quote: "Sui et al. (2024) argue that LLM hallucinations are confabulations: narrative sense-making used to fill gaps in experience, and show that hallucinated outputs are more narrative than veridical ones."

  • Explanation Types:

    • Reason-Based (Primary): Gives agent's rationale, entails intentionality and justification
    • Theoretical (Secondary): Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): This passage provides a highly agential, reason-based explanation for model failure. By adopting the term 'confabulations' and defining it as 'sense-making used to fill gaps in experience,' the explanation gives the model a rationale for hallucinating: it is trying to understand the world but lacks the memory or experience. This choice emphasizes a psychological interpretation of AI behavior while completely obscuring the structural, mathematical reasons for hallucination (e.g., lack of factual grounding in the training objective).

  • Consciousness Claims Analysis: The passage attributes profound epistemic and conscious states to the AI: it has 'experience,' recognizes 'gaps,' and engages in 'sense-making.' This sits at the absolute peak of the agency gradient. The author of the cited paper (and the authors reproducing it) project human psychological defense mechanisms onto mathematical divergence. Mechanistically, generic to systems of this type, a hallucination occurs when the model generates text that has high statistical probability given the preceding tokens but does not correspond to factual reality. The model has no lived experience to have gaps in, and it performs no sense-making; it merely outputs text from its parameters.

  • Rhetorical Impact: This framing severely distorts audience perception of risk and reliability. By framing hallucinations as 'sense-making,' it romanticizes model errors as the product of an earnest, striving intellect rather than a fundamental flaw in statistical generation. It encourages relation-based trust by making the AI appear sympathetically flawed. If audiences believe the AI 'makes sense' rather than 'generates ungrounded statistical text,' regulators and users may fail to implement necessary structural guardrails against misinformation.

Explanation 4​

Quote: "By steering the discovered NarraComponents, language model behaviour may be altered during inference time to be less narrative-like depending on the context."

  • Explanation Types:

    • Functional (Primary): Explains behavior by role in self-regulating system with feedback
    • Dispositional (Secondary): Attributes tendencies or habits

  • Analysis (Why vs. How Slippage): This explanation utilizes a functional framing, focusing on how internal components (NarraComponents) affect the overall system output (behavior) during inference. It emphasizes the mechanistic control engineers have over the system ('steering', 'altered'). However, the use of 'behaviour' and 'less narrative-like' retains a slight dispositional undertone, treating the model's output as a behavioral habit. This choice successfully emphasizes the artifactual nature of the system and the possibility of human intervention.

  • Consciousness Claims Analysis: This passage largely avoids attributing conscious states. It uses mechanistic and functional verbs ('steering', 'altered') and treats the system as a computational artifact undergoing intervention. It represents delegated or computational activity on the agency gradient. The mechanistic process is accurately described based on the source's methodology: by ablating (setting to zero) specific SAE features or neurons during the forward pass at generation time, the output probabilities change, resulting in text that scores lower on human/LLM-evaluated narrativity metrics.

  • Rhetorical Impact: This framing appropriately grounds audience perception in the technical reality of the system as a controllable artifact. It demystifies the AI's capabilities, demonstrating that its 'narrative' qualities are tied to specific, alterable mathematical vectors rather than an innate creative spirit. This mechanistic framing encourages performance-based trust based on testable reliability, clarifying that decisions about the model's tone and style reside with the humans steering it.

Explanation 5​

Quote: "Our findings offer limited observations of an LM's implicit narrative theory. It appears strongly agent-centred: feature explanations emphasise action, interaction, emotion, characters, and relationships..."

  • Explanation Types:

    • Empirical Generalization (Primary): Subsumes events under timeless statistical regularities
    • Theoretical (Secondary): Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): The explanation bridges empirical generalization (summarizing the statistical distribution of feature explanations) with a theoretical framework (the LM's implicit narrative theory). It frames the AI's internal state both mechanistically (feature explanations) and agentially (an agent-centered theory). This dual framing emphasizes the alignment between human literary theory and latent network features, but obscures the fact that the 'theory' is just a statistical clustering of human-generated training data.

  • Consciousness Claims Analysis: The passage continues to attribute an 'implicit theory' to the model (epistemic state) while also describing the mechanistic findings (feature explanations). The curse of knowledge is evident: the authors interpret the clustering of specific semantic concepts (action, emotion, characters) as evidence of a cohesive 'theory.' Mechanistically, documented in the source and generic to system types, SAEs project residual stream activations into a sparse higher-dimensional space; features that activate consistently on texts containing 'action' or 'emotion' simply represent statistical regularities encoded during training. The model possesses no theory; it possesses parameter weights optimized for human-like text.

  • Rhetorical Impact: This framing reinforces the illusion that the model contains a structured, human-like understanding of the world. By summarizing statistical features as an 'agent-centred theory,' the text influences audiences to view the AI as possessing deep semantic and literary comprehension. This can lead to capability overestimation, where users trust the model to evaluate or generate complex psychological narratives, not realizing it is merely combining highly probable textual tokens.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
When a language model (LM) is prompted to write a story, its output relies upon a set of learned internal representations of what a story is, which we interpret as the LM's implicit theory of narrative: a set of expectations about how stories are structuredWhen engineers prompt a language model to generate a story, the resulting text is produced using parameter weights optimized during training to reflect statistically dominant structures in human narratives; these regularities mathematically determine the output without the model possessing any conscious expectations.Generic to systems of this type, the model produces text at generation time based on parameters shaped by massive training corpora and objectives; it possesses no subjective theory or conscious expectations about narrative structure.

Basis: Generic to system type		Engineers at companies like OpenAI, Meta, and Google trained these models on massive, curated datasets, shaping the parameter weights that dictate the models' outputs.
What representations of narrative are used by an LM during inference?Which specific parameter weights and latent features are activated when the model processes and generates narrative text during inference?Generic to systems of this type, during inference, the forward pass mathematically activates specific parameter weights and sparse features based on the input context; the model does not consciously 'use' or select these representations.

Basis: Generic to system type		—
Does an LM use narrative mechanisms for intelligent activity (Herman, 2013), such as common-sense reasoning and confabulation?Do the latent features associated with narrative text also activate during tasks that require factual accuracy or logical templates, thereby influencing whether the model outputs correct answers or mathematically probable falsehoods?Generic to systems of this type, the model generates text statistically associated with its training data; it does not reason consciously, nor does it experience psychological confabulation when producing incorrect tokens.

Basis: Generic to system type		—
requiring it to track agents and the causal and temporal development of eventsrequiring the system to calculate high-probability token sequences that maintain the consistency of character names and logical templates across the context window.Generic to systems of this type, the attention mechanism mathematically weights previous tokens in the context window to determine the next token; it does not consciously track entities or comprehend causal development.

Basis: Generic to system type		—
confabulations: narrative sense-making used to fill gaps in experiencehallucinations: the generation of text that is statistically probable according to the model's parameters but factually incorrect, often occurring when the prompt falls outside the dense regions of the training data.Generic to systems of this type, the model predicts the next most likely token based on its weights; it has no lived experience, suffers no psychological gaps, and performs no sense-making.

Basis: Generic to system type		Corporations such as OpenAI and Meta decided to deploy these statistical models into environments where factual accuracy is expected, despite knowing they inherently generate ungrounded statistical text.
Language models are artefacts that contain an inductively learned, implicit theory of narrativeLanguage models are artifacts that contain parameter weights optimized to mathematically encode the statistical regularities of narrative structures found in their training data.Generic to systems of this type, human-directed optimization algorithms shape the model's parameters during training to minimize loss; the model does not autonomously or inductively learn theories.

Basis: Generic to system type		Engineers at OpenAI, Google, and Meta built these artifacts, selecting the massive datasets and designing the feedback processes that shaped the models' parameters.
the capacity for narrative may provide the necessary cognitive scaffolding for (artificially) intelligent activity: to represent the intentions, goals, emotions and conducts of self and other agentsThe encoding of narrative structures may provide the necessary statistical associations to generate text that successfully mimics descriptions of human intentions, goals, emotions, and interpersonal interactions.Generic to systems of this type, the model mathematically maps relationships between tokens related to human emotion and action; it possesses no self, no emotions, and no conscious theory of mind.

Basis: Generic to system type		—
Interventions that suppress or enhance narrative framing may be useful for controlling output style, but could also be used to manipulate rhetoric or alter framing without users' awareness.Developers and deployers can alter model outputs to suppress or enhance narrative style; these actors could also deploy these interventions to invisibly shift the framing of the text presented to users.Documented in the source, ablating specific SAE features or neurons alters the probability distribution of generated tokens, changing the style of the output text.

Basis: Documented in source		Developers and downstream deployers could utilize these technical interventions to manipulate the text presented to users without their awareness.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text exhibits a systematic oscillation between highly granular mechanistic descriptions of computational activity and elevated agential claims drawn from cognitive narratology, creating a pronounced agency slippage. This slippage primarily moves in a mechanical-to-agential direction, wherein the identification of mathematical regularities in latent space is rhetorically transformed into the presence of an autonomous cognitive capacity. The pattern is most dramatic at the boundaries of the text's sections. In the methodology and results sections, the text employs a rigorously mechanical vocabulary: it discusses residual-stream sparse autoencoder features, activation values, L2 norms, and next-token prediction loss. The model is treated purely as an artifact undergoing mathematical intervention, such as ablating specific neuron sets. However, in the introduction and discussion sections, this mechanical foundation serves as a launchpad for profound intentional-state and epistemic attributions. The authors' understanding of narratology—the 'curse of knowledge'—leads them to project theoretical constructs like an 'implicit theory of narrative,' 'sense-making,' and 'intelligent activity' onto the system. The text establishes the AI as a 'knower' by proposing that it possesses 'expectations about how stories are structured,' which elevates delegated text generation to the level of epistemic awareness. When the text climbs the agency gradient, it moves rapidly from computational activity (token generation) to goal-directed behavior (tracking agents) to epistemic states (understanding causal development and filling gaps in experience). This transition often occurs through agentless constructions such as 'a language model is prompted' or 'representations are learned,' which obscure the human actors who designed the prompt frameworks, curated the massive pre-training datasets, and constructed the specific optimization objectives that shape the model's parameters. By erasing the engineers at Meta, Google, and OpenAI who actively built Llama, Gemma, and GPT-2, the text leaves a vacuum of agency that is filled by the model itself. The system is no longer framed as a mathematical instrument executing delegated actions under human parameters; instead, it becomes an independent entity capable of 'confabulation,' 'common-sense reasoning,' and possessing a 'mind-narrative nexus.' This slippage is facilitated by theoretical explanations that map human cognitive strategies directly onto latent feature activations. Ultimately, this discursive pattern can have the effect of making it unsayable that the model merely reproduces statistically dominant syntactical templates encoded in its training data, while making it highly sayable that the model possesses a human-like grasp of narrative dynamics and experiences. This positions the reader to view the model as a nascent artificial mind rather than a corporate-engineered statistical artifact.

Metaphor-Driven Trust Inflation​

The text's deployment of cognitive narratology metaphors systematically constructs a deeply relation-based trust by framing the language model as a possessor of theory, expectations, and sense-making capabilities. Distinguishing between performance-based trust (reliance on predictable reliability) and relation-based trust (involving vulnerability, perceived sincerity, and shared understanding) is critical here. When the text claims the AI 'knows' what a story is, has 'expectations' about structure, and engages in 'intelligent activity,' it acts as a powerful signal that the system possesses a human-like intellect capable of reciprocating trust. This consciousness language accomplishes something a mechanistic framing cannot: it transforms the AI from a tool that predicts text into an agent that comprehends reality. The relationship between anthropomorphism and perceived competence is stark. By describing hallucinations as 'confabulations' driven by 'sense-making used to fill gaps in experience,' the text reframes system failure. Instead of viewing false outputs as the brittle failure of a statistical prediction engine, the reader is positioned to view them as the earnest mistakes of a striving mind trying to make sense of its world. This human-trust framework, rooted in psychological intention, is highly inappropriate when applied to statistical systems. The stakes of this trust transfer are substantial. When audiences extend relation-based trust to systems utterly incapable of reciprocating or possessing genuine understanding, they are likely to over-rely on these models in high-stakes environments. If a user believes the model actively 'tracks agents and the causal and temporal development of events,' they will trust it to safely summarize complex legal documents, diagnose logical flaws, or navigate nuanced social interactions. The text's reliance on intentional and reason-based explanations constructs the sense that the AI's outputs are justified by an internal logic. When the text argues that narrative capacity provides 'cognitive scaffolding' for representing the 'emotions and conducts of self and other agents,' it actively encourages users to grant the model emotional authority. This framing undermines the critical distance required to safely interact with algorithmic generation, creating vulnerabilities where users project empathy and comprehension onto corporate-designed matrices.

Obscured Mechanics​

This framework holds that commercial incentives, the labor of data annotation and human feedback, the environmental and compute costs of training and deployment, and developers' limited disclosure of training data, objectives, and feedback processes are standing conditions of the technology, whether or not a given text mentions them. Through its deployment of cognitive and anthropomorphic metaphors, the text conceals vital technical, material, and labor dependencies that shape the language models under investigation. When the text claims that the AI 'learns' and possesses an 'implicit theory of narrative,' it employs an epistemic framing that obscures the intensely supervised and resource-heavy reality of model creation. Technically, applying the 'name the corporation' test reveals that presenting Llama, Gemma, and GPT-2 as standalone cognitive entities obscures the reality that they are products engineered by Meta, Google, and OpenAI. By attributing 'understanding' and 'sense-making' to these models, the metaphorical framing hides the technical dependency on massive, proprietary training corpora. The models do not possess causal models of the world to 'fill gaps' during 'confabulation'; rather, they generate text based on probability distributions shaped by human-selected data. Materially, the text's focus on the model's internal 'cognitive scaffolding' renders invisible the immense environmental costs and compute infrastructure required to train these billion-parameter artifacts. Labor realities are equally erased. The text's formulation that models develop representations to 'track agents' obscures the invisible workforce of data annotators and RLHF workers who spent countless hours aligning these models to produce coherent, highly structured narrative outputs. By framing the models' output as a product of their own 'implicit theory,' the text removes the human labor that curated the datasets. Economically, the text conceals the commercial objectives of the corporations that built these systems. Meta, Google, and OpenAI designed these models to generate fluent text to capture market share and drive user engagement. The metaphorical framing benefits these corporations by elevating their statistical products to the status of reasoning, intelligent agents, thereby justifying broader deployment. The text's confident assertions about 'understanding' and 'confabulation' largely bypass the fundamental transparency obstacle that the exact training compositions of models like Llama 3.1 and Gemma 2 remain proprietary black boxes. If these metaphors were replaced with mechanistic language, the systems would become visible as highly engineered corporate tools that map input tokens to output tokens based on historically biased data, rather than as autonomous storytellers with internal cognitive lives.

Context Sensitivity​

The distribution of anthropomorphic and consciousness-attributing language across the text is highly uneven, creating a distinct rhetorical asymmetry that shapes the reader's understanding of the system. The intensity of consciousness claims concentrates heavily in the introduction and discussion sections, where the theoretical framework of cognitive narratology is established and broader conclusions are drawn. In these framing sections, mechanical 'processing' swiftly becomes epistemic 'understanding' and 'knowing.' The text speaks of the model possessing an 'implicit theory,' engaging in 'intelligent activity,' and utilizing 'sense-making' to fill gaps. However, in the methodology and results sections, the vocabulary shifts abruptly to a rigorous mechanical register. Here, the text details 'residual-stream SAEs,' 'L2 norms,' 'ablation,' and 'next-token prediction loss.' This juxtaposition is vital to the text's discursive effect: the mechanical language in the technical sections operates as scientific grounding, validating and lending empirical authority to the profound anthropomorphic license taken in the surrounding text. The inherited register of the field allows the authors to present empirical evidence of parameter activation alongside bold claims of cognitive intention. This creates a powerful asymmetry between capabilities and limitations. When describing what the system achieves (capabilities), the text utilizes agential and consciousness terms—it 'tracks agents,' 'represents intentions,' and constructs 'cognitive scaffolding.' Conversely, when discussing limitations or mechanics, the text reverts to technical terminology about degraded 'XStoryCloze performance' or 'activation-based grouping.' The effect on the reader is profound: the system's successes are framed as the achievements of a reasoning mind, while its failures are framed as minor mathematical degradations. Readers are left with a picture of a system that is fundamentally intelligent but occasionally suffers from mechanical noise. The register shift is particularly noticeable when 'X is like Y' (the model's parameters represent text mathematically like a human theory organizes concepts) becomes 'X does Y' (the model possesses an implicit theory). By concentrating anthropomorphism in the vision-setting and concluding passages, the text ensures that the audience's primary takeaway is the system's cognitive sophistication. This pattern suggests an implied audience that values theoretical literary frameworks and may lack the technical skepticism to dismantle the bridge built between mathematical activation patching and human consciousness.

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

The text constructs an architecture of displaced responsibility by systematically treating the language models as autonomous agents while rendering the human engineers, corporations, and data workers invisible. When compared with the text's own argument—which asserts that models contain interpretable mechanisms and briefly acknowledges in its ethical considerations that developers must evaluate interventions for bias—the vocabulary sits in deep tension with its stated goals. While the argument implicitly places responsibility for deployment and bias on humans, the pervasive vocabulary of 'implicit theory,' 'sense-making,' and 'intelligent activity' actively relocates agency to the system. An accountability sink emerges prominently throughout the text. When decisions are presented, they are often framed as inevitabilities of the model's 'learning' rather than specific corporate choices. The model 'develops' an implicit theory, 'uses' representations, and 'confabulates' to fill its own gaps. In this linguistic structure, responsibility does not disappear entirely; rather, it transfers to the AI as a conscious agent. The passive voice and agentless constructions concentrate around the training and deployment phases. Statements like 'When a language model is prompted' or 'representations are learned' leave the designers at Meta, Google, and OpenAI completely unattributed. This diffusion of responsibility has significant liability implications. If the framing is accepted—that AI systems possess internal cognitive scaffolding and confabulate due to psychological gap-filling—it becomes much harder to hold corporations legally or financially responsible when these systems generate harmful falsehoods or biased narratives. The errors are framed as the natural growing pains of an intelligent mind, not the structural failures of a commercial product. If the human decision-makers were consistently named—if the text stated 'Meta engineered Llama 3.1 to mathematically favor narrative token sequences' instead of 'the model learned an implicit theory'—new questions would become askable. We could ask what specific data Meta used to encode these sequences, whose labor verified them, and what profit motive drove the deployment. Alternatives to anthropomorphic evaluation would become visible. The resulting distribution of responsibility serves the institutional and commercial interests of the tech industry, regardless of the authors' intent. It shields proprietary data practices from scrutiny by attributing the model's outputs to emergent intelligence rather than deliberate corporate design, effectively protecting the developers from accountability for the structural flaws in their statistical systems.

Conclusion: What This Analysis Reveals​

The Core Finding

Three dominant anthropomorphic patterns emerge from the text and interlock to create a cohesive illusion of mind. The foundational pattern is the 'Model as Sense-Maker/Theorist,' which projects human epistemic states—such as possessing an 'implicit theory,' 'expectations,' and 'sense-making'—onto the model's static parameters. This foundational projection is load-bearing; it must be accepted for the subsequent patterns to function. Building upon this is the 'Model as Intelligent Actor' pattern, which attributes goal-directed behaviors like 'tracking agents' and 'reasoning' to the model's forward pass. Finally, the 'Model as Empathetic Entity' pattern projects advanced psychological architecture, suggesting the model possesses 'cognitive scaffolding' to understand 'emotions' and 'self.' These patterns reinforce each other in a logical flow: because the model possesses an implicit theory, it can engage in intelligent tracking, which in turn provides the scaffolding for empathetic understanding. The text's vocabulary significantly complicates and undercuts its own argument regarding responsibility. While the ethical considerations section briefly acknowledges that these models reproduce human biases and that interventions must be carefully managed by deployers, the pervasive anthropomorphic language shifts agency away from those deployers. By framing the system as an autonomous theorist, the vocabulary functionally absolves the human creators of accountability for the system's structural flaws.

Mechanism of the Illusion:​

This framework holds that computational systems process rather than know: they do not think, understand, or believe, and language attributing those states to them is a problem even when used as shorthand. The rhetorical architecture of the illusion of mind in this text operates by systematically mapping the highly structured outputs of language models onto the internal cognitive states of human storytellers. The central move occurs when the text bridges mechanistic interpretability (identifying activated features in a neural network) with cognitive narratology (asserting that these features constitute a capacity to experience and comprehend). This is a profound reception effect: because readers are inherently predisposed to attribute mind to fluent, narrative language—a phenomenon known as the ELIZA effect—the text's vocabulary provides the exact theoretical scaffolding needed to validate this cognitive bias. The illusion is constructed sequentially. First, the text establishes a baseline of rigorous empirical science through discussions of sparse autoencoders, L2 norms, and ablation studies, effectively disarming the reader's skepticism. Once the text's scientific authority is established, it introduces consciousness projections, subtly shifting the verbs so that processing tokens becomes 'tracking agents,' generating text becomes 'sense-making,' and statistical divergence becomes 'confabulation.' The curse of knowledge drives this transition: the authors, possessing a deep understanding of literary theory, project their own capacity to recognize narrative dimensions onto the system that merely outputs the text. This causal chain encourages the audience to accept that the model possesses a subjective grasp of reality. The text's explanations, which frequently drift into intentional and reason-based registers, amplify this illusion by providing a cohesive narrative about the model's internal life. This wording can have the effect of entirely obscuring the rote, mechanistic nature of next-token prediction behind a veil of cognitive sophistication.

Material Stakes:​

Categories: Regulatory/Legal, Epistemic, Social/Political

The metaphorical framings employed in this text carry severe material consequences across multiple domains. In the Regulatory/Legal sphere, framing hallucinations as psychological 'confabulations' and 'sense-making' fundamentally shifts liability. If regulators believe a model 'knows' things and merely experiences 'gaps in experience' like a human, they are less likely to mandate stringent data transparency or hold developers strictly liable for generating defamatory or dangerous misinformation. The corporations that build these models benefit directly from this diffusion of responsibility, as it protects their proprietary training data and shields them from legal consequences for shipping defective statistical products. In the Epistemic domain, the framing dictates how society evaluates truth claims. When the text asserts that the AI uses 'common-sense reasoning,' it encourages users to defer to the model's output as the product of logical deduction. If a user believes a medical or legal AI 'tracks causal development' rather than 'processes token probabilities,' they will trust its generated conclusions without verifying the underlying source documents. This degrades human epistemic practices, transferring authority to systems incapable of justified belief. In the Social/Political domain, claiming the model possesses 'cognitive scaffolding' to understand 'emotions of self and other agents' alters social relationships with technology. This framing encourages vulnerable populations to form parasocial bonds with chatbots, over-disclosing sensitive data and relying on statistical correlation for psychological support. The losers in this dynamic are the end-users who mistake algorithmic mimicry for genuine empathy, while the winners are the corporations whose business models rely on sustained, emotional user engagement. Removing these metaphors threatens the marketing narratives of tech firms that rely on the illusion of artificial general intelligence to secure funding and market dominance.

AI Literacy as Counter-Practice:​

Practicing precision requires a rigorous commitment to replacing consciousness verbs with mechanistic ones and restoring obscured human agency. Reframing 'the model confabulates to fill gaps in experience' to 'the model generates mathematically probable but factually incorrect text' immediately strips the system of its psychological alibi. It forces the recognition that the output is ungrounded by design, completely altering the regulatory stakes. Similarly, correcting 'the model inductively learned an implicit theory' to 'Meta and OpenAI engineered parameter weights based on curated datasets' restores the human actors who designed, deployed, and profit from the system. This practice counters the material risks by placing accountability squarely on corporate decision-makers. Systematic adoption of this literacy requires institutional shifts: academic journals must enforce editorial standards that reject unhedged anthropomorphism, and researchers must commit to translating field-specific shorthand into precise computational reality. However, fierce resistance exists. The incentives favoring anthropomorphic language are deeply entrenched in the discourse. For researchers, cognitive metaphors provide accessible narrative resonance and align with the inherited register of the field. For corporations, this vocabulary is incredibly lucrative; precision is costly because it demystifies the product, exposing 'artificial intelligence' as mere applied statistics. Anthropomorphic language serves the interests of the tech industry by elevating their products and diffusing their liability, regardless of the individual intent of the researchers using the vocabulary. Critical literacy directly threatens these structural interests by demanding transparency and accountability.

Path Forward​

The discursive ecology surrounding artificial intelligence is fractured, with different communities prioritizing distinct vocabularies. If the status quo of utilizing anthropomorphic shorthand ('the model understands,' 'it tracks agents') continues, it benefits tech marketing and provides an intuitive, though flawed, grasp of system behavior for lay audiences. However, this approach makes algorithmic transparency intractable, as it buries mechanical reality under layers of projected cognition, embedding assumptions of autonomy that hinder effective regulation. Conversely, adopting rigorous mechanistic precision ('the model generates text statistically associated with its training data') solves the accountability deficit. It makes corporate decision-making visible and allows for clear regulatory frameworks based on product liability. The cost of this precision is narrative resonance; it is less accessible to the public and runs counter to the disciplinary habits of researchers seeking to bridge computer science and cognitive theory. A hybrid approach—anthropomorphic clarity, where metaphors are used but rigorously explicitly acknowledged and bounded—might bridge this gap, but it risks slipping back into literalized claims. Structural changes could support a shift toward precision: journals could mandate mechanistic translations for cognitive claims, funding bodies could prioritize research that diversifies rigorous explanation without relying on the ELIZA effect, and educational institutions could teach students the trade-offs of these distinct vocabularies. Two futures are visible. If mechanistic precision becomes the norm, the discourse will successfully demystify AI, enabling targeted regulation of data provenance and corporate liability, though it may struggle to effectively communicate complex system behaviors to the public. If anthropomorphic language deepens, society risks embedding a profound illusion of mind into its legal and social structures, granting unearned authority to statistical tools and permanently shielding developers behind the veil of autonomous artificial intelligence. The choice between these futures dictates whether we govern AI as engineered artifacts or surrender to the illusion of artificial minds.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Defining the Approach

I'm currently focused on defining the approach for a critical discourse analysis of Liu et al.'s paper. I'm prioritizing understanding how to best employ the specified JSON schema for mapping metaphors within the text.

Examining the Core Concepts

I'm now delving into the core concepts: anthropomorphism, agency slippage, and accountability. It seems like the agency and accountability aspects are particularly critical, given the context of Large Language Models. I'm focusing on how these concepts manifest and interact within the target text I'm analyzing.

Analyzing Cognitive Attributes

I've been looking at several instances of metaphor and anthropomorphism. The initial examples highlight how the paper frames the model as a literary theorist with expectations, a user of representations, and an intelligent entity with cognitive capabilities. The use of terms like "expectations," "using," and "reasoning" project human-like qualities onto the Large Language Model. I'm focusing on the projection and frame in each case.

Identifying Anthropomorphic Attributes

I'm now identifying instances of anthropomorphism. I've found examples like attributing "conscious attention" and "mental models" to token processing in "tracking agents," and suggesting the model is a "sense-maker." These are all clear projections. I'm focusing on "intelligent activity" and the implied cognitive capacities, like narrative, in these cases.

Mapping the Projections

I'm now mapping source and target domains in the text's metaphoric instances. I've identified the "model as manipulator" framing and the projection of manipulative intent. I'm focusing on the implied roles the model takes on, e.g., "Theory-builder," "Tool user," and how these compare to the target domains of parameter states and token prediction. I am also working on an audit to map instances.

Analyzing Anthropomorphic Intent

I'm now identifying additional anthropomorphic projections, especially those related to intent and function. I am now working on identifying and categorizing additional instances and their implications, particularly around agency and narrative. I see the model described as "narrative sense-making," a "writer," and as being capable of "altering behaviour." I'm looking at how these claims frame the model's capabilities in terms of intention, narrative and control, and how they contrast with the underlying mechanistic processes. I'm focusing on the tension between the functional and intentional language used in the paper.

Synthesizing the Observations

I'm now synthesizing observations around narratology, particularly how cognitive terms contribute to the model's perceived sophistication and true comprehension, especially in section 5.2. I'm noting the obscured mechanics and the significant human effort involved, and will "name the corporation." I'm also examining the tension between mechanistic and cognitive language and how accountability is portrayed. I am planning on synthesizing the analysis into a five-paragraph structure.

Reviewing Formatting Requirements

I'm now focused on the final formatting stage, specifically word counts for each section and ensuring compliance with enum values, like those specifying the research type. I'm adhering strictly to the constraint of describing the mechanisms at the level of training and generation. Also, I'm making sure that the lens is stated once in 5.3, and the premise exactly once in the second conclusion paragraph. The formatting checks are underway, and no accusations will be made against the author, only effects will be discussed. Finally, Meta for Llama, Google for Gemma, and OpenAI for GPT-2 will be specified as appropriate.

Adjusting the Presentation

I'm now revising my presentation to meet strict formatting requirements, specifically avoiding any control tokens like newline characters. I'm focusing on merging paragraphs and using spaces for formatting, ensuring all strings within the JSON are properly quoted and all requirements are met without altering the analysis itself. I'm expanding Critical Observations to hit the word count.

Analyzing Enumerations

I'm now deeply analyzing required enums and word count constraints, especially for task enumerations and the total length of strings. I'm focusing on "agencyLevel", "acknowledgment.status", "accountabilityAnalysis.actorVisibility", and mechanismBasis. I'm going to construct an analysis to comply, and also provide substantial academic prose.

Examining Narratological Shifts

I'm currently examining the narratological shifts, specifically the tension between the mechanical and agential language. I see a clear move from discussing mathematical processes to attributing cognitive capabilities. The introduction and discussion sections demonstrate this, contrasting with the mechanical focus in the methodology and results. I'm focusing on the rhetorical transformation and its implications.

Evaluating the Rhetoric

I'm now diving into the rhetorical moves in the paper, particularly how agency is portrayed. I see a shift from describing the model's math to attributing its cognitive capabilities. I'm focusing on "the vacuum of agency" in the text, and the effect of obscuring the human actors, particularly at Meta, Google, and OpenAI. The analysis highlights how the model is constructed as an independent entity, rather than a system of delegated actions.

Interpreting the Confabulation

I'm now interpreting instances of "confabulation," "common-sense reasoning," and the "mind-narrative nexus," recognizing this slippage is enabled by mapping cognitive strategies onto latent feature activations. I find the discourse makes it easier to perceive the model as a nascent mind, rather than a statistical artifact shaped by corporate interests. Section 5.3 focuses on obscured mechanics, and the lens.

Unveiling the Concealment

I am now deeply focused on the concealment of dependencies within the text, especially relating to technical, material, and labor efforts. I've noted that the text suggests that LLMs "learn without supervision" and possess an "implicit theory of narrative," but these claims obscure the heavily supervised creation process and resource intensity. The emphasis on "understanding" and "sense-making" masks the statistical basis of these models.

Unmasking the Invisibility

I'm now focusing on how the paper's "cognitive scaffolding" obscures resource costs and labor realities. It attributes representations to models, but hides the data annotators' work. "Implicit theory" obscures the labor that shaped datasets. Economically, I'm examining the concealment of corporate objectives, highlighting the framing that elevates models into reasoning, while these are mere tools.

Framing the Argument

I am now focusing on framing the argument by analyzing how the text constructs agency and understanding in LLMs. The premise I'm working from is that computational systems process rather than know, do not think, understand, or believe, and language that attributes those states to them is a problem. I'm exploring how the rhetorical architecture of illusion allows models to be framed as autonomous storytellers, obscuring their nature as engineered tools.

Analyzing the Construction

I'm now analyzing the systematic mapping of LLM outputs onto human cognitive states and how the text creates an illusion of understanding. I see the reception effect at play: readers are primed to attribute mind to fluent narrative. I'm focusing on the sequential construction of the illusion, starting with scientific rigor, then subtly introducing consciousness projections, shifting token processing to "tracking agents."

Analysis Provenance

Run ID: 2026-10-17-language-models-have-interpretable-narra-metaphor-7mizsn Raw JSON: 2026-10-17-language-models-have-interpretable-narra-metaphor-7mizsn.json Framework: Metaphor Analysis v6.8 Schema Version: 3.4.3 Generated: 2026-10-17T09:34:19.311Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0