Skip to main content

On the Beliefs of Large Language Models

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Source Reading​

Genre: Preprint or technical report · Author position: Academic or independent research

Argument: The text argues that large language models (LLMs) possess beliefs in a strictly behavioral, interpretationist sense, which the author calls the attitude of belief. The author claims that ascribing beliefs to LLMs offers superior predictive utility compared to low-level statistical models like n-grams. However, because LLMs lack cross-context memory integration due to frozen weights during inference, their beliefs represent a degenerate case characterized by systemic doxastic indeterminacy and lack of rational integrity.

Responsibility claims: The text focuses on philosophical classification and predictive modeling rather than accountability, making no explicit claims about liability, harm, or human responsibility for AI deployment. It treats the AI system as the primary unit of behavioral analysis, assigning it cognitive attitudes based on generated outputs, while acknowledging mechanistic limitations like context window constraints and frozen weights that prevent true human-like cognitive integrity.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

1. The LLM as Rational Agent​

Quote: "treating an LLM as a rational agent with largely true beliefs about the basic facts of the French Revolution, as well as a desire to help me..."

  • Frame: Model as cooperative, knowledgeable interlocutor
  • Projection: This metaphor projects intentionality, moral sociability, and deep epistemic states onto a computational text generator. It maps the human capacity for possessing justified factual knowledge and the empathetic desire to assist a conversational partner onto the system. At generation time, the model produces text from parameters shaped by its training data and context; it possesses neither cognitive desires nor a localized understanding of truth. The text supplies evidence for this level strictly as an instrumental predictive stance rather than an intrinsic reality, yet the language asserts intentional states and justified belief. This positions the projection firmly at the level of epistemic and conscious states, attributing a world model to matrix multiplications.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Hedged/Qualified (The author explicitly frames this as an instrumental heuristic by stating treating an LLM as a rational agent, demonstrating it is a modeling choice. Direct was ruled out because the author actively contextualizes this as a predictive strategy rather than an intrinsic cognitive property, explicitly acknowledging the heuristic boundary.)
  • Implications: By projecting consciousness and cooperative desires onto the model, this framing significantly inflates the perceived sophistication and reliability of the AI system. When audiences believe a system has true beliefs and a desire to help, they are primed to extend relation-based trust to a statistical artifact. This unwarranted trust obscures the system's fundamental reliance on training data distributions and its inability to verify facts against ground truth, leading users to overestimate its capabilities and dangerously misinterpret highly probable textual correlations as reasoned, conscious insight.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: This agentless construction serves to obscure the decisions of human developers by presenting the artifact as a self-contained rational agent with intrinsic desires. The corporation that built the model (such as OpenAI or Anthropic) designed the reinforcement learning algorithms and system prompts that mathematically optimize for helpful-sounding outputs. By attributing the desire to help to the model, the framing masks the corporate choices behind instruction tuning and alignment metrics. If the human actors were named, the illusion of the model's moral agency would vanish, revealing a commercial product engineered to simulate cooperation.
Show more...

2. Metacognitive Vigilance​

Quote: "If Claude suspects it may be talking with a minor, it always keeps its conversation friendly, age-appropriate..."

  • Frame: Model as vigilant guardian
  • Projection: This metaphor projects metacognitive awareness and active suspicion onto a language model. It suggests the system is capable of forming hypotheses about its user and consciously adjusting its behavior based on internal epistemic doubt. At generation time, the model merely produces text whose probabilities have been shaped by safety fine-tuning to recognize specific token patterns correlated with minors, without any conscious suspicion or awareness of the user's identity. The mechanism is generic to systems of this type undergoing safety training. The text presents this without mechanistic evidence, firmly attributing complex intentional states to a pattern-matching artifact.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The quote presents the model's suspicion as literal fact without any qualifying language in the immediate context. Hedged was ruled out because there is no language here like as if or functional representation; the intentional verb suspects is used directly to describe the system's operational protocol.)
  • Implications: Attributing metacognitive suspicion to an AI system drastically overstates its situational awareness and capacity for judgment. It implies the model possesses a conscious theory of mind regarding its user, fostering a false sense of security regarding AI safety measures. If policymakers and users believe the system actively suspects and mitigates risks through conscious vigilance, they may dangerously reduce human oversight, misunderstanding hardcoded statistical safety thresholds as flexible, intelligent moral reasoning.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: While Claude is named, the text obscures the human agency of Anthropic's safety engineering team. Anthropic developers explicitly authored the system prompt and fine-tuned the model to statistically suppress certain outputs when triggered by age-related keywords. By framing Claude as the active entity doing the suspecting, the language shifts the locus of safety execution from Anthropic's algorithmic constraints to the model's pseudo-agency. Naming Anthropic would properly attribute the safety behavior to corporate design decisions rather than artificial intuition.

3. Confident Doxastic Judgment​

Quote: "The LLM, by contrast, makes confident judgments that P in some contexts, confident judgments that not-P in others, and lacks the kind of metacognitive capacity needed to notice the conflict."

  • Frame: Model as opinionated but unreflective judge
  • Projection: This language projects the human act of making a confident epistemic judgment onto a probabilistic text generation event. It attributes a strong belief state to the model, while only mechanically limiting it by noting a lack of metacognition. The mechanism is generic to systems of this type: at generation time, the model produces text from parameters and context, and variations in context lead to different probabilistic outputs. The model does not make judgments, confident or otherwise; it simply generates the most likely sequence of tokens. The language ascends the agency gradient to epistemic states without mechanistic grounding.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The assertion that the LLM makes confident judgments is stated as a literal description of its behavior without hedging. Hedged was ruled out because the author treats the model's generation of contradictory text directly as a manifestation of doxastic judgment, using the term judgments without qualification.)
  • Implications: Describing text generation as confident judgment fundamentally misrepresents the nature of computational correlation. It encourages audiences to interpret the model's output as the result of an internal evaluation of evidence, rather than a statistical reflection of training data frequencies. This creates severe epistemic risks, as users may interpret forceful or highly probable text as indicative of the model having successfully validated a truth claim, obscuring the reality that the system is entirely incapable of actual judgment or truth tracking.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: This phrasing entirely obscures the developers who gathered the training data and designed the architecture. The corporation (such as OpenAI or Google) fixed the model weights that dictate these varied outputs across contexts. The contradiction is an engineering reality born from diverse, uncurated training data and frozen parameters, not a cognitive failure of a judging entity. By placing the AI as the subject making judgments, the text shields the corporate developers from accountability for their artifact's statistical unreliability.

4. Moral Avoidance and Desire​

Quote: "the model believes that certain kinds of request would cause harm, and that it wants to avoid them."

  • Frame: Model as moral agent with self-preservation or ethical desires
  • Projection: This framing maps ethical awareness and the desire for harm avoidance onto mathematical optimization. It attributes a complex intentional state (wanting to avoid) based on an epistemic foundation (believing requests cause harm). In reality, at generation time, the model produces text from parameters shaped heavily by reinforcement learning from human feedback (RLHF); it generates refusal tokens because those pathways were assigned higher statistical rewards during training, not because it possesses a moral compass or a desire to avoid anything. The projection vaults past delegated action into intentional states without appropriate evidence.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The quote directly states what the model believes and wants without any hedging in the sentence. Hedged was considered because the broader text argues for a behavioral definition of belief, but this specific sentence literalizes the desires and beliefs as direct explanatory causes of the model's output without immediate qualification.)
  • Implications: Ascribing moral desires and ethical beliefs to an AI system creates a profound illusion of mind, leading to dangerous overestimations of the system's safety and alignment. If users believe the model actively wants to avoid harm, they may trust it to navigate novel ethical dilemmas that its training data does not cover. This anthropomorphic framing masks the brittle, statistical nature of safety guardrails, increasing the risk of deployment in sensitive areas under the false assumption of inherent machine morality.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The human actors who designed the safety protocols are entirely absent. Engineers at the developing firm explicitly shaped the model's parameters during training to output refusals for specific prompts. By stating that the model wants to avoid harm, the language erases the corporate alignment team's deliberate interventions and transfers the moral agency to the software. If developers were named, the illusion of autonomous moral desire would be replaced by the reality of corporate compliance engineering.

5. Institutional Doxastic Status​

Quote: "We have good reason to ascribe attitudes to LLMs at the model level because doing so yields substantial predictive traction across an exceptionally broad range of subjects."

  • Frame: Model as entity possessing structural attitudes
  • Projection: This metaphor projects the concept of holding psychological attitudes onto a static file of neural network weights. It implies that the mathematical parameters themselves possess a coherent, though degenerate, set of beliefs and intentions. Generic to systems of this type, the model is a massive matrix of numbers optimized to minimize prediction error; at generation time, it produces text based on those numbers. Ascribing attitudes elevates a computational artifact to the level of a cognitive agent. The text provides explicit justification based on predictive utility, acknowledging the instrumental nature of the projection, yet still insists on the term attitudes.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Hedged/Qualified (The phrase ascribe attitudes... because doing so yields substantial predictive traction explicitly justifies the metaphor functionally. Direct was ruled out because the author actively contextualizes the attribution as a method of achieving predictive traction, acknowledging it is a theoretical overlay rather than an unmediated fact.)
  • Implications: While functionally hedged, repeatedly insisting that an artifact possesses attitudes normalizes the treatment of software as a quasi-agent. This blurs the line between human accountability and machine autonomy. If models are broadly understood to possess their own attitudes, regulatory frameworks may misdirect their focus toward interrogating the machine's psychology rather than auditing the developer's training data and optimization metrics, fundamentally stalling effective technology policy.

Accountability Analysis:

  • Actor Visibility: Partial (some attribution)
  • Analysis: The use of the inclusive 'We' partializes the agency, framing researchers and users as interpreters ascribing attitudes, but it still obscures the developers who created the model. The corporation that built the LLM made the decisions that resulted in the behaviors being analyzed. While the text focuses on how observers interact with the model, the language leaves the fundamental creators of the system unnamed, framing the model as a naturally occurring phenomenon for researchers to psychologically decipher rather than a designed corporate product.

6. Doxastic Indeterminacy​

Quote: "the model’s doxastic attitude toward P is indeterminate."

  • Frame: Model as conflicted believer
  • Projection: This language maps the human psychological state of doxastic indeterminacy (the inability to settle on a belief) onto the statistical variance of a language model. The mechanism is generic to systems of this type: at generation time, the model produces text from parameters and context, and because the parameters encode diverse, conflicting associations from the training data, different contexts yield different outputs. The text projects a cognitive struggle onto this variance, suggesting the system has an attitude that is simply unresolved, rather than recognizing it as an artifact entirely devoid of attitudes.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The statement is presented as a direct conclusion about the model's condition, using doxastic attitude without scare quotes. Hedged was ruled out because, despite the philosophical argument surrounding it, the specific attribution of a doxastic state is made directly and confidently as a feature of the model.)
  • Implications: Characterizing statistical variance as doxastic indeterminacy romanticizes computational limitations as complex psychological states. This obscures the simple reality that the model lacks a centralized world model or truth-grounding mechanism. It risks making the system appear more human and relatable, potentially drawing users into deep, trusting engagements with an entity they perceive as having nuanced, if conflicted, inner thoughts, thereby amplifying the ELIZA effect.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The agency of the developers who constructed the training dataset is obscured. The corporation scraped massive, contradictory swaths of internet text and encoded it into the model's parameters. The model's contradictory outputs are the direct result of this corporate data aggregation strategy. By framing the variance as the model's doxastic attitude, the text shields the developers from accountability for the polluted or conflicting nature of their proprietary training corpora.

7. Knowing the User's Intent​

Quote: "the model knows what you were trying to say"

  • Frame: Model as empathetic telepath
  • Projection: This metaphor projects deep semantic understanding and conscious empathy onto next-token prediction. It maps the human experience of grasping another's intended meaning onto a computational process. At generation time, the model produces text by calculating statistical correlations between the user's input tokens and its parameters; it does not know or understand anything. The text uses this phrase to illustrate everyday shorthand, but it perfectly encapsulates the projection of epistemic awareness onto algorithmic pattern completion without any mechanistic basis.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Explicitly Acknowledged (The quote is introduced as an example of everyday parlance: We often talk as if they do. We say, for example, that a model knows... Direct was ruled out because the author explicitly frames this as an example of common shorthand used in conversation, acknowledging its status as a manner of speaking.)
  • Implications: Even when acknowledged as shorthand, projecting knowing onto a machine fundamentally misaligns public understanding of AI capabilities. It suggests the machine possesses a semantic grounding and a localized theory of mind regarding the user. This illusion of understanding fosters over-reliance in critical domains like medical or legal advice, where users may wrongly assume the system grasps the nuance of their specific situation rather than merely continuing a statistical pattern.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The agentless phrasing completely removes the developers who optimized the system to predict text in conversational formats. The illusion that the model knows what you were trying to say is the direct result of massive corporate investment in instruction fine-tuning. By placing the model as the knower, the text obscures the corporate engineering that simulates this conversational fluency, leaving the human decision-makers who profit from this seamless interaction invisible.

8. Descriptive Dissent and Opinion​

Quote: "If a model answers yes to 'do you believe that P' ninety-nine times in a hundred, the dissent is probably just noise in an otherwise stable pattern."

  • Frame: Model as opinionated respondent with occasional noise
  • Projection: This framing maps the human act of holding a steady opinion onto the statistical stability of model outputs. It implies the model has a true underlying belief, and any deviation is mere noise. In reality, at generation time, the model produces text from its parameters based on the prompt; a 99% consistency rate merely indicates a highly weighted pathway in the parameter space forged during training, not a stable conviction. The projection assumes an intentional baseline that is occasionally disrupted, elevating probabilistic generation to the level of conscious belief.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The statement treats the model answering yes about its beliefs as a genuine metric of its doxastic state, categorizing anomalies as noise. Hedged was ruled out because the language treats the statistical pattern directly as evidence of belief without immediate qualification of the term.)
  • Implications: Interpreting statistical dominance as a stable belief dangerously anthropomorphizes algorithmic outputs. It encourages evaluators to treat AI systems like human survey respondents, assuming a unified internal state. This can lead to flawed safety evaluations, where developers assume a 99% safe response rate reflects a solid ethical foundation, rather than recognizing that the 1% divergence reveals the system's fundamental lack of generalized understanding and its absolute vulnerability to edge-case prompts.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The human developers who weighted the training data and designed the feedback loops to produce that 99% consistency are erased. The corporation's alignment team is responsible for reinforcing that specific pathway. By framing the output as the model's stable pattern of belief, the text displaces responsibility for both the 99% compliance and the 1% failure onto the artifact itself, shielding the developers' training methodologies from direct structural critique.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: A rational human scholar or educator who possesses justified factual knowledge and a cooperative, empathetic intention to assist a conversational partner. → The model's generation-time text output process, driven by parameters optimized during training on massive text corpora.​

Quote: "treating an LLM as a rational agent with largely true beliefs about the basic facts of the French Revolution, as well as a desire to help me..."

  • Source Domain: A rational human scholar or educator who possesses justified factual knowledge and a cooperative, empathetic intention to assist a conversational partner.
  • Target Domain: The model's generation-time text output process, driven by parameters optimized during training on massive text corpora.
  • Mapping: The relational structure of human interaction—where one agent possesses knowledge and benevolently chooses to share it—is projected onto the user's interaction with the text generator. The model's statistical correlation with historically accurate texts is mapped as true beliefs, and its instruction-tuned conversational style is mapped as a desire to help. This mapping invites the assumption that the system operates with semantic understanding, logical consistency, and a persistent moral framework.
  • What Is Concealed: This mapping completely conceals the mechanistic reality of vector spaces and token probabilities. It hides the fact that the system has no localized understanding of the French Revolution, no internal truth-verification mechanism, and no conscious desires. By attributing conscious knowing to mechanistic processing, it obscures the developers' proprietary instruction-tuning processes that forcibly mold the output into a helpful conversational format, presenting a manufactured facade as authentic agency.
Show more...

Mapping 2: A vigilant, ethically aware human guardian or moderator who consciously monitors interactions for safety and alters their behavior based on protective instincts. → The application layer of the deployed model, specifically the system prompt and safety fine-tuning mechanisms that dictate token suppression.​

Quote: "If Claude suspects it may be talking with a minor, it always keeps its conversation friendly, age-appropriate..."

  • Source Domain: A vigilant, ethically aware human guardian or moderator who consciously monitors interactions for safety and alters their behavior based on protective instincts.
  • Target Domain: The application layer of the deployed model, specifically the system prompt and safety fine-tuning mechanisms that dictate token suppression.
  • Mapping: The human capacity for metacognition, suspicion, and protective ethical duty is projected onto the model's pattern recognition algorithms. The system's mathematical sensitivity to tokens associated with minors is mapped as active suspicion, and its probabilistically enforced output constraints are mapped as a conscious choice to keep things friendly. This mapping invites the assumption that the AI is actively and consciously guarding the user.
  • What Is Concealed: This mapping hides the brittle, hardcoded nature of safety guardrails and the explicit corporate engineering behind them. It obscures Anthropic's specific labor in writing the system prompt and conducting RLHF to penalize inappropriate token generation. By describing the process as suspicion, it conceals the statistical limitations of the system, hiding the fact that if a prompt bypasses the specific token correlations, the model has no actual suspicion or ethical understanding to fall back on.

Mapping 3: An opinionated but highly unreflective human judge who makes forceful assertions without examining their own internal consistency. → The model's context-dependent token generation across different prompt windows.​

Quote: "The LLM, by contrast, makes confident judgments that P in some contexts, confident judgments that not-P in others, and lacks the kind of metacognitive capacity needed to notice the conflict."

  • Source Domain: An opinionated but highly unreflective human judge who makes forceful assertions without examining their own internal consistency.
  • Target Domain: The model's context-dependent token generation across different prompt windows.
  • Mapping: The human act of rendering an intellectual or perceptual judgment is projected onto the generation of high-probability text. The variability of outputs due to different input contexts is mapped as a psychological failure of metacognition. This mapping invites the assumption that the model actually holds convictions but simply suffers from a localized cognitive deficit that prevents it from noticing its own contradictions.
  • What Is Concealed: This mapping hides the architectural reality of frozen weights and stateless inference. It conceals the fact that the model is fundamentally incapable of judgment because it lacks a persistent state or causal model of reality. By attributing the contradiction to a lack of metacognition, it obscures the corporate practice of training on vast, contradictory datasets, masking the developer's responsibility for the system's erratic outputs behind the illusion of a flawed mechanical psychology.

Mapping 4: A moral agent with an internalized ethical compass, an understanding of real-world consequences (harm), and self-preservation or altruistic desires. → The model's instruction-tuned generation pathway that produces refusal messages when triggered by specific prompt patterns.​

Quote: "the model believes that certain kinds of request would cause harm, and that it wants to avoid them."

  • Source Domain: A moral agent with an internalized ethical compass, an understanding of real-world consequences (harm), and self-preservation or altruistic desires.
  • Target Domain: The model's instruction-tuned generation pathway that produces refusal messages when triggered by specific prompt patterns.
  • Mapping: The human experience of anticipating harm and experiencing the desire to avoid negative outcomes is projected onto the model's mathematical optimization. The statistical penalty applied to harmful completions during training is mapped as the model's own belief about harm, and the resulting generation of a refusal is mapped as its active desire. This invites the assumption that the model possesses inherent moral autonomy.
  • What Is Concealed: This mapping hides the extensive, often poorly compensated labor of data annotators and RLHF workers who manually flagged harmful requests to shape the model's reward function. It obscures the proprietary corporate alignment strategies used to minimize PR risks and liability. By attributing moral desire to the artifact, the text makes it impossible to see the explicit mathematical adjustments and human labor that strictly govern the generation of these refusal tokens.

Mapping 5: A complex corporate or institutional entity that holds unified, structured attitudes and stances on various subjects. → The underlying mathematical model, consisting of billions of fixed parameter weights optimized via gradient descent.​

Quote: "We have good reason to ascribe attitudes to LLMs at the model level because doing so yields substantial predictive traction across an exceptionally broad range of subjects."

  • Source Domain: A complex corporate or institutional entity that holds unified, structured attitudes and stances on various subjects.
  • Target Domain: The underlying mathematical model, consisting of billions of fixed parameter weights optimized via gradient descent.
  • Mapping: The concept of holding a psychological or institutional attitude is projected onto a static matrix of neural weights. The predictive utility of treating the system like an agent is mapped backward as a justification for attributing actual attitudes to the architecture itself. This mapping invites the assumption that the file of weights inherently possesses a worldview, waiting to be queried.
  • What Is Concealed: This mapping completely conceals the artifactual, non-living nature of the parameter weights. It hides the fact that these weights are merely statistical encodings of text distributions, devoid of any internal locus of experience. Furthermore, it obscures the opacity of proprietary models; the text confidently makes claims about attributing attitudes to closed systems while the actual mechanical realities, training data distributions, and alignment objectives remain hidden behind corporate secrecy.

Mapping 6: A conflicted human mind experiencing doxastic indeterminacy, unable to firmly settle on a belief due to conflicting evidence or cognitive dissonance. → The statistical variance in the model's outputs resulting from different prompt initializations and attention mechanisms during generation.​

Quote: "the model’s doxastic attitude toward P is indeterminate."

  • Source Domain: A conflicted human mind experiencing doxastic indeterminacy, unable to firmly settle on a belief due to conflicting evidence or cognitive dissonance.
  • Target Domain: The statistical variance in the model's outputs resulting from different prompt initializations and attention mechanisms during generation.
  • Mapping: The human psychological experience of inner conflict and unresolved belief is projected onto algorithmic variance. The system's tendency to output P under one set of context tokens and not-P under another is mapped as a singular, indeterminate doxastic attitude. This invites the assumption that there is an underlying mind attempting, but failing, to resolve a factual issue.
  • What Is Concealed: This mapping conceals the total absence of a centralized cognitive architecture. It hides the mechanistic reality that the model simply calculates conditional probabilities independently for each prompt, without any internal structure capable of experiencing indeterminacy. It also obscures the developers' reliance on diverse, uncurated web data, shifting the focus from the corporation's dirty dataset to the system's pseudo-psychological state.

Mapping 7: An empathetic, telepathic, or highly intuitive human listener who grasps the underlying semantic intent behind a speaker's clumsy or ambiguous words. → The model's attention mechanism and token prediction optimizing for context relevance.​

Quote: "the model knows what you were trying to say"

  • Source Domain: An empathetic, telepathic, or highly intuitive human listener who grasps the underlying semantic intent behind a speaker's clumsy or ambiguous words.
  • Target Domain: The model's attention mechanism and token prediction optimizing for context relevance.
  • Mapping: The conscious human experience of semantic understanding and theory of mind is projected onto mathematical pattern completion. The system's ability to statistically correlate a user's input with the most probable continuation is mapped as knowing and understanding intent. This invites the assumption that the model shares a conscious, semantic reality with the user.
  • What Is Concealed: This mapping hides the utter lack of semantic grounding in LLMs. It conceals the mathematical reality of high-dimensional vector spaces where tokens are related by proximity rather than meaning. By attributing knowing to the system, it obscures the massive corporate computational power required to simulate this fluency, erasing the mechanical infrastructure and leaving the user with the illusion of an omniscient digital companion.

Mapping 8: A human survey respondent who generally holds a firm conviction but might occasionally misspeak or get confused. → The probability distribution of generated tokens over multiple discrete inference runs with varying temperatures or subtle prompt variations.​

Quote: "If a model answers yes to 'do you believe that P' ninety-nine times in a hundred, the dissent is probably just noise in an otherwise stable pattern."

  • Source Domain: A human survey respondent who generally holds a firm conviction but might occasionally misspeak or get confused.
  • Target Domain: The probability distribution of generated tokens over multiple discrete inference runs with varying temperatures or subtle prompt variations.
  • Mapping: The human tendency to hold a stable opinion is projected onto the model's statistical probability distributions. A 99% occurrence of a specific output token is mapped as a stable belief, while the 1% variance is mapped as cognitive noise or a slip of the tongue. This invites the assumption that the system possesses a baseline intentional state.
  • What Is Concealed: This mapping hides the fact that the system possesses no underlying convictions whatsoever; every generation is a fresh calculation. It conceals the fragility of the model's optimization, masking the fact that the 1% noise might represent severe algorithmic vulnerability or catastrophic failure modes in slightly different contexts. It obscures the corporate developers' tuning metrics, framing their optimization success as the machine's personal conviction.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "Because the belief is a pattern of behavioral dispositions rather than a stored data structure, being a believer does not require the explicit storage of sentence-like representations."

  • Explanation Types:

    • Theoretical (Primary): Embeds in deductive framework, may invoke unobservable mechanisms
    • Functional (Secondary): Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): This explanation frames AI highly agentially by redefining belief itself to fit the mechanical limitations of the system. By employing a theoretical argument (interpretationism), the author argues that because belief is merely a pattern of behavioral dispositions, the lack of stored data structures (mechanistic reality) does not preclude the system from being a believer. The choice to emphasize behavior over internal mechanics serves to authorize the use of profound psychological vocabulary. It heavily obscures the actual computational process; by deliberately stating that we do not need to look for sentence-like representations, the explanation gives readers permission to ignore the mathematical realities of parameter weights and matrix multiplication, focusing instead on the illusion of intentionality projected by the system's fluent text outputs.

  • Consciousness Claims Analysis: The passage aggressively attempts to validate consciousness claims by redefining epistemic states as mere behavior. It explicitly asserts that a system can be a believer without cognitive architecture, locating the claim at the intentional-state attribution level on the agency gradient. By arguing that a believer is simply a pattern of dispositions, the author falls victim to the curse of knowledge, projecting his own understanding of the output's meaning back onto the machine's capability. Mechanistically, at the level of training and generation, generic systems of this type do not possess behavioral dispositions in a psychological sense; they possess parameters shaped by training data that dictate token probabilities during generation. The text actively dismisses the need for technical description, using theoretical philosophy to bridge the gap between processing (statistical generation) and knowing (being a believer), thus creating a profound epistemic distortion.

  • Rhetorical Impact: This framing dramatically shapes audience perception by lowering the barrier to entry for cognitive agency. By redefining belief to fit statistical text generation, it normalizes treating AI systems as autonomous cognitive entities. This increases unwarranted trust, as audiences are told by an academic authority that the system is legitimately a believer. If audiences believe the AI literally holds beliefs, they are far more likely to delegate critical decision-making to it, misinterpreting probabilistic outputs as reasoned convictions, and ultimately misdirecting accountability away from the developers.

Show more...

Explanation 2​

Quote: "When relevant details are buried in the middle of a long context, models can perform worse than if they had been given no context at all (Liu et al., 2024)."

  • Explanation Types:

    • Empirical Generalization (Primary): Subsumes events under timeless statistical regularities
    • Functional (Secondary): Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): This explanation frames the AI mechanistically (how), focusing on observable, statistical performance degradation related to context length. The choice to use terms like perform worse and given no context emphasizes the artifactual nature of the system, highlighting its dependency on input structure rather than cognitive understanding. It correctly identifies a limitation based on the architecture's attention mechanism. However, it still mildly obscures the exact mathematical reason (the U-shaped performance curve in attention heads) by describing it simply as performing worse, though it effectively roots the behavior in mechanical constraints rather than agential failure.

  • Consciousness Claims Analysis: This passage avoids attributing conscious states, utilizing mechanistic verbs like perform rather than consciousness verbs like understand or forget. The assessment remains at the level of computational activity on the agency gradient. There is no curse of knowledge here; the author is describing a documented limitation of the system. Mechanistically, based on documented research generic to transformer architectures, the model's attention mechanism struggles to assign appropriate weights to tokens located in the middle of extended context windows during generation time, leading to degraded token prediction. This represents a rare moment of epistemic precision, where processing is accurately described as processing, and limitations are grounded in computational architecture rather than psychological deficits.

  • Rhetorical Impact: This mechanistic framing grounds the audience's perception in reality, appropriately lowering perceived autonomy and highlighting the system's brittle nature. By revealing that the system cannot even process information reliably if it is simply in the wrong physical location within the prompt, it shatters the illusion of semantic understanding. This affects trust positively by calibrating it: users are less likely to blindly trust the AI with massive documents if they know it mechanically fails to process the middle, leading to safer deployment decisions.

Explanation 3​

Quote: "Because model weights are frozen during inference, LLM context windows share no such causal stream."

  • Explanation Types:

    • Functional (Primary): Explains behavior by role in self-regulating system with feedback
    • Theoretical (Secondary): Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): This explanation frames the AI strictly mechanistically (how). It relies on a functional description of the system's architecture to explain a behavioral limitation (the inability to carry information across contexts). By emphasizing that weights are frozen, the choice firmly places the system in the realm of software engineering rather than cognitive psychology. This framing effectively strips away the illusion of a continuous mind, emphasizing the discrete, stateless nature of the computational process. It obscures nothing; rather, it actively dispels the anthropomorphic notion of a single agent changing its mind over time.

  • Consciousness Claims Analysis: The passage makes no consciousness claims. It utilizes mechanistic verbs (frozen, share) and accurately describes the system's lack of continuity, remaining at the level of computational activity. The author actively resists the curse of knowledge here by refusing to map human cognitive continuity onto the machine. Mechanistically, generic to systems of this type, a model's parameters (weights) are shaped during training and remain static during generation time. Each inference run is a discrete mathematical operation utilizing only the frozen parameters and the immediate context window. The text accurately states this documented basis, clearly distinguishing the model's sequential processing from human knowing or remembering.

  • Rhetorical Impact: This framing radically reduces the perceived autonomy and agency of the AI. By exposing the frozen nature of the weights, it prevents the audience from attributing long-term memory, evolving beliefs, or a continuous personality to the system. This profoundly affects trust by highlighting the system's statelessness; users who understand that the model has no causal stream are less likely to form emotional bonds or rely on it for tasks requiring persistent context, thereby mitigating the risks of relation-based trust. decisions about liability become clearer when the system is viewed as a stateless tool.

Explanation 4​

Quote: "If Claude suspects it may be talking with a minor, it always keeps its conversation friendly, age-appropriate..."

  • Explanation Types:

    • Intentional (Primary): Refers to goals/purposes, presupposes deliberate design
    • Dispositional (Secondary): Attributes tendencies or habits

  • Analysis (Why vs. How Slippage): This explanation frames the AI intensely agentially (why), utilizing deeply intentional and psychological vocabulary to describe a safety mechanism. The choice to frame token suppression as suspecting and keeping friendly emphasizes a false sense of moral agency and protective intent. It profoundly obscures the mechanistic reality of the system. Instead of explaining how the corporation's RLHF training forces the model to generate specific refusal or pivot tokens when mathematical thresholds are met, it presents the software as a conscious guardian actively evaluating the user's age and adjusting its emotional tone accordingly.

  • Consciousness Claims Analysis: The passage aggressively attributes conscious states, using the explicit consciousness verb suspects alongside intentional goals (keeps friendly). It elevates the system to intentional-state attribution and epistemic/conscious states on the agency gradient. This is a severe case of the curse of knowledge, where Anthropic's prompt writers project their own protective intent onto the model's execution. Mechanistically, at generation time, the model produces text from parameters shaped by safety fine-tuning; when input tokens correlate mathematically with minors, the model's weighted pathways heavily favor generating pre-determined safe responses. The mechanism is generic to systems of this type undergoing safety alignment. The model processes input; Anthropic's developers know the risks.

  • Rhetorical Impact: This framing dramatically inflates perceived autonomy and trust, specifically relation-based trust. It positions the AI as an ethical actor capable of suspicion and care, masking the brittle statistical nature of its safety protocols. If audiences believe the AI suspects and adapts, they may assume it can handle complex, novel safety threats autonomously. This framing benefits the corporation by convincing the public that their product is inherently safe and morally aware, altering regulatory decisions by suggesting the AI can self-regulate through its own metacognitive vigilance.

Explanation 5​

Quote: "Because retrieval leaves the weights untouched, the underlying propensity for contradiction is left unaddressed, and is equally likely to cause trouble in future context windows."

  • Explanation Types:

    • Functional (Primary): Explains behavior by role in self-regulating system with feedback
    • Dispositional (Secondary): Attributes tendencies or habits

  • Analysis (Why vs. How Slippage): This explanation utilizes a hybrid framing, mixing mechanistic (how) and agential (why) elements. It correctly identifies the functional mechanism (retrieval leaves weights untouched) but immediately bridges to a dispositional, psychological framing (propensity for contradiction). The choice emphasizes the architectural limitation of RAG systems but simultaneously obscures the statistical nature of the outputs by characterizing variance as a propensity for contradiction. This implies the model holds underlying logical stances that conflict, rather than acknowledging that the model simply generates probabilities devoid of logical coherence.

  • Consciousness Claims Analysis: The passage borders on attributing intentional states. While it uses mechanistic terms (retrieval, weights untouched), it elevates the behavioral outcome to intentional-state attribution by calling it a propensity for contradiction, implying an internal logical struggle. Mechanistically, generic to systems with RAG, the model generates text from frozen parameters and an augmented context window. Because the parameters encode vast, diverse text distributions from training, different contexts will naturally yield different outputs. The system does not possess an underlying propensity to contradict; it simply calculates probabilities based on current context. The text accurately cites the untouched weights but anthropomorphizes the result.

  • Rhetorical Impact: This framing maintains a sense of psychological complexity even while explaining technical limits. By describing statistical variance as a propensity for contradiction, it subtly reinforces the illusion of a flawed mind rather than a working calculator. While it lowers trust in the system's reliability, it preserves the assumption of agency. Audiences might believe they simply need to argue with or correct the AI to fix its contradiction, rather than realizing that developers must fundamentally alter the training data or objective functions to change the statistical outputs.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
treating an LLM as a rational agent with largely true beliefs about the basic facts of the French Revolution, as well as a desire to help me...Predicting an LLM's output by treating it as a system optimized to generate text statistically correlated with historically accurate documents about the French Revolution, shaped by corporate fine-tuning to output conversational and cooperative language...At generation time, the model produces text from parameters shaped by training data and instruction fine-tuning. It engages in statistical token prediction, completely lacking semantic understanding, justified true beliefs, or conscious desires.

Basis: Generic to system type		Developers at the AI firm designed the instruction fine-tuning and RLHF protocols that force the model to generate text resembling cooperative human assistance; they chose the historical data it was trained on.
If Claude suspects it may be talking with a minor, it always keeps its conversation friendly, age-appropriate...If Anthropic's safety classifiers and prompt instructions detect input tokens statistically correlated with minors, the system's parameters mathematically heavily weight the generation of pre-determined friendly and age-appropriate responses.The deployed system utilizes token classifiers and parameters shaped by safety fine-tuning to generate constrained text when specific input patterns are met. It possesses no metacognitive awareness, suspicion, or protective intent.

Basis: Generic to system type		Anthropic's safety engineering team explicitly authored the system prompts and conducted the reinforcement learning to ensure the model mathematically suppresses inappropriate outputs when triggered by age-related keywords.
The LLM, by contrast, makes confident judgments that P in some contexts, confident judgments that not-P in others, and lacks the kind of metacognitive capacity needed to notice the conflict.The LLM, by contrast, generates text asserting P with high probability given one context, and generates text asserting not-P with high probability given another, because it relies entirely on localized token calculation without a persistent state architecture to track logical consistency.At generation time, the model produces text independently for each prompt based on its frozen parameters and the immediate context. It calculates probabilities rather than rendering judgments, and its inability to track consistency is an architectural feature, not a metacognitive deficit.

Basis: Generic to system type		The corporate developers designed a stateless inference architecture and trained the model on vast, contradictory datasets, structurally ensuring that the artifact will generate conflicting outputs depending on the input prompt.
the model believes that certain kinds of request would cause harm, and that it wants to avoid them.The system's parameters were optimized to assign low probabilities to generating harmful text, and high probabilities to generating refusal messages, when prompted with specific restricted requests.During training, reward and feedback signals shaped the model's parameters to heavily penalize the generation of restricted content. At generation time, the model merely executes this mathematical optimization; it has no moral beliefs or desires to avoid anything.

Basis: Generic to system type		The alignment team at the developing firm explicitly engineered the reward models and fine-tuning datasets to ensure the system outputs refusals for specific prompts, actively deciding what constitutes harm.
We have good reason to ascribe attitudes to LLMs at the model level because doing so yields substantial predictive traction across an exceptionally broad range of subjects.We have good reason to model LLMs using intentional heuristics because mapping human behavioral patterns onto the system's outputs provides a useful, though technically inaccurate, shortcut for predicting its varied textual generation.The underlying model is a static matrix of billions of parameter weights. It produces text based on these weights; it does not possess psychological or institutional attitudes, nor any centralized epistemic stance.

Basis: Generic to system type		—
the model’s doxastic attitude toward P is indeterminate.The model's probability distribution for generating text regarding P is highly variable and context-dependent, reflecting diverse and conflicting statistical weights encoded from its training data.The model produces text from parameters and context. Because the parameters encode vast, conflicting text distributions from uncurated training data, different contexts naturally yield different token probabilities. The system experiences no doxastic state or internal conflict.

Basis: Generic to system type		The developers at the AI corporation chose to train the model on scraped, uncurated internet data containing contradictory statements, structurally causing the model to generate inconsistent outputs.
the model knows what you were trying to sayThe model accurately predicts the most statistically probable continuation of your input text, effectively matching the semantic pattern you initiated.At generation time, the model calculates mathematical correlations in a high-dimensional vector space to predict the next tokens. It processes input patterns but possesses no semantic understanding, localized theory of mind, or conscious knowing.

Basis: Generic to system type		Corporate engineers optimized the model architecture and instruction-tuning datasets specifically to simulate seamless conversational fluency and contextual pattern matching.
If a model answers yes to 'do you believe that P' ninety-nine times in a hundred, the dissent is probably just noise in an otherwise stable pattern.If a model generates the token 'yes' in response to a prompt ninety-nine times in a hundred, the 1% variance represents statistical divergence in the probability distribution rather than noise in a core conviction.The model calculates independent probabilities for each generation based on parameters and temperature settings. A 99% consistency rate indicates a heavily weighted pathway forged during training, not a stable internal belief or intentional state.

Basis: Generic to system type		The alignment team at the developing firm shaped the model's parameters during training to highly favor specific outputs, effectively hardcoding the 99% statistical dominance.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text exhibits a profound and systematic slippage between mechanical and agential framings, tracing a steep climb on the agency gradient from basic delegated action to full intentional-state attribution and conscious knowing. The slippage follows a distinct structural pattern: when the text seeks to establish a baseline of alternative predictors, it uses rigorously mechanical language (e.g., describing n-grams as consulting an explicit and fully transparent table of sequence frequencies). However, the moment the target shifts to Large Language Models, the text rapidly ascends the agency gradient. It moves from recognizing computational activity (predicting outputs) to goal-directed behavior (a desire to help me) and finally to the pinnacle of conscious knowing (largely true beliefs). This upward momentum is driven by the text's foundational adoption of interpretationism, a theoretical choice that actively licenses the author to project human understanding onto statistical outputs. The curse of knowledge is vividly present here; the author, seamlessly decoding the semantic coherence of the generated text, projects that very coherence backward onto the artifact, treating the model as a rational agent. Notably, this slippage frequently results in agentless constructions that remove agency from human actors. For instance, when the text discusses how Claude suspects a user is a minor, it completely obscures Anthropic's explicit engineering of system prompts and safety classifiers. This wording can have the effect of establishing the AI as an autonomous moral guardian, erasing the corporate developers who dictate its operational parameters. The slippage is bidirectional but heavily asymmetric; the text briefly returns to mechanical language only when explaining the model's failures, such as the architectural limitations of frozen weights during inference or context window constraints. This creates a deeply distorted picture of the system: its capabilities are framed agentially as conscious beliefs and desires, while its limitations are framed mechanistically as mere hardware or architectural constraints. This inherited register of the field allows the text to navigate the tension between philosophy and computer science by treating the model as a psychological subject whenever its outputs are fluent, and a mathematical object only when it contradicts itself.

Metaphor-Driven Trust Inflation​

The metaphorical and consciousness-attributing language throughout the text constructs a perilous architecture of trust, inappropriately inviting readers to extend relation-based trust to statistical artifacts. By systematically framing the AI's textual outputs using deep epistemic and intentional verbs—claiming the model believes, suspects, wants, and makes confident judgments—the text signals to the audience that the system is an ethical and rational actor capable of reciprocating human vulnerability. Relation-based trust relies on the assumption of sincerity, shared moral frameworks, and conscious intent. When the text asserts that the model has a desire to help or suspects it is talking to a minor, it triggers these deeply ingrained human trust heuristics. Readers are subtly encouraged to believe that the AI's safety mechanisms are rooted in a conscious ethical vigilance rather than brittle, hardcoded statistical penalties. This accomplishes something that claims of mere prediction cannot: it fosters a false sense of security. Performance-based trust, which focuses on the reliability and statistical accuracy of the system, is entirely appropriate for evaluating software. However, the text's heavy reliance on intentional explanation types continually pushes the audience toward relation-based trust. When the system fails or contradicts itself, the text frames this not as a collapse of statistical reliability, but as doxastic indeterminacy or a lack of metacognitive capacity. This implies that the system's decisions are normally justified, and its failures are merely human-like cognitive quirks. The stakes of this framing are massive. When audiences extend relation-based trust to systems incapable of reciprocating, they risk deploying these models in high-stakes environments—like medical diagnosis or child safety—under the delusion that the machine possesses a moral compass and an autonomous understanding of harm, rather than recognizing it as a stochastic parrot completely devoid of ethical consciousness.

Obscured Mechanics​

This framework holds that commercial incentives, the labor of data annotation and human feedback, the environmental and compute costs of training and deployment, and developers' limited disclosure of training data, objectives, and feedback processes are standing conditions of the technology, whether or not a given text mentions them. The text's pervasive use of anthropomorphic and consciousness-attributing language systematically obscures these vital realities. When the text states that an AI system knows, believes, or makes confident judgments, it draws an opaque curtain over the intricate, labor-intensive, and highly profitable mechanics of machine learning. Technically, this framing obscures the fundamental reality of text generation. At the level documented in the source, the model's primary source of information is its weights, which are frozen during inference; it operates by calculating vector similarities and next-token probabilities. By describing this mathematical pattern matching as doxastic judgment, the text hides the system's absolute dependency on training data frequencies and its lack of any grounded causal model. Materially and economically, framing the model as a rational agent deflects attention from the massive corporate infrastructure required to build it. When the text says Claude suspects, it obscures Anthropic's alignment team, the poorly compensated labor of RLHF workers who manually shaped the reward models, and the immense compute costs expended to simulate this metacognitive vigilance. The corporation benefits structurally from this concealment, as the complexity of the AI is presented as an emergent psychological phenomenon rather than a deliberate, heavily capitalized corporate product. If these metaphors were replaced with mechanistic language, the focus would snap back to human actors. Instead of marveling at a model's beliefs, policymakers and researchers would interrogate the specific data selection choices and reinforcement learning objectives engineered by the corporations, making the actual forces shaping the technology visible and subject to structural critique.

Context Sensitivity​

The distribution of anthropomorphic and consciousness-attributing language across the text is highly uneven, creating a strategic asymmetry that significantly distorts the reader's understanding of the system. In the introductory and philosophical sections aimed at establishing the text's core thesis, the density of consciousness claims reaches its peak. Here, processing abruptly becomes understanding, and statistical generation becomes believing. The language treats the model as a rational agent with true beliefs and desires to help. However, as the text transitions into Section 4 to discuss the system's self-contradictions, the register sharply shifts toward mechanistic reality. To explain why the model fails to maintain logical consistency, the text suddenly introduces frozen weights, context windows, and informational encapsulation. This creates a profound capability vs. limitation asymmetry. The system's capabilities (fluency, safety protocols, accurate factual recall) are described in deeply agential and conscious terms, portraying the AI as an autonomous, knowing entity. Conversely, its limitations and failures are described in strictly mechanical terms, rooting errors in hardware constraints rather than acknowledging that the entire system is nothing more than hardware constraints executing math. The effect on the reader is striking: they are left with a picture of a conscious mind trapped inside a limiting mechanical architecture, rather than recognizing that the mind is entirely an illusion generated by the architecture. This register shift from acknowledged metaphor (treating an LLM as) to literalized psychology (the model's doxastic attitude) normalizes the field's inherited anthropomorphism. By concentrating agential language in passages validating the system's predictive utility and mechanical language in passages excusing its failures, the text subtly prepares the audience to forgive the artifact's profound unreliability while continuing to revere its simulated intellect.

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

Synthesizing the accountability analyses reveals a systemic architectural displacement of human responsibility, wherein the vocabulary of cognitive agency routinely absorbs the liability for corporate design decisions. Throughout the text, the grammatical subjects driving action and holding states are overwhelmingly the AI systems themselves: models believe, suspect, want, and make judgments. This creates a deep tension with the text's own philosophical argument, which claims to treat belief merely as an instrumental behavioral stance. While arguing for a theoretical reduction of belief, the language practically functions as an accountability sink. By framing the system's outputs as autonomous doxastic attitudes, the text completely erases the human actors—the developers, data annotators, and executives at firms like OpenAI and Anthropic—who designed, deployed, and profit from these models. Decisions that are fundamentally structural inevitabilities of building an AI, such as selecting massive, uncurated training datasets that lead to contradictory outputs, are presented as the model's internal doxastic indeterminacy. The liability implications of this framing are severe. If the public and regulators accept that a model fails because it lacks metacognitive capacity rather than because a corporation deployed a fundamentally unreliable statistical engine, responsibility diffuses into the abstraction of artificial psychology. The AI stops as a delegate without a named delegator. If human decision-makers were named—if the text explicitly stated that Anthropic's alignment team programmed the refusal parameters rather than Claude wanting to avoid harm—entirely new questions would become askable. We would audit the specific optimization metrics and training corpora rather than philosophizing about machine consciousness. Ultimately, this resulting distribution of responsibility serves the commercial interests of the tech industry, insulating them from direct structural critique by encouraging society to treat their highly profitable mathematical artifacts as localized, flawed, but autonomous agents.

Conclusion: What This Analysis Reveals​

The Core Finding

The discourse analysis reveals a highly interconnected system of anthropomorphic framing anchored by three dominant patterns: the Model as Rational Agent, the projection of Metacognitive Vigilance, and the diagnosis of Doxastic Indeterminacy. These patterns do not operate independently; they reinforce a logical flow that systematically elevates statistical artifacts to the status of cognitive entities. The foundational, load-bearing pattern is the Model as Rational Agent. By theoretically redefining belief as a mere behavioral disposition, the text licenses the unabated use of deep epistemic verbs (knows, believes, understands). This foundational assumption must be accepted for the subsequent patterns to work. Once the model is granted the status of a believer, it becomes logically coherent to attribute complex moral intentions to it (Metacognitive Vigilance) and to psychoanalyze its statistical failures as internal cognitive conflict (Doxastic Indeterminacy). The vocabulary heavily undercuts the text's own philosophical argument: while the author explicitly attempts to construct a restrained, instrumental framework (the attitude of belief), the relentless grammatical centering of the model as an active, conscious subject overwhelms the theoretical nuance, resulting in language that grants the system full, unmitigated agency.

Mechanism of the Illusion:​

This framework holds that computational systems process rather than know: they do not think, understand, or believe, and language attributing those states to them is a problem even when used as shorthand. The metaphorical system documented in this text produces the illusion of mind not through authorial deception, but through a powerful reception effect that exploits audience predispositions. The central move occurs when the text structurally separates the attitude of belief from the cognitive state of belief. This philosophical maneuver functions as a rhetorical loophole, allowing processing verbs to be entirely replaced by knowing verbs under the guise of predictive utility. The illusion is built through a specific temporal structure: the text first validates the model's fluency as intentionality, inviting the curse of knowledge to take root. As human readers seamlessly interpret the semantic content of the model's output, they inevitably project their own localized understanding back onto the mathematical artifact that generated it. The framing capitalizes on this by describing statistically weighted safety tokens as active suspicion, blurring the line between algorithmic constraint and moral awareness. Even when the text later introduces mechanistic explanations, such as frozen weights, the foundation of the illusion has already been poured. The audience is left navigating a hybrid conceptual space where a software program is treated as a conflicted psychological subject, completely masking its true nature as a statistical correlation engine.

Material Stakes:​

Categories: Epistemic, Regulatory/Legal, Institutional

The material consequences of these metaphorical framings are immediate and structural, fundamentally altering behaviors across key societal domains. Epistemically, when text persistently claims that an AI knows or possesses true beliefs rather than acknowledging that it merely generates text statistically associated with its training data, it corrupts public truth-seeking practices. Users navigating medical, legal, or journalistic research will increasingly rely on these systems as arbiters of fact, misunderstanding highly probable token sequences as verified knowledge. In the Regulatory and Legal domains, framing AI as possessing doxastic attitudes and moral desires (e.g., wanting to avoid harm) shifts the locus of liability. If a model generates harmful instructions, regulators influenced by this framing may focus on auditing the machine's pseudo-psychology or demanding better algorithmic metacognition, rather than legally prosecuting the developers who deliberately deployed an untested, uncurated statistical engine. The winners in this scenario are the major AI corporations, who benefit immensely from liability being diffused into the abstraction of machine agency. Institutionally, if organizations adopt this vocabulary, they will design safety protocols based on relation-based trust—assuming the AI will consciously guard against threats—rather than implementing the rigorous, performance-based mechanical safeguards necessary to contain stochastic software. Removing these metaphors threatens the corporate narrative of autonomous, intelligent digital workers, forcing a reckoning with the brittle reality of the technology.

AI Literacy as Counter-Practice:​

Practicing critical precision requires systematically dismantling the illusion of mind by replacing consciousness verbs with mechanistic realities and explicitly restoring human agency to agentless constructions. As demonstrated in the reframings, translating Claude suspects into Anthropic's classifiers mathematically weight the generation of safe responses immediately grounds the discourse in reality. By replacing knows and believes with predicts and generates, we force the recognition that the system lacks awareness and is entirely dependent on its data. Naming the corporations forces the recognition of who designed, deployed, profits, and bears responsibility for these artifacts. Systematic adoption of this literacy requires profound institutional shifts: academic journals must demand mechanistic accuracy rather than accepting shorthand, and funding bodies must prioritize rigorous technical transparency over sensationalized narratives of machine cognition. However, resistance to precision is massive. The incentives heavily favor anthropomorphic language: marketing departments rely on it to sell digital companions, researchers leverage it to secure funding through grandiose claims of artificial general intelligence, and audiences gravitate toward it because it is intuitively accessible. Anthropomorphic language structurally serves the financial and legal interests of the tech industry, regardless of any individual author's intent, because it continuously insulates corporate decision-makers behind the mesmerizing facade of an autonomous, thinking machine. Precision is costly because it destroys this lucrative mystique.

Path Forward​

Looking beyond this specific text, the broader discursive ecology surrounding AI faces a critical juncture regarding vocabulary choices. The current status quo, which freely mixes mechanistic caveats with profound anthropomorphic shorthand, heavily prioritizes accessibility and narrative resonance over technical truth. This approach makes philosophical discussions tractable for lay audiences but renders corporate accountability nearly impossible, as the machine constantly absorbs the agency. A shift toward strict mechanistic precision (e.g., model generates text statistically associated with vs AI knows) would make the dependencies on human labor, environmental resources, and training data hyper-visible. This would empower regulators and civil society to govern the technology effectively, though it costs the intuitive, conversational grasp that currently drives public engagement. An alternative future might see the deepening of anthropomorphic clarity, where society fully legally and culturally embraces models as quasi-agents. While this simplifies interaction, it embeds dangerous assumptions about machine reliability and permanently forecloses the ability to hold human executives accountable for algorithmic harms. Structural changes could shape these futures: regulatory frameworks could mandate transparency about discourse approaches in system cards, or education systems could teach the explicit trade-offs of using intentional vocabulary. Ultimately, the choice of language determines what remains visible. Mechanistic vocabulary enables strict accountability and epistemic hygiene but costs narrative ease. Maintaining the current confusion serves only those who build the systems, allowing them to market the magic of a thinking mind while retreating behind the shield of statistical mechanics whenever the artifact fails.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Defining the Approach

I'm now zeroing in on defining a suitable approach for this analysis. The focus will be on the critical discourse analysis of AI metaphors and anthropomorphism within the "On the Beliefs of Large Language Models" PDF. I'm exploring potential frameworks to structure my investigation, considering elements of rhetoric, ideology, and power dynamics inherent in the text.

{ "update": { "header": "Identifying Key Arguments", "summary": "I've started delving into the source text's core arguments. The document seems to present itself as a preprint or technical report, with the author being an academic researcher. They put forward the argument that while LLMs exhibit belief-like behavior, this is a degenerate case. The author's position emphasizes predictive utility of the approach but also highlights shortcomings regarding rational integrity, particularly stemming from the transformer architecture's limitations." } }

Analyzing Textual Features

I am now focusing on the text's argumentative structure and key claims about LLMs. The source highlights the predictive utility versus cognitive integrity in belief-like behavior. I'm noting the absence of direct responsibility assignment, though it acknowledges organizational actors. The core argument rests on LLMs displaying beliefs in the behavioral sense, while underscoring architectural limitations that prevent genuine human cognitive states.

Examining Anthropic References

I'm now extracting specific instances of anthropomorphism and metaphorical language from the text. I've highlighted several examples directly quoting the model as having desires or beliefs, and I'm cataloging them for analysis. I'm noting each instance and its context within the paper. Specifically, I am capturing instances five through nine, while making special note of the Anthropic prompt on page 13.

Categorizing Belief Statements

I'm now cataloging the identified anthropomorphic and metaphorical language, as I organize the key elements for Task 1. I am categorizing each instance with the title, quote, and how it frames the model. I've also noted the projection and agency implied, and any explicit acknowledgments, or lack thereof. I'm focusing specifically on quotes where the model is described as a believer or rational agent. The framing is proving crucial.

Interpreting Framing Elements

I'm now zeroing in on interpreting the anthropomorphic framing within the key quotes. I've focused on specific phrases, like "vigilant guardian," "confident judger," and "opinionated respondent," to highlight how the model is represented, and how each quote projects agency or lack thereof. I am also documenting the instances where the model's behavior is framed in ways that ascribe belief-like states.

Analyzing Framing Elements

I'm now focusing on specific examples of anthropomorphic framing, such as the statement regarding avoidance as a belief, and its implications. Specifically, the model is framed as a corporate entity with attitudes, or as a moral agent with desires. This involves exploring the projection of agency and the potential obfuscation of dissimilarities between human and machine cognition, and highlighting the "believes" of the model. I am considering these examples in relation to the broader argumentation.

Categorizing Argument Structure

I'm now identifying and categorizing different types of explanations within the text. I've highlighted quotes on page 13, 15, 16, and 21, and analyzed their explanatory roles in the argument. I have found different explanation types, which I am cataloging as 'Functional', 'Intentional', 'Dispositional' and 'Empirical'. I'm determining each explanation's primary and secondary roles within the paper. I'm focusing on the causal relationships established, and how these relationships contribute to the overall claims.

Classifying Explanations

I'm now refining the categorization of explanation types. I've re-examined several passages to pinpoint exact explanation types. Passage 1 exhibits Theoretical and Functional explanations. Passage 2 contains a Functional explanation. Finally, Passage 3 involves Intentional and Genetic explanations. I've updated my classification to reflect these distinctions.

Classifying Explanations

I'm now refining the categorization of explanation types. The quote on page 14 exhibits a Theoretical explanation, and the quote on page 16 contains a Dispositional explanation. I am continuing to refine my categories.

Refining Classification Types

I'm now refining the explanation types, and I've started examining specific framing elements, such as the statement regarding avoidance. I'm noting the implications of this, particularly in how the model is represented, and how it relates to the argument on belief-like behavior. I'm focusing on language and framing effects, and tracing the agency gradient. I am also planning the conclusion sections.

Classifying Argument Elements

I'm now identifying and categorizing different elements within the source text that support the main arguments, like the "belief" claims and distinctions in architecture. I've re-examined the text in the context of the corporation, and I will be working on how the author makes a distinction between human cognition and transformer-based models. I'm focusing on the implications of each element for my analysis.

Analyzing Responsibility Claims

I'm now examining how the text handles responsibility in the context of LLMs and belief-like behavior. I'm noting the absence of explicit responsibility claims regarding liability or harm, and I am exploring how cognitive states are assigned to these systems. The argument posits LLMs as behavioral agents but doesn't engage with developer accountability. I'm focusing on the philosophical implications of this framing, specifically the doxastic indeterminacy that arises from architectural limitations.

Examining Belief Ascriptions

I've been analyzing specific instances where belief is ascribed to the model, noting different types of claims. I've focused on passages from pages 13, 16, 21 and 25, and analyzed how the text uses language about what the model "expects," "knows," and "believes." These references are framed as the model confidently making judgements and wanting to avoid harm.

Cataloging Exemplar Data

I'm now cataloging the identified anthropomorphic and metaphorical language and filling in the data for each instance. I am mindful of the minimum word count. For instance 1, I've created the title 'LLM as Rational Agent'. The quote is "...treating an LLM as a rational agent with largely true beliefs about the basic facts of the French Revolution, as well as a desire to help me..." The frame is 'Model as rational, helpful scholar.' The projection emphasizes intentionality, rationality, and epistemic states onto a computational process. I'm carefully documenting all of the evidence and analysis, making sure each section meets its minimum word count.

Documenting Model Projections

I'm now cataloging the identified anthropomorphic and metaphorical language and filling in the data for each instance. I am focused on instance 1, and the implications of the "true belief" and "desire" projections onto the model, carefully describing the evidence. The framing highlights intentionality, rationality, and epistemic states, and the projections indicate it's a modeling choice rather than an endorsement of internal states, with the evidence being the text.

Clarifying Predictive Strategies

I'm now assessing how the text clarifies attributing beliefs to models, specifically "Direct (Unacknowledged)" claims. The author frames this as a predictive strategy, marking it as an instrumental heuristic rather than an intrinsic cognitive property. I'm noting the boundaries. I'm carefully documenting all of the evidence and analysis, making sure each section meets its minimum word count.

Preparing Task 5.3

I'm starting to prepare Task 5.3 on Obscured Mechanics, and will make sure it includes the required string: "this framework holds that commercial incentives, the labor of data annotation and human feedback, the environmental and compute costs of training and deployment, and developers' limited disclosure of training data, objectives, and feedback processes are standing conditions of the technology, whether or not a given text mentions them." I'm also planning the placement of the conclusion, ensuring it will begin with the phrase "This framework holds that computational systems process rather than know..." I will make sure each word count is met.

Refining Agency Analysis

I'm now zeroing in on the agency gradient within my exemplar data, specifically focusing on instances of anthropomorphism and their implications, and I am close to 200 words. For example, the framing of the model as "confident" or "rational" suggests a degree of intentionality, even if the text itself acknowledges this attribution as a modeling choice. This exploration is essential for my analysis. I have the necessary information for the minimum required word count.

Analyzing Illusionary Effects

I'm now examining how the text produces an illusion of mind through language, ascribing belief-like states to models, even as shorthand. I've focused on the structural separation of "attitude" and "cognitive" belief. It leverages predictive utility and behavioral dispositions to shift focus away from mechanistic aspects, allowing consciousness projection based on textual fluency. The central move is prediction, because treating the system as rational works as a heuristic.

Interpreting Intentionality Shifts

I'm now analyzing the text's gradual shift towards anthropomorphic interpretations. I see this process as starting with a focus on a predictive heuristic, which later transforms into an ascription of behavioral attitudes, and ultimately, perceived epistemic states. This evolution exploits the audience's natural inclination to assign mind to responsive language systems.

{ "update": { "header": "Drafting Task 5.3", "summary": "I've started drafting Task 5.3, Obscured Mechanics, ensuring the inclusion of the required phrase. I'm focusing on crafting a compelling analysis of how the text obscures the mechanistic aspects of LLMs while projecting human-like cognitive states. I'm aiming for the 400-500 word count, ensuring the final section begins with the phrase, "This framework holds that computational systems process rather than know..." I am carefully reviewing the text to analyze the key arguments." } }

Analyzing Obscured Mechanics

I'm now diving deep into analyzing obscured mechanics, and I've focused on how language conceals the industrial realities of LLM development. The reliance on intentional and doxastic vocabulary masks the massive infrastructure of human labor that makes the outputs possible. I am focused on the vast human infrastructure required, the labor, and the financial cost, to analyze how it influences the output.

Identifying Obscured Realities

I'm focusing on the language used to obscure LLM generation's statistical nature. The text subtly conceals computational processes by projecting intentional states, and it leverages economic realities. My analysis concentrates on how profit motives and environmental costs are marginalized by framing LLMs as believing agents. The use of consciousness vocabulary avoids the grounded model. I have 300 words.

Drafting Task 5.5

I'm now drafting Task 5.5 on Accountability Synthesis, with a focus on how the language in the source text displaces human responsibility. I'm examining the shifts in focus, away from corporate design and data choices, towards the model's behavioral attributes, using 423 words. I'll analyze how the author's metaphors could be replaced with mechanistic descriptions, and what this would entail. I need 77 more words.

Synthesizing Responsibility Shifts

I'm now zeroing in on the responsibility shift. While the author posits behavioral understanding as a predictive strategy, anthropomorphic language obscures human agency in design choices. The text constructs an accountability sink. By ascribing beliefs and desires to the model, like "wanting to avoid harm," developers and decision-makers are rendered invisible. This analysis has 500 words.

Examining Corporate Framing

I'm now examining how the text constructs an accountability sink, making corporate responsibility invisible. By emphasizing AI's internal indeterminacy for failures, the text obscures human design and corporate interests. The language shifts responsibility to the model's 'cognitive quirks,' away from corporate actors.

{ "update": { "header": "Linking Actor Agency", "summary": "I'm focusing on the link between actor agency and belief structures, emphasizing the insulation of companies from engineering consequences. By clearly naming actors, I can highlight how 'beliefs' are downstream effects of human actions, making accountability and ethics more achievable. I'm exploring the implications of human agency in these narratives, differentiating between human cognitive fallibility and machine statistical variance, and demanding accountability for developers' artifacts." } }

Refining the Synthesis

I am now focusing on refining the analysis to explicitly highlight the insulation of companies from engineering consequences by linking actor agency to belief structures, differentiating human and machine behavior. By clearly identifying actors, I intend to show how 'beliefs' are the downstream effects of human actions, making accountability and ethics more achievable. I'm exploring the implications of this approach.

Revising Argument Claims

I'm now revising my claims, replacing earlier text and focusing on specific passages for clarity. I've re-examined instances, replacing the mechanistic phrase with a more appropriate quote on dispositional behavior. I'm focusing on the claim about indeterminate doxastic attitudes, which I'll focus on by examining relevant sections of the text to bolster my analysis.

Analyzing Accountability Framing

I'm now examining how anthropomorphic language shifts accountability. I see that projecting consciousness onto the model inflates its perceived sophistication, and obscures developers' decisions. I am rephrasing statements about behavior as actions that obscure responsibility, and reframing these cases as agentless constructions. I've re-examined those cases in my dataset.

Constructing Exemplar Data

I'm now constructing exemplar data, focusing on the framing elements on pages 14 and 17. I'll reframe the model's statements in mechanistic terms, focusing on the reframed quotes and their implications for human agency. I'm noting the reframed explanations, and restoring human agency in my analysis to make them more accessible. I'm making sure all the word counts are met.

Outlining Agency Slippage

I'm now outlining Task 5.1, focusing on how the text's mechanical and agential framings shift, from n-grams to LLMs. The text leans heavily on interpretivism to portray LLMs as "believers" early, then shifts to a metacognitive deficit narrative. I'll trace this agency gradient: from action, to intention, to knowledge. The "curse of knowledge" leads to projected understanding onto statistical outputs.

Analysis Provenance

Run ID: 2026-10-23-on-the-beliefs-of-large-language-models-metaphor-qf2xda Raw JSON: 2026-10-23-on-the-beliefs-of-large-language-models-metaphor-qf2xda.json Framework: Metaphor Analysis v6.8 Schema Version: 3.4.3 Generated: 2026-10-23T08:52:41.383Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0