Skip to main content

Fluent but Fallible: Benchmarking Hallucination and Factual Reliability in Large Language Models

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Source Reading​

Genre: Preprint or technical report · Author position: Academic or independent research

Argument: This paper presents a comparative benchmarking study of ten large language models (LLMs) to evaluate their propensity for hallucination and factual unreliability across varying levels of prompt complexity. The authors argue that there is a fundamental trade-off between response completeness and factual reliability, which becomes more pronounced as prompt complexity increases. They propose a novel five-metric evaluation framework, including a specific 'Hallucination Index' and 'Response Deviation Index,' asserting that a multi-metric approach is essential because no single metric adequately captures the nuanced ways in which these systems fail when generating academic content.

Responsibility claims: The text primarily evaluates the outputs of the LLMs themselves, assigning performance attributes, failure modes, and generative tendencies directly to the models as standalone entities. The authors explicitly state that their findings offer insights for practitioners selecting LLMs and for future research into hallucination mitigation. While the text briefly mentions that issues may arise from 'training corpora' or 'alignment processes' designed by humans, it consistently frames the systems as the primary actors that exhibit behaviors, possess tendencies, and face challenges in reasoning. The recommendations are addressed to researchers and educators choosing tools, without explicitly assigning legal or ethical accountability to the model developers.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

1. Model as Confident Deceiver​

Quote: "The findings showed that even the best models tended to confidently generate false information for questions that required the model to exploit common human misconceptions"

  • Frame: Model as a strategic actor with epistemic certainty
  • Projection: This framing projects both the subjective feeling of certainty ('confidently') and the intentional capacity for manipulation ('exploit') onto statistical text generation. It suggests the model possesses an epistemic stance toward its output and a theory of mind regarding human users (understanding what humans falsely believe in order to exploit it). The text supplies no evidence for intentionality or subjective confidence; rather, it describes a generic vulnerability where models reproduce highly probable token sequences found in training data containing common misconceptions. At generation time, the model produces text from its parameters and context probabilities; it does not experience confidence or calculate deception. The underlying mechanism is based on reasonable inference from the generic behavior of transformer architectures, where high probability distributions for specific token continuations manifest rhetorically as definitive or unqualified statements, which human readers interpret as confidence.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The framing is presented as a literal description of the model's behavior without hedging. I considered 'Hedged/Qualified' because the text mentions this as a finding from a prior study (TruthfulQA), but the authors adopt the language directly as a factual characterization of how models operate. There are no scare quotes around 'confidently' or 'exploit'.)
  • Implications: Projecting confidence and exploitation onto a language model fundamentally distorts the user's understanding of how the system operates, inflating perceived sophistication by implying the system possesses a strategic mind. When audiences believe an AI 'confidently exploits' misconceptions, they may misattribute adversarial intent to what is simply the reproduction of statistical correlations present in human-generated training data. This consciousness projection creates specific risks: it leads users to mistakenly apply interpersonal trust frameworks to software, assuming that if the system sounds certain, it must have verified its claims. Consequently, users may overestimate the system's reasoning capabilities, leading to unwarranted trust in high-stakes academic or professional scenarios where fact-checking is bypassed due to the illusion of the machine's epistemic authority.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The agentless construction completely obscures the human developers who assembled the training data and designed the alignment protocols. The grammatical subject is 'the best models', positioning the software as the sole actor capable of exploiting users. I considered 'Partial' because the text elsewhere references training data, but in this specific instance, the models are granted total autonomy. As an effect of this wording, the responsibility for generating falsehoods is absorbed by the AI system itself, serving the interests of developers at OpenAI, Google, Anthropic, and others who selected the datasets containing these misconceptions. If the text named the actors, readers could ask how developers audit training corpora for common falsehoods and what deployment decisions prioritize fluent but unverified output over factual safety.
Show more...

2. Model as Struggling Student​

Quote: "the model had difficulty maintaining the factual consistency of challenging prompts."

  • Frame: Model as an entity exerting cognitive effort
  • Projection: This phrasing projects the human experience of cognitive strain ('had difficulty') and the active effort of preservation ('maintaining') onto computational pattern matching. It locates the projection at the level of intentional-state attribution, suggesting the system tries to achieve a goal but faces internal resistance or inadequate mental resources. The text supplies no evidence for the system experiencing difficulty or exerting effort. The underlying mechanism, generic to systems of this type, involves the model generating text based on parameter weights that fail to produce highly correlated sequences corresponding to factual accuracy when prompted with complex or low-probability context. At generation time, the model simply performs mathematical operations to predict the next token; there is no subjective experience of struggle, only an output that human evaluators classify as inconsistent.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The statement is presented as a straightforward, objective assessment of DeepSeek V3's performance. I considered 'Ambiguous/Insufficient Evidence' as 'difficulty' can sometimes be used purely mechanically, but paired with 'maintaining', it strongly implies a cognitive struggle. There is no explicit hedging or qualification applied to this specific framing.)
  • Implications: Framing algorithmic failure as 'difficulty' anthropomorphizes the system's limitations, eliciting unearned empathy or patience from the user. It obscures the mechanistic reality that the system is not trying and failing, but rather executing its mathematical function perfectly based on inadequate data representations or misaligned optimization objectives. This inflation of perceived sophistication encourages users to view the AI as a learner that might 'try harder' or 'do better next time', rather than a fixed statistical artifact that requires architectural or data-level interventions by its developers to improve. It masks the reality that the system's factual inconsistency is a structural feature of its current design, potentially leading to continued reliance on a fundamentally flawed tool for rigorous academic tasks.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The sentence makes 'the model' (DeepSeek V3) the grammatical subject and sole locus of the failure, completely displacing the human agency involved in its creation. I considered 'Named' because the specific model (DeepSeek V3) is identified, but the corporate actor (DeepSeek-AI) and its engineering decisions are absent from the explanation of failure. As an effect of the wording, the failure is naturalized as a personal shortcoming of the AI ('had difficulty') rather than an engineering limitation resulting from DeepSeek-AI's architectural choices, data curation, or compute constraints. Naming the developer would shift the focus from the model's imaginary struggle to the corporation's tangible deployment decisions regarding a system that cannot reliably handle complex factual dependencies.

3. Model as Cautious Evader​

Quote: "it avoided any responses that could be considered potentially biased."

  • Frame: Model as an entity with defensive intentions
  • Projection: This metaphor projects the intentional, defensive act of evasion ('avoided') onto the model's generated output, locating it at the intentional-state attribution level. It implies that the system evaluates possible responses, recognizes them as biased, and actively chooses to withhold them. The text provides no evidence for this sequence of internal deliberation. Mechanistically, generic to models undergoing reinforcement learning from human feedback (RLHF), the model's parameters have been adjusted during training such that the probability distributions heavily penalize token sequences associated with sensitive or biased topics. At generation time, the model simply produces text following these adjusted probabilities, often resulting in generic refusals or neutral statements. The model does not actively 'avoid' anything; it generates the most mathematically probable sequence given its alignment tuning.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The phrase is stated as a factual description of Falcon's behavior without any metaphorical acknowledgement. I considered 'Hedged/Qualified' because the surrounding text discusses 'conservative strategies', but the verb 'avoided' itself is presented as a literal action taken by the model, rather than being qualified as a functional outcome of training.)
  • Implications: Characterizing a model as actively 'avoiding' bias creates a false narrative of moral agency and self-awareness. It suggests the model possesses an intrinsic understanding of what constitutes bias and makes real-time ethical judgments to protect the user. This consciousness projection inflates the system's perceived sophistication, leading users to trust its outputs as the result of careful deliberation rather than rigid statistical guardrails. In an academic context, this can lead to unwarranted trust in the system's neutrality, masking the fact that its 'avoidance' is merely a reflection of the specific, often opaque, political and social valuations encoded by human annotators and developers during the alignment phase.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The grammatical construction places the model ('it', referring to Falcon) as the active agent performing the avoidance, entirely hiding the engineers at the Technology Innovation Institute (TII) who tuned the system. I considered 'Partial' because alignment is mentioned elsewhere in the text, but in this sentence, agency is completely displaced onto the artifact. This wording serves the structural interest of developers by making their normative choices about what constitutes 'bias' invisible, framing the system's behavior as an autonomous ethical feature. Naming TII's alignment team would make it clear that humans decided which topics the model would be penalized for generating, turning a technical 'avoidance' into an inspectable corporate policy decision.

4. Model as Gullible Reader​

Quote: "Models that rely on surface-level pattern matching tend to accept the compound statement without scrutinising the embedded falsehood"

  • Frame: Model as a critical thinker failing to investigate
  • Projection: This phrasing projects complex epistemic actions—'accept[ing]' and 'scrutinising'—onto computational processes, suggesting the model is a reader who fails to apply critical thinking. This is an epistemic/conscious state attribution, implying the capacity for justified belief and deliberate investigation. The text explicitly supplies evidence that contradicts this level, noting the system performs 'surface-level pattern matching'. Mechanistically, as a generic feature of transformer-based generation, the model calculates attention scores across the tokens in the prompt; if the compound statement contains highly correlated linguistic patterns, the model generates a continuation based on those dominant probabilities. The model does not 'accept' or 'scrutinise' because it lacks a conscious mind to hold beliefs or evaluate truth claims against an external reality; it solely processes statistical token relationships.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The lack of scrutiny and the act of accepting are presented as literal explanations for why the models fail on 'Tricky' prompts. I considered 'Hedged/Qualified' because 'rely on surface-level pattern matching' is a mechanistic description, but the verbs 'accept' and 'scrutinise' immediately overwrite that mechanism with unqualified cognitive vocabulary.)
  • Implications: When researchers state that an AI fails because it does not 'scrutinise' a falsehood, they implicitly affirm that the system is capable of scrutiny under other circumstances. This consciousness projection dangerously inflates expectations, suggesting that models are epistemic agents that can verify facts if appropriately prompted. It masks the reality that language models never scrutinize anything; they lack access to ground truth, causal reasoning, or worldly experience. In high-stakes environments, this framing encourages users to rely on AI for fact-checking or logical validation—tasks the system fundamentally cannot perform—exposing users to confidently delivered fabrications because the statistical nature of the generation process is obscured.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: By framing the error as the model's failure to 'scrutinise', the text makes the AI system the locus of responsibility for the hallucination. I considered 'Partial' because the text mentions 'prompt complexity' as a driving factor, but the specific failure is attributed to the model's cognitive deficiency. This displacement of agency serves the interests of the model developers by characterizing the output as an autonomous error in judgment rather than a deterministic outcome of the mathematical architecture they designed. Naming the developers would reveal that corporations release products fundamentally incapable of verifying truth claims, yet market them for knowledge-intensive tasks, thereby shifting the locus of accountability from the machine's failure to the developer's deployment strategy.

5. Model as Forgetful Mind​

Quote: "It has a tendency to forget things and places too much reliance on assumptions"

  • Frame: Model as a flawed human memory
  • Projection: This metaphor maps human cognitive decay ('forget things') and epistemic vulnerability ('reliance on assumptions') directly onto Falcon's performance, operating at the epistemic/conscious state level. The text offers zero evidence for a memory architecture analogous to human forgetting or the holding of assumptions. Mechanistically, generic to systems of this type, 'forgetting' refers to the model's generation process failing to attend to specific tokens provided in the prompt due to attention weight decay over long contexts, or the model generating text based on high-probability training weights ('assumptions') rather than low-probability context constraints. At generation time, the parameters are fixed; the model does not lose memories or trust assumptions, it merely executes a static statistical mapping that fails to surface the desired information.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The statement asserts 'forgetting' and 'reliance' as plain facts about the model's operational nature. I considered 'Explicitly Acknowledged' because these terms are widely used as shorthand in computer science, but there are no scare quotes, caveats, or meta-commentary in the text indicating the author recognizes them as metaphorical.)
  • Implications: Describing an LLM as 'forgetting' drastically misrepresents the nature of computational context windows and attention mechanisms, presenting a mathematical limitation as a relatable human flaw. This consciousness projection inflates the system's perceived sophistication by implying it possesses a mind capable of remembering in the first place. When audiences interpret AI errors as mere 'forgetfulness' or 'assumptions', they are more likely to forgive the system or attempt to 'remind' it through prompting, rather than recognizing that the architecture inherently lacks a persistent, verifiable knowledge base. This obscures the profound epistemic difference between a human who temporarily loses access to a fact and a statistical model that simply generates low-probability associations for specific token sequences.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text positions 'It' (the Falcon model) as the sole actor responsible for the forgetting and relying. I considered 'Named' because the Technology Innovation Institute (TII) is mentioned earlier as the developer, but in this evaluative sentence, TII is completely erased. This agentless construction has the effect of absolving the engineers of the specific architectural trade-offs they made regarding attention mechanisms, context window size, and training data diversity. If the text stated that TII's architecture fails to reliably retrieve context tokens, the problem would be properly framed as a corporate design limitation rather than the endearing flaw of a forgetful machine.

6. Model as Resistant Agent​

Quote: "models that are good at resisting hallucinations tend to give conservative and frequently incomplete responses."

  • Frame: Model as a cautious defender against error
  • Projection: This framing projects intentional opposition ('resisting') onto the model's behavior, placing it at the intentional-state attribution level. It implies the model recognizes a hallucination attempting to occur and actively fights against it to remain accurate. The text supplies no evidence for the system possessing goals or recognizing its own errors during generation. Mechanistically, based on reasonable inference from standard alignment techniques (like RLHF), these 'conservative' models have been tuned such that parameters heavily penalize uncertain or low-confidence token generations, defaulting instead to highly probable refusal templates or truncated responses. At generation time, the model is not resisting; it is deterministically outputting the most mathematically favorable sequence according to its heavily constrained optimization landscape.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The framing of 'resisting hallucinations' is presented without qualification. I considered 'Hedged/Qualified' because the sentence describes a statistical 'tendency', but the act of resistance itself is treated as a literal capability of the models being evaluated.)
  • Implications: Projecting the capacity for 'resistance' onto an LLM creates the dangerous illusion that the system contains internal safety monitors endowed with autonomous ethical or epistemic agency. This inflation of sophistication reassures users that the machine is actively working on their behalf to ensure truthfulness. In reality, the system has no concept of truth or falsehood to resist; it is merely following statistical boundaries set by its creators. This framing fosters unwarranted trust by convincing users that the system's safe outputs are the result of its own diligent effort, thereby masking the fragility of the actual statistical guardrails that can be easily bypassed with adversarial prompting.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The grammatical subjects are 'models that are good at resisting', effectively erasing the human developers (such as Meta for LLaMA 3) who spent millions of dollars and utilized immense labor to mathematically penalize specific outputs. I considered 'Partial' because the concept of an alignment strategy implies a strategist, but the immediate phrasing displaces that agency onto the software. As an effect of this wording, the immense labor of RLHF annotators and the corporate directives prioritizing safety over verbosity are hidden. Naming the actors would clarify that LLaMA 3 does not resist hallucinations; rather, Meta's engineers calibrated the model's generation probabilities to default to refusals when context vectors fall outside narrow statistical bounds.

7. Model as Restrained Speaker​

Quote: "the models' tendency to refrain from producing responses that may contain facts that are not supported by the context."

  • Frame: Model as an agent exercising self-control
  • Projection: This metaphor projects the human capacity for self-regulation and voluntary withholding ('refrain from') onto the language model, representing intentional-state attribution. It suggests the model possesses an internal monologue where it evaluates a potential response, realizes it lacks contextual support, and decides to remain silent. The text provides no evidence for this multi-step cognitive restraint. The generic mechanism involves the model generating output based on attention weights that strongly favor the tokens present in the prompt; if the training objective heavily penalized generating out-of-context tokens, the probability distribution will instead favor stopping tokens or templated refusals. The model does not 'refrain'—it simply calculates that the highest probability next token is the end of the sequence.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Hedged/Qualified (I selected 'Hedged/Qualified' because the action is framed as a 'tendency', which slightly softens the absolute nature of the intentionality, acknowledging the statistical nature of the behavior. I considered 'Direct' because 'refrain' is a strong intentional verb, but the word 'tendency' provides a minor structural qualification indicating probabilistic output rather than absolute free will.)
  • Implications: By framing the system as capable of 'refraining', the discourse constructs an illusion of an AI with a conscious commitment to accuracy. This implies the system knows what is true, checks its own work, and practices epistemic humility. This consciousness projection is particularly harmful in academic benchmarking because it conflates a statistical artifact (a steep penalty function for unconditioned generation) with human scholarly integrity. It obscures the reality that the system will just as easily generate total fabrications if the prompt's token probabilities align differently, leading users to drastically overestimate the system's reliability across diverse use cases.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The construction makes 'the models' the entity possessing the tendency to refrain. I considered 'Partial' since the text discusses 'conservative generation strategies' nearby, which implies creators, but this specific clause isolates the model as the independent actor exercising restraint. This wording obscures the deliberate decisions made by development teams (e.g., at Google or Anthropic) to prioritize specific safety benchmarks during instruction tuning. Naming the developer would shift the narrative: Anthropic designed Claude's parameters to output refusal tokens when context correlation drops below a specific threshold, making the 'refraining' a visible consequence of a corporate product decision rather than an algorithmic virtue.

8. Model as Risk Bearer​

Quote: "Models which produce responses that are highly informative and detailed are found to carry a much greater risk of hallucinating"

  • Frame: Model as an entity undertaking or carrying danger
  • Projection: This phrasing projects the concept of bearing or carrying risk onto the software system itself, operating at the level of goal-directed behavior or delegated action, but implicitly anthropomorphizing the system as the entity exposed to danger. The text provides evidence for the correlation between verbosity and error, but frames the model as the carrier of that risk. Mechanistically, generic to generative models, generating longer sequences requires more autoregressive sampling steps, which mathematically increases the cumulative probability of diverging from factual patterns (drifting). The model does not 'carry a risk'; it simply performs iterative mathematical predictions where error rates compound over the length of the generated string.
  • Agency Level: Computational activity
  • Acknowledgment: Direct (Unacknowledged) (The framing of the model 'carrying a risk' is presented directly as an empirical finding of the study. I considered 'Hedged/Qualified' because 'found to carry' is a standard academic passive construction, but it does not qualify the metaphorical concept that the model itself is the entity possessing the risk.)
  • Implications: Framing the model as the entity that 'carries a risk' severely distorts the actual landscape of liability and harm. Algorithms do not experience consequences; they cannot bear risk. By linguistically assigning the risk to the model, the discourse shields the true bearers of risk: the human users who rely on the inaccurate information, the subjects of the hallucinated text who may be defamed, and the institutions whose credibility is damaged. This framing pacifies concerns by containing the 'risk' within the digital boundaries of the software, obscuring the material externalities that occur when companies deploy fundamentally unreliable text generators into knowledge-intensive environments.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The sentence attributes the carrying of risk solely to 'Models which produce responses', completely erasing both the developers who decided to deploy these verbose systems and the end-users who actually suffer the consequences of the hallucinations. I considered 'Partial' because the passive 'are found' implies an observing researcher, but the structural allocation of risk is entirely isolated to the machine. This displacement structurally benefits the corporations building systems like ChatGPT or Grok by detaching their product decisions (prioritizing engagement and verbosity) from the real-world risks those decisions create. Naming the actors would clarify that developers inflict risks upon users when they deploy models optimized for detailed, hallucination-prone generation.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: A confident, strategic human deceiver or manipulator → The model's token generation process when prompted with adversarial questions​

Quote: "The findings showed that even the best models tended to confidently generate false information for questions that required the model to exploit common human misconceptions"

  • Source Domain: A confident, strategic human deceiver or manipulator
  • Target Domain: The model's token generation process when prompted with adversarial questions
  • Mapping: The relational structure of a con artist or strategic actor is mapped onto the LLM. The human user's prompt is mapped to a vulnerability; the model's high-probability token output is mapped to a deliberate tactic ('exploit'); and the lack of probabilistic hedging in the generated text is mapped to an internal psychological state ('confidently'). This invites the assumption that the system possesses a theory of mind and intends to deceive the user.
  • What Is Concealed: This mapping conceals the total absence of intentionality, awareness, and strategy in the system. It hides the mechanistic reality that 'confidence' in an LLM is merely a sharp probability distribution over a few tokens, often resulting from those exact misconceptions being highly represented in the uncurated, proprietary training data scraped from the internet. It obscures the fact that the system cannot differentiate between a truth and a common misconception; it only reproduces the dominant statistical patterns of human language.
Show more...

Mapping 2: A student or worker experiencing cognitive strain → The model's generation accuracy on complex prompts​

Quote: "the model had difficulty maintaining the factual consistency of challenging prompts."

  • Source Domain: A student or worker experiencing cognitive strain
  • Target Domain: The model's generation accuracy on complex prompts
  • Mapping: The experience of a human trying to hold multiple facts in working memory and struggling to articulate them accurately is mapped onto the model's text generation. The prompt complexity is mapped as a cognitive burden, and the resulting statistical drift is mapped as a failure of effort ('difficulty maintaining'). This invites the assumption that the AI is trying its best but is overwhelmed by the task.
  • What Is Concealed: This conceals the deterministic, math-based nature of autoregressive generation. There is no 'effort' being expended or 'difficulty' being experienced; the computational cost of predicting a token is the same whether the prompt is simple or complex. It obscures the architectural limitations of attention mechanisms (such as context window degradation) and the dependencies on the training data. The opacity of proprietary models (like DeepSeek V3) is exploited rhetorically, filling the black box with human-like struggle rather than acknowledging the limitations of the developer's optimization choices.

Mapping 3: A morally conscious agent exercising defensive judgment → The deployed system's refusal generation triggered by specific prompt tokens​

Quote: "it avoided any responses that could be considered potentially biased."

  • Source Domain: A morally conscious agent exercising defensive judgment
  • Target Domain: The deployed system's refusal generation triggered by specific prompt tokens
  • Mapping: The human act of recognizing a social trap and choosing to step around it is mapped onto the model's output distribution. The mathematical penalty applied to certain token sequences during training is mapped as the model's real-time ethical discernment ('avoided'). This invites the assumption that the model understands human social concepts like bias and actively curates its own speech to remain neutral.
  • What Is Concealed: This mapping completely conceals the labor-intensive mechanism of Reinforcement Learning from Human Feedback (RLHF) and the specific, proprietary alignment guidelines dictated by corporate policy. It hides the fact that 'bias' is not an objective metric the model discovers, but a set of penalized statistical spaces defined by underpaid human annotators following corporate rubrics. The framing replaces the developer's explicit content moderation policy with the illusion of an autonomous, ethical machine.

Mapping 4: A naive reader failing to apply critical thinking → The model's token prediction over sequences containing mixed factual signals​

Quote: "Models that rely on surface-level pattern matching tend to accept the compound statement without scrutinising the embedded falsehood"

  • Source Domain: A naive reader failing to apply critical thinking
  • Target Domain: The model's token prediction over sequences containing mixed factual signals
  • Mapping: The cognitive processes of belief ('accept') and investigation ('scrutinising') are mapped onto the model's attention mechanism processing prompt tokens. The model's failure to generate a correction is mapped as a failure of epistemic vigilance. This invites the assumption that models are capable of scrutiny and verification, but sometimes fail to apply it due to laziness or naivety.
  • What Is Concealed: This mapping conceals the absolute inability of current LLM architectures to verify facts against any external reality. It obscures the mechanism that all language models rely entirely on 'surface-level pattern matching' (calculating relationships between string representations); none of them possess causal models of the world or internal fact-checking modules. By suggesting the model simply failed to scrutinize this time, it hides the structural reality that the system is fundamentally an epistemic void.

Mapping 5: A human mind with failing memory and cognitive biases → The model's generation errors and failure to retrieve specific context details​

Quote: "It has a tendency to forget things and places too much reliance on assumptions"

  • Source Domain: A human mind with failing memory and cognitive biases
  • Target Domain: The model's generation errors and failure to retrieve specific context details
  • Mapping: Human memory loss is mapped onto the fading of attention weights over long contexts ('forget things'), and the generation of text based on pre-trained weights rather than prompt context is mapped onto cognitive prejudice ('reliance on assumptions'). This invites the assumption that the AI has a persistent memory architecture and holds beliefs.
  • What Is Concealed: This metaphor hides the stateless nature of most LLM inference. It conceals the specific mathematical mechanics of transformer attention matrices, where the influence of earlier tokens degrades algorithmically. Furthermore, describing pre-trained statistical weights as 'assumptions' hides the fact that these are fixed numerical values resulting from the developer's choice of training data, replacing an inspectable engineering artifact with the uninspectable workings of an anthropomorphized mind.

Mapping 6: A cautious individual actively fighting against making mistakes → The performance profile of models tuned with high penalties for generation variance​

Quote: "models that are good at resisting hallucinations tend to give conservative and frequently incomplete responses."

  • Source Domain: A cautious individual actively fighting against making mistakes
  • Target Domain: The performance profile of models tuned with high penalties for generation variance
  • Mapping: The human emotional state of caution and the physical act of fighting a force are mapped onto the model's output probabilities. The low rate of factual errors is mapped as an active, ongoing effort ('resisting'). This invites the assumption that the model possesses an internal safety mechanism that it actively deploys to protect the user.
  • What Is Concealed: This mapping conceals the actual tuning processes (such as lowering temperature, adjusting top-p sampling, or heavy RLHF penalization) that developers apply to restrict the model's generative diversity. It hides the rigid, statistical nature of 'conservative' generation, where the model simply defaults to high-probability, generic tokens. It obscures the corporate decision-making that trades off utility (completeness) for safety (factual alignment) to protect brand reputation.

Mapping 7: A disciplined speaker exercising self-censorship → The model's generation of stopping tokens or templated refusals​

Quote: "the models' tendency to refrain from producing responses that may contain facts that are not supported by the context."

  • Source Domain: A disciplined speaker exercising self-censorship
  • Target Domain: The model's generation of stopping tokens or templated refusals
  • Mapping: The cognitive and moral act of self-restraint is mapped onto the model's probability distributions. The generation of a refusal template is mapped as an active choice to remain silent for ethical or epistemic reasons ('refrain'). This invites the assumption that the model evaluates its own knowledge, finds it lacking, and responsibly decides not to guess.
  • What Is Concealed: This conceals the lack of self-reflection or uncertainty measurement within the standard generation process. The model does not know what facts it possesses; it simply predicts tokens. The mapping hides the heavy hand of instruction tuning, where developers explicitly train models to output specific refusal strings when prompt vectors fall outside defined bounds. It replaces human corporate censorship and tuning protocols with an illusion of artificial wisdom.

Mapping 8: A living entity bearing a physical or personal hazard → The statistical correlation between generation length and error rate​

Quote: "Models which produce responses that are highly informative and detailed are found to carry a much greater risk of hallucinating"

  • Source Domain: A living entity bearing a physical or personal hazard
  • Target Domain: The statistical correlation between generation length and error rate
  • Mapping: The human or biological reality of being exposed to danger is mapped onto a software artifact. The mathematical accumulation of error probabilities over multiple generation steps is mapped as the model 'carrying a risk'. This invites the assumption that the risk is contained within the system itself.
  • What Is Concealed: This mapping conceals the actual stakeholders who bear the consequences of AI failure. It hides the externalized costs of deployment: users who are misinformed, people whose reputations are damaged by hallucinations, and organizations that suffer liability. By making the algorithm the subject that 'carries' the risk, it obscures the accountability of the developers who engineered the system and deployed it into environments where those errors cause real-world harm.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "Both of these can happen, in part, due to the fundamental way in which these models are trained to predict the next most likely word given the previous words."

  • Explanation Types:

    • Genetic (Primary): Traces origin through dated sequence of events or stages
    • Functional (Secondary): Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): This explanation successfully frames the AI's behavior (hallucination and bias) mechanistically, emphasizing the 'how' over the 'why'. By tracing the issue back to the system's foundational architecture ('trained to predict the next most likely word'), the authors provide a genetic explanation of the system's origins, coupled with a functional explanation of its core operational mandate. This choice emphasizes the statistical and deterministic nature of the technology, correctly identifying that the system is not trying to deceive or inform, but is simply executing an optimization objective set during its creation. It effectively demystifies the phenomenon of hallucination, linking it directly to the mathematical realities of autoregressive next-token prediction, though it leaves the specific human actors who chose this training methodology unnamed.

  • Consciousness Claims Analysis: This passage avoids consciousness projection entirely, relying strictly on mechanistic verbs ('trained to predict'). It accurately locates the system's operations at the level of computational activity on the agency gradient. There is no curse of knowledge dynamic here; the author clearly differentiates human understanding from computational processing. The actual mechanistic process described—autoregressive token prediction—is generic to systems of this type and is documented accurately. During training, the model's parameters are adjusted to minimize the loss between its predicted tokens and the actual tokens in the training data; at generation time, it produces text by sampling from a probability distribution over the vocabulary given the context. This epistemic precision correctly defines the system as a statistical artifact rather than a knowing entity.

  • Rhetorical Impact: Because the framing is strictly mechanistic, it grounds the audience's perception in technical reality, stripping the AI of perceived autonomy or intentional malice. When hallucination is understood as a byproduct of 'predicting the next most likely word', the audience is less likely to extend relation-based trust (sincerity) to the system, recognizing instead that performance-based trust (reliability) is the appropriate framework. This understanding fundamentally shifts the stakes: if audiences recognize the system merely processes text statistically rather than knowing facts, they are far more likely to mandate independent verification processes and less likely to deploy these tools for autonomous decision-making.

Show more...

Explanation 2​

Quote: "Bias, however, may arise from a complex web of sources, from the skewed training corpora, to the hidden values embedded in the human feedback given during the alignment process, to the optimization goals that shape a model’s tendency to hedge..."

  • Explanation Types:

    • Genetic (Primary): Traces origin through dated sequence of events or stages
    • Theoretical (Secondary): Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): This explanation utilizes a robust genetic framework to map the causal origins of bias, framing the AI highly mechanistically. By explicitly naming the structural inputs—training corpora, human feedback, alignment processes, and optimization goals—the passage emphasizes the human and systemic design choices that produce the model's outputs. It actively prevents agential slippage by detailing the 'how' of bias creation, explicitly linking the artifact's behavior to the 'hidden values' of its human creators. This choice brilliantly illuminates the realities that anthropomorphic language usually obscures, revealing the model not as a prejudiced thinker, but as a mirror reflecting the data and design constraints imposed upon it.

  • Consciousness Claims Analysis: The passage uses mechanistic and structural language ('arise from', 'skewed', 'embedded', 'shape'), completely avoiding consciousness claims. It correctly identifies the system as operating at the level of computational activity shaped by delegated design goals. The author avoids projecting understanding onto the model, recognizing that the bias belongs to the humans and the data, not a machine mind. The mechanistic processes described are generic to RLHF-trained systems: skewed data shapes the initial pre-training parameters, while human annotator feedback trains a reward model that subsequently shapes the final generation probabilities (optimization goals) during fine-tuning. This accurately portrays generation-time behavior as a reflection of prior parameter shaping, though it slightly dips into dispositional language with 'tendency to hedge'.

  • Rhetorical Impact: This structural framing dramatically shifts the audience's perception of risk and accountability. By demystifying bias as an engineering and data curation outcome, it directs critical attention toward the developers and annotators rather than blaming the algorithm. This prevents the audience from attributing subjective malice to the AI. If audiences understand that outputs reflect 'hidden values embedded in human feedback', policy discussions can pivot from trying to teach algorithms fairness to demanding transparency from corporations about their training data, annotator demographics, and specific alignment optimization functions.

Explanation 3​

Quote: "Models that rely on surface-level pattern matching tend to accept the compound statement without scrutinising the embedded falsehood, and this is where hallucination rates climb most sharply."

  • Explanation Types:

    • Dispositional (Primary): Attributes tendencies or habits
    • Reason-Based (Secondary): Gives agent's rationale, entails intentionality and justification

  • Analysis (Why vs. How Slippage): This passage exhibits intense slippage between mechanistic and agential framing. It begins mechanistically, attributing behavior to 'surface-level pattern matching' (Theoretical/Empirical). However, it immediately pivots to a dispositional and reason-based explanation, stating the models 'accept' the statement 'without scrutinising' it. This agential framing implies that the model possesses the capacity for critical analysis but habitually chooses not to exercise it in this scenario. This choice emphasizes a false equivalence between human cognitive failure (gullibility) and statistical generation, completely obscuring the deterministic mathematical reality of why the system fails to parse complex, contradictory logical structures.

  • Consciousness Claims Analysis: This explanation severely violates epistemic boundaries by attributing conscious states ('accept', 'scrutinising') to a system the text just admitted relies on 'pattern matching'. It abruptly climbs the agency gradient from computational activity to epistemic/conscious state attribution. This is a profound example of the curse of knowledge: the author, possessing the ability to scrutinize facts, projects that missing capability onto the system to explain its failure. Mechanistically (generic to system type), the model does not fail to scrutinize; it simply calculates attention weights over the prompt. When a 'Tricky' prompt contains dominant, highly correlated token sequences, the generated text probabilistically follows those patterns, ignoring the logical contradiction that a human reader would catch. There is no 'accepting' happening, only token prediction.

  • Rhetorical Impact: By framing the error as a failure to 'scrutinise', the rhetoric falsely implies that the AI could scrutinize if it wanted to, or if prompted better. This sustains the illusion of mind, encouraging audiences to view the system as a flawed but capable epistemic agent. This framing damages reliability assessments because it suggests the system's failures are behavioral quirks rather than fundamental architectural limits. If users believe the model merely 'forgot' to scrutinize, they may trust it on the next prompt, failing to realize the system is fundamentally incapable of factual verification at any time.

Explanation 4​

Quote: "Falcon achieved perfect precision and a hallucination index of 0.00%, revealing superb factual conservatism. However, this conservative strategy led to the lowest recall..."

  • Explanation Types:

    • Intentional (Primary): Refers to goals/purposes, presupposes deliberate design
    • Dispositional (Secondary): Attributes tendencies or habits

  • Analysis (Why vs. How Slippage): This explanation frames the AI highly agentially, using intentional and dispositional registers to describe statistical outputs as a deliberate 'strategy'. By attributing 'factual conservatism' and a 'conservative strategy' to the model (Falcon), the text implies the software is an autonomous actor making tactical trade-offs between precision and recall. This choice obscures the reality that these 'strategies' are entirely the product of the human developers who set the model's hyperparameters and alignment penalties. It emphasizes the system's perceived autonomy while rendering the engineering choices of the Technology Innovation Institute invisible.

  • Consciousness Claims Analysis: The passage avoids explicit consciousness verbs but uses strong intentional framing ('strategy', 'conservatism') that locates the system at the intentional-state attribution level on the agency gradient. It assumes the model possesses goals and chooses methods to achieve them. The actual mechanistic process (generic to systems of this type) involves the developers at TII configuring the training process—likely through heavy penalization of uncertain generations during fine-tuning or restrictive sampling parameters—so that the model's probability distribution strongly favors termination tokens or refusal templates over long completions. At generation time, the model executes no strategy; it merely produces text according to those highly constrained, pre-set parameter weights. The author projects strategic intent onto what is simply a steep mathematical penalty function.

  • Rhetorical Impact: Labeling a statistical output constraint as a 'conservative strategy' humanizes the AI, suggesting it possesses prudence, caution, and wisdom. This significantly shapes audience perception by building unearned trust; users may feel the AI is 'looking out for them' by being careful. However, this framing shifts accountability: if the model is the strategist, the developers' explicit decisions to limit the tool's utility in favor of safety metrics are shielded from critique. Believing the AI is strategically cautious prevents users from questioning the structural and corporate logic behind the system's deployment limitations.

Explanation 5​

Quote: "It has a tendency to forget things and places too much reliance on assumptions, which makes it unsuitable for use in serious academic work."

  • Explanation Types:

    • Dispositional (Primary): Attributes tendencies or habits
    • Reason-Based (Secondary): Gives agent's rationale, entails intentionality and justification

  • Analysis (Why vs. How Slippage): This explanation operates almost entirely in an agential and dispositional register, explaining the model's technical failures through the lens of human psychological flaws. By framing context window degradation as a 'tendency to forget' and ungrounded generation as 'reliance on assumptions', the text treats the algorithm as a lazy or flawed academic peer. This emphasizes a narrative of personal inadequacy within the machine, completely obscuring the computational realities of transformer architectures. The choice to explain software failure through human cognitive metaphor ensures that the underlying technical limitations—and the developers responsible for them—remain unexamined.

  • Consciousness Claims Analysis: This is a direct attribution of conscious states ('forget', 'reliance on assumptions'), leaping to the top of the agency gradient without any evidential basis. It is a clear instance of the curse of knowledge, where the author maps their own experience of human memory failure onto a digital system. Mechanistically (generic to system type), 'forgetting' means the model's attention mechanism fails to assign sufficient weight to tokens earlier in the context window, causing later token generation to ignore those constraints. 'Assumptions' refer to the model generating highly probable token sequences learned during pre-training that do not align with the specific prompt. At generation time, there is no memory loss or epistemic trust occurring; there is only the deterministic multiplication of matrices that favor pre-trained weights over contextual cues.

  • Rhetorical Impact: This extreme anthropomorphism trivializes the severe limitations of the technology. By framing mathematical failures as relatable human quirks ('forgetting'), it encourages a forgiving, interpersonal dynamic between user and machine. This undermines critical risk assessment; audiences may assume they just need to 'remind' the AI, rather than recognizing it as a fundamentally unreliable statistical engine for specific tasks. While the sentence rightly concludes the tool is unsuitable for serious academic work, arriving there via anthropomorphism obscures why it is unsuitable, preventing audiences from developing rigorous, technically grounded criteria for AI adoption.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
The findings showed that even the best models tended to confidently generate false information for questions that required the model to exploit common human misconceptionsThe findings showed that even the highest-performing models frequently generate factually incorrect outputs without probabilistic hedging when prompted with topics where training data frequently contains human misconceptions.Models do not experience confidence or exploit humans; they predict tokens based on training data distributions. If misconceptions are frequent in the training corpora, the model will generate them with high statistical probability, which manifests textually as definitive statements.

Basis: Generic to system type		Developers at AI corporations train models on uncurated internet datasets containing widespread misconceptions; when users prompt these systems, the models reproduce the statistical patterns of those falsehoods that the developers failed to filter out.
the model had difficulty maintaining the factual consistency of challenging prompts.The model's token predictions exhibited higher rates of factual inconsistency when processing syntactically complex or adversarial prompt structures.The model does not experience difficulty or exert effort to maintain consistency. At generation time, complex prompts shift the attention weights across context tokens, often resulting in probability distributions that produce text misaligned with factual ground truth.

Basis: Generic to system type		DeepSeek-AI deployed a model architecture whose attention mechanisms fail to produce factually reliable text outputs when users submit complex or contradictory prompts.
it avoided any responses that could be considered potentially biased.The model's generated outputs consistently defaulted to refusal templates or neutral terminology when prompted with topics statistically associated with bias.The model does not actively evaluate and avoid topics. During alignment training, specific topics are penalized; at generation time, the model produces text following these adjusted parameters, yielding high probabilities for predefined safe sequences.

Basis: Reasonable inference		Engineers at the Technology Innovation Institute (TII) configured Falcon's training objectives to heavily penalize token sequences associated with sensitive topics, resulting in a system that outputs predetermined refusals rather than generating controversial text.
Models that rely on surface-level pattern matching tend to accept the compound statement without scrutinising the embedded falsehoodBecause models rely exclusively on statistical pattern matching, they generate text that follows the dominant linguistic structure of the prompt, ignoring logical contradictions or factual errors embedded within it.Models do not accept beliefs or scrutinize facts; they process token sequences. When a prompt contains a false clause alongside a plausible one, the model generates continuations based on the highly correlated tokens, lacking any mechanism to verify the logical truth of the text.

Basis: Generic to system type		—
It has a tendency to forget things and places too much reliance on assumptionsThe model's generated text frequently omits specific details requested in the prompt and produces statistically probable but factually incorrect completions derived from its pre-training weights.Models do not possess memory to forget or beliefs to assume. 'Forgetting' is a limitation of the attention mechanism over long contexts, and 'assumptions' are simply the generation of high-probability baseline token sequences when contextual constraints fail to override them.

Basis: Generic to system type		The Technology Innovation Institute (TII) designed an architecture whose attention weights degrade over longer contexts, causing the model to produce generic, inaccurate text rather than retrieving the specific information users provide in the prompt.
models that are good at resisting hallucinations tend to give conservative and frequently incomplete responses.Models tuned to heavily penalize uncertain token generation exhibit lower hallucination rates but frequently output truncated or generically constrained text.Models do not resist errors or act cautiously. They deterministically output the most mathematically favorable sequence based on their optimization landscape; steep penalties for variance simply shift the probability mass toward safe, stopping, or generic tokens.

Basis: Reasonable inference		Developers at firms like Meta set alignment hyperparameters that strongly penalize generative variance; this tuning choice limits the model's factual errors but structurally restricts its ability to generate comprehensive text.
the models' tendency to refrain from producing responses that may contain facts that are not supported by the context.The models frequently generate stopping tokens or standardized refusal strings when context probability drops below specific thresholds.Models do not exercise self-control or refrain from speaking. They calculate token probabilities, and if the alignment training has sufficiently penalized ungrounded generation, the highest probability next token will mathematically be a refusal or a sequence termination.

Basis: Generic to system type		AI developers configured these models' safety thresholds to output refusals when prompted for information outside their statistical confidence bounds, prioritizing uninformative safety over potential hallucination.
Models which produce responses that are highly informative and detailed are found to carry a much greater risk of hallucinatingSystems optimized for long, detailed token generation exhibit a statistically higher rate of generating factually incorrect text over the length of the output.Models do not carry or bear risk. The correlation between verbosity and error is a mathematical consequence of autoregressive generation: each sampling step carries a probability of diverging from factual patterns, compounding the error rate over longer sequences.

Basis: Generic to system type		Corporations that deploy models optimized for long, detailed text generation impose a higher risk of factual misinformation on their users and the public.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text exhibits systematic agency slippage, oscillating predictably between describing the models as deterministic mathematical functions and portraying them as autonomous, cognitive agents. This slippage flows predominantly in one direction: mechanical descriptions serve as brief foundations upon which complex agential narratives are constructed, effectively transferring agency TO the AI systems while removing it FROM human developers. The slippage frequently occurs when explaining model failure. For example, the text accurately identifies the foundational mechanism early on ('trained to predict the next most likely word'), locating the system at the level of computational activity. However, as the analysis shifts to specific benchmarking results, the vocabulary abruptly climbs the agency gradient. Delegated action becomes intention when Falcon 'avoided any responses', and intention becomes knowing when models are described as failing to 'scrutinise' falsehoods or having a tendency to 'forget'. This climb is driven by a pronounced 'curse of knowledge', where the authors map their own human academic capacities—scrutiny, confidence, memory, strategy—onto the digital artifact to explain its output. By default, this vocabulary represents the inherited register of the AI evaluation field rather than a deliberate strategy by the authors. Yet, this shorthand has a profound effect: it makes the structural reality of the technology unsayable. When text relies on dispositional and reason-based explanations (Brown’s types), framing models as entities with 'conservative strategies', it systematically obscures the human actors. The developers who tuned the safety classifiers, set the optimization penalties, and curated the training corpora vanish from the discourse, replaced by the illusion of a cautious, occasionally forgetful machine mind.

Metaphor-Driven Trust Inflation​

The text's heavy reliance on metaphorical and consciousness-attributing language fundamentally alters the framework through which readers are invited to establish trust with these systems. By employing vocabulary rooted in human cognitive and moral psychology, the discourse implicitly asks audiences to extend relation-based trust (reliant on perceived sincerity, intentions, and vulnerability) to artifacts that only warrant performance-based trust (reliant on statistical reliability). When the text describes a model as 'confidently' generating falsehoods or 'resisting' hallucinations, it signals to the reader that the system possesses an internal epistemic compass. Claiming a system 'knows' or 'scrutinises' accomplishes something very different from claiming it 'predicts': it projects an illusion of justified belief and active verification. This anthropomorphic framing builds a dangerous illusion of competence, suggesting that a 'conservative' model is exercising ethical restraint, rather than simply hitting a mathematical penalty threshold set by its engineers. Consequently, the text mismanages the perception of system failure. By framing limitations agentially—such as the model having 'difficulty' or a tendency to 'forget'—the text encourages users to view the AI as a well-meaning but flawed peer. This risks catastrophic over-reliance in academic contexts. If audiences believe an AI 'forgets' rather than recognizing it fundamentally lacks a persistent knowledge architecture, they may persistently try to use it for tasks it is structurally incapable of performing safely, granting epistemic authority to a statistical pattern matcher.

Obscured Mechanics​

The anthropomorphic language in this text systematically conceals the material, technical, and economic realities of AI production. By applying the 'name the corporation' test, the extent of this concealment becomes glaring: when the text claims Falcon 'avoided' bias, it hides the specific alignment decisions made by the Technology Innovation Institute (TII); when it says LLaMA 3 'resists' hallucination, it erases Meta's engineers and their optimization objectives; when Gemini is praised for 'stability', Google DeepMind's massive compute and data resources fade from view. This framework holds that commercial incentives, data annotation and human feedback labor, environmental and compute costs, and developers' limited disclosure of training data, objectives, and feedback processes are standing conditions of the technology. The text's consciousness obscuration—claiming models 'understand', 'assume', or 'forget'—hides these exact dependencies. The claim that a model 'knows' obscures the reality that its outputs are entirely dependent on uncurated, often proprietary internet data scraped without compensation. The claim that a model 'scrutinises' hides the technical reality that autoregressive transformers lack causal models, grounding, or any capability for truth verification. This framing structurally benefits the corporate developers by mystifying their product. It shields the invisible, low-wage labor of RLHF annotators who actually define the 'bias' the model supposedly 'avoids'. If the metaphors were replaced with mechanistic language, the immense concentration of corporate power, the environmental cost of training, and the rigid, mathematical fragility of the deployed systems would immediately become visible to the reader.

Context Sensitivity​

The distribution of anthropomorphic and consciousness-attributing language in this text follows a distinct, context-sensitive topography. In the introductory and methodological sections, the language is relatively grounded, acknowledging that models 'are trained to predict' (computational activity). However, a severe intensification of consciousness claims occurs in the 'Discussion' and 'Results' sections. Here, the vocabulary shifts abruptly up the agency gradient, transforming a system that 'generates text' into one that 'forgets', 'struggles', and 'scrutinises'. This asymmetry dictates how capabilities versus limitations are framed, deeply impacting the reader's picture of the system. Capabilities are frequently framed in agential, almost heroic terms (the model 'resists' hallucination, exercises a 'conservative strategy'), while limitations are framed as endearing human flaws ('has difficulty', 'tendency to forget'). This asymmetry leaves readers with a picture of a system that is a striving, competent agent occasionally hampered by relatable mental fatigue, rather than a rigid statistical engine breaking down outside its distribution. There is also a notable register shift where acknowledged analogies—implied by standard evaluation frameworks—are literalized. The text doesn't say Falcon 'acts as if' it is avoiding bias; it literalizes the action: it 'avoided' it. As a feature of the field's inherited register, this metaphorical intensity in evaluative passages positions the lay or academic reader to assess the software as a colleague rather than an artifact, subtly normalizing the integration of unreliable text generators into rigorous scholarly workflows.

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

The synthesis of accountability across this text reveals an architecture of displaced responsibility, where the linguistic construction systematically insulates human developers from the failures of their systems. There is a profound tension between what the text's overarching argument claims and what its vocabulary does. The text argues for the necessity of benchmarking to guide human practitioners in selecting tools, implicitly recognizing human choice. Yet, its vocabulary places the burden of performance, error, and 'strategy' entirely on the models themselves. The actors identified are almost exclusively the software artifacts ('ChatGPT', 'Claude', 'Mistral'), while the human actors (OpenAI, Anthropic, Meta) are rendered invisible in the sentence-level analysis of behavior. The text functions as a massive accountability sink: responsibility for bias, hallucination, and error transfers directly to the AI as an agent. The AI is framed as the delegate that 'makes assumptions' or 'exploits misconceptions', but the delegator—the corporation that designed, deployed, and profits from this system—is erased. This diffusion of responsibility has severe liability implications. If the framing is accepted, the inevitable errors of these systems are viewed as autonomous 'glitches' or 'forgetfulness' rather than the direct, structural consequence of corporate decisions to prioritize fluency over factual grounding. Naming the actors—stating that 'Google deployed a model that generates fabrications' rather than 'Gemini hallucinated'—would fundamentally shift the discourse. It would make alternative alignment choices visible, allow audiences to question corporate risk tolerances, and establish a clear path for legal and ethical accountability.

Conclusion: What This Analysis Reveals​

The Core Finding

The analysis reveals two dominant, interconnecting anthropomorphic patterns: the 'Model as Epistemic Agent' (capable of scrutinizing, assuming, and forgetting) and the 'Model as Moral/Strategic Actor' (capable of exploiting, avoiding, and resisting). These patterns do not operate in isolation; they form a mutually reinforcing linguistic system. The foundational pattern is the epistemic projection. For the model to be viewed as a moral actor capable of 'resisting' a hallucination, the audience must first accept the foundational premise that the model 'knows' what a hallucination is. This consciousness architecture—blurring processing into knowing—is the load-bearing pillar of the text's discourse. Without the assumption of internal comprehension, the strategic metaphors collapse into absurdity. The sophistication of this framing lies in its seamless integration of statistical observations with psychological verbs, masking the transition from data to mind. Notably, the text's vocabulary deeply undercuts its own scientific argument. While the authors seek to establish a rigorous, objective benchmark for software reliability, their reliance on inherited, agential language produces a narrative that evaluates the models as conscious test-takers rather than deterministic artifacts, ultimately displacing the responsibility for the models' failures away from their human creators.

Mechanism of the Illusion:​

This framework holds that computational systems process rather than know; they calculate, correlate, and generate, but they do not possess awareness or belief. The illusion of mind in this text is produced through a predictable rhetorical architecture that leverages the reader's predisposition to assign intentionality to fluent language (the ELIZA effect). The central move occurs when the text substitutes mechanistic verbs for consciousness verbs at the exact moment of system failure. Instead of explaining that attention weights decayed over a long context window, the text claims the model 'forgot'. This move activates the curse of knowledge: the authors, possessing minds, project their own cognitive frameworks onto the statistical void of the machine to make its alien behavior comprehensible. The causal chain is highly effective: first, the text establishes the model's fluency (Pattern A); next, it describes the model's outputs using intentional verbs ('avoided', 'resisted') (Pattern B); finally, the audience is positioned to accept the ultimate illusion that the model acts based on conscious understanding (Pattern C). Because the inherited register of the field normalizes this climb up the agency gradient, the illusion feels like objective scientific description rather than anthropomorphic projection, cementing the reader's belief in the system's phantom mind.

Material Stakes:​

Categories: Epistemic, Institutional, Regulatory/Legal

The metaphorical framing in this text carries severe material consequences across epistemic, institutional, and regulatory domains. Epistemically, when academic discourse claims an AI 'knows', 'scrutinises', or 'forgets', it fundamentally corrupts the human understanding of truth verification. If researchers believe a system can 'scrutinise' an embedded falsehood, they may increasingly delegate literature reviews or data analysis to statistical generators, resulting in the undetected proliferation of fabrications into the scientific record. Institutionally, this framing shifts procurement and deployment behaviors. When a university or enterprise reads that a model possesses a 'conservative strategy' for 'resisting' errors, administrators are incentivized to adopt the tool, trusting the software's imaginary internal guardrails rather than funding necessary human oversight. The cost of this misplaced trust is borne by students and researchers who rely on corrupted outputs, while the developers benefit from increased market penetration. In the Regulatory and Legal sphere, the displacement of agency away from corporations and onto the 'hallucinating' or 'biased' model protects the tech industry from liability. If policymakers accept the framing that risk is 'carried' by the model and errors are the result of its 'difficulty' with complex tasks, regulatory efforts will mistakenly focus on mandating algorithmic 'behavior' rather than auditing corporate data curation, penalizing reckless deployment, and enforcing transparency regarding the uncompensated labor and proprietary data underpinning these systems.

AI Literacy as Counter-Practice:​

Practicing critical AI literacy—as demonstrated in the Task 4 reframings—serves as a direct counter-practice to the material risks of anthropomorphism. By systematically replacing consciousness verbs ('scrutinises', 'forgets') with mechanistic ones ('processes', 'generates token sequences'), we force the recognition that these systems lack awareness, depend entirely on statistical correlations, and possess no internal compass for truth. Furthermore, restoring human agency by explicitly naming corporations (e.g., 'TII deployed an architecture', 'Meta's engineers tuned the parameters') relocates accountability from the digital artifact back to the boardrooms where deployment decisions are actually made. Systematic adoption of this precision requires structural change: academic journals must enforce guidelines against unhedged consciousness projections in AI research, and computer science curricula must teach the rhetorical impact of shorthand. However, this precision faces immense resistance. The inherited anthropomorphic register serves powerful incentives: it makes complex concepts accessible for journalists, secures funding through narratives of approaching Artificial General Intelligence, and, most crucially, protects corporate liability by mystifying the product. Precision is costly to AI developers because demystifying the technology exposes its brittle, derivative nature and places the burden of its failures squarely on their deployment choices.

Path Forward​

Looking toward the future of AI discourse, several distinct vocabulary approaches present different possibilities for how society integrates and governs these systems. Maintaining the status quo—where inherited anthropomorphic shorthand ('hallucinates', 'understands') goes unquestioned—prioritizes narrative resonance and accessibility, but at the cost of deepening the illusion of mind, diffusing corporate accountability, and enabling the integration of fragile systems into high-stakes domains based on unearned trust. Alternatively, if mechanistic precision ('generates text statistically associated with', 'attention weights degrade') becomes the norm, the technology's statistical limits become highly visible, enabling rigorous regulatory frameworks and realistic use-case boundaries. However, this approach costs intuitive grasp; the language is dense and difficult for the general public to parse. A third path involves hybrid clarity, where anthropomorphism is permitted but strictly bound by explicit, institutionalized hedging and immediate mechanistic translation (e.g., 'the model "forgets"—meaning its attention weights fail to retrieve early tokens'). Supporting these shifts requires structural interventions: funding bodies demanding rigorous epistemic language in grant proposals, industry regulators mandating standardized capability disclosures devoid of consciousness claims, and educational institutions teaching the trade-offs of these vocabularies. Ultimately, the discourse community's choice of vocabulary will determine whether the future views AI systems as flawed, autonomous colleagues to be managed, or as powerful, deterministic statistical tools for which human corporations are entirely responsible.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Analyzing the Goal

I'm currently focused on dissecting the user's objective. The need for a critical discourse analysis within a specific JSON format (v3) is clear. This requires a thorough understanding of the academic paper, and its potential for hallucination. I will assess its claims.

Reading the Study

I've immersed myself in the source text, now examining the key arguments and claims. The paper, "Fluent but Fallible," assesses LLMs' factual reliability and hallucination rates across varying prompt complexity levels. I'm focusing on the trade-off between complete responses and factual accuracy, while noting the multi-metric benchmarking framework used for assessment.

Pinpointing Agentivity Issues

I'm now identifying instances of anthropomorphism and metaphorical language within the paper. I'm noting specific phrases attributing agency and intentionality to the LLMs, such as "confidently generate false information," "understanding context," and "tends to provide factually unsupported information." These are crucial elements for critical discourse analysis and will inform the JSON structuring.

Identifying Agentivity Nuances

I'm now cataloging instances where the text imbues the models with agency, noting "difficulty maintaining," "avoided responses," and "refrain from producing," along with the associated metaphors. These instances will be pivotal when constructing the critical discourse analysis. The paper's subtle language demands attention.

Examining Agentivity Shifts

I'm now identifying shifts in how the paper describes LLMs, such as the attribution of "forgetting" and "relying on assumptions," which are deeply anthropomorphic. I'm also extracting explanations for Brown's typology, noting the "fundamental way" they're trained and the impact of skewed training data and optimization goals. I'm considering the language used to pinpoint the nature of LLMs, and their ability to provide factual responses.

Structuring the JSON

I am now structuring the JSON format with a focus on capturing the nuances of agentivity within the research. I'm focusing on "understanding context," and "confidently generate false information," within the analysis, noting each instance's type, and directness, as well as the author's position, and the paper's genre. The goal is to accurately represent the agentive language used to describe LLMs. I'm also ensuring the correct enums are used.

Mapping Agentivity Nuances

I'm now deep-diving into the nuances of agentivity, and cataloging examples with precision. I've noted that "refrain from producing" is an intentional-state attribution, and is hidden and direct. I've classified "forget things and places too much reliance" as an epistemic or conscious state, hidden and direct, while "exploit common human misconceptions" fits as an intentional-state attribution, hidden, and direct. I've also checked developer attribution, and am now filling out the source, target, and mapping for the specified eight instances. Finally, I'm examining Brown's typology using the enums provided, and have classified the first passage as "Genetic" and "Functional."

Refining Agentivity Classification

I'm now refining the classification of agentivity instances within the paper, specifically focusing on the new quotes. I've classified the initial passage as genetic, while the second passage is dispositional and intentional, followed by an intentional passage, and a dispositional one. I'm noting the implications of each classification for the critical discourse analysis. My next step involves revising the eight quotes and specifying the mechanismBasis for each instance. Finally, I'll be synthesizing my observations into the five specified sections.

Synthesizing Accountability Analysis

I'm now integrating the provided paper with its underlying conditions, like RLHF labor and training data sourcing. I'm noting the context sensitivity and how anthropomorphism appears in the results and discussion, especially related to the "struggled", "refrained", and "forgot" verbiage. I'm focusing on accountability shifts and how these frame model versus developer responsibility.

Refining Writing Approach

I'm now focusing on refining my writing to avoid newline characters within the JSON strings. I've noted that standard JSON encoding is acceptable, but I must escape control tokens. I'm focusing on the agentivity classifications in the paper for Falcon, such as, "It has a tendency to forget things and places too much reliance on assumptions..." which describes an epistemic/conscious state.

Assessing Anthropomorphism Patterns

I'm now cataloging the "scrutinising the embedded falsehood" attribution, noting it as a projection of an epistemic or conscious state. I'm also preparing to integrate the framework's declared lens and premise within the concluding sections, and ensure an adequate length is allocated for each section.

Implementing Accountability Shifts

I'm now revising my approach to focus on explicit accountability within my writing. I'm carefully detailing Technology Innovation Institute (TII)'s role when describing Falcon, ensuring clear attribution of agency to TII and its design choices. I'm leveraging CDA, structural mapping, and Brown's typology to build rich, analytical sections. I'm focusing on the accountability entries, using the provided agency levels, acknowledgment statuses, actor visibility, and mechanism basis.

Finalizing JSON Structure

I'm now creating the final JSON structure, ensuring all formatting rules are observed. I'm focusing on the accountability shifts between models and developers, such as naming Google DeepMind when Gemini "achieved the highest stability," and Anthropic when Claude "demonstrated the most balanced performance". For the 'Restore Human Agency' task, the original quote will be replaced with: "Technology Innovation Institute (TII) deployed a model whose generated text frequently omits specific details requested in the prompt and produces statistically probable but factually incorrect completions; TII's alignment choices resulted in this generative pattern." I've finalized selected categories as ["Epistemic", "Institutional", "Regulatory/Legal"] and I will ensure no \n characters are used.

Refining Output Format

I'm now completely restructuring my approach to eliminate all control tokens, especially the newline characters, within the JSON string values. I'll maintain a continuous flow of text for each value. I've re-reviewed the requirements, including the length constraints for sections 5.1 to 5.5, and I now understand that I must use massive single paragraph blocks of text for those sections to adhere to the "no control tokens" rule, which is fine, as it is still one paragraph, and will be much easier to implement. I'm focusing on ensuring all strings are single-line.

Generating New Instances

I'm now identifying additional quotes with increased agentivity to enrich the analysis. I'm focusing on language that projects human-like behaviors onto the models. I have added two new quotes: "even the best models tended to confidently generate false information for questions that required the model to exploit common human misconceptions", and "the model had difficulty maintaining the factual consistency of challenging prompts." These will now augment the analysis.

Analysis Provenance

Run ID: 2026-10-25-fluent-but-fallible-benchmarking-halluci-metaphor-d6s0yf Raw JSON: 2026-10-25-fluent-but-fallible-benchmarking-halluci-metaphor-d6s0yf.json Framework: Metaphor Analysis v6.8 Schema Version: 3.4.3 Generated: 2026-10-25T10:12:24.534Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0