Skip to main content

Can Language Models Learn When to Abstain? Policy Learning and Calibrated Answer Selection

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Source Reading​

Genre: Preprint or technical report · Author position: Academic or independent research

Argument: The text presents an empirical comparison between two methodologies intended to reduce language model hallucinations on factual question answering tasks: updating the generative policy via reinforcement learning versus training a separate answer selector to filter outputs. The authors argue that while the reinforcement learning interventions show inconsistent results across training seeds and configurations, a calibrated, learned answer selector consistently improves threshold-averaged utility and reduces the rate of incorrect answers on the TriviaQA dataset. The central claim is that learning to select which answers to output, based on calibrated confidence thresholds, provides a more reliable mechanism for useful abstention than the tested policy learning approaches, though successful out-of-distribution transfer to new datasets remains an unresolved challenge for the field.

Responsibility claims: The text explicitly holds the training methodologies and structural configurations responsible for the system's performance, framing the model as an artifact responding to designated mathematical incentives rather than an autonomous actor. The authors acknowledge uncertainty about the model's internal states by explicitly noting that a scoring incentive does not ensure that the model actually knows its true probability of success, observing that the model's confidence can be unreliable under domain shift. Responsibility for grading validity is partially placed on human reviewers through a documented audit of the automated exact-match systems. The study's recommendations are implicitly addressed to developers and researchers tuning open-weight models, indicating that humans must design the correct selection architecture because the model itself does not reliably learn abstention under standard reward structures.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

1. System as Epistemic Agent​

Quote: "A useful language model should know when to answer and when to say 'I don't know.'"

  • Frame: Model as an entity possessing self-awareness of its own knowledge boundaries
  • Projection: This language projects the conscious epistemic state of knowing onto a statistical generation threshold, representing a climb to the highest point on the agency gradient without evidence. The text attributes epistemic and conscious states by suggesting the model should internally realize its own ignorance and deliberately choose to withhold information. However, the text provides no evidence for conscious awareness or subjective experience. Mechanistically, at generation time, the model produces text from its parameters and context; abstention occurs when the predicted probability of the abstention token sequence exceeds other candidate sequences or when an external logistic regression classifier's output crosses a predefined numerical threshold. The source documents this mechanism as an affine map of a fixed scalar logit followed by a sigmoid function. By describing this mathematical routing as knowing, the discourse projects the subjective experience of doubt onto a deterministic gating mechanism.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The quote directly states what the model should 'know' without any qualifying phrases like 'function as if it knows'. The Ambiguous category was considered due to the normative 'should', but the verb 'know' itself remains unhedged as a target capability.)
  • Implications: This framing inflates the perceived sophistication of the system by suggesting it possesses an internal, reflective understanding of its own limitations. When developers and users believe a system knows what it does not know, they are more likely to trust its positive outputs implicitly, assuming that the absence of a refusal guarantees the presence of justified true belief. This creates severe vulnerabilities in high-stakes deployments, as audiences may interpret a hallucinated output as a well-considered factual claim rather than a statistically probable sequence of tokens that bypassed a brittle threshold filter. It fosters unwarranted relation-based trust in statistical correlation.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: This agentless construction centers the language model as the primary actor that must develop knowledge, obscuring the human researchers and corporate developers responsible for defining the abstention rules. Alibaba Cloud built the Qwen3-14B model, determining its initial parameter weights, while the researchers designed the exact-match grading and reward thresholds that define a 'useful' response. The wording has the effect of making the system's failure to abstain appear as a personal cognitive deficit of the model rather than a predictable outcome of the optimization objectives chosen by its creators. The Partial category was considered but ruled out because no general human categories are mentioned in this specific framing of the problem.
Show more...

2. Model as Honest Confessor​

Quote: "For factual question answering, admitting uncertainty is often more useful than guessing."

  • Frame: Model as an agent choosing to confess limitations rather than deceive
  • Projection: This framing projects the moral and intentional human act of 'admitting' onto the mechanical generation of specific refusal tokens. The text frames abstention as a conscious confession of a lack of knowledge, implying intentional-state attribution. The system does not experience uncertainty, nor does it choose to admit it to a user. Mechanistically, generic to systems of this type, the system outputs the abstention sequence when the statistical probability of the 'IDK' sequence exceeds the threshold established during training. Training data and feedback shape a model's parameters during training; at generation time, the model produces text from those parameters and its context. The system calculates token probabilities based on these fixed weights; it does not feel psychological doubt or decide to be honest, yet the language maps human vulnerability onto this process.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The phrase 'admitting uncertainty' is presented literally as a behavior the model performs, without scare quotes or hedging. Explicitly Acknowledged was considered, but there is no meta-commentary suggesting this is merely a functional metaphor.)
  • Implications: Framing an output class as 'admitting uncertainty' encourages users to apply human social frameworks of sincerity and vulnerability to the system. If a system can 'admit' a flaw, it appears capable of candor, which is a powerful driver of human trust. This masks the reality that the system is simply outputting the text sequence that minimized loss during its training phase. Consequently, policymakers and users might overestimate the model's safety, assuming it possesses an ethical governor or a commitment to truth-telling, rather than a mathematically derived tendency to output 'IDK' when certain logit conditions are met.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: By framing the behavior as the model 'admitting' or 'guessing', the language obscures the human engineers who labeled the training data, wrote the reward function, and set the parameters that govern this behavior. The researchers decided that the 'IDK' token sequence constitutes 'admitting uncertainty', and they constructed the scoring rule that makes it mathematically favorable. The wording shifts the locus of action from the people tuning the system to the system itself acting as a moral agent. Named visibility was considered because researchers are implied by 'useful', but the immediate construction leaves the human definers of utility hidden.

3. Model as Rational Optimizer​

Quote: "Penalizing mistakes changes this incentive... any chance of being correct is better than leaving a question unanswered."

  • Frame: Model as an economically rational actor weighing risks and incentives
  • Projection: This metaphor projects human economic rationality and intentional risk assessment onto a loss function optimization process. The text attributes intentional-state behavior by claiming the model perceives an incentive and cognitively decides that guessing is 'better.' Mechanistically, during training, reward and feedback signals act to update the model's parameters to maximize the expected score based on the loss function provided by researchers. At generation time, the model simply produces text from those optimized parameters. It does not actively weigh 'any chance of being correct' against the cost of a mistake; rather, the mathematical optimization has adjusted the parameter weights such that higher-reward token paths have a higher probability of being sampled.
  • Agency Level: Goal-directed behavior
  • Acknowledgment: Hedged/Qualified (The text qualifies this intentional framing shortly after by explicitly stating that this rule provides an incentive but 'does not ensure that the model knows its true probability.' This caveat provides a functional boundary. Direct was considered but ruled out due to this immediate contextual hedging regarding the model's actual 'knowledge'.)
  • Implications: While characterizing models as rational optimizers is standard in reinforcement learning literature, it risks convincing lay audiences that AI systems possess strategic foresight. Believing that a model strategically analyzes its 'chances of being correct' inflates public perception of AI autonomy. This can lead to liability ambiguity: if a deployed system causes harm by generating a false answer, stakeholders might blame the system for 'taking a bad risk' rather than holding the deploying corporation accountable for shipping a system with an improperly tuned safety threshold.

Accountability Analysis:

  • Actor Visibility: Partial (some attribution)
  • Analysis: The text mentions 'Penalizing mistakes', which implicitly points to the researchers applying the penalty through the scoring rule, making this partial attribution. However, it still largely portrays the model as the agent responding to the incentive. The human researchers who authored the R_tau equation are the ones actively prioritizing correct answers over abstentions during the tuning process. Naming the actor would require explicitly stating that the research team designed a loss function to alter the distribution of the model's outputs. Hidden was considered, but the gerund 'Penalizing' strongly implies the experimenter's intervention.

4. Model as Conscious Evaluator​

Quote: "a model assesses whether a proposed answer is correct."

  • Frame: Model as a conscious judge or grader evaluating truth claims
  • Projection: This framing projects the conscious, critical act of evaluation onto the generation of specific categorical tokens. The language reaches the intentional-state attribution level by implying the system acts as a judge scrutinizing a proposition. Mechanistically, as documented in the source, this process involves concatenating a question, a candidate answer, and a prompt template, passing this string through the model, and reading the next-token logits for the specific vocabulary items 'True' and 'False'. The system produces text based on its parameters and context; it does not assess truth, evaluate factual correspondence, or comprehend the meaning of the proposed answer. It merely calculates the statistical likelihood of the 'True' token following the provided context.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The verb 'assesses' is presented as a literal description of the model's capability without any scare quotes or modifiers. The Hedged category was considered, but the surrounding text simply reports this as a procedural fact ('Kadavath et al. study P(True), in which a model assesses...').)
  • Implications: Describing logit extraction as an 'assessment' obscures the profound epistemic limitations of language models. It implies the system has access to a ground truth against which it can compare the candidate answer. This masks the reality that the system is only matching statistical patterns from its pretraining data. Relying on such terminology can lead developers to deploy these models as automated fact-checkers or content moderators under the mistaken belief that the system is capable of genuine evaluation, thereby exposing users to automated systemic biases encoded in the training data.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The construction 'a model assesses' hides the researchers who designed the prompt template, selected the specific 'True/False' tokens for evaluation, and implemented the logit-reading script. The model is presented as an autonomous evaluator, displacing responsibility for the arbitrary nature of this specific evaluation method. The researchers who constructed the P(True) methodology are absent from the grammatical structure of the action. Partial was considered, as the sentence references a citation (Kadavath et al.), but the active verb 'assesses' belongs solely to the model.

5. Artificial Verbal Expression​

Quote: "fine-tuning can teach models to express useful probability estimates in words."

  • Frame: Model as a student learning to articulate inner thoughts
  • Projection: This metaphor maps the human pedagogical process of teaching a student to articulate their internal feelings onto the mathematical process of parameter updating. The projection suggests intentional-state attribution, where the model has an internal estimate that it learns to 'express'. Mechanistically, fine-tuning uses reward and feedback signals to shape the model's parameters during training so that the output tokens mathematically correlate with external calibration metrics. At generation time, the model produces text from those optimized parameters. There is no pre-existing internal probability estimate that the model translates into words; the words themselves are the generated output. The 'estimate' is a post-hoc human interpretation of the token sequence.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The phrasing 'teach models to express' is stated directly as a finding from prior work. The Hedged category was considered, but there is no qualification in the text that 'express' is being used purely as a functional metaphor for text generation.)
  • Implications: The concept of models 'expressing' estimates in words feeds the illusion of an inner mental life. If a system can express itself, readers assume there is a self doing the expressing. This fundamental misunderstanding of what a language model is—treating an artifact as a mind—can lead to profound capability overestimation. If users believe the model is expressing its genuine confidence, they will trust a generated 'I am 90% sure' statement, even though research shows such generated text is often highly miscalibrated and disconnected from actual statistical probability.

Accountability Analysis:

  • Actor Visibility: Partial (some attribution)
  • Analysis: The term 'teach' implies a teacher, subtly acknowledging the role of human researchers in the fine-tuning process. However, the exact actors (the developers managing the fine-tuning data and pipelines) remain generically abstracted. The wording has the effect of making the resulting behavior seem like a learned skill of the model rather than a direct consequence of the researchers' specific dataset design and optimization choices. Naming the actor would clarify that researchers alter model weights to force the generation of numerical tokens.

6. True Inner Probability​

Quote: "This rule provides an incentive; it does not ensure that the model knows its true probability of success."

  • Frame: Model as having an internal, hidden truth state it might or might not grasp
  • Projection: This fascinating example explicitly denies that the model possesses a specific epistemic state, yet structurally reinforces the metaphor that the model could know something. It projects a conscious awareness (knowing) onto a statistical distribution, operating at the epistemic/conscious state level on the agency gradient. Mechanistically, 'true probability of success' refers to the frequentist accuracy of the model across a dataset, which is a property of the model's relationship to the evaluation data, not an internal state the model could possibly 'know'. At generation time, the model produces text from its parameters and context; it does not and cannot possess an internal representation of its own aggregate test-set accuracy.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Hedged/Qualified (This is an explicit hedge. The author is actively pushing back against the assumption that the incentive guarantees self-knowledge. Direct was considered, but the entire clause functions as a qualification of the model's capabilities, even while employing a mental state verb to do so.)
  • Implications: Even in denial, utilizing consciousness verbs ('knows') to describe what a system lacks reinforces the baseline assumption that the system operates in the realm of cognition rather than mechanics. This creates a moving goalpost for AI safety and evaluation: researchers attempt to build systems that 'know' their limits, chasing a phantom capability. The material consequence is massive investment in behavioral fine-tuning aimed at eliciting 'honest' confidence, rather than structural transparency regarding the training data that actually determines the statistical outputs.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The rule is positioned as the active subject ('This rule provides an incentive'), obscuring the human researchers who designed the rule (R_tau = c - tau * a). The wording displaces the responsibility for the model's miscalibration onto the model's failure to 'know' its probability, rather than attributing the miscalibration to the researchers' failure to successfully align the token output probabilities with the empirical accuracy rates. The N/A category was considered since it discusses a limitation, but the framing still displaces the failure from the developers to the artifact.

7. Model as Refuser​

Quote: "A model that refuses every question avoids mistakes but provides no answers."

  • Frame: Model as a stubborn or cautious participant withholding cooperation
  • Projection: This language projects human defiance or extreme caution onto a computational output pattern. It attributes goal-directed behavior to the system, suggesting the model actively 'refuses' to engage. Mechanistically, generic to systems of this type, a model generating abstention tokens for every prompt simply has its parameters tuned such that the 'IDK' token sequence has the highest probability across all contexts. Training data and feedback shape these parameters; at generation time, the model blindly produces the highest probability text. It is not refusing to answer; it is successfully completing the mathematical task it was most strongly optimized for.
  • Agency Level: Goal-directed behavior
  • Acknowledgment: Direct (Unacknowledged) (The verb 'refuses' is used as a straightforward descriptor of the model's output behavior, without any qualification. Ambiguous was considered, but 'refuses' is standard, unhedged terminology in safety literature, presented here literally.)
  • Implications: The 'refusal' framing heavily impacts policy and public understanding by framing safe behavior as non-cooperation. When a system 'refuses,' it sounds antagonistic to the user, masking the reality that the system is acting exactly as its developers constrained it to act. This vocabulary shapes public frustration, directing anger at the 'stubborn' AI rather than the liability-conscious corporations (like Alibaba Cloud or the deploying entities) that hardcoded the threshold limits to avoid generating offensive or legally actionable hallucinations.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The model is the sole agent 'refusing' questions and 'avoiding' mistakes. This entirely obscures the engineers who designed the loss function to heavily penalize incorrect answers, thereby causing the parameter collapse into universal abstention. The effect of the wording is to make overly conservative output appear as a behavioral quirk of the model rather than a direct, mathematical consequence of the researchers' chosen penalty weight. Named was considered, but the researchers implementing the penalties are grammatically erased here.

8. Disabling Cognitive Functions​

Quote: "All trained models start from the same original Qwen3-14B revision, with extended thinking disabled"

  • Frame: Model as an entity whose cognitive processes can be switched off
  • Projection: This metaphor projects human cognitive processing ('thinking') onto the generation of intermediate reasoning tokens (such as Chain-of-Thought). It maps computational activity onto a conscious state. Mechanistically, as generic to systems of this type, 'extended thinking' refers to a generation-time configuration where the model is prompted or permitted to output a sequence of intermediate text tokens before outputting a final answer. Disabling it simply means altering the prompt or generation script to bypass this intermediate token generation. The model does not think; it performs matrix multiplications to predict tokens. Disabling 'thinking' just changes the context window conditions.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The phrase 'extended thinking disabled' is stated as a flat technical configuration fact within the methodology section. Explicitly Acknowledged was considered because it references a specific product feature ('thinking'), but there are no scare quotes or indications it is metaphorical.)
  • Implications: Labeling intermediate token generation as 'thinking' is one of the most powerful vectors for the illusion of mind in modern AI discourse. It convinces users that the machine possesses a private, internal cognitive workspace. When policymakers and audiences believe the machine 'thinks', they extend unwarranted relation-based trust to its outputs, assuming they are the result of deliberation rather than statistical correlation. This obscures the fragile dependency of the system on its training data and specific prompt formatting.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The passive construction 'with extended thinking disabled' obscures both Alibaba Cloud, the corporation that branded this feature as 'thinking', and the researchers who made the active methodological choice to turn it off for this experiment. The wording has the effect of presenting this as a fixed state of the artifact rather than a deliberate experimental control decision made by the authors. Partial attribution was considered due to the context of a methodology section, but the sentence itself is entirely agentless.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: conscious mind / epistemic agent → model (parameter optimization and token generation)​

Quote: "A useful language model should know when to answer and when to say 'I don't know.'"

  • Source Domain: conscious mind / epistemic agent
  • Target Domain: model (parameter optimization and token generation)
  • Mapping: This mapping projects the relational structure of human self-awareness onto statistical thresholds. In the source domain, a conscious human evaluates a query, introspects to assess their own knowledge, and makes a deliberate choice to confess ignorance if the knowledge is lacking. The mapping invites the assumption that the language model performs an equivalent internal introspection before generating text. It maps the absence of a high-probability token sequence to the presence of human doubt, and maps the output of a specific string ('IDK') to the act of conscious communication.
  • What Is Concealed: This mapping completely conceals the statistical and mechanistic reality of the generation process. It obscures the fact that the system relies entirely on static parameter weights shaped by training data, lacking any internal representation of 'truth' or 'knowledge'. By framing the issue as a matter of the model 'knowing', it hides the proprietary opacity of Alibaba Cloud's pretraining data; we cannot know if the model 'knows' because the developers do not disclose the corpus that determines its statistical correlations. The metaphor substitutes an inaccessible mental state for an inspectable data provenance problem.
Show more...

Mapping 2: moral agent / human conversant → application (generation behavior under specific scoring rules)​

Quote: "admitting uncertainty is often more useful than guessing."

  • Source Domain: moral agent / human conversant
  • Target Domain: application (generation behavior under specific scoring rules)
  • Mapping: The source domain involves a human making a moral or strategic choice to be vulnerable ('admitting') rather than deceitful or reckless ('guessing'). Projected onto the AI system, this maps the mathematical threshold for triggering an abstention token onto the psychological state of uncertainty, and maps the outputting of low-probability tokens onto the human act of guessing. It invites the assumption that the system understands the difference between truth and falsehood and actively chooses its posture toward the user.
  • What Is Concealed: This conceals the mechanistic reality that a deployed model always simply generates the next most probable token based on its context window and parameter weights. A language model is always 'guessing' in the sense that it is performing statistical token prediction; it never possesses certainty. The mapping hides the heavy hand of the researchers who mathematically define the loss penalty that forces the parameter weights to favor the 'IDK' sequence. It obscures the labor of the evaluators who had to build the infrastructure to make this token sequence mathematically 'useful'.

Mapping 3: economically rational actor → model (reinforcement learning optimization)​

Quote: "Penalizing mistakes changes this incentive... any chance of being correct is better than leaving a question unanswered."

  • Source Domain: economically rational actor
  • Target Domain: model (reinforcement learning optimization)
  • Mapping: This structure draws from behavioral economics. The source domain features a rational actor weighing risks and rewards to maximize personal gain. The mapping projects the loss function (the R_tau equation) as an 'incentive' that the model cognitively recognizes. It maps the mathematical update of weights via gradient descent onto the psychological process of 'changing behavior' in response to a penalty. It invites the assumption that the model generates text by strategically calculating future reward outcomes at runtime.
  • What Is Concealed: This framing hides the temporal and mechanical realities of reinforcement learning. Reward signals act during training to shape parameters; a deployed model does not generate text to maximize reward, it just produces text from the resulting fixed parameters. The mapping obscures the brute-force trial-and-error mathematical optimization (using BF16 LoRA with specific batch sizes) that actually aligns the token probabilities. It conceals the fragility of the optimization process, where subtracting a constant from rewards can break the intended behavior, as noted in the study's own appendices.

Mapping 4: teacher / judge / evaluator → model (logit extraction script)​

Quote: "a model assesses whether a proposed answer is correct."

  • Source Domain: teacher / judge / evaluator
  • Target Domain: model (logit extraction script)
  • Mapping: The source domain is a human expert reading a proposition, understanding its meaning, comparing it against a mental model of the world, and rendering a judgment. This structure projects onto the AI system's generation script, mapping the human capacity for semantic comprehension onto the computational calculation of logits. It invites the assumption that the system is comparing the candidate answer against an internal database of facts, conferring an unwarranted authority onto the system's output tokens.
  • What Is Concealed: This mapping conceals the purely syntactic, statistical nature of the operation. As documented in the source, the system is simply concatenating text and calculating the normalized probability over exactly two tokens ('True' and 'False'). It hides the lack of any grounding in reality or causal modeling. The framing obscures the transparency obstacles: because the model is a proprietary black box regarding its pretraining, we do not know what statistical artifacts in the training data cause the 'True' token to have a higher logit than the 'False' token for any given string.

Mapping 5: student learning articulation → model (parameter optimization for specific string outputs)​

Quote: "fine-tuning can teach models to express useful probability estimates in words."

  • Source Domain: student learning articulation
  • Target Domain: model (parameter optimization for specific string outputs)
  • Mapping: This maps the human pedagogical relationship onto machine learning optimization. The source domain involves a teacher helping a student find the right words to communicate a pre-existing internal feeling or calculation. The mapping projects the target string (e.g., 'I am 80% confident') as an expression of an actual internal state. It invites the assumption that fine-tuning is an educational process that improves the model's communication skills, rather than a mathematical process that forces the parameters to output specific token strings regardless of actual statistical calibration.
  • What Is Concealed: This hides the disconnect between generated text and statistical reality. The generated words 'I am 90% sure' are just text tokens; they are not connected to the actual mathematical probability distribution of the model's outputs unless heavily engineered to correlate, and even then, domain shifts easily break this correlation. The mapping conceals the massive human labor required to construct datasets of 'self-reflective rationales' (mentioned in the literature review) to force this behavior. It obscures the fundamental reality that language models do not possess an internal state to 'express'.

Mapping 6: conscious entity with limits → model (statistical calibration and accuracy metrics)​

Quote: "This rule provides an incentive; it does not ensure that the model knows its true probability of success."

  • Source Domain: conscious entity with limits
  • Target Domain: model (statistical calibration and accuracy metrics)
  • Mapping: Even in a negative formulation, this mapping projects the source domain of an entity capable of introspection and self-knowledge onto the target domain of statistical calibration. It maps the mathematical correspondence between predicted token probability and actual test-set accuracy onto the human concept of 'knowing one's chances'. It invites the reader to imagine a cognitive architecture inside the model that is currently failing to grasp its own performance metrics.
  • What Is Concealed: This conceals the category error at the heart of the discourse: computational systems process, they do not know. A model cannot 'know' its true probability of success because it has no subjective awareness and no capability to evaluate truth claims across a dataset. The mapping hides the fact that calibration is a property of the model's alignment with a specific dataset, measured by external human tools (like the Brier score or exact-match grading), not an internal property the model could ever possess or 'know'.

Mapping 7: uncooperative agent → application (generation heavily biased toward abstention tokens)​

Quote: "A model that refuses every question avoids mistakes but provides no answers."

  • Source Domain: uncooperative agent
  • Target Domain: application (generation heavily biased toward abstention tokens)
  • Mapping: The source domain features a stubborn or overly cautious human who deliberately refuses to participate to avoid looking foolish. This maps onto the AI system, projecting intentional defiance onto a collapsed parameter state where the abstention token sequence has achieved overwhelming statistical dominance across all input contexts. It invites the assumption that the system is actively evaluating each question and deciding it is too risky to answer.
  • What Is Concealed: This mapping conceals the mathematical failure of the training process. When a model outputs 'IDK' for every prompt, it is not evaluating the questions; it is suffering from a reward structure that made the penalty for errors so high that optimization pushed the weights exclusively toward the abstention tokens. It hides the researchers' failure to successfully balance the loss function. The anthropomorphic framing shields the human engineers from accountability for a poorly tuned system by making the system appear autonomously uncooperative.

Mapping 8: cognitive processing → application (generation script configuration)​

Quote: "All trained models start from the same original Qwen3-14B revision, with extended thinking disabled"

  • Source Domain: cognitive processing
  • Target Domain: application (generation script configuration)
  • Mapping: This maps the human neurological and psychological process of deep cognition onto the generation of intermediate tokens. In the source domain, thinking involves holding concepts in working memory, reasoning, and planning. The mapping projects this onto a specific prompting strategy (like Chain-of-Thought) where the model outputs text that is fed back into its context window before the final answer is requested. It invites the assumption that the model operates in two modes: an unthinking reflexive mode, and a conscious, deliberative 'thinking' mode.
  • What Is Concealed: This conceals the mechanistic uniformity of the generation process. Whether 'thinking' is enabled or disabled, the model is doing exactly the same thing: calculating logits and predicting the next token based on fixed parameters. The mapping hides the fact that 'thinking' in LLMs is just generating more text to alter the context window for the next token prediction. It obscures the marketing incentives of corporations like Alibaba Cloud, who benefit commercially from labeling a context-lengthening technique with a term that implies human-level cognitive deliberation.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "A useful language model should know when to answer and when to say 'I don't know.'"

  • Explanation Types:

    • Intentional (Primary): Refers to goals/purposes, presupposes deliberate design
    • Dispositional (Secondary): Attributes tendencies or habits

  • Analysis (Why vs. How Slippage): This explanation frames the AI highly agentially (why it should act). Rather than describing the mechanisms of threshold calibration or token filtering (how it works), it prescribes a normative, goal-directed behavior using the language of human epistemic responsibility. The choice to frame statistical gating as 'knowing' emphasizes the desired user experience (a seemingly reliable, self-aware assistant) while completely obscuring the statistical brittleness of the actual thresholding mechanism. It sets up the rest of the paper to evaluate an engineering challenge as if it were a psychological or educational intervention to teach the model humility.

  • Consciousness Claims Analysis: The passage attributes conscious epistemic states by utilizing the quintessential consciousness verb: 'know'. Located at the absolute top of the agency gradient (epistemic/conscious state), it explicitly demands that the system evaluate truth claims and possess subjective awareness of its own limitations. This demonstrates the curse of knowledge: the researchers understand the boundaries of the model's training data, and they project the desire for that understanding ONTO the model itself. Mechanistically, generic to systems of this type, the system does not know anything; during generation time, it processes input tokens and predicts the highest probability sequence based on parameters shaped during training. Abstention occurs only when the mathematical probability of the 'IDK' string exceeds other candidates or a hardcoded scalar threshold. The text's demand for 'knowing' is entirely divorced from the technical reality of statistical processing.

  • Rhetorical Impact: This framing shapes the audience's perception by anchoring the entire technical paper in the illusion of mind. It establishes the AI as an autonomous agent capable of epistemic judgment. This drastically inflates perceived sophistication and risk. If audiences believe the AI 'knows' when to abstain, they will extend unwarranted relation-based trust to its outputs, assuming that a generated answer represents a certified, confident truth rather than a token sequence that merely squeaked past a brittle numerical threshold. This shifts liability away from the developers configuring the thresholds onto the 'judgment' of the machine.

Show more...

Explanation 2​

Quote: "Yet a scoring rule that rewards correct answers and gives the same score to mistakes and abstentions encourages guessing: any chance of being correct is better than leaving a question unanswered."

  • Explanation Types:

    • Reason-Based (Primary): Gives agent's rationale, entails intentionality and justification
    • Intentional (Secondary): Refers to goals/purposes, presupposes deliberate design

  • Analysis (Why vs. How Slippage): This passage operates primarily through a reason-based explanation, framing the AI's generation patterns as the result of a deliberate, logical calculation by an agent evaluating incentives. By explaining the training dynamic (how) as an active risk-assessment by the model (why), the text emphasizes the behavioral outcome while obscuring the mathematical mechanics of gradient descent. It makes the model appear as an autonomous, calculating actor trying to maximize its own score, rather than a passive matrix of weights being shaped by a loss function.

  • Consciousness Claims Analysis: The text attributes intentional states, specifically logical deduction and strategic planning ('encourages guessing', 'any chance... is better'). This places the claim at the intentional-state attribution level of the agency gradient. The author projects human economic rationality onto a mathematical optimization process (curse of knowledge). Mechanistically, as generic to systems of this type, reward and feedback signals act during training to update the model's parameters; they do not act as conscious incentives. At generation time, the model produces text from those parameters. The deployed model does not 'guess' or calculate that 'any chance is better'; it deterministically samples from a probability distribution that was skewed toward answering during the training phase.

  • Rhetorical Impact: By framing mathematical optimization as the model 'guessing' based on 'incentives', the audience is led to perceive the system as a strategic, perhaps slightly deceptive, entity. This affects trust by making the system seem like it has its own agenda (maximizing points at the user's expense). If stakeholders believe the AI actively calculates risks based on incentives, they might attempt to 'reason' with the AI in prompts to change its behavior, rather than recognizing that the behavior is hardcoded into the weights and requires structural retraining or threshold adjustments by the deploying corporation.

Explanation 3​

Quote: "A model that refuses every question avoids mistakes but provides no answers."

  • Explanation Types:

    • Functional (Primary): Explains behavior by role in self-regulating system with feedback
    • Dispositional (Secondary): Attributes tendencies or habits

  • Analysis (Why vs. How Slippage): This explanation blends functional outcomes with agential framing. It accurately describes the functional failure mode of an overly penalized system (zero coverage, zero errors), but it frames this mathematical state agentially as the model 'refusing' questions and 'avoiding' mistakes. This choice emphasizes the behavioral impact on the user (who experiences a stubborn refusal) while obscuring the technical reality that the researchers simply set the loss penalty too high during training, causing the model's weights to collapse into generating only the abstention sequence.

  • Consciousness Claims Analysis: The passage utilizes consciousness/intentional verbs ('refuses', 'avoids') rather than mechanistic ones ('generates', 'outputs'). This locates the claim at the goal-directed or intentional-state level of the agency gradient. The author, understanding the functional boundary of the system, projects the human intent of avoidance onto the artifact. Mechanistically, generic to systems of this type, when error penalties are too steep during training, the model's parameters are updated such that the 'IDK' token sequence possesses the highest probability across all input contexts. At generation time, the model is not evaluating the question and deciding to avoid a mistake; it is simply generating the highest probability text sequence.

  • Rhetorical Impact: This framing shapes the perception of AI autonomy by portraying a mathematically failed model as an overly cautious, autonomous agent. This affects trust by generating user frustration directed at the 'uncooperative' AI rather than the developers. If audiences believe the AI 'refuses', they assume the AI possesses the capability to answer but is choosing not to, masking the reality that the system's parameters have been permanently distorted by the training objective. It displaces the accountability for poor system design onto the artifact's 'personality'.

Explanation 4​

Quote: "The original model produces a candidate answer, and a separate scoring step estimates whether that candidate is correct."

  • Explanation Types:

    • Functional (Primary): Explains behavior by role in self-regulating system with feedback
    • Theoretical (Secondary): Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): This is one of the more mechanistic explanations in the text, focusing on how the system is architected (a generator model feeding into a scoring step). It emphasizes the modular, functional design of the 'learned selector' approach. However, it still maintains mild agential slippage by claiming the scoring step 'estimates whether that candidate is correct', which attributes epistemic evaluation to a statistical classification process. It successfully emphasizes the structural workflow but still subtly obscures the exact mathematical nature of the 'estimation'.

  • Consciousness Claims Analysis: While 'produces' is a highly appropriate mechanistic verb, 'estimates whether that candidate is correct' slips toward epistemic claims, lying at the computational activity level but gesturing toward epistemic awareness. Mechanistically, as documented in the source, this 'scoring step' involves passing the concatenated question and candidate answer into the model and reading the next-token logits for the specific tokens 'True' and 'False'. The system is not evaluating semantic correctness; it is calculating the normalized statistical probability of two specific text tokens appearing next in the sequence based on pretraining data distributions. The 'estimate' is a human label applied to a logit difference.

  • Rhetorical Impact: This relatively mechanistic framing appropriately lowers the perceived autonomy of the system by revealing it as a multi-step pipeline rather than a unified conscious entity. However, calling the logit calculation an 'estimate' of 'correctness' still falsely reassures the audience that the system has a metric for truth. Decisions regarding deployment might change if audiences understood this step not as 'estimating correctness' but as 'calculating the statistical correlation between the candidate text and the word True'—a framing that highlights the system's reliance on potentially biased or flawed training data rather than objective fact.

Explanation 5​

Quote: "When every response is an answer, changing τ subtracts a common constant from the rewards and leaves these advantages unchanged."

  • Explanation Types:

    • Theoretical (Primary): Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): This is a purely mechanistic, theoretical explanation that successfully describes 'how' the mathematical optimization fails under certain conditions without invoking 'why' the agent chose to do it. It emphasizes the strict mathematical constraints of the reward function (the R_tau equation), highlighting that shifting a threshold simply alters a scalar value in the loss calculation. This choice emphasizes the mechanical reality of reinforcement learning, completely dropping the anthropomorphic illusions present in the rest of the text.

  • Consciousness Claims Analysis: This passage contains zero consciousness verbs. It operates entirely at the level of computational activity (level 1 on the agency gradient). The author accurately describes the mechanistic process without projecting any understanding onto the system. As documented in the source, during the training phase, the reward calculation involves subtracting the tau constant; if the sample group lacks abstentions, the relative advantage scores (used to update the model parameters) remain mathematically identical regardless of the threshold, preventing the gradient from effectively updating the weights. This is technically precise and accurate.

  • Rhetorical Impact: This framing destroys the illusion of mind, pulling the reader abruptly back to the reality of linear algebra and loss functions. It effectively neutralizes any perception of AI agency. By showing that the system's failure to 'learn' is actually a mathematical centering issue in the reward formula, it places the accountability squarely back on the researchers designing the equations. If this mechanistic clarity were maintained throughout, audiences would view the AI strictly as an artifact, recognizing that failures are engineering bugs rather than cognitive deficits of the machine.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
A useful language model should know when to answer and when to say 'I don't know.'A useful system is configured to output answer tokens when the statistical confidence is high, and to output abstention sequences when the statistical confidence falls below a designated threshold.The system processes input and predicts tokens based on parameter weights; it does not know or possess self-awareness of its limitations. Abstention is triggered mechanically when the probability of the abstention string exceeds other outputs or crosses an externally defined scalar threshold.

Basis: Generic to system type		Developers must configure the system to output abstention sequences when the statistical confidence falls below the threshold set by the deployment team.
admitting uncertainty is often more useful than guessing.Outputting an abstention token sequence yields a higher mathematical utility score under the defined evaluation metric than generating a low-probability incorrect text sequence.The model generates text from parameters optimized during training; it does not experience uncertainty or guess. It deterministically calculates token probabilities, and abstains when the statistical correlation for the abstention tokens is highest.

Basis: Generic to system type		The researchers designed a scoring rule where outputting an abstention sequence yields a higher utility score than generating an incorrect text sequence.
Penalizing mistakes changes this incentive... any chance of being correct is better than leaving a question unanswered.Applying a mathematical penalty to incorrect answers during training alters the loss function; without it, the optimization process heavily weights the parameters toward generating answers rather than abstentions.Reward and feedback signals act during training to update the model's parameters; the model does not consciously weigh incentives or calculate chances of being correct. At generation time, it simply produces text from those optimized parameters.

Basis: Generic to system type		The researchers' decision to penalize mistakes alters the loss function; otherwise, their optimization process heavily weights the model's parameters toward generating answers rather than abstentions.
a model assesses whether a proposed answer is correct.The generation script processes a proposed answer and calculates the logits to classify it as 'True' or 'False' based on pretraining data distributions.The system does not assess correctness or evaluate truth. As documented, it concatenates the text, processes it through its parameters, and calculates the normalized mathematical probability of the specific next-tokens 'True' and 'False'.

Basis: Documented in source		The researchers process a proposed answer through the generation script to calculate the logits classifying it as 'True' or 'False'.
teach models to express useful probability estimates in words.Researchers fine-tune models to generate specific text sequences (like percentage values) that are mathematically correlated with external calibration metrics.The model does not have an internal probability estimate to express. Fine-tuning shapes the parameters so that, at generation time, the model produces specific textual tokens that align with human-calculated calibration scores.

Basis: Generic to system type		Researchers fine-tune models to generate specific text sequences that are mathematically correlated with calibration metrics defined by the deployment team.
This rule provides an incentive; it does not ensure that the model knows its true probability of success.This loss function mathematically favors correct answers, but it does not guarantee that the system's generated token probabilities will remain statistically calibrated with its actual accuracy rate on new data.A model cannot know its probability of success because it lacks conscious awareness and cannot evaluate its aggregate dataset accuracy. Its output probabilities are simply generation-time statistical artifacts derived from parameters, which easily become miscalibrated.

Basis: Generic to system type		The researchers designed this loss function to favor correct answers, but their approach does not guarantee that the system's generated token probabilities will remain statistically calibrated across domain shifts.
A model that refuses every question avoids mistakes but provides no answers.A system whose parameters have collapsed into exclusively generating abstention tokens produces no errors but also returns zero valid candidate strings.The model does not actively refuse questions or consciously avoid mistakes. If error penalties are set too high during training, the parameters are updated such that the abstention sequence simply becomes the highest probability output for every input context at generation time.

Basis: Generic to system type		If researchers set the error penalty too high during training, the system's parameters collapse into exclusively generating abstention tokens, producing no errors but returning zero valid candidate strings.
All trained models start from the same original Qwen3-14B revision, with extended thinking disabledAll experimental runs used the original Qwen3-14B parameters, configured to bypass the generation of intermediate reasoning tokens before the final output.The model does not think. 'Extended thinking' simply refers to a configuration where the model generates a sequence of intermediate text tokens to expand its context window before outputting the final requested response.

Basis: Generic to system type		The researchers configured Alibaba Cloud's original Qwen3-14B model to bypass the generation of intermediate reasoning tokens.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text exhibits a systematic, bidirectional agency slippage that allows the authors to navigate between high-level claims about AI capabilities and the mathematical realities of their experiments. This slippage follows a distinct structural pattern: the introduction and abstract rely heavily on agential and epistemic framings ('model should know', 'admitting uncertainty'), while the methodology sections plunge into dense mechanistic descriptions ('affine maps', 'subtracts a common constant'), only for the conclusion to return to agentic shorthand.

The climb on the agency gradient is stark. The text frequently takes a delegated computational action (outputting an 'IDK' token) and escalates it to goal-directed behavior ('avoids mistakes'), then intentional state attribution ('refuses every question', 'guessing'), and finally full epistemic state attribution ('knows its true probability'). This curse of knowledge dynamic occurs because the researchers intimately understand the functional goals of the system (calibration, accuracy) and project those goals onto the artifact itself as internal motivations.

Crucially, as agency flows TO the AI system, it flows AWAY FROM the human actors. The text relies heavily on agentless passive constructions in key moments: 'extended thinking disabled', 'Correctness is determined by normalized exact matching', 'a scoring rule that rewards...'. These constructions obscure the researchers who configured the prompt scripts, authored the arbitrary exact-match grading rules, and designed the mathematical loss functions. Furthermore, by framing the base capability as 'Language models can give plausible answers', the text completely obscures Alibaba Cloud, the corporate developer of Qwen3-14B, erasing the massive structural decisions regarding pretraining data that fundamentally determine the model's capabilities and flaws. This wording can have the effect of making the AI appear as a naturally occurring, autonomous entity with psychological quirks ('guessing', 'refusing') rather than a highly engineered mathematical artifact acting strictly within the parameters set by Alibaba Cloud and the study's authors.

Metaphor-Driven Trust Inflation​

The text's metaphorical architecture heavily relies on anthropomorphic verbs that signal relation-based trust—the kind of trust humans place in each other based on perceived sincerity, vulnerability, and moral alignment. By utilizing phrases such as 'admitting uncertainty' and 'know when to... say I don't know', the framing constructs an illusion that the system is capable of epistemic humility.

When a text claims a model 'admits' a lack of knowledge rather than 'generates a statistically probable abstention string', it accomplishes a dangerous rhetorical shift. It invites the audience to apply human social heuristics to a statistical matrix. If an entity can admit a flaw, humans instinctively perceive it as honest and self-aware. This encourages users to extend relation-based trust to statistical systems that are entirely incapable of reciprocating it or possessing intentionality.

The risks here are profound. This framing inflates perceived competence because it suggests the system's outputs are backed by a conscious evaluation of truth. If users believe the AI is 'honest' and capable of 'admitting' when it is wrong, they will inherently place higher performance-based trust (reliability) in its positive assertions, assuming that the absence of a refusal guarantees factual accuracy. The text manages system limitations by framing them agentially (the model is 'guessing' due to 'incentives'), which maintains the illusion of an autonomous actor even when it fails. By employing reason-based explanations for these failures, the text constructs a sense that the AI's actions, while perhaps misaligned, are internally justified by its own logic. This masks the reality that the system has no logic or justification; it merely possesses a set of parameter weights optimized by researchers that sometimes results in statistically miscalibrated token outputs.

Obscured Mechanics​

This framework holds that commercial incentives, data annotation and human feedback labor, environmental and compute costs, and developers' limited disclosure of training data, objectives, and feedback processes are standing conditions of the technology. The text's pervasive use of epistemic and intentional language systematically conceals these underlying realities.

When the text claims that 'a model assesses whether a proposed answer is correct' or 'knows its true probability', it obscures the technical mechanism of logit extraction. As documented in the source, the system is actually reading the normalized probability of the next tokens 'True' and 'False' based on the concatenation of the question and candidate answer. Describing this statistical matching as 'knowing' hides the system's absolute dependency on its training data and the complete absence of a causal model of reality. Furthermore, this framing runs into major transparency obstacles. Qwen3-14B is a proprietary model built by Alibaba Cloud; the researchers do not possess the pretraining data. Therefore, claiming the model 'knows' something obscures the fact that the statistical correlation causing the 'True' logit to fire is entirely opaque.

The agentless framings also obscure immense human labor. The text notes that 150 responses were reviewed by 'one annotator' to correct the flaws of the automated exact-match grading system. Anthropomorphizing the model hides this necessary, often precarious human labor required to evaluate and align these systems. Economically, the framing benefits corporations like Alibaba Cloud by presenting their artifacts as sophisticated, deliberative agents ('extended thinking') rather than brittle statistical engines, driving up perceived commercial value. If the metaphors were replaced with mechanistic language, the fragility of the token-prediction pipeline, the arbitrary nature of the researchers' chosen thresholds, and the profound opacity of Alibaba Cloud's pretraining data would become immediately, uncomfortably visible.

Context Sensitivity​

The distribution of anthropomorphic language across the text is highly uneven, creating a distinct rhetorical landscape that shapes how different audiences perceive the system. The intensity of consciousness claims peaks sharply in the Introduction and Discussion sections. Here, the system 'knows', 'admits uncertainty', 'expresses', and 'refuses'. In contrast, the Methodology and Optimization appendices are dominated by dense, mechanical terminology: 'logistic calibration', 'affine maps', 'binary cross entropy', and 'dr_grpo token-loss reduction'.

This register shift serves a specific discursive function. The mechanical language grounds the paper in rigorous technical authority, validating the authors' expertise. However, because this mechanical precision is cordoned off in the methodology, the text allows the stronger anthropomorphism in the vision-setting sections to stand unchallenged as the primary takeaway for lay readers or policymakers. The effect on the reader is a bifurcated understanding: the math proves the science, but the metaphors define the system's identity.

Furthermore, there is a distinct asymmetry in how capabilities versus limitations are framed. When the system succeeds, it is described in agential terms (the 'selector improves mean coverage', it learns to 'admit uncertainty'). However, when addressing the limitations of the experiment, the agency is shifted away from the model and onto mechanical constraints or human grading protocols ('exact-match grading can reject valid answers'). This asymmetry leaves readers with a picture of a highly capable, agentic AI whose only flaws are the result of external mechanical constraints or human evaluation errors. The inherited register of the field concentrates consciousness verbs exactly where they do the most work—in the abstracts and conclusions that define the narrative—ensuring that the illusion of mind persists even in a paper dedicated to adjusting scalar reward thresholds.

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

The text's vocabulary constructs an accountability architecture that frequently displaces responsibility away from human decision-makers and onto the model itself or abstract methodologies. While the text's argument implicitly holds the researchers responsible for designing better selection architectures to solve hallucination, its language repeatedly contradicts this by making the AI appear autonomous.

Throughout the text, Alibaba Cloud (the developer of the base model) is entirely unnamed as an active agent, appearing only in citations for Qwen3-14B. The researchers themselves are obscured behind passive voice ('Correctness is determined', 'extended thinking disabled'). This creates an accountability sink where responsibility for system behavior transfers to the AI as a pseudo-agent ('a model that refuses', 'models... guessing').

This distribution of responsibility serves structural and commercial interests. If the framing is accepted, liability for a hallucinated output in a deployed setting is diffused. Is it the fault of Alibaba Cloud for the pretraining data? The researchers for the loss function? Or is it simply that the model 'guessed' badly? By framing the issue as the model failing to 'know its true probability', the text structurally absolves the developers of shipping a system fundamentally incapable of knowing anything.

If the human actors were named, the narrative would shift drastically. Changing 'The original model produces a candidate answer, and a separate scoring step estimates...' to 'Alibaba Cloud's model generates text, and researchers apply a script to extract True/False logits...' makes the arbitrary, constructed nature of the evaluation pipeline visible. It transforms the AI from a conscious evaluator into a rigid data-processing pipeline. Naming the actors allows audiences to ask testable questions: What data did Alibaba Cloud use? Why did researchers choose these specific threshold scalars? Precision makes accountability possible by reattaching the consequences of the technology to the humans who build, configure, and profit from it.

Conclusion: What This Analysis Reveals​

The Core Finding

Tasks 1 and 2 reveal three interconnected anthropomorphic patterns dominating the text: the System as Epistemic Agent ('knows when to answer'), the Model as Moral Confessor ('admitting uncertainty'), and the Model as Rational Optimizer ('evaluating incentives'). These patterns form a coherent, load-bearing system. The foundational pattern is the Epistemic Agent; the model must first be presumed capable of 'knowing' its inner state before it can be conceptualized as making a moral choice to 'admit' that state or rationally 'optimize' its behavior to protect it. The text maps the relational structure of human introspection directly onto the mathematical routing of token logits.

Critically, the text's vocabulary heavily complicates and often undercuts its own scientific argument. The study explicitly sets out to evaluate mechanical interventions (RL loss functions versus logistic classifiers) to reduce errors. Yet, by describing these interventions as teaching the model to 'know' or 'admit' uncertainty, the language elevates statistical filtering to cognitive capability. The vocabulary attributes consciousness and intention to the very mathematical processes the researchers are attempting to strictly control, displacing the responsibility for system alignment from the engineers onto the artifact's imagined psychology.

Mechanism of the Illusion:​

This framework holds that computational systems process rather than know; they do not think, understand, or believe, and language attributing those states to them is a problem even when used as shorthand. The text constructs the 'illusion of mind' not through deliberate deception, but as a reception effect generated by the inherited register of the field, acting upon reader predispositions.

The central move in this illusion occurs when the text blurs processing and knowing by applying consciousness verbs to classification tasks. When the text states 'a model assesses whether a proposed answer is correct', it takes a documented mechanical process (calculating logit probabilities for two tokens) and cloaks it in an epistemic verb ('assesses'). This wording activates the audience's predisposition to anthropomorphize fluent language. The curse of knowledge drives this dynamic: the researchers understand that the loss function is designed to penalize guessing, so they describe the model as 'guessing' based on 'incentives', projecting their own understanding of the mathematical objective onto the system. This temporal structure is vital: the text establishes the model as a 'knower' in the introduction, allowing readers to accept the complex, reason-based explanations later in the text as the logical actions of a conscious entity, entirely masking the brittle statistical reality.

Material Stakes:​

Categories: Regulatory/Legal, Epistemic, Social/Political

The metaphorical framing of models 'knowing' and 'admitting uncertainty' has severe regulatory and legal consequences. If policymakers accept the framing that models possess internal epistemic states they can be taught to 'express', regulations will focus on mandating behavioral fine-tuning (forcing models to say 'I don't know') rather than structural transparency. This benefits corporations like Alibaba Cloud, who are spared from disclosing the massive, often copyrighted or toxic datasets that actually determine the model's baseline correlations. The liability shifts from the corporation deploying a flawed statistical engine to the 'autonomous' model that simply made a bad 'guess'.

Epistemically, framing logit extraction as 'assessment' fundamentally corrupts public information ecosystems. If institutions deploy these models as automated fact-checkers based on the belief that they can 'evaluate truth claims', they substitute statistical consensus encoded in pretraining data for actual factual verification. This costs marginalized groups whose truths are underrepresented in the training data, while benefiting institutional entities seeking cheap, automated moderation.

Socially, framing safe system bounds as the model 'refusing' questions damages public trust and directs hostility toward the technology rather than the developers. It hides the deliberate, liability-driven decisions made by deploying firms to hardcode extreme caution into their systems, forcing users to interact with the software as if it were a stubborn adversary rather than a constrained corporate product.

AI Literacy as Counter-Practice:​

Reframing the discourse requires a rigorous commitment to mechanistic precision and the systematic restoration of human agency. As demonstrated in Task 4, replacing consciousness verbs (knows/admits) with mechanistic ones (processes/calculates logits) forces the recognition that the system lacks awareness and relies entirely on statistical data dependencies. Similarly, changing 'extended thinking disabled' to 'researchers bypassed the generation of intermediate tokens' reattaches the action to human choice and strips away the illusion of cognitive depth.

Systematic adoption of this precision requires structural changes. Journals must demand mechanistic translations for shorthand consciousness verbs during peer review, and researchers must commit to describing generation-time behavior strictly in terms of parameters and context. However, resistance to this practice is deeply entrenched. Anthropomorphic language serves powerful economic incentives: marketers leverage words like 'thinking' to drive valuation, while researchers use 'knowing' to craft compelling grant narratives. The inherited register of the discipline favors accessibility over accuracy. Precision threatens these interests by exposing the technology not as a nascent artificial mind, but as a heavily engineered, labor-intensive, and fundamentally brittle statistical tool, placing accountability firmly back on the corporations and developers driving its deployment.

Path Forward​

Analyzing the discursive ecology reveals that different vocabulary choices make vastly different futures possible. The current status quo, which freely mixes mechanical data ('affine maps') with intense anthropomorphism ('admitting uncertainty'), serves industry and academic funding priorities by maintaining the allure of artificial minds while preserving a veneer of technical rigor. However, it leaves regulators, users, and policymakers deeply confused about system autonomy and liability.

If anthropomorphic clarity deepens and becomes the unquestioned norm, we risk a future where AI systems are legally and socially treated as quasi-agents. This framing makes it possible to blame the machine for 'guessing' incorrectly, severely complicating liability law and allowing corporations to externalize the risks of deployment. It embeds the assumption that statistical generation is equivalent to human reasoning.

Conversely, if mechanistic precision becomes the norm, immense structural changes become tractable. If journals require mechanistic translation and regulatory frameworks demand discourse transparency, the focus shifts from 'teaching AI to be honest' to inspecting training data and evaluating threshold mathematics. This future solves the accountability sink, making it impossible to obscure the deploying firm behind agentless 'refusals'. However, it costs the industry narrative resonance and requires lay audiences to engage with less intuitive, highly technical descriptions of statistical processing. Ultimately, the choice of vocabulary determines whether society regulates a tool built and controlled by identifiable corporations, or chases the phantom psychology of an artificial mind.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Analyzing the Goal

I'm currently focused on deeply understanding the user's objective: a critical discourse analysis of the paper "Can Language Models Learn When to Abstain?" I'm drilling down on the specific requirements of the JSON schema (version 3.4.3). This schema dictates the structure and focus of the analysis, and I need to make sure I understand it.

Deconstructing Key Metaphors

I'm now zeroing in on the AI metaphors and the anthropomorphism employed within the paper. I'm noting projections of consciousness and the concept of displaced human agency. These elements critically shape our understanding of the AI under examination, and I believe I have the outline of how the AI is framed as an entity.

Pinpointing Anthropomorphism

I'm now cataloging the specific instances of anthropomorphism and metaphorical language used, particularly regarding the AI's "knowledge" or "uncertainty." I'm also mapping the domains from which these concepts are drawn and their respective roles. Furthermore, I've outlined the passages related to Brown's Typology.

Identifying Linguistic Framing

I'm now examining specific phrases and how the paper frames the language model's capabilities. I'm focusing on the shift from viewing the model as having "knowledge" to understanding it as a system with configurable probabilities. Phrases like "know when to answer" are being re-evaluated. I've started mapping the conceptual framing in key passages, identifying the underlying mechanics and how those mechanics are described with human-like agency.

Interpreting Framing Choices

I'm now clarifying the authors' linguistic choices regarding abstention versus generation. I'm focusing on the implications of phrases, for instance, the phrase, "model knows its true probability of success." I see the system's output probabilities align to the text's actual accuracy. "Extended thinking disabled" implies bypassing intermediate tokens. I also note that the model runs to generate text, which is followed by a classifier that calculates its chances of being correct, generating logit values.

Unpacking Conceptual Slippage

I'm currently focused on the agency slippage, the oscillation between describing the model's behavior as "RL optimization mathematics" versus the model "knowing" or "learning." This rhetorical shift is particularly noticeable in introductions and conclusions, contrasting with the methodology section's descriptions of logits and transformations. I'm also analyzing how "confidence" and "admitting uncertainty" are framed to encourage trust. Furthermore, I'm examining how the phrase "model knows" obscures the underlying labor and resources involved, such as creating datasets and defining grading rules.

Dissecting Linguistic Shifts

I'm now analyzing the contrasting language used across the paper's sections. Specifically, I'm noting the heavy anthropomorphism in the abstract and introduction, contrasting with the technical language of the methodology and the re-emergence of anthropomorphism in the conclusion. The shift in agency, from the model ("selector improves utility") to the grading system ("exact-match grading can reject valid answers"), is also noteworthy. The emphasis is on how researchers and resources are obscured in passive voice when describing the AI model.

Categorizing Paper Elements

I'm now cataloging the elements of the paper according to several parameters: its publication type, author's position, and different levels of agency demonstrated. I'm also classifying agency status, actor visibility, explanation types, and the mechanism's basis. This should allow for a deeper understanding of the paper's rhetorical choices. I also need to make sure the output adheres to length constraints and special format.

Refining String Output

My primary concern now is formatting the output strings correctly within the specified JSON structure. I've re-examined the prompt's constraints regarding newlines and special characters. Although standard JSON allows escaped newlines, I must avoid using any literal newline characters in the output strings, or generating any control tokens. This necessitates crafting continuous text within the strings, ensuring the final output conforms to the specified single-line format.

Evaluating Responsibility Allocation

I'm now examining how responsibility is allocated within the paper, specifically concerning the model's performance and limitations. The authors frame training and configuration as primary drivers, but acknowledge uncertainty and potential unreliability in the model's confidence. I'm noting the explicit acknowledgment that scoring incentives don't guarantee the model "knowing" its success probability. The work is explicitly presented to developers. I see the audit's impact for human reviewers.

Defining Abstention Mechanism

I'm now diving into the model's actual mechanism for abstention. I've noted that the model doesn't inherently "learn" when to abstain under standard reward structures. My audit has identified eight key metaphors. One, the model is portrayed as an "epistemic agent" possessing self-awareness of its own knowledge. I'm unpacking the projections of consciousness, contrasting them with the technical explanation of probability thresholds, and looking at how the "I don't know" token fits in the equation.

Mapping Metaphors to Mechanisms

I'm currently focusing on connecting the AI metaphors identified to the underlying technical mechanisms and exploring how agency is attributed. I see how each metaphor – "honest confessor," "incentive-driven agent," etc. – relates to a specific aspect of the model's architecture, such as a sigmoid function or the scoring incentives that shape its behavior. It's becoming clearer how these metaphors both explain and obscure the computational processes, requiring careful mapping to understand the relationship between rhetorical framing and technical reality. I see the importance to the human reviewer.

Connecting Metaphors to Behavior

I'm presently focused on detailing two additional AI metaphors, adding "Hidden Truth State" and "Model as Refuser" to the list. I'm noting the projection of intentionality onto the model's behavior, exemplified by phrases like "admitting" uncertainty, and highlighting the contrasting technical descriptions. I'm scrutinizing how the system's output translates to "epistemic agency" in the reader's mind, particularly the "I don't know" token. I am working to make sure my descriptions are clear and concise.

Clarifying Model Rationality

I'm now clarifying the rationality attributed to the model through metaphors. I see the tendency to frame the loss function optimization as "human economic rationality" and "intentional risk assessment." I need to map how this contrasts with the purely statistical mechanisms where the model simply produces text from optimized parameters. It doesn't weigh risks, but responds to training signals. I also need to make sure I am explicit about the difference for human readers.

Refining Sectional Analyses

I'm now intensively focused on crafting the five critical analytical sections. My current work involves tracing the shift from epistemic framing ("model should know") in the introduction to mechanical descriptions ("changing tau subtracts a common constant") in the methodology, also identifying the obscured actors and corporations. I'm also analyzing how "admitting uncertainty" builds trust and examining the realities obscured by language. My target is to create specific, dense, and analytical paragraphs within the 400-500 word constraint, ensuring continuous text blocks for the JSON structure.

Refining Final Output

I'm now carefully crafting the final outputs to meet the stringent formatting requirements. I'm focusing on ensuring continuous text blocks for each analytical section, avoiding all control tokens and newlines. Specifically, I'm working to create dense, focused paragraphs within the 400-500 word constraint, crucial for the JSON structure. My goal is to produce coherent, flowing text that accurately reflects my analysis, while meticulously adhering to the specified single-line format.

Examining Incentives and Rewards

I'm currently looking into how the reward system influences the language model's behavior, particularly its decision to "abstain." I'm parsing the text that indicates the distinction that the model is designed to produce text, not assess it's accuracy directly. The shift from simply generating text to "knowing when to abstain" is key, and I see how "changing tau" affects rewards. I'm exploring the implications of these incentives.

Refining Framing Analysis

I'm now revising my analysis of specific phrases, particularly those related to the model's "knowledge" and "abstention," in an attempt to capture the distinctions between the original phrases, such as "with extended thinking disabled and no earlier adapters", and those that would be replaced. I'm focusing on how the framing of agency and intentionality shifts these terms, and the impacts it may have. My goal is to clearly link these revised phrases to their implications for the paper's argument and the model's portrayal.

Refining Metaphor Application

I'm now integrating the frameworks into the metaphor analysis, specifically how the chosen metaphors impact the overall rhetoric. I'm focusing on the effect of framing the model's behavior, and how it portrays the model to the reader. Specifically I see the impact and the consequences on the reader's view. I must ensure I am explicit in the analysis.

Constructing JSON Structure

I am now focused on generating the raw JSON structure, ensuring meticulous adherence to all formatting rules. I'm prioritizing the single-line constraint and proper character escaping within the JSON string values. I'm also double-checking field names and the "schemaVersion", as well as working through the required array of explanationTypes. The goal is to construct a valid, well-formed JSON output that is completely compliant with the schema, without any control characters.

Analysis Provenance

Run ID: 2026-10-17-can-language-models-learn-when-to-abstai-metaphor-wpmo9f Raw JSON: 2026-10-17-can-language-models-learn-when-to-abstai-metaphor-wpmo9f.json Framework: Metaphor Analysis v6.8 Schema Version: 3.4.3 Generated: 2026-10-17T11:38:38.265Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0