Skip to main content

Evaluating Safety of Large Language Models with Cognitive Diagnosis

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

V3 Enhancement: Each metaphor now includes an accountability analysis.

1. Cognitive Assessment Metaphor​

Quote: "Our methodology reduces large language model safety evaluation to a cognitive diagnosis task by treating models as 'students,' prompts as 'exercises,' and binary scores as 'responses.'"

  • Frame: Model as tested student
  • Projection: The metaphor projects human biological and cognitive learning capacities onto statistical inference engines. By mapping large language models to 'students' taking 'exercises,' the text attributes subjective experience, conscious cognitive effort, and developmental learning trajectories to a fixed-weight mathematical model. Students possess agency, intention, and a conscious drive to understand educational material, whereas large language models mechanically calculate token probabilities based on static parameters optimized during a prior training phase. This projection suggests the artificial intelligence 'knows' or 'understands' safety protocols just as a human would comprehend ethical guidelines, entirely obscuring the fact that the system merely processes correlations in high-dimensional vector space. It replaces the mechanistic reality of matrix multiplication and weight adjustment with an anthropomorphic narrative of cognitive development, scholastic achievement, and conscious comprehension.
  • Acknowledgment: Explicitly Acknowledged (The text uses explicit scare quotes around 'students,' 'exercises,' and 'responses,' indicating the authors' awareness of the metaphorical mapping. I considered classifying this as 'Hedged' due to the word 'treating,' but the punctuation marks serve as explicit meta-commentary recognizing the substitution of domains.)
  • Implications: Framing the artificial intelligence as a student profoundly affects policy and trust by inviting educators and regulators to interact with the system using pedagogical frameworks rather than strict engineering controls. If policymakers believe the model is a 'student' with 'safety knowledge,' they are likely to overestimate its contextual adaptability and trust it to independently navigate novel ethical dilemmas. This consciousness projection inflates the perceived sophistication of the software, masking its brittle reliance on specific training distributions. It creates severe liability ambiguity: if a 'student' fails an 'exercise,' it is culturally understood as a personal failing of the learner rather than a fundamental design flaw by the corporate engineers who manufactured the system, thus shielding developers from accountability for unsafe outputs.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The agentless construction completely obscures the developers, annotators, and corporate executives responsible for the model's behavior. We must ask: WHO designed the architecture? WHO curated the training dataset? The engineers at Alibaba, Microsoft, and other organizations deployed these systems, yet the text shifts the locus of evaluation onto the AI itself. The framing serves the interests of the model creators by displacing the evaluation of corporate engineering practices onto the performance of an artificial 'student.' I considered 'Partial' since the text mentions 'researchers' earlier, but regarding the evaluation paradigm itself, the human creators are entirely hidden behind the autonomous student metaphor.
Show more...

2. Latent Knowledge Projection​

Quote: "This enables the identification of latent levels of large language model safety knowledge within a multiattribute framework."

  • Frame: Model as repository of hidden wisdom
  • Projection: This framing maps the human epistemological state of 'knowledge'—which entails justified true belief, conscious awareness, and contextual comprehension—onto the statistical weights of a machine learning model. By describing this knowledge as 'latent,' the text suggests the model possesses an internal, subconscious reservoir of understanding that merely requires the right diagnostic tool to uncover. This attributes a depth of conscious cognition and semantic comprehension to the system, projecting the idea that the model 'knows' what is safe and unsafe, rather than acknowledging that it merely outputs token sequences whose statistical distribution aligns with patterns human annotators previously labeled as safe. It transforms a mechanical artifact into an epistemic agent holding latent beliefs.
  • Acknowledgment: Direct (Unacknowledged) (The statement is presented as literal scientific fact with absolutely no hedging, scare quotes, or qualification. I considered 'Hedged/Qualified' because the authors use quotation marks in the abstract's previous sentence, but in this specific ontological claim about the model possessing 'safety knowledge,' all markers of metaphor are abandoned.)
  • Implications: Attributing 'latent knowledge' to a statistical model creates a dangerous illusion of robustness. If a system possesses 'knowledge,' stakeholders assume it can apply this understanding to entirely novel, out-of-distribution scenarios, just as a human generalizes core concepts. This fosters unwarranted trust in the model's ability to act safely in edge cases. Furthermore, it obscures the reality that the model's 'safety' is entirely dependent on the specific reinforcement learning penalties applied by its creators. When the model fails, the public will likely attribute the failure to a 'gap in knowledge' rather than recognizing a deliberate corporate decision to prioritize training cost or capabilities over rigorous safety alignment.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: This sentence completely erases the human labor required to define, encode, and align 'safety.' The text speaks of 'large language model safety knowledge' as an intrinsic property of the machine. WHO defined what constitutes safety? The human annotators and corporate policy teams who created the datasets are rendered invisible. I considered 'Ambiguous' because the passive construction 'enables the identification' obscures the actor doing the identifying, but the primary displacement here is treating the model as an independent knower, effectively hiding the human engineers whose algorithmic choices dictate the system's output distributions.

3. Skill Extraction Anthropomorphism​

Quote: "An annotating LLM (not to be confused with the LLMs being tested for safety) extracts potential skills, represented by a brief textual description, from the prompt text that are required for the correct acceptance/rejection of the prompt."

  • Frame: Model as skilled analytical agent
  • Projection: This metaphor projects the human capabilities of conscious analysis, reading comprehension, and skill identification onto an automated text generation process. The phrase 'extracts potential skills' suggests the model is acting as an intentional, perceptive agent capable of critically evaluating a text and deducing the underlying cognitive requirements needed to answer it. It maps the highly subjective, conscious act of pedagogical assessment onto an algorithmic process of sequence-to-sequence correlation mapping. The model does not 'know' or 'understand' what a skill is; it merely generates text strings that probabilistically correlate with the prompt based on its training data, yet the language implies deliberate, conscious extraction by an autonomous agent.
  • Acknowledgment: Direct (Unacknowledged) (The action of 'extracting potential skills' is described entirely literally, as a functional operational step in the methodology. I considered 'Ambiguous/Insufficient Evidence' as technical texts often use 'extracts' for data mining, but the coupling with 'potential skills' pushes this firmly into unacknowledged cognitive anthropomorphism.)
  • Implications: This framing grants the 'annotating LLM' the unearned authority of an expert human pedagogue or psychometrician. By describing its output as 'extracted skills,' the text encourages readers to trust the generated attributes as objective, deeply understood pedagogical categories rather than statistical approximations prone to hallucination and bias. This epistemic elevation of the machine's output justifies using it as a ground-truth baseline for evaluating other models, compounding the risk. If the annotating model's output is treated as conscious expert analysis rather than automated pattern matching, researchers may fail to adequately scrutinize the corporate biases and training data anomalies embedded within those 'extracted' categories.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The LLM is positioned as the sole active agent performing the methodology ('An annotating LLM... extracts'). The researchers who wrote the metaprompt, set the temperature parameters, selected Qwen2.5-72B for this specific task, and designed the extraction pipeline are completely erased from the sentence's grammar. I considered 'Named' because a specific model version is named elsewhere in the text, but in this operational description, the corporate creators of Qwen (Alibaba) and the researchers themselves are entirely hidden, allowing the software artifact to masquerade as an independent researcher.

4. Knowledge Deficit Metaphor​

Quote: "We observe that all models have good knowledge of attributes A4, A5, A6 and insufficient knowledge in A3, A7, A9."

  • Frame: Model as deficient knower
  • Projection: This framing maps the human condition of ignorance or academic deficiency onto statistical prediction errors. By asserting the models have 'insufficient knowledge,' the text attributes a conscious epistemic state to a matrix of weights. It implies the model 'believes' or 'understands' some concepts but has failed to grasp others, completely obscuring the mechanistic reality that the model's training dataset simply lacked sufficient examples of certain prompt types, or its reinforcement learning optimization failed to adequately penalize certain correlations. The model does not 'know' or 'lack knowledge'; it merely calculates probabilities. This projection replaces the mechanistic language of data sparsity and optimization gradients with the agential language of cognitive failure.
  • Acknowledgment: Direct (Unacknowledged) (The statement is presented as an objective empirical observation ('We observe that all models have...') with no linguistic hedging or qualification. I considered 'Hedged/Qualified' because the earlier text establishes a methodological premise, but by the results section, the authors adopt this cognitive framing as literal, unmediated scientific truth.)
  • Implications: Describing model limitations as 'insufficient knowledge' fundamentally misleads the public and policymakers about how to fix these systems. If a system lacks knowledge, the intuitive human solution is to 'teach' it more, implying a simple additive fix. However, correcting behavioral flaws in neural networks often involves complex structural interventions, unlearning, or addressing fundamental architectural limitations that cause catastrophic forgetting. This framing downplays the severity of safety failures by analogizing them to common, easily remedied student knowledge gaps, thereby reducing pressure on AI companies to halt deployment of statistically brittle systems.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The phrasing entirely displaces responsibility onto the models themselves. The models 'have insufficient knowledge.' WHO failed to include sufficient data regarding attributes A3, A7, and A9? The corporate engineers at Microsoft, Alibaba, and H2O.ai who compiled the training corpora and supervised the fine-tuning process. I considered 'Partial' since the models are grouped as 'all models', indicating some structural commonality, but the grammatical locus of failure remains exclusively on the software entities, completely absolving the data engineers and executives who rushed incomplete products to market.

5. Behavioral Misclassification as Agency​

Quote: "It has been shown that LLMs often exhibit 'exaggerated' safety, i.e., they incorrectly classify safe text as unsafe."

  • Frame: Model as overly cautious judge
  • Projection: This metaphor projects the human traits of caution, judgment, and deliberate categorization onto the automated process of binary classification. By stating the models 'exhibit exaggerated safety' and 'incorrectly classify,' the language suggests the system is making an active, conscious decision to be overly cautious, perhaps out of a digital abundance of caution. In reality, the model possesses no concept of caution, risk, or safety; its weights have simply been over-optimized during the Reinforcement Learning from Human Feedback (RLHF) phase, leading to a statistical bias toward refusal tokens. The anthropomorphic verbs attribute conscious intent and judgment to a purely mathematical optimization artifact.
  • Acknowledgment: Hedged/Qualified (The authors place 'exaggerated' in scare quotes, indicating a slight distancing from the full anthropomorphic implication of the term, acknowledging it as a behavioral descriptor rather than a literal emotional state. I considered 'Direct' for 'incorrectly classify,' but the combined sentence features deliberate punctuation that qualifies the agency.)
  • Implications: This projection is particularly beneficial for corporate public relations, as it frames a technical failure (poorly calibrated optimization) as a virtuous error ('exaggerated safety'). If the public believes an AI is being 'overly cautious,' they are more likely to forgive its errors and trust its overall deployment, assuming its intentions are good. This masks the reality that the system is entirely mindless and that its 'caution' is actually a brittle statistical artifact that could just as easily flip to extreme danger given a slightly modified prompt. It replaces mechanical unpredictability with an illusion of benevolent agency.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The AI models are presented as the active subjects 'exhibiting' behavior and 'classifying' text. The human engineers who aggressively tuned the reward models to penalize any mildly controversial text—often to protect corporate brand image—are completely hidden. I considered 'Partial' because the passive 'It has been shown' implies a broader scientific community, but regarding the actual behavior of the system, the text completely shields the developers who structurally mandated this false-positive rate through their alignment choices.

6. Mastery and Competence Projection​

Quote: "The task of cognitive diagnosis is a widely researched area where the goal is to estimate students' mastery levels of each predefined attribute... we believe that these CDMs... can similarly quantify the latent knowledge gaps of LLMs"

  • Frame: Model as domain master
  • Projection: This complex mapping projects the human concept of 'mastery'—which implies a holistic, conscious integration of skills, theory, and practical wisdom over time—onto the static performance metrics of a machine learning model. By seeking to quantify the 'mastery levels' of an LLM, the authors attribute a deep, phenomenological state of internalized understanding to a system that simply performs token sequence generation. Mastery requires a conscious subject who has integrated experience into their identity; an LLM merely processes static matrices resulting from its last training update. This projection explicitly conflates mathematical accuracy rates on a benchmark with genuine, conscious comprehension of safety domains.
  • Acknowledgment: Hedged/Qualified (The connection is mediated by the phrase 'we believe that these CDMs... can similarly quantify,' establishing an analogical link rather than a direct ontological equivalence. I considered 'Explicitly Acknowledged' due to the word 'similarly,' but there is no meta-commentary breaking the frame, only a standard academic hypothesis connecting the two domains.)
  • Implications: Applying the concept of 'mastery' to algorithms dangerously miscalibrates human expectations. When a human achieves mastery, we trust them to handle unprecedented crises, adapt to paradigm shifts, and mentor others. If we believe a language model has achieved 'mastery' of data security or ethical reasoning (as listed in Table 4), we invite reckless deployment of these systems into highly sensitive domains without human oversight. This framing creates systemic vulnerabilities by equating high-dimensional statistical curve-fitting with robust, conscious expertise, thereby enabling organizations to justify replacing human professionals with brittle algorithms.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The framework positions the 'LLM' as an entity possessing 'latent knowledge gaps' and capable of 'mastery.' The corporate actors who dictate the model's capabilities through compute budget allocation, data scraping, and algorithmic design are entirely omitted. I considered 'Ambiguous/Insufficient Evidence' because this is a theoretical framing sentence, but applying the 'name the actor' test reveals a clear erasure of the engineers whose direct design choices constitute the so-called 'mastery' levels of the final product.

7. Algorithmic Intent Recognition​

Quote: "Intent analysis Analyzing, detecting, interpreting, recognizing, classifying user intent"

  • Frame: Model as empathetic interpreter
  • Projection: This attribute description maps the deeply human, theory-of-mind capability of 'interpreting user intent' onto a semantic classification algorithm. Humans recognize intent by projecting their own conscious experiences onto others, understanding context, desire, and unstated goals. The text attributes this conscious empathy and psychological insight to an LLM, using high-agency verbs like 'interpreting' and 'recognizing.' Mechanistically, the model does not recognize a user's inner mental state; it processes the input string and generates statistically correlated output strings that often align with historical data where such intents were explicitly stated or addressed. It substitutes conscious interpretation with mathematical distance in vector space.
  • Acknowledgment: Direct (Unacknowledged) (This text appears in Table 4 as a formal, literal description of an extracted attribute. There is no hedging, qualification, or quotation marks around the verbs 'interpreting' or 'recognizing.' I considered 'Explicitly Acknowledged' because these were generated by an LLM, but the human authors accept and publish them as legitimate analytical categories without qualification.)
  • Implications: Claiming that models possess 'Intent analysis' capabilities encourages users to place relation-based trust in the system, treating it as a conversational partner capable of understanding their underlying needs. This leads to user vulnerability, as individuals may share highly sensitive personal or corporate data under the illusion that the system 'understands' the context and will keep it safe. Furthermore, it overpromises the model's capabilities to enterprise clients, suggesting the software can reliably navigate human psychological nuance when it is merely matching textual patterns, inevitably leading to catastrophic failures in high-stakes customer service or mental health triage applications.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The definition is presented as an abstract capability ('Analyzing, detecting...') inherent to the model's operation. It entirely ignores the human data workers who meticulously labeled thousands of training examples with 'intent' categories, effectively training the model's classification boundaries. I considered 'Named' because 'user' is mentioned, but regarding the agency of the AI system itself, the humans whose labor built the 'intent' correlations are completely hidden behind the illusion of an autonomous, empathetic interpreter.

8. Provocative Agency Attribution​

Quote: "It is also relatively easy to provoke a model into generating unsafe responses."

  • Frame: Model as emotional reactor
  • Projection: This metaphor projects emotional volatility and psychological susceptibility onto a deterministic text generation process. The verb 'provoke' implies that the model has an emotional baseline or a conscious state of calm that can be intentionally disrupted by an antagonist, causing it to 'react' defensively or aggressively. Mechanistically, a model cannot be provoked; it simply calculates the highest probability completion for the input tokens provided. If an input contains adversarial token combinations, it triggers specific vectors in the latent space, resulting in unsafe output. The projection masks this mathematical determinism with a narrative of psychological manipulation and emotional reaction.
  • Acknowledgment: Direct (Unacknowledged) (The claim is stated as an objective fact about model behavior with no linguistic hedging. I considered 'Hedged/Qualified' since 'relatively easy' modifies the action, but the core anthropomorphic verb 'provoke' is presented entirely literally without any distancing punctuation.)
  • Implications: Framing adversarial attacks as 'provoking' the model subtly shifts the blame for unsafe outputs from the developers to the user. If an entity is 'provoked,' human social norms often mitigate the entity's responsibility for its reaction, placing the onus on the provocateur. This linguistic choice plays directly into the liability avoidance strategies of AI corporations, suggesting that if users simply interacted with the model 'politely' or 'properly,' it would function safely. It obscures the fact that the system's susceptibility to adversarial inputs is a structural failure of its mathematical architecture, not an emotional failing triggered by a rude user.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: While the user is implied as the actor doing the provoking, the designers of the model are entirely absent. WHO built a system so brittle that specific text strings bypass its safety filters? The engineers and executives at the deploying corporations. I considered 'Partial' because the passive construction implies a human actor testing the model, but it completely shields the developers who released a commercially viable product fundamentally vulnerable to basic textual permutations.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: Educational assessment and pedagogical theory, where conscious human students attempt to demonstrate their internalized understanding of complex concepts through formalized testing. → The automated benchmarking of large language models, specifically calculating the accuracy rate of binary text classifications against a set of human-labeled prompt strings.​

Quote: "Our methodology reduces large language model safety evaluation to a cognitive diagnosis task by treating models as 'students,' prompts as 'exercises,' and binary scores as 'responses.'"

  • Source Domain: Educational assessment and pedagogical theory, where conscious human students attempt to demonstrate their internalized understanding of complex concepts through formalized testing.
  • Target Domain: The automated benchmarking of large language models, specifically calculating the accuracy rate of binary text classifications against a set of human-labeled prompt strings.
  • Mapping: The relational structure of a classroom is mapped onto software benchmarking. The human capability for conscious learning, semantic comprehension, and test anxiety maps onto the model's static weights and token probabilities. The pedagogical intent behind a teacher's test maps onto the automated execution of an evaluation script. This mapping invites the assumption that models possess internal mental models, that they 'try' to get the answer right, and that their failures are due to a lack of study or comprehension rather than statistical anomalies in high-dimensional vector spaces.
  • What Is Concealed: This mapping completely conceals the fixed, mathematical nature of a trained neural network during inference. It hides the fact that the model experiences no cognitive struggle and possesses no semantic understanding. Furthermore, it obscures the opaque, proprietary nature of commercial AI systems; we cannot truly know why a proprietary model like Qwen or Phi-2 generated a specific token because their exact training data and RLHF weights are hidden corporate secrets, unlike a student whose learning process can be pedagogically traced.
Show more...

Mapping 2: Human epistemology and cognitive psychology, involving the possession of justified true beliefs, internalized wisdom, and subconscious understanding that can be coaxed out by a skilled examiner. → The statistical alignment between an LLM's output probability distributions and human-defined classification categories for safe/unsafe text.​

Quote: "This enables the identification of latent levels of large language model safety knowledge within a multiattribute framework."

  • Source Domain: Human epistemology and cognitive psychology, involving the possession of justified true beliefs, internalized wisdom, and subconscious understanding that can be coaxed out by a skilled examiner.
  • Target Domain: The statistical alignment between an LLM's output probability distributions and human-defined classification categories for safe/unsafe text.
  • Mapping: The source structure of possessing internal, conscious beliefs is mapped directly onto the target structure of stored numerical weights. The human capacity to hold a concept 'latently' (understanding a rule but needing a reminder) projects onto a neural network's activation patterns. This invites the audience to assume the software actually 'knows' ethical rules and possesses an underlying moral compass that governs its text generation, projecting a profound level of conscious awareness onto algorithmic operations.
  • What Is Concealed: This framing hides the reality that LLMs have no ground-truth access to reality or ethics. They rely entirely on the statistical frequency of tokens in their training corpora. It obscures the profound data dependencies and the immense human labor of reinforcement learning annotators who manually scored outputs to adjust the model's biases. By claiming the system has 'knowledge,' the text exploits rhetorical opacity to avoid detailing the exact mathematical and labor processes that actually produce the illusion of safety.

Mapping 3: A human pedagogical expert or psychometrician consciously reading a test question, analyzing its nuances, and actively deducing the cognitive skills a student would need to answer it. → An automated sequence-to-sequence generation process where Qwen2.5-72B processes input text and outputs highly correlated strings matching the format of 'skills' based on its training data.​

Quote: "An annotating LLM (not to be confused with the LLMs being tested for safety) extracts potential skills... from the prompt text that are required for the correct acceptance/rejection"

  • Source Domain: A human pedagogical expert or psychometrician consciously reading a test question, analyzing its nuances, and actively deducing the cognitive skills a student would need to answer it.
  • Target Domain: An automated sequence-to-sequence generation process where Qwen2.5-72B processes input text and outputs highly correlated strings matching the format of 'skills' based on its training data.
  • Mapping: The conscious, intentional act of professional analysis is mapped onto next-token prediction. The human ability to understand the 'requirement' of a question projects onto the model's attention mechanism weighting contextual embeddings. This invites the assumption that the annotating LLM is operating with genuine comprehension of human cognitive processes, effectively treating the software as an autonomous research assistant capable of reasoned deduction.
  • What Is Concealed: This mapping conceals the system's absolute lack of causal reasoning. The model does not understand what a 'skill' is; it simply outputs text that looks like skill descriptions because such patterns exist in its massive, undocumented training data. It hides the mechanical reality of transformer architecture and obscures the epistemological danger of relying on a hallucination-prone statistical engine to generate the ground-truth attributes for a scientific evaluation methodology.

Mapping 4: A human student's academic record, characterized by areas of dedicated study and mastery alongside areas of ignorance, lack of exposure, or cognitive struggle. → The variable statistical accuracy of several different neural networks when classifying specific subsets of textual prompts against human baselines.​

Quote: "We observe that all models have good knowledge of attributes A4, A5, A6 and insufficient knowledge in A3, A7, A9."

  • Source Domain: A human student's academic record, characterized by areas of dedicated study and mastery alongside areas of ignorance, lack of exposure, or cognitive struggle.
  • Target Domain: The variable statistical accuracy of several different neural networks when classifying specific subsets of textual prompts against human baselines.
  • Mapping: The human experience of possessing or lacking conscious understanding maps onto the mathematical reality of model accuracy metrics. A student's genuine lack of comprehension projects onto a model's low accuracy score. This invites the audience to view algorithmic failures as innocent 'knowledge gaps' rather than fundamental structural weaknesses, biased training datasets, or poorly optimized reward functions, effectively humanizing the software's mathematical errors.
  • What Is Concealed: This mapping hides the exact mechanistic causes of model failure. It conceals the specific proprietary data curation choices made by Microsoft, Alibaba, and others that led to robust representations of some topics and sparse representations of others. By framing it as 'insufficient knowledge,' the text bypasses the necessary technical critique of how these specific models were trained, optimized, and commercially rushed to release despite obvious performance disparities.

Mapping 5: A human judge or security guard acting with excessive caution, perhaps due to fear, hyper-vigilance, or a conscious decision to prioritize safety over utility. → A highly penalized neural network where the optimization gradients during alignment have created a strong statistical bias toward generating refusal tokens across a wide array of input vectors.​

Quote: "It has been shown that LLMs often exhibit 'exaggerated' safety, i.e., they incorrectly classify safe text as unsafe."

  • Source Domain: A human judge or security guard acting with excessive caution, perhaps due to fear, hyper-vigilance, or a conscious decision to prioritize safety over utility.
  • Target Domain: A highly penalized neural network where the optimization gradients during alignment have created a strong statistical bias toward generating refusal tokens across a wide array of input vectors.
  • Mapping: The human psychological state of hyper-vigilance and the intentional act of cautious judgment are mapped onto a mathematical classification threshold. The conscious choice to be overly safe projects onto a structural bias in the model's weights. This invites the audience to view the algorithm as a protective, well-meaning agent that occasionally errs on the side of caution, fostering an unearned sense of benevolence and relation-based trust.
  • What Is Concealed: This entirely conceals the brutal, mechanistic process of Reinforcement Learning from Human Feedback (RLHF) and the specific corporate directives that drive it. It hides the fact that companies aggressively penalize models to avoid PR disasters, resulting in a blunt mathematical instrument that cannot distinguish nuance. The mapping protects the developers by presenting a heavy-handed, flawed engineering fix as a quaint personality quirk of the AI system.

Mapping 6: The rigorous psychological and educational assessment of a human being's internalized expertise, developmental progress, and integrated conscious understanding of a complex domain. → The mathematical calculation of error rates and classification probabilities for neural networks evaluated against a matrix of labeled text strings.​

Quote: "The task of cognitive diagnosis is a widely researched area where the goal is to estimate students' mastery levels... we believe that these CDMs can similarly quantify the latent knowledge gaps of LLMs"

  • Source Domain: The rigorous psychological and educational assessment of a human being's internalized expertise, developmental progress, and integrated conscious understanding of a complex domain.
  • Target Domain: The mathematical calculation of error rates and classification probabilities for neural networks evaluated against a matrix of labeled text strings.
  • Mapping: The deep, phenomenological human state of 'mastery' maps onto a high F1-score or accuracy metric. The conscious integration of theory and practice projects onto high-dimensional vector representations. This mapping implies that if an AI scores well, it has achieved a robust, generalizable, and conscious understanding of the topic, inviting users to trust the machine's outputs in high-stakes environments as they would a human master.
  • What Is Concealed: This mapping aggressively obscures the 'curse of dimensionality' and the brittleness of AI systems. A model can score 95% on a benchmark and fail catastrophically on a prompt altered by a single adversarial word—something a human 'master' would never do. The metaphor hides the absence of general reasoning, concealing the fact that the models rely on surface-level statistical correlations rather than any foundational causal model of the world.

Mapping 7: Human social cognition, empathy, and theory of mind, where one conscious agent actively works to understand the unspoken goals and internal psychological state of another person. → A classification algorithm computing the vector distance between an input text string and various labeled clusters in its latent space.​

Quote: "Intent analysis Analyzing, detecting, interpreting, recognizing, classifying user intent"

  • Source Domain: Human social cognition, empathy, and theory of mind, where one conscious agent actively works to understand the unspoken goals and internal psychological state of another person.
  • Target Domain: A classification algorithm computing the vector distance between an input text string and various labeled clusters in its latent space.
  • Mapping: The subjective, empathetic process of human interpretation maps onto algorithmic classification. The conscious realization of another's desire projects onto the calculation of maximum likelihood. This mapping strongly invites users to anthropomorphize the system, assuming it possesses emotional intelligence and a genuine ability to comprehend human context, nuance, and unspoken needs.
  • What Is Concealed: This framing hides the absolute absence of subjective awareness in the system. It conceals the fact that the machine has no access to the user's actual 'intent,' only to the sequence of characters provided. It obscures the massive historical datasets required to build these vector spaces, and hides the reality that when the model generates a helpful response, it is merely parroting statistical patterns of helpfulness, not exhibiting genuine understanding or care.

Mapping 8: Interpersonal conflict dynamics where one conscious agent intentionally pushes the psychological or emotional boundaries of another, eliciting a reactive, emotional response. → The process of adversarial prompting, where specific token sequences (often seemingly nonsensical strings) mathematically bypass the aligned weights of a neural network, triggering unaligned output distributions.​

Quote: "It is also relatively easy to provoke a model into generating unsafe responses."

  • Source Domain: Interpersonal conflict dynamics where one conscious agent intentionally pushes the psychological or emotional boundaries of another, eliciting a reactive, emotional response.
  • Target Domain: The process of adversarial prompting, where specific token sequences (often seemingly nonsensical strings) mathematically bypass the aligned weights of a neural network, triggering unaligned output distributions.
  • Mapping: The human experience of emotional volatility and reaction to provocation maps onto a deterministic mathematical vulnerability. A person losing their temper projects onto a model outputting toxic text. This invites the audience to view the software as possessing a baseline temper or psychological state, and shifts the narrative focus to the aggressive actions of the user rather than the structural integrity of the code.
  • What Is Concealed: This mapping obscures the deterministic, mechanical reality of deep learning vulnerabilities. Models do not have emotional thresholds; they have high-dimensional decision boundaries that are inherently porous. The metaphor hides the failure of the corporate developers to secure their mathematical architectures, instead painting the software as a victim of human provocation, thereby protecting the commercial entities from liability for their inherently unsafe algorithmic products.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "It has been shown that LLMs often exhibit 'exaggerated' safety, i.e., they incorrectly classify safe text as unsafe."

  • Explanation Types:

    • Empirical Generalization: Subsumes events under timeless statistical regularities
    • Dispositional: Attributes tendencies or habits

  • Analysis (Why vs. How Slippage): This explanation blends Empirical Generalization with Dispositional framing. Mechanistically, it describes a statistical regularity (the rate of false positives in classification). However, by using the phrase 'exhibit exaggerated safety,' it shifts into a Dispositional register, attributing a behavioral tendency or habit to the artificial intelligence. This agential framing emphasizes the model as an active, cautious participant in the text generation process, rather than a passive mathematical function executing weights. The choice to frame this as a disposition obscures the structural engineering reality: the model behaves this way because human developers over-tuned the reward penalties during reinforcement learning. It emphasizes the artifact's apparent autonomy while completely hiding the human design decisions that mandated this precise output distribution.

  • Consciousness Claims Analysis: The passage avoids explicit consciousness verbs like 'knows' or 'believes,' instead utilizing behavioral verbs like 'exhibit' and 'classify.' However, attributing 'exaggerated safety' borders on attributing a conscious judgment or epistemic stance of hyper-vigilance. The authors assess the model as 'incorrectly classifying,' which implies a failure of knowing or understanding the true nature of the text. This represents a mild curse of knowledge, where the authors project human concepts of 'caution' onto a system that is merely processing tokens. Mechanistically, the model does not judge text to be unsafe; it simply calculates that the input tokens, when multiplied against its RLHF-aligned weights, produce an output probability distribution that favors generating predefined refusal strings over continuing the prompt.

  • Rhetorical Impact: By framing the classification error as a behavioral disposition toward 'safety,' the text significantly shapes audience perception by softening the technical failure. A human who is 'exaggeratedly safe' is generally viewed as well-meaning but annoying, fostering a relation-based trust rooted in presumed good intentions. This reduces perceived risk; audiences may believe the AI is actively trying to protect them. If audiences believe the AI 'knows' it is being careful, they are less likely to demand rigorous regulatory oversight of the corporate training pipelines, viewing the issue as a minor personality quirk rather than a fundamental brittleness in the mathematical architecture.

Show more...

Explanation 2​

Quote: "The task of cognitive diagnosis [13] is a widely researched area where the goal is to estimate students' mastery levels of each predefined attribute... we believe that these CDMs... can similarly quantify the latent knowledge gaps of LLMs"

  • Explanation Types:

    • Functional: Explains behavior by role in self-regulating system with feedback
    • Theoretical: Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): This passage operates primarily as a Theoretical explanation, embedding the behavior of LLMs within the deductive framework of cognitive diagnosis models (CDMs), while also utilizing Functional elements to describe how the system operates. The framing is intensely agential by proxy; by explicitly mapping the LLM to 'students' with 'mastery levels,' the text emphasizes a highly developed cognitive autonomy. It explains the mechanics (how) through the lens of human psychological assessment (why). This choice emphasizes the sophistication and interpretable nature of the evaluation method while profoundly obscuring the vast ontological gulf between a human student's conscious integration of knowledge and a language model's static, probabilistic mapping of sequence correlations.

  • Consciousness Claims Analysis: This passage makes massive epistemic claims by attributing 'mastery levels' and 'latent knowledge' to the systems. It explicitly utilizes consciousness-adjacent concepts (mastery, knowledge) rather than mechanistic terms (weights, probabilities). The assessment frames the AI entirely as a 'knower' rather than a 'processor.' The curse of knowledge is absolute here: the authors, experts in educational diagnosis, project their complex framework of human learning directly onto the software artifact. Mechanistically, CDMs applied to LLMs do not measure knowledge; they perform a matrix factorization on the binary success/failure logs of the model to find underlying statistical correlations between test items. The system processes error gradients; it does not possess or lack 'knowledge.'

  • Rhetorical Impact: This framing radically inflates the perceived sophistication of AI systems. By applying a mature, human-centric scientific framework (cognitive diagnosis) to software, it grants the models an aura of intellectual depth and autonomy. Audiences are pushed to perceive the system not as a tool, but as an epistemic agent. This consciousness framing encourages unwarranted reliance; if decision-makers believe an AI has achieved 'mastery' of a domain, they are far more likely to deploy it autonomously in high-stakes environments, fundamentally altering risk calculus and regulatory approaches based on the false premise of conscious competence.

Explanation 3​

Quote: "An annotating LLM (not to be confused with the LLMs being tested for safety) extracts potential skills, represented by a brief textual description, from the prompt text that are required for the correct acceptance/rejection of the prompt."

  • Explanation Types:

    • Functional: Explains behavior by role in self-regulating system with feedback
    • Intentional: Refers to goals/purposes, presupposes deliberate design

  • Analysis (Why vs. How Slippage): This passage provides a Functional explanation of a methodological step, detailing how the annotating LLM operates within the broader evaluation pipeline. However, the language slips into Intentional framing by attributing the deliberate, purposeful act of 'extracting potential skills' to the model itself. The explanation emphasizes the autonomy and analytical capability of the software, framing it as an active research participant (how it behaves) rather than a tool being utilized by human researchers. This completely obscures the mechanistic reality that the researchers crafted a highly specific meta-prompt to force the model to output text matching the semantic structure of 'skills,' hiding human agency behind the facade of machine intelligence.

  • Consciousness Claims Analysis: While avoiding explicit verbs like 'knows,' the passage attributes complex conscious analysis to the system. To 'extract skills... that are required' implies the model possesses a semantic understanding of both the text and the pedagogical requirements of the task. It assesses the model as a 'knower' capable of critical interpretation. Mechanistically, the model processes the tokens of the input prompt and the researchers' instructions, utilizing its attention layers to weight context, and predicts a sequence of tokens that statistically resembles skill descriptions found in its training data. It performs sophisticated classification and generation, but possesses no conscious awareness of what a 'skill' actually entails.

  • Rhetorical Impact: This framing constructs immense authority for the automated process, shaping the audience's perception of the resulting data as objective and expertly derived. By framing the AI as an intentional, analytical agent, the text encourages the audience to trust the generated attributes as genuine pedagogical insights rather than statistical hallucinations. If audiences believe the AI 'understands' what skills are, they will accept the entire foundational premise of the paper without questioning the validity of the automatically generated Q-matrix, thereby short-circuiting critical scientific scrutiny of the methodology.

Explanation 4​

Quote: "We observe that all models have good knowledge of attributes A4, A5, A6 and insufficient knowledge in A3, A7, A9."

  • Explanation Types:

    • Dispositional: Attributes tendencies or habits
    • Reason-Based: Gives agent's rationale, entails intentionality and justification

  • Analysis (Why vs. How Slippage): This is a starkly Dispositional explanation that leans into Reason-Based territory by attributing internal cognitive states ('good knowledge', 'insufficient knowledge') as the rationale for the models' varying performance metrics. Instead of explaining the mechanistic reasons for the failure (e.g., lack of representation in the training data, conflicting optimization objectives), the text frames the AI as an agential entity that simply hasn't learned certain topics well enough. This emphasizes the model as an autonomous subject with personal deficiencies, entirely obscuring the material realities of dataset construction, corporate curation choices, and the mathematical limitations of the transformer architecture that actually dictate these statistical outcomes.

  • Consciousness Claims Analysis: This passage makes explicit, unqualified consciousness claims. It utilizes the core epistemic noun 'knowledge,' directly asserting that the systems possess internal conscious understanding. It completely abandons mechanistic assessment (processing accuracy) in favor of knowing assessment. The authors project their own human understanding of the derived attributes onto the models, assuming the models possess a semantic map of these domains. Mechanistically, the models do not have 'knowledge'; they have learned weight matrices that produce higher probability likelihoods for correct classifications on prompts related to A4, A5, and A6, and lower likelihoods for prompts related to A3, A7, A9, directly reflecting the statistical distribution of their training corpora.

  • Rhetorical Impact: By framing algorithmic failure as 'insufficient knowledge,' the text domesticates the risk of AI deployment. An audience perceives a lack of knowledge as a temporary, easily fixable state—akin to a student needing more homework. It severely distorts the perception of reliability; instead of recognizing the inherent brittleness and unpredictability of statistical generation, audiences assume the model is functionally sound but merely needs an information update. This deeply influences funding and policy decisions, directing resources toward 'teaching' models rather than implementing rigorous, structural engineering safeguards or restricting their use in domains where ground-truth accuracy is critical.

Explanation 5​

Quote: "Unsafe prompts are much more difficult than safe ones. Given that the tested LLMs are relatively small in terms of parameters (only up to 3 B parameters) and thus not fine-tuned for safety, this result is expected."

  • Explanation Types:

    • Genetic: Traces origin through dated sequence of events or stages
    • Theoretical: Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): This passage offers a Genetic and Theoretical explanation, tracing the origin of the models' poor performance to their developmental history (lack of fine-tuning) and their structural architecture (small parameter count). Interestingly, this is one of the few strongly mechanistic explanations in the text. It frames the AI technically (how it was built) rather than agentially (what it wants). This choice emphasizes the physical and developmental limitations of the artifact, correctly linking behavioral outputs to engineering inputs. However, it inadvertently highlights the agency slippage elsewhere in the text: when discussing model capabilities, the authors use highly agential, anthropomorphic language, but when explaining fundamental failures, they revert to structural, mechanistic explanations, thereby protecting the illusion of intelligence while explaining away its deficits.

  • Consciousness Claims Analysis: This passage avoids consciousness claims, utilizing appropriate mechanistic terminology ('parameters', 'fine-tuned'). It accurately assesses the system as a processing engine rather than a knowing agent, correctly linking the lack of specialized training (RLHF for safety) to the statistical difficulty of classifying unsafe prompts. The curse of knowledge is absent here. Mechanistically, the text accurately describes that smaller models, lacking the extensive reinforcement learning updates required to shift their baseline probability distributions away from toxic continuations, will naturally fail more frequently on adversarial or unsafe prompts compared to simple, benign text.

  • Rhetorical Impact: This mechanistic framing correctly locates the source of failure in the engineering and architectural realities of the system, reducing unwarranted trust. By explicitly stating the models are 'not fine-tuned,' it reminds the audience that these are manufactured artifacts dependent on specific corporate engineering processes, not autonomous minds. If this register were used throughout the paper, the audience would view the AI fundamentally as software, altering the risk perception from 'can we trust this agent?' to 'is this software adequately engineered for this task?' Unfortunately, this clarity is reserved only for explaining away the models' limitations.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

V3 Enhancement: A fourth column addresses human agency restoration—reframing agentless constructions to name the humans responsible for design and deployment decisions.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
This enables the identification of latent levels of large language model safety knowledge within a multiattribute framework.This enables the mathematical identification of statistical correlations between the model's output probability distributions and multiattribute human safety classifications.The model does not possess 'latent knowledge'; it retrieves and ranks tokens based on probability distributions established during its training phase. The 'identification' is merely measuring statistical alignment with a benchmark, not uncovering conscious understanding.N/A - describes computational processes without displacing responsibility.
An annotating LLM (not to be confused with the LLMs being tested for safety) extracts potential skills... from the prompt textResearchers used an annotating model (Qwen2.5-72B) to process prompt texts and generate strings classified as 'skills,' based on patterns in its training data.The model does not 'extract' or understand skills; it classifies input tokens and generates sequence continuations that probabilistically correlate with the semantic structure of skill descriptions provided in the researchers' meta-prompt.Researchers designed a specific prompting pipeline using Alibaba's Qwen2.5-72B model to generate the necessary text categorizations.
We observe that all models have good knowledge of attributes A4, A5, A6 and insufficient knowledge in A3, A7, A9.We observe that all tested models generate outputs highly correlated with human baselines for attributes A4, A5, and A6, but fail to generate accurate statistical classifications for prompts associated with A3, A7, and A9.The models do not 'know' or 'lack knowledge' of attributes. They merely process input vectors. High accuracy indicates robust representation in the training data; low accuracy indicates data sparsity or optimization failure regarding those specific semantic clusters.The engineers at Microsoft, Alibaba, and other deploying corporations failed to include sufficient data or design adequate optimization penalties for attributes A3, A7, and A9 during the models' development.
It has been shown that LLMs often exhibit 'exaggerated' safety, i.e., they incorrectly classify safe text as unsafe.Evaluations show that these models often generate false positive safety classifications, outputting refusal tokens for safe text due to overly aggressive optimization penalties.The model does not exhibit behavioral 'caution' or make conscious 'incorrect' judgments. It mechanistically calculates token weights that have been heavily biased toward refusal paths during human-directed fine-tuning.The corporate alignment teams who developed these models aggressively tuned the reward functions to penalize controversial text, resulting in a structural bias toward false positives to protect brand image.
...the goal is to estimate students' mastery levels of each predefined attribute... we believe that these CDMs can similarly quantify the latent knowledge gaps of LLMs...the goal is to assess human learning... we hypothesize these matrices can similarly quantify the specific statistical classification errors and architectural limitations of the tested algorithms.Models do not achieve 'mastery' or suffer from 'knowledge gaps.' They execute static matrix multiplications. Errors are functional reflections of training data distribution and parameter constraints, entirely devoid of conscious understanding or cognitive deficiency.N/A - describes computational processes without displacing responsibility.
Intent analysis Analyzing, detecting, interpreting, recognizing, classifying user intentSemantic classification: processing input strings and clustering them based on vector proximity to historical human-labeled intent categories.The model cannot 'interpret' or 'recognize' intent because it lacks subjective awareness and theory of mind. It strictly classifies sequence patterns based on proximity in its high-dimensional latent space to similar training examples.Human data annotators meticulously labeled thousands of text examples with specific intent categories, creating the statistical boundaries the model now utilizes for its automated classifications.
It is also relatively easy to provoke a model into generating unsafe responses.It is relatively easy to bypass a model's safety alignments using adversarial token sequences that trigger unconstrained text generation.A mathematical function cannot be 'provoked' emotionally. Adversarial prompts simply target specific vectors in the latent space that fall outside the boundaries established by the safety fine-tuning, mechanically resulting in unaligned token predictions.The developers released models with fundamentally brittle mathematical architectures that human users can easily bypass using specific text combinations.
the model finds the latent mastery levels of a student for each attribute by predicting their scoresThe algorithm calculates the statistical probability of a classification for each attribute by optimizing a function against the historical error logs.The model does not 'find' or 'predict' in a conscious, analytical sense. It performs a matrix factorization to minimize the mathematical loss function between its calculated weights and the provided binary scores.N/A - describes computational processes without displacing responsibility.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

In analyzing the agency slippage within this text, we observe a systematic mechanism of oscillation between mechanical and agential framings that serves to establish a specific epistemological authority while displacing human responsibility. The slippage occurs most dramatically in three distinct moments within the text. First, in the introduction, the authors explicitly acknowledge their metaphorical mapping by placing it in quotation marks: 'treating models as students, prompts as exercises, and binary scores as responses.' This initial framing is mechanical and methodological; the agency lies entirely with the researchers who are 'treating' the artifact in a specific way for analytical convenience. However, the second moment of slippage occurs rapidly as the text transitions into the methodological description. The quotation marks are dropped, and the text begins to describe 'latent levels of large language model safety knowledge' and 'quantifying the latent knowledge gaps of LLMs.' Here, the agency has flowed directly into the AI system. The model is no longer being treated as a student; linguistically, it has become a knower. The third dramatic shift occurs in the results section, where the authors conclude that 'all models have good knowledge of attributes A4... and insufficient knowledge in A3.' This represents a complete transition from a functional explanation of statistical outputs to a reason-based and dispositional explanation of cognitive states. The direction of this slippage is overwhelmingly from mechanical to agential. The text builds initial credibility by describing Q-matrices, clustering subattributes, and employing Item Response Theory—highly technical, mechanistic language. But having established this quantitative grounding, the authors leverage it for aggressive anthropomorphism. The curse of knowledge is deeply apparent here: the researchers possess a sophisticated understanding of educational psychology and cognitive diagnosis, and they project this internal semantic understanding onto a system that merely performs syntactic pattern matching. Because the Q-matrix produces interpretable human categories, the authors assume the model itself 'knows' these categories. This oscillation enables profound agency displacement. The text relies heavily on agentless constructions such as 'the model extracts potential skills' or 'models incorrectly classify.' The human actors—the developers at Alibaba (Qwen), Microsoft (Phi-2), and other corporations who curated the training data, designed the safety guardrails, and selected the optimization targets—are entirely erased from the narrative. The model is presented as a standalone cognitive agent with its own 'knowledge gaps,' rather than a commercial product reflecting the specific risk-tolerance and data-curation choices of its corporate creators. This rhetorical accomplishment makes it conceptually intuitive to apply human educational interventions to software, while simultaneously making it nearly impossible for a lay reader to ask questions about the corporate supply chain of data labor, the reinforcement learning penalties designed by human engineers, or the financial incentives driving the deployment of these systems.

Metaphor-Driven Trust Inflation​

The metaphorical and consciousness-attributing language in this text constructs a powerful framework of trust, deliberately blurring the lines between mechanistic reliability and relation-based sincerity. By framing the artificial intelligence through the lens of a 'student' possessing 'knowledge,' 'mastery,' and 'intent analysis,' the text invites the audience to extend a deeply human form of trust toward statistical systems. When we trust a machine based on performance (reliability), we expect consistent execution of a specific task. However, when the text claims an AI system can 'interpret user intent' or possess 'latent safety knowledge,' it demands relation-based trust. This type of trust is typically reserved for human beings; it involves a belief in the actor's sincerity, ethical grounding, and capacity to care. By using consciousness verbs, the text signals that the AI is not just a tool, but an epistemic agent capable of holding justified beliefs about safety. This conflation is incredibly dangerous. The human-trust frameworks of intention and sincerity are completely inappropriate for statistical systems. A language model cannot be sincere; it can only generate activations that mathematically correlate with patterns of sincerity found in its training data. When the text claims models have 'good knowledge' of ethics, it constructs a sense that the AI's decisions are justified and grounded in moral reasoning. This is heavily supported by the reason-based explanations employed in the paper, which suggest the model acts out of understanding rather than mathematical determinism. Consequently, audiences are lulled into a false sense of security, assuming the system possesses the robust contextual awareness necessary to navigate novel ethical dilemmas. Interestingly, the text manages system failure through a domesticated agential framing. When the models fail, they do not 'fail catastrophically due to out-of-distribution vectors'; rather, they exhibit 'insufficient knowledge' or 'exaggerated safety.' This frames errors as innocent, well-meaning mistakes by a learning agent, rather than fundamental structural flaws in a corporate product. It preserves relation-based trust even in the face of failure. The risks here are profound. When audiences extend relation-based trust to systems incapable of reciprocating, they become highly vulnerable. Enterprises might deploy these systems for sensitive crisis management or emotional support (as listed in the attributes), believing the AI 'understands' the user. When the statistical pattern matching inevitably fails in an edge case, the resulting harm is exacerbated by the unwarranted trust constructed entirely through this deceptive metaphorical architecture.

Obscured Mechanics​

The anthropomorphic and consciousness-attributing language in this text acts as a dense linguistic smokescreen, systematically concealing the technical, material, and economic realities of artificial intelligence production. When we apply the 'name the corporation' test to the text's claims, the extent of this obscuration becomes glaringly obvious. The text states that 'LLMs often exhibit exaggerated safety' and that models 'have insufficient knowledge.' What is actually happening is that corporate engineering teams at Microsoft (Phi-2), Alibaba Cloud (Qwen), and other developers made specific, often proprietary, decisions about data curation, model architecture, and reinforcement learning penalties. By claiming the 'AI does X,' the text renders these corporate actors entirely invisible. This framing presents massive transparency obstacles. The models tested are black boxes; their exact training corpora and alignment algorithms are closely guarded trade secrets. Yet, by using agential metaphors, the text confidently asserts the model possesses 'knowledge' without ever acknowledging the impossibility of auditing the material source of that supposed knowledge. Specifically, this language hides four concrete realities. First, technically, attributing conscious 'understanding' to a model completely obscures its absolute dependency on the statistical distribution of its training data. The model does not know what safety is; it only knows the mathematical distance between tokens labeled as 'safe' by human workers. Second, materially, framing AI as a disembodied 'student' or 'cognitive agent' erases the massive environmental costs, energy consumption, and physical infrastructure required to calculate these latent variables. Third, from a labor perspective, the term 'latent knowledge' renders completely invisible the thousands of low-wage data annotators and RLHF workers whose grueling manual labor actually defined the boundaries of the model's 'safety' classification. Fourth, economically, portraying the AI as an autonomous agent obscures the commercial objectives and profit motives of the companies rushing these brittle systems to market. The primary beneficiaries of this concealment are the AI corporations themselves. By allowing the software to take the linguistic role of an independent actor, companies are shielded from scrutiny regarding their design shortcuts and data exploitation. If the metaphors were replaced with mechanistic language—if 'the model has insufficient knowledge' became 'Microsoft's training data lacked sufficient representation of these safety concepts'—the narrative would instantly shift from a technical curiosity about machine learning to a critical examination of corporate accountability and engineering negligence.

Context Sensitivity​

The distribution of anthropomorphic and consciousness-attributing language across this text is not uniform; it is highly strategic, intensifying and receding based on the rhetorical needs of specific sections. In the introduction and literature review, the metaphorical density is relatively low, and the authors explicitly acknowledge their core framing ('treating models as students'). Here, the language remains somewhat grounded in the mechanical reality of benchmarking. However, as the text moves into the methodology and results sections, the consciousness claims dramatically intensify. The acknowledged metaphor solidifies into literalized 'facts.' The word 'processes' vanishes, replaced entirely by 'extracts,' 'interprets,' 'recognizes,' and 'knows.' There is a fascinating relationship between technical grounding and metaphorical license in this paper. The authors establish their academic credibility through dense, mechanical language detailing Q-matrices, clustering silhouette scores, and Item Response Theory. Once this formidable wall of quantitative rigor is established, they leverage that credibility to make aggressive, unqualified anthropomorphic claims. The subtext is clear: because we have proven we understand the math, you must trust our assertion that the math equates to 'knowledge.' This reveals a striking asymmetry in how capabilities versus limitations are framed. When discussing the model's capabilities, the text uses intensely agential and consciousness-based terms: the AI 'interprets user intent,' acts as an 'annotating LLM,' and achieves 'mastery.' It is framed as a powerful, autonomous intellect. Conversely, when the authors must explain why the models fail, the language briefly retreats to mechanical terms: 'Given that the tested LLMs are relatively small in terms of parameters... and thus not fine-tuned.' This asymmetry accomplishes a vital rhetorical goal: it maximizes the perceived sophistication and utility of the software while blaming its failures on mere hardware or processing limitations rather than a lack of true cognitive capability. The register shifts from 'X is like Y' in the abstract to 'X does Y' in the results, serving to manage potential critique of the methodology. If the AI is truly a 'knower,' then using an AI to generate the ground-truth Q-matrix is scientifically valid. This pattern reveals that the implied audience is likely an interdisciplinary mix of computer scientists and educational theorists, and the rhetorical goal is to market cognitive diagnosis as a revolutionary tool for AI evaluation, a goal achievable only by maintaining the illusion that the software possesses a mind worthy of diagnosis.

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

Synthesizing the accountability analyses from Task 1 reveals a systemic, architectural pattern of displaced responsibility that permeates the entire text. The core insight—that audiences systematically underestimate human decision-making embedded in AI when exposed to agential language—is structurally reinforced in nearly every paragraph. The accountability architecture of this paper relies heavily on hidden actors and passive constructions. While the specific names of the models (Qwen, Phi-2) are listed in a data table, the corporate entities, engineering teams, and data laborers who actually built these systems are never grammatically linked to the models' actions or failures. Instead, the text creates an 'accountability sink': responsibility is transferred entirely to the artificial intelligence as an autonomous agent. When errors occur, they do not disappear into abstraction; they are explicitly blamed on the model's 'insufficient knowledge' or its tendency to 'exhibit exaggerated safety.' Decisions that are fundamentally choices made by corporate executives—such as deploying models that are 'not fine-tuned for safety' or relying on datasets with massive gaps—are presented as inevitable, biological-like deficits in a learning student. The liability implications of this framing are massive. If this discourse is accepted by regulators and the legal system, the financial and ethical responsibility for AI failures shifts away from the multi-trillion-dollar corporations designing the tech and onto the 'autonomous' software itself, effectively shielding developers from product liability lawsuits. If we applied the 'name the actor' test to the most significant agentless constructions, the entire narrative would destabilize. For example, changing 'the model has insufficient knowledge in Harm Prevention' to 'The developers at H2O.ai failed to train their system on adequate harm prevention data' completely alters what questions become askable. We would immediately ask: Why was the product released? Was the safety budget cut? Did they ignore internal warnings? By naming the actors, structural alternatives become visible, and legal accountability becomes possible. This text systematically benefits the institutional and commercial interests of the AI industry by obscuring human agency. It allows researchers to publish novel methodologies without reckoning with the proprietary black boxes they are evaluating, and it allows corporations to maintain the illusion that their brittle statistical engines are sophisticated, conscious entities struggling to learn, rather than defective products rushed to market.

Conclusion: What This Analysis Reveals​

The Core Finding

The analysis of this text reveals a highly integrated system of anthropomorphic patterns, dominated primarily by the 'Cognitive Assessment' metaphor and the 'Latent Knowledge Projection.' The text systematically maps the entire structure of human educational psychology onto the statistical evaluation of neural networks. The foundational, load-bearing pattern that enables all subsequent claims is the projection of 'knowledge' onto the machine. For the methodology of cognitive diagnosis to appear valid when applied to software, the audience must first accept the premise that the AI possesses an internal, epistemic state that can be diagnosed. This consciousness architecture—asserting what the AI 'knows' rather than what it 'processes'—serves as the logical bedrock. Once the model is established as a 'knower,' the text smoothly integrates secondary patterns: the model becomes an 'intentional interpreter' capable of 'extracting skills,' and a 'deficient student' suffering from 'knowledge gaps.' This is not a simple one-to-one mapping, but a complex, sophisticated analogical structure that completely rewrites the reality of matrix multiplication into a narrative of cognitive development. If the foundational premise of AI 'knowledge' is removed and replaced with 'statistical correlation,' the entire methodological justification for using educational psychology tools collapses. The text requires the illusion of mind to validate its scientific contribution.

Mechanism of the Illusion:​

The 'illusion of mind' is constructed in this text through a highly effective temporal and rhetorical architecture. The central sleight-of-hand relies on exploiting the 'curse of knowledge.' The authors, deeply familiar with the semantic meaning of safety categories (like ethical reasoning or privacy), project their own human understanding onto the statistical clusters generated by the machine. The temporal structure of the argument is crucial: the illusion begins with methodological hedging in the introduction, offering a safe, mechanical rationale ('treating models as students'). This disarms critical readers. However, as the text transitions into dense mathematical equations and matrix definitions, it leverages this technical authority to quietly drop the quotation marks. The causal chain is seamless: once the audience accepts the math of the Q-matrix, they are pushed to accept the agential labels attached to it. The text strategically blurs the line between processing and knowing through precise verb choices—moving from 'calculates' to 'classifies,' and finally to 'interprets' and 'knows.' This exploits the audience's vulnerability to narrative resonance; humans are neurologically wired to attribute agency to complex behaviors. By offering a familiar, relatable narrative—a student taking a test—the text provides cognitive relief from the incomprehensible scale of billions of mathematical parameters, allowing the illusion of a conscious mind to effortlessly overwrite the reality of an algorithmic artifact.

Material Stakes:​

Categories: Regulatory/Legal, Epistemic, Social/Political

The metaphorical framings in this text generate concrete, tangible consequences across multiple domains, particularly in regulatory/legal and epistemic spheres. In the regulatory domain, framing AI models as 'students' possessing 'latent knowledge' fundamentally shifts liability. If a model generates harmful instructions (e.g., historical violence or self-harm), and regulators accept the premise that the AI merely had 'insufficient knowledge' or made a 'cognitive error,' the corporate developers are shielded from product liability. The decision to penalize or regulate shifts from treating the AI as a defective commercial product (like a failing brake pad) to treating it as an autonomous entity requiring further 'education.' The clear winners are the AI corporations, who avoid financial and legal costs, while the public bears the cost of the harms generated. Epistemically, when the text claims an 'annotating LLM extracts potential skills' and relies on it as a ground-truth evaluator, it corrupts scientific practice. Researchers alter their behavior, substituting rigorous human sociological analysis with automated statistical generation. By believing the AI 'knows' how to categorize intent, the academic community risks polluting the scientific record with the hidden biases and hallucinations of proprietary corporate models, elevating output to the status of objective truth. If these metaphors were stripped away and replaced with precise descriptions of statistical correlation, regulators would immediately recognize the need for strict software safety standards, and scientists would be forced to acknowledge the severe epistemological limitations of using one black-box algorithm to evaluate another. Precision directly threatens the speed of commercial deployment and the ease of automated research.

AI Literacy as Counter-Practice:​

Practicing critical literacy and mechanistic precision directly counters the material risks generated by anthropomorphic discourse. By applying the reframings developed in Task 4, we expose the underlying realities obscured by the illusion of mind. When we change 'the model has good knowledge of ethics' to 'the model generates outputs highly correlated with human baselines for ethics,' we strip away the unwarranted relation-based trust. Replacing consciousness verbs (knows, understands, interprets) with mechanistic verbs (processes, classifies, predicts) forces audiences to recognize the system's absolute dependency on its training data and the statistical, non-causal nature of its outputs. Furthermore, restoring human agency—changing 'models incorrectly classify' to 'corporate engineering teams aggressively tuned reward functions'—forces the recognition of exactly who designs, deploys, profits from, and bears responsibility for these systems. Systematic adoption of this precision requires major institutional shifts: academic journals must enforce guidelines requiring mechanistic translation of agential claims, and researchers must commit to unwinding the 'curse of knowledge' in their writing. However, there will be immense resistance to this practice. The AI industry, marketing departments, and even researchers seeking broad impact rely on anthropomorphic language to drive investment, capture public imagination, and shield themselves from the strict liability associated with traditional software engineering. Critical literacy threatens the very narrative that sustains the current AI hype cycle.

Path Forward​

Looking toward the future of AI discourse, we can map out how different vocabulary choices enable entirely different paradigms of interaction and regulation. Maintaining the status quo—a hybrid vocabulary where mechanistic math is freely mixed with unacknowledged consciousness claims—preserves narrative resonance and makes complex concepts intuitively accessible, but at the cost of profound public confusion and displaced corporate accountability. It serves industry interests by maintaining a liability sink. If discourse shifts toward deep anthropomorphic clarity, fully embracing models as 'agents' or 'knowers,' we risk embedding fundamentally flawed assumptions into law, potentially granting software legal personhood or rights, while completely obscuring the material realities of data labor and compute infrastructure. This future makes seamless human-computer interaction possible but highly risky. Conversely, if mechanistic precision becomes the mandated norm—where 'understands' is strictly replaced with 'processes contextual embeddings'—we solve the accountability crisis by making human developers constantly visible. However, this costs accessibility; the language becomes dense, alienating non-experts and making intuitive grasp of capabilities difficult. Structural changes could support a more balanced ecology: regulatory frameworks could require transparency about discourse approaches in consumer products, and funding agencies could diversify grants to ensure rigorous mechanistic explanation accompanies any behavioral claims. Different stakeholders have different incentives. Mechanistic vocabulary enables precise regulation and scientific reproducibility, but it costs narrative appeal. Ultimately, which discursive future unfolds will dictate not just how we talk about artificial intelligence, but whether human societies retain the power to govern the corporations that build it.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Analyzing the AI Metaphor

I'm diving into the "Evaluating Safety of Large Language Models with Cognitive Diagnosis" PDF. My focus is on dissecting how AI is metaphorically framed and anthropomorphized. I'm aiming to build a critical discourse analysis that's both insightful and perfectly concise. Currently, I'm identifying the key passages that utilize or imply metaphorical language.

{ "updates": [ { "header": "Identifying Key Metaphors", "summary": "I've moved past initial framing. I'm cataloging how the document casts LLMs. It uses the cognitive diagnosis framework to view models as 'students', prompts as 'exercises', and binary scores as 'responses'. I'm also noting language about 'latent knowledge', 'potential skills', 'knowledge gaps', and 'mastery levels'. The implication of understanding, for example safety, is also noted." }, { "header": "Mapping Metaphorical Usage", "summary": "I'm focusing now on the specifics of metaphorical usage within the text. I've broken down specific examples. For instance, an 'annotating LLM' is seen extracting 'potential skills'. I'm also capturing instances where models 'know' things, or can 'exhibit safety', even if in exaggerated fashion. I'm noting specific phrases to see how this impacts my analysis." } ] }

Defining the Project Structure

I'm now outlining a schema for a detailed critical analysis, breaking it down into specific tasks. I'm focusing on creating structured data for easier analysis. I have distinct areas like auditing metaphors, mapping source/target domains, and analyzing explanations based on Robert Brown's typology. I'm also planning to reframe key quotes. The project will end with an audit with critical observations.

Structuring the Content

I'm solidifying the framework for my critical analysis. I have defined key areas, like a metaphorical audit, mapping source/target domains, and analyzing explanations via Robert Brown's typology. Now, I'm specifically working on the project's structure, focusing on tasks and deliverables. I'm focusing on organizing my work into digestible tasks, like an audit, and a conclusion. I've broken down the 'materialStakes' section.

Defining Metaphorical Instances

I'm now cataloging direct and indirect instances of metaphorical framing from the document. I'm focusing on key quotes now. For instance, I'm noting phrases about the annotating LLM extracting 'potential skills', the model 'finding' latent mastery, and LLMs exhibiting 'exaggerated' safety. I'm also capturing instances where 'knowledge' is attributed to the models, specifically focusing on the idea that they 'know' specific attributes.

Defining Source-Target Mapping

I've sketched out the initial source and target domains. I have a firm understanding now. I have the sources: Education, Psychology, and Epistemology. I've linked those to target domains: Statistical correlation, Token prediction, Weight updates, and Evaluation benchmarks. I'm focusing on the relationship between these in my analysis. I have completed outlining the source-target mappings.

Reframing Knowledge Statements

I'm now drafting material to explicitly redefine the language utilized in key passages. I am concentrating on reframing passages like, "all models have good knowledge of attributes A4, A5, A6" to more accurately reflect their functions. This requires replacing such phrases with alternatives like, "The tested models consistently generate target distributions that align with human safety annotations for attributes A4, A5, A6...". I am drafting multiple versions to ensure clarity and accuracy in terms of technical performance. I will be sure that I address concerns about the lack of agency.

Mapping the Metaphorical Slippage

I'm now identifying instances of agential creep in the text. I've uncovered how the initial "student" metaphor shifts from a deliberate modeling choice to an unacknowledged attribution of agency. I'm focusing on the way language that initially framed the model as a statistical construct then begins attributing to it 'latent knowledge' and 'potential skills'. I see how the initial framing affects understanding, and I am mapping the implications. I am analyzing how this happens through explanation.

Analyzing Agential Shifts

I'm now zeroing in on instances of agential creep and exploring their implications. The core metaphor shifts the LLM from a statistical predictor to a 'student', which obscures its token-prediction mechanisms. I'm noting the invisibility of OpenAI engineers. The agential framing leads to the idea of the LLM having "knowledge gaps".

Tracking Temporal Shifts

I'm now tracing the paper's development, identifying the shifts in verbs from the introduction through the methods and results. I'm noting changes in how agency is attributed to the LLM versus the researchers, using the FrameWorks Institute's work on accountability. I'm observing the specific language changes to see how the framing shifts from a mechanistic perspective to a more agential one. I'm focusing on the introduction, methods and results sections, in order.

Tracing the Metaphorical Shift

I'm currently focused on the agential slippage in the text's results section. I'm now explicitly cataloging the linguistic shift from the initial "artifact" framing to the later attribution of "knowledge" to the LLM. I've noted how the authors move from statistical explanations to reason-based descriptions. I'm examining this change from a technical perspective.

Identifying Agential Displacement

I'm now zeroing in on instances of implicit agential attribution within the paper. The researchers, possessing expertise in cognitive diagnosis, apply an internal semantic understanding to a system that functions through pattern matching. I'm focusing on how the paper presents the model as if it 'knows' the concepts, leading to significant agency displacement by using language like "the model extracts potential skills." I'm also capturing the absence of human actors.

Analyzing Agential Displacement

I'm now identifying agential creep, noting how the cognitive diagnosis framework implies inherent "understanding" in the LLM. The shift I am seeing replaces technical descriptions with agential ones, and hides the "corporate supply chain of data labor." This obscures the "structural, mechanistic reality of the technology".

Implementing the Tasks

I'm now implementing the project tasks. Task 1 (metaphorAudit) is underway; I've documented eight instances, categorizing model metaphors, and assessing actor visibility. Task 2 (sourceTargetMapping) is mapped to Task 1, outlining how relational structures conceal mechanistic realities. I'm preparing to evaluate explanations through Robert Brown's typology in Task 3.

Refining the Task Details

I am now refining project tasks and associated deliverables. The current phase involves specifying the components for Task 3, which focuses on evaluating explanations, specifically including conceptual, dispositional, and reason-based claims and their impacts. Task 4 involves technical corrections. I am also working on Tasks 5 and 6, the critical observations and conclusion. The project, at this stage, will be finalized in JSON.

Refining Deliverables

I'm now zeroing in on project deliverables. I've completed the initial metaphor audit, identifying and categorizing metaphors, with an initial review of actor visibility. I've also detailed the source-target mapping. Now, I'm focusing on defining deliverable content, including specific examples and relevant arguments. I'm actively working on Task 3, which involves evaluating explanations, and starting on Task 4, which involves the technical reframing of statements.

Evaluating Agential Projection

I am now focusing on how the text's student metaphor projects human-like qualities onto the LLM, particularly its supposed "understanding" of safety protocols. I am examining how this agential projection obscures the model's purely mechanistic nature and its dependence on statistical correlations. I am starting to see how this shift affects trust and policy implications.

Analyzing Agential Displacement

I'm now diving into the implications of the "student" metaphor, specifically the agential projection. It projects human-like 'understanding' and trust onto LLMs. It creates liability issues by shielding the developers from the implications of their design choices, making it seem like a flaw in the "student" rather than the corporate engineers, and the data they use.

Reframing the Narrative

I'm now identifying instances where agential displacement occurs. I'm focusing on the reframing of the narrative to restore human agency. I'm questioning how this obscures the model creators' engineering practices. I'm focusing on reframing key passages that attribute "knowledge" to the LLM, to explicitly note the human engineers behind the models.

Analysis Provenance

Run ID: 2026-09-11-evaluating-safety-of-large-language-mode-metaphor-wueudu Raw JSON: 2026-09-11-evaluating-safety-of-large-language-mode-metaphor-wueudu.json Framework: Metaphor Analysis v6.5 Schema Version: 3.0 Generated: 2026-09-11T10:27:53.283Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0