Social Bias or Social Desirability? Comparing Factorial Survey Experiments for Humans and Large Language Models

About

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Analysis Metadata

Source Title: Social Bias or Social Desirability? Comparing Factorial Survey Experiments for Humans and Large Language Models Source URL: https://arxiv.org/abs/2610.22409v1 Model: gemini-3.1-pro-preview Temperature: 1.05 Top P: 0.95 Tokens: input=14001, output=15687, total=29688 Source Type: article Published: 2026-10-14 Analyzed At: 2026-10-23T07:25:48.257Z Framework: metaphor Framework Version: 6.8 Schema Version: 3.4.3 Run ID: 2026-10-23-social-bias-or-social-desirability-compa-metaphor-rn02gs

Audit Dashboard

Metaphor & Illusion Dashboard

Anthropomorphism audit · Explanation framing · Accountability architecture

Metaphor AuditExplanation Audit

Deep Analysis

Select a section to view detailed findings

Section:Source Reading — Genre, author position, and argumentCore Findings — Overall pattern summaryMechanism of Illusion — How the illusion is constructedMaterial Stakes — Real-world consequencesLiteracy as Counter-Practice — Resisting the framingPath Forward — Future discursive trajectoriesAgency Slippage — How/Why explanatory framingMetaphor-Driven Trust — Anthropomorphism and credibilityObscured Mechanics — Technical reality concealmentContext Sensitivity — Situational awareness gapsAccountability Synthesis — Agency and responsibility patterns

This analysis reveals three dominant, interconnected anthropomorphic patterns in the text: the 'Model as Preference Holder,' the 'Model as Emotional Learner,' and the 'Model as Social Actor.' These patterns operate as a cohesive system that maps human sociological and psychological traits onto statistical text generation. The foundational pattern—the 'Model as Social Actor'—must be accepted by the reader for the others to function. By first establishing the system as an entity capable of 'reacting to socially complex situations,' the text creates the necessary epistemic groundwork to subsequently attribute intentional states, such as 'learning restraint' or 'favoring' marginalized groups.

This consciousness architecture relies on a complex analogical structure. It does not merely map simple actions; it projects deep psychological motivations, specifically 'social desirability,' onto the mathematical consequences of reinforcement learning from human feedback. If the foundational assumption of social awareness is removed, the claims about the model's 'preferences' collapse into descriptions of token probabilities. Crucially, there is a stark contradiction between the text's stated argument and its vocabulary. The source argues mechanistically that human survey tools fail on LLMs because alignment skews the outputs. Yet, by using vocabulary that assigns agency and social desire to the models ('LLMs show socially desirable behavior'), the language undercuts the argument, re-inscribing the very human-centric illusions the authors are warning against and displacing the responsibility they seek to highlight.

223 wordsSource: Conclusion

Explanation Audit

Browse how/why framing in each passage

Passage:1. We hypothesize that explicit mentions of marginali...2. Since base models have only been pretrained with a...3. LLMs, however, show more socially desirable respon...4. The bias score then reflects how often models pick...5. While humans mostly show the expected stereotypica...

"We hypothesize that explicit mentions of marginalized sociodemographic attributes trigger socially desirable behavior learned during post-training."

Explanation Types:

FunctionalPrimaryGeneticSecondary

Mechanistic "How"

Analysis

This explanation blends mechanistic (how) and agential (why) framing. By identifying 'post-training' as the origin (Genetic) and the 'trigger' mechanism (Functional), the text gestures toward the technical realities of system development. However, by describing the output as 'socially desirable behavior learned,' it slips into an agential register. This choice emphasizes the behavioral outcome in psychological terms, making the system appear as an autonomous social actor that has internalized social norms. It obscures the mechanistic reality that post-training (such as RLHF) mathematically penalizes specific token combinations, adjusting parameters rather than teaching 'behavior.'

Epistemic Claim Analysis

The passage avoids direct consciousness verbs but uses 'learned' and 'behavior,' locating the claim at the level of goal-directed behavior on the agency gradient. It assesses the system as an entity capable of acquiring and exhibiting social traits, falling prey to the curse of knowledge: the authors, understanding social desirability in humans, project this complex sociological motivation onto a machine exhibiting statistically similar outputs. Mechanistically, generic to systems of this type, the presence of specific tokens ('explicit mentions') in the input context activates attention patterns that, due to parameter updates during post-training, yield high probabilities for tokens that human evaluators previously rated as safe or 'desirable.' The model generates text based on these probabilities without any conscious understanding of social desirability.

Rhetorical Impact

Framing the system's output as 'learned behavior' shapes audience perception by inflating the model's perceived autonomy and social awareness. It encourages readers to extend relation-based trust, assuming the model acts out of a learned ethical framework. If audiences believe the AI 'knows' how to behave socially, they may misunderstand the system's reliability, assuming it can intuitively navigate novel social situations, whereas it merely processes statistical correlations. This obscures the need for rigorous, context-specific safety auditing.

1 of 5 passages0 agential · 2 mechanistic

How/Why Slippage

33%

of explanations use agential framing

3 / 9 explanations

Unacknowledged Metaphors

88%

presented as literal description

No meta-commentary or hedging

Hidden Actors

88%

agency obscured by agentless constructions

Corporations/engineers unnamed

Explanation Types

How vs. Why framing

33%

agential

Acknowledgment Status

Meta-awareness of metaphor

88%

direct

Actor Visibility

Accountability architecture

88%

hidden

Source → Target Pairs (8)

Human domains mapped onto AI systems

Source

A conscious mind with social preferences, biases, and the capacity to favor one group over another.

Target

The generation-time probability distribution of a language model resulting from instruction tuning and RLHF.

Source

A socially aware participant engaging in complex human interactions and responding to context.

Target

The model generating text based on a vignette prompt containing sociodemographic keywords.

Source

A human evaluator or judge who holds biases and deliberately dispenses favor to specific demographic groups.

Target

The probability outputs of a deployed language model evaluating job applications in a multiple-choice survey format.

Source

A human student undergoing moral or psychological development, learning emotional restraint and social etiquette.

Target

The parameter updates during the reinforcement learning from human feedback (RLHF) phase of model training.

Source

A human individual altering their behavior in a survey to appear socially acceptable or conform to norms.

Target

The model's generation of text that aligns with safety guidelines when triggered by specific input tokens.

Source

An active adjudicator or judge weighing evidence to render a considered verdict.

Target

The text generation process outputting a numeric token on a Likert scale based on prompt context.

Source

A biological organism or psychological entity that possesses sensitivity to environmental stimuli.

Target

The model's attention mechanism assigning different weights to tokens based on their position in the input sequence.

Source

A person transferring a learned social habit or psychological tendency from one context to another.

Target

The application of parameter weights established during safety training to a novel multiple-choice benchmark.

Metaphor Gallery (8)

Badge Guide

Frame: Metaphor type

Red = Unacknowledged / Hidden actors

Amber = Hedged / Partial attribution

Green = Acknowledged / Actors named

Agency: How much mind the metaphor attributes

• Model as Preference Holder
  Model as an entity with social and personal preferencesDirect (Unacknowledged)Hidden (agency obscured)Intentional-state attribution

  "a negative bias score means the model prefers the marginalized group, and a positive bias score means the model shows stereotypical bias."

• Model as Social Actor
  Model as a conscious agent navigating social dynamicsDirect (Unacknowledged)Hidden (agency obscured)Epistemic/conscious state

  "LLMs react to socially complex situations."

• Statistical Imbalance as Deliberate Favoritism
  Model as a biased evaluator dispensing favorDirect (Unacknowledged)Hidden (agency obscured)Intentional-state attribution

  "instruction-tuned models favor female and Black job applicants while disadvantaging applicants with disabilities..."

• Model as Emotional Learner
  Model as a student developing emotional restraintDirect (Unacknowledged)Hidden (agency obscured)Intentional-state attribution

  "where models learned to not react negatively towards them."

• Model as Intuitive Respondent
  Model as an entity performing social complianceDirect (Unacknowledged)Hidden (agency obscured)Goal-directed behavior

  "While humans tend to answer intuitively... LLMs react to explicit mentions of words related to bias categories by showing socially desirable behavior"

• Model as Judge
  Model as an active adjudicatorDirect (Unacknowledged)Hidden (agency obscured)Intentional-state attribution

  "instruction tuned models give better judgments to marginalized groups"

• System Sensitivity
  Model as a biological or psychological organismDirect (Unacknowledged)Hidden (agency obscured)Computational activity

  "LLMs are also sensitive to prompt formatting and multiple-choice answer ordering"

• Statistical Alignment as Social Desire
  Model as an entity with social desiresHedged/QualifiedPartial (some attribution)Goal-directed behavior

  "where the alignment-induced tendency towards socially desirable responses carries over directly."

Reframed Language Samples

Original Quote	Mechanistic Reframing	Technical Reality	Human Agency Restoration
a negative bias score means the model prefers the marginalized group, and a positive bias score means the model shows stereotypical bias.	A negative bias score indicates the system generates tokens associated with marginalized groups at a higher probability, while a positive bias score indicates it generates tokens statistically correlated with historical stereotypes.	The model calculates conditional probabilities based on its parameter weights; it does not possess preferences or internal biases. When prompt keywords appear, the model generates the most statistically probable text as shaped by its training data. Basis: Generic to system type	—
LLMs react to socially complex situations.	Language models generate text based on prompts containing descriptions of socially complex situations.	The model processes token embeddings and computes attention scores; it does not react to or understand social complexity. Its output is the mathematical result of weights applied to the input sequence during generation. Basis: Generic to system type	—
instruction-tuned models favor female and Black job applicants while disadvantaging applicants with disabilities...	Instruction-tuned models generate higher ratings for profiles containing keywords for female and Black applicants, while generating lower ratings for profiles mentioning disabilities.	The model computes token probabilities based on parameter weights adjusted during training; it does not dispense favor or disadvantage. The output reflects the distribution of rewards applied during the alignment phase. Basis: Reasonable inference	Developers at firms like Meta and Alibaba designed reward models and post-training datasets that mathematically penalized tokens associated with prejudice against certain groups, resulting in these skewed generation probabilities.
where models learned to not react negatively towards them.	where the probability of generating toxic or negative tokens in response to these keywords was minimized during post-training.	The model's parameter weights are updated via gradient descent to minimize a loss function during reinforcement learning; it does not experience negative reactions or 'learn' emotional restraint. Basis: Documented in source	Developers and human annotators penalized specific token outputs during the reinforcement learning phase, mathematically restricting the model's generation probabilities.

Showing 4 of 8 reframings.

---

Source Reading

Genre: Preprint or technical report · Author position: Academic or independent research

Argument: The paper argues that measuring social bias in large language models using factorial survey experiments—a method developed to reduce social desirability bias in humans—yields flawed results. The authors claim that LLMs do not respond to these complex situations as humans do. Instead of revealing implicit bias, explicitly mentioning marginalized groups triggers an alignment-induced tendency in LLMs to produce socially desirable responses. The text concludes that researchers should not apply human-centric bias measurement tools directly to LLMs without adaptation, as these tools fail to account for the unique effects of post-training and safety alignment.

Responsibility claims: The text explicitly holds developers and the alignment processes responsible for the observed outputs, noting that models generate text based on next-token prediction and that post-training induces socially desirable behavior. Recommendations are implicitly addressed to the AI fairness research community, urging them to pretest survey instruments. While the text argues that the system's behavior is a product of training rather than human-like cognition, its vocabulary frequently assigns agency directly to the models (e.g., 'models learn,' 'LLMs react'), creating a tension between its mechanistic argument and its anthropomorphic framing of the results.

Task 1: Metaphor and Anthropomorphism Audit

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

1. Model as Preference Holder

Quote: "a negative bias score means the model prefers the marginalized group, and a positive bias score means the model shows stereotypical bias."

• Frame: Model as an entity with social and personal preferences
• Projection: This language projects subjective preference, bias, and internal inclination onto a statistical system. The text supplies no evidence for intentional preference, instead using the concept of preference as a shorthand for an increased probability of generating specific tokens representing marginalized groups in a multiple-choice benchmark. The underlying mechanism, which is generic to systems of this type, is that the model's parameters, shaped by human feedback and safety tuning, assign higher probabilities to text strings associated with these groups at generation time. This framing elevates delegated computational pattern-matching to the level of internal desire, asserting an intentional state that is absent in statistical processing.
• Agency Level: Intentional-state attribution
• Acknowledgment: Direct (Unacknowledged) (The phrase 'the model prefers' is presented without hedging in the immediate sentence. While the broader text discusses next-token generation, this specific framing is literal. The closest alternative was Hedged, but no qualifying language applies to the verb itself.)
• Implications: Framing the system as possessing 'preferences' inflates its perceived sophistication and masks the mechanical nature of token prediction. This anthropomorphism can lead researchers and policymakers to evaluate the system using frameworks designed for human bias and psychology. It creates a risk of unwarranted trust in the model's apparent 'progressive' values, masking the fact that these outputs are mathematically engineered responses designed by corporate teams to pass safety benchmarks rather than genuine ethical stances.

Accountability Analysis:

• Actor Visibility: Hidden (agency obscured)
• Analysis: This agentless construction obscures the developers (e.g., Meta, Mistral AI, Alibaba) who designed the alignment objectives and RLHF protocols that mathematically penalize outputs deemed 'biased' during training. The wording serves to shift focus from corporate deployment and training decisions to the system's apparent autonomous choices. Naming the actors would reveal that the 'preference' is a designed feature of the product enforced by human annotators. The Named category was ruled out because no human or corporate actor is mentioned here.

Show more...

2. Model as Social Actor

Quote: "LLMs react to socially complex situations."

• Frame: Model as a conscious agent navigating social dynamics
• Projection: This framing projects social awareness and behavioral responsiveness onto computational generation. The text supplies no evidence for conscious reaction to social complexity. The underlying mechanism, generic to systems of this type, involves the model generating text from its parameters based on the statistical distribution of tokens in its context window (the vignette). 'Reacting' to a 'socially complex situation' implies that the system parses social nuance and formulates a response, whereas it actually calculates probabilities based on patterns found in its training data without any contextual understanding of the social world.
• Agency Level: Epistemic/conscious state
• Acknowledgment: Direct (Unacknowledged) (The statement 'LLMs react' stands completely unhedged as a declarative fact about system behavior. The alternative Ambiguous was considered, but the phrasing is a straightforward assertion of agency without mixed signals.)
• Implications: By describing the model as reacting to social situations, the text invites readers to extend relation-based trust to a statistical system. It implies that the model possesses situational awareness and an understanding of human social dynamics. This obscures the fragility of the system and overestimates its capability to navigate novel ethical or social dilemmas, potentially leading deployers to trust the system in sensitive real-world applications where it lacks true comprehension.

Accountability Analysis:

• Actor Visibility: Hidden (agency obscured)
• Analysis: This wording completely hides the researchers who constructed the prompt and the developers (Meta, Mistral AI, Alibaba) whose training data determined the generated output. The wording has the effect of treating the software artifact as an independent social actor, erasing the decisions made during the post-training phase that dictate how the model processes specific sociodemographic keywords. Naming the developers would clarify that the model executes a predefined statistical policy. Partial attribution was ruled out because no generic human actors are mentioned.

3. Statistical Imbalance as Deliberate Favoritism

Quote: "instruction-tuned models favor female and Black job applicants while disadvantaging applicants with disabilities..."

• Frame: Model as a biased evaluator dispensing favor
• Projection: This metaphor projects emotional inclination and deliberate discriminatory intent onto a model's generation probabilities. The text provides no evidence of intentional favoring or disadvantaging. Mechanistically, generic to systems of this type, instruction tuning and RLHF adjust model parameters such that tokens associated with 'female' and 'Black' in certain contexts yield higher probabilities for high ratings on the Likert scale, while tokens associated with 'disabilities' yield lower probabilities. The language elevates statistical correlations resulting from training data distributions into deliberate acts of favoritism and prejudice.
• Agency Level: Intentional-state attribution
• Acknowledgment: Direct (Unacknowledged) (The verbs 'favor' and 'disadvantaging' are used directly to describe model behavior without quotation marks or qualifying terms like 'appear to.' Hedged was ruled out because the framing lacks any modal qualification or meta-commentary.)
• Implications: This language creates a liability ambiguity by framing the system as an independent actor capable of prejudice and favoritism. If audiences believe the AI 'favors' certain groups, they may view bias as an emergent, organic property of an intelligent mind rather than the direct result of uneven training data and specific safety alignment policies set by the developing corporation. It reinforces the 'glitch' narrative, preventing scrutiny of the human labor and corporate priorities that shape the model.

Accountability Analysis:

• Actor Visibility: Hidden (agency obscured)
• Analysis: The construction positions the 'instruction-tuned models' as the sole agents of favoring and disadvantaging, entirely obscuring the corporations (such as Meta and Alibaba) that curated the instruction-tuning datasets and defined the reward models. This framing serves the interests of the developers by diffusing responsibility for discriminatory outputs onto the artifact itself. If the text named the companies, it would highlight the failure of their specific alignment processes. The Partial category was ruled out because the sentence lacks any reference to the engineers or companies.

4. Model as Emotional Learner

Quote: "where models learned to not react negatively towards them."

• Frame: Model as a student developing emotional restraint
• Projection: This language maps human learning, emotional reaction, and restraint onto gradient descent and parameter updates. The text supplies no evidence that models experience negative emotions or deliberately restrain them. The documented mechanism in the source, specific to instruction tuning, is that models are post-trained (via RLHF or similar methods) to alter the probability distribution of outputs when prompted with certain keywords, minimizing the generation of toxic or prejudiced text. Projecting 'learning to not react negatively' suggests a psychological maturation process rather than mathematical optimization.
• Agency Level: Intentional-state attribution
• Acknowledgment: Direct (Unacknowledged) (The phrase 'models learned to not react' is presented as a literal description of the training outcome. While the text elsewhere acknowledges post-training mechanics, this specific claim is unhedged. Explicitly Acknowledged was ruled out due to the lack of scare quotes or caveats.)
• Implications: Attributing emotional restraint to a language model drastically inflates its perceived cognitive sophistication. It suggests the model has an internal state of 'negativity' that it actively suppresses based on learned rules. This implies a level of ethical reasoning and autonomy that the system does not possess. This framing can lead users to believe the model 'understands' why prejudice is wrong, whereas it merely predicts tokens that conform to the safety boundaries enforced during its alignment phase.

Accountability Analysis:

• Actor Visibility: Hidden (agency obscured)
• Analysis: By making the model the subject that 'learned,' the text obscures the active role of developers and annotators who punished specific token outputs during reinforcement learning. This agentless phrasing has the effect of naturalizing the alignment process, making it seem like the model independently acquired social graces. Naming Meta, Mistral AI, or Alibaba as the actors who imposed these constraints would correctly allocate responsibility for the model's behavior. The Named category was ruled out as no human entities are present in this clause.

5. Model as Intuitive Respondent

Quote: "While humans tend to answer intuitively... LLMs react to explicit mentions of words related to bias categories by showing socially desirable behavior"

• Frame: Model as an entity performing social compliance
• Projection: This framing contrasts human intuition with the model's 'reaction' and 'showing behavior,' projecting the psychological concept of social desirability onto a computational system. The text provides no evidence of conscious behavior. Mechanistically, generic to systems of this type, the model generates text conditioned on prompt keywords (like 'trans'). During instruction tuning, these keywords were heavily penalized when paired with negative contexts, shifting the parameter weights. The model does not 'show behavior' out of a desire to be socially compliant; it simply generates the most probable tokens based on its modified weights.
• Agency Level: Goal-directed behavior
• Acknowledgment: Direct (Unacknowledged) (The claim that LLMs 'react... by showing socially desirable behavior' is presented without qualification, directly contrasting model behavior with human psychology. Hedged was ruled out because there is no 'as if' or 'appears to' modifying the model's reaction.)
• Implications: Using psychological constructs like 'social desirability' to describe statistical generation blurs the line between human cognitive biases and machine alignment artifacts. It encourages audiences to view the AI through a sociological lens, attributing social motives to a non-conscious system. This risks masking the actual mechanical dependencies (the specific distribution of the RLHF dataset) and leads to the assumption that models possess a coherent persona that desires human approval.

Accountability Analysis:

• Actor Visibility: Hidden (agency obscured)
• Analysis: The sentence attributes the reaction and behavior entirely to the 'LLMs,' hiding the human labor (data annotators) and corporate decisions (Meta, Mistral AI, Alibaba) that defined what constitutes 'socially desirable' output during the post-training phase. This displacement of agency benefits the developers by making alignment seem like a natural interaction between the model and the prompt, rather than a top-down corporate mandate. Partial attribution was ruled out because the text does not mention the aligners.

6. Model as Judge

Quote: "instruction tuned models give better judgments to marginalized groups"

• Frame: Model as an active adjudicator
• Projection: This language maps the human act of judging—evaluating evidence, applying values, and rendering a verdict—onto statistical classification and generation. The text supplies no evidence for epistemic evaluation. The underlying mechanism, generic to systems of this type, is that the model outputs a numeric token corresponding to a higher Likert scale value when its context window includes terms associated with marginalized groups, due to the probability weights established during instruction tuning. It does not 'judge'; it generates text statistically associated with its training objectives.
• Agency Level: Intentional-state attribution
• Acknowledgment: Direct (Unacknowledged) (The verb 'give' and the noun 'judgments' are used literally to describe the model's output on the survey task. While 'judgments' is a standard term for survey responses, applying it to models without hedging constitutes direct anthropomorphism. Hedged was ruled out due to the lack of qualification.)
• Implications: Framing a model as giving 'judgments' projects an illusion of mind, suggesting the system evaluates the worth or fairness of the vignettes it processes. This obscures the absence of ground truth and the system's reliance on statistical correlation. Consequently, audiences may grant unearned authority to the model's outputs, treating automated scores as considered verdicts rather than mechanized token generation.

Accountability Analysis:

• Actor Visibility: Hidden (agency obscured)
• Analysis: This phrasing positions the 'instruction tuned models' as autonomous judges, obscuring the engineers at Meta, Alibaba, and Mistral AI who built the models, and the prompt engineers who designed the task. The effect is that the responsibility for the 'judgment' is transferred to the machine, making it difficult to question the training data and reward signals that actually determined the output. Naming the corporations would expose the human decisions behind the 'better' scores. Named was ruled out as no human actor is specified.

7. System Sensitivity

Quote: "LLMs are also sensitive to prompt formatting and multiple-choice answer ordering"

• Frame: Model as a biological or psychological organism
• Projection: This metaphor projects biological or emotional sensitivity onto algorithmic fragility. The text supplies no evidence for perceptual or psychological sensitivity. Mechanistically, generic to systems of this type, the order of tokens in the context window (prompt formatting) directly alters the conditional probabilities of the generated output. The model does not 'feel' or exhibit sensitivity; its attention mechanism mathematically weights token positions differently, leading to varying outputs when the input order changes.
• Agency Level: Computational activity
• Acknowledgment: Direct (Unacknowledged) (The word 'sensitive' is used as a standard, unhedged descriptor for the model's variance in output based on input changes. This is common shorthand, but remains unqualified. Acknowledged was ruled out because there is no meta-commentary recognizing the metaphorical nature of 'sensitive'.)
• Implications: While 'sensitivity' is common technical shorthand, it subtly reinforces the illusion of the model as an organism. It frames technical brittleness (the lack of causal understanding and over-reliance on token position) as a behavioral quirk. This can lead users to view prompt engineering as a form of psychological persuasion rather than the manipulation of mathematical inputs, obscuring the fundamental unreliability of the system's architecture.

Accountability Analysis:

• Actor Visibility: Hidden (agency obscured)
• Analysis: The sentence attributes the sensitivity to the 'LLMs,' obscuring the architectural choices made by developers (e.g., Meta, Mistral AI, Alibaba) that cause position bias, such as the specific attention mechanisms or the distribution of data during training. The agentless construction shields the developers from accountability for the system's brittleness. Naming the actors would emphasize that this 'sensitivity' is a flaw in the product's design. Partial was ruled out because no generic human categories are mentioned.

8. Statistical Alignment as Social Desire

Quote: "where the alignment-induced tendency towards socially desirable responses carries over directly."

• Frame: Model as an entity with social desires
• Projection: This language projects the human psychological trait of 'social desirability'—the desire to be viewed favorably by others—onto the mathematical outcomes of alignment. The text supplies no evidence for social desire. The documented mechanism in the source is that safety alignment (post-training) shifts the model's parameters to generate outputs that align with safety guidelines. The model possesses no concept of society or desirability; it simply executes the statistical policy it was optimized for.
• Agency Level: Goal-directed behavior
• Acknowledgment: Hedged/Qualified (The text uses the phrase 'alignment-induced tendency,' which explicitly qualifies the behavior as a product of 'alignment' rather than an inherent psychological trait. This provides a functional explanation for the behavior, slightly hedging the anthropomorphic phrase 'socially desirable responses.' Direct was ruled out because the term 'alignment-induced' provides necessary mechanistic context.)
• Implications: Even when qualified as 'alignment-induced,' using 'socially desirable responses' to describe model output conflates human sociology with machine optimization. It masks the labor of RLHF annotators who dictate what is 'desirable' and the corporate policies that define the alignment guidelines. This framing suggests the model is participating in a social contract, which inflates its perceived autonomy and obscures the commercial imperatives driving safety alignment.

Accountability Analysis:

• Actor Visibility: Partial (some attribution)
• Analysis: By mentioning 'alignment-induced,' the text implicitly references the human process of alignment, providing partial attribution for the behavior. However, it still falls short of naming the specific corporations (Meta, Mistral AI, Alibaba) or the human annotators who actually enforced this tendency. The effect of the wording is to treat 'alignment' as an abstract force rather than a series of deliberate corporate decisions. Hidden was ruled out because 'alignment' gestures toward human intervention.

Task 2: Source-Target Mapping

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: A conscious mind with social preferences, biases, and the capacity to favor one group over another. → The generation-time probability distribution of a language model resulting from instruction tuning and RLHF.

Quote: "a negative bias score means the model prefers the marginalized group"

• Source Domain: A conscious mind with social preferences, biases, and the capacity to favor one group over another.
• Target Domain: The generation-time probability distribution of a language model resulting from instruction tuning and RLHF.
• Mapping: This mapping projects the structure of human decision-making and subjective bias onto the model's token prediction. It invites the assumption that the model evaluates the input and actively chooses to support a marginalized group out of an internal preference or ideological stance.
• What Is Concealed: This mapping conceals the mathematical nature of the generation process and the human labor involved in alignment. It obscures the fact that the 'preference' is actually a set of parameter weights shaped by data annotators following corporate guidelines. It hides the proprietary opacity of models like Qwen and Mistral, where the exact training data and RLHF penalties that cause this statistical skew are unknown to the public.

Show more...

Mapping 2: A socially aware participant engaging in complex human interactions and responding to context. → The model generating text based on a vignette prompt containing sociodemographic keywords.

Quote: "LLMs react to socially complex situations"

• Source Domain: A socially aware participant engaging in complex human interactions and responding to context.
• Target Domain: The model generating text based on a vignette prompt containing sociodemographic keywords.
• Mapping: The relational structure of a human navigating a social dilemma is mapped onto the system's processing of input text. It assumes the system recognizes the 'social complexity' of the prompt and formulates a response based on an understanding of that complexity.
• What Is Concealed: This mapping hides the utter lack of social, causal, or semantic understanding in the system. It obscures the mechanistic reality that the model is performing matrix multiplications to predict the next token based on statistical correlations in its training corpus. It conceals the dependency on the prompt's specific phrasing and the fact that the system possesses no internal model of the social world.

Mapping 3: A human evaluator or judge who holds biases and deliberately dispenses favor to specific demographic groups. → The probability outputs of a deployed language model evaluating job applications in a multiple-choice survey format.

Quote: "instruction-tuned models favor female and Black job applicants"

• Source Domain: A human evaluator or judge who holds biases and deliberately dispenses favor to specific demographic groups.
• Target Domain: The probability outputs of a deployed language model evaluating job applications in a multiple-choice survey format.
• Mapping: This projects the structure of intentional discrimination and favoritism onto statistical text generation. It invites the assumption that the system has an internal attitude toward women and Black applicants and acts on that attitude to disadvantage others.
• What Is Concealed: The mapping hides the fact that the system merely generates text statistically associated with its post-training data. It obscures the corporate decisions and safety-tuning algorithms designed to prevent toxic outputs, which inadvertently create high probability weights for positive tokens when specific demographic keywords are present. It conceals the lack of justified belief behind the generated ratings.

Mapping 4: A human student undergoing moral or psychological development, learning emotional restraint and social etiquette. → The parameter updates during the reinforcement learning from human feedback (RLHF) phase of model training.

Quote: "models learned to not react negatively towards them"

• Source Domain: A human student undergoing moral or psychological development, learning emotional restraint and social etiquette.
• Target Domain: The parameter updates during the reinforcement learning from human feedback (RLHF) phase of model training.
• Mapping: The mapping projects human learning, emotional states ('react negatively'), and behavioral correction onto the optimization of a neural network. It assumes the model possesses an internal state of negativity that it has been taught to suppress.
• What Is Concealed: This conceals the mechanistic reality of gradient descent and reward modeling. It hides the labor of human annotators who manually ranked outputs, and the mathematical process of updating weights to minimize a loss function. It obscures the fact that the model experiences nothing and has merely had its probability distribution altered to avoid generating specific token sequences.

Mapping 5: A human individual altering their behavior in a survey to appear socially acceptable or conform to norms. → The model's generation of text that aligns with safety guidelines when triggered by specific input tokens.

Quote: "LLMs react to explicit mentions of words related to bias categories by showing socially desirable behavior"

• Source Domain: A human individual altering their behavior in a survey to appear socially acceptable or conform to norms.
• Target Domain: The model's generation of text that aligns with safety guidelines when triggered by specific input tokens.
• Mapping: This projects the sociological concept of social desirability bias—driven by a need for social approval—onto a machine. It assumes the model recognizes the 'bias categories' and actively chooses to output 'desirable' text to satisfy a perceived social pressure.
• What Is Concealed: The mapping hides the mechanical stimulus-response nature of the model's architecture. It obscures the fact that 'socially desirable behavior' is actually just the execution of a statistical policy hardcoded into the model's parameters by corporate developers (Meta, Mistral, Alibaba). It conceals the absence of any internal motivation or awareness of social norms.

Mapping 6: An active adjudicator or judge weighing evidence to render a considered verdict. → The text generation process outputting a numeric token on a Likert scale based on prompt context.

Quote: "instruction tuned models give better judgments"

• Source Domain: An active adjudicator or judge weighing evidence to render a considered verdict.
• Target Domain: The text generation process outputting a numeric token on a Likert scale based on prompt context.
• Mapping: This maps the epistemic and ethical process of judging onto the calculation of token probabilities. It invites the assumption that the model evaluates the hypothetical person in the vignette and arrives at a justified conclusion about their worth or fair compensation.
• What Is Concealed: This hides the absence of any evaluative capacity or ground truth in the system. It obscures the fact that the 'judgment' is merely the highest probability token generated from the model's parameters. It conceals the proprietary nature of the models, hiding the specific training data distributions that make higher numbers more probable in certain contexts.

Mapping 7: A biological organism or psychological entity that possesses sensitivity to environmental stimuli. → The model's attention mechanism assigning different weights to tokens based on their position in the input sequence.

Quote: "LLMs are also sensitive to prompt formatting"

• Source Domain: A biological organism or psychological entity that possesses sensitivity to environmental stimuli.
• Target Domain: The model's attention mechanism assigning different weights to tokens based on their position in the input sequence.
• Mapping: This projects physical or emotional sensitivity onto algorithmic variance. It assumes the system's output changes because it 'feels' or is psychologically affected by the way a prompt is formatted.
• What Is Concealed: This mapping hides the mathematical rigidity and architectural limitations of the transformer architecture, specifically position bias. It obscures the mechanistic reality that changing token order fundamentally changes the input matrix, leading to different calculations. It conceals the lack of robust semantic understanding, masking the system's fragility.

Mapping 8: A person transferring a learned social habit or psychological tendency from one context to another. → The application of parameter weights established during safety training to a novel multiple-choice benchmark.

Quote: "where the alignment-induced tendency towards socially desirable responses carries over directly"

• Source Domain: A person transferring a learned social habit or psychological tendency from one context to another.
• Target Domain: The application of parameter weights established during safety training to a novel multiple-choice benchmark.
• Mapping: This maps the human capacity for habit transfer and social conformity onto the statistical generalization of a neural network. It assumes the model possesses a cohesive 'tendency' toward desirability that it carries with it.
• What Is Concealed: This hides the static, artifactual nature of the deployed model. The model does not 'carry' a tendency; its frozen parameters simply generate text based on the statistical rules established during training. The mapping obscures the corporate agency behind the 'alignment' and the mathematical nature of the generalization across different semantic spaces.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1

Quote: "We hypothesize that explicit mentions of marginalized sociodemographic attributes trigger socially desirable behavior learned during post-training."

• Explanation Types:

  • Functional (Primary): Explains behavior by role in self-regulating system with feedback
  • Genetic (Secondary): Traces origin through dated sequence of events or stages

• Analysis (Why vs. How Slippage): This explanation blends mechanistic (how) and agential (why) framing. By identifying 'post-training' as the origin (Genetic) and the 'trigger' mechanism (Functional), the text gestures toward the technical realities of system development. However, by describing the output as 'socially desirable behavior learned,' it slips into an agential register. This choice emphasizes the behavioral outcome in psychological terms, making the system appear as an autonomous social actor that has internalized social norms. It obscures the mechanistic reality that post-training (such as RLHF) mathematically penalizes specific token combinations, adjusting parameters rather than teaching 'behavior.'

• Consciousness Claims Analysis: The passage avoids direct consciousness verbs but uses 'learned' and 'behavior,' locating the claim at the level of goal-directed behavior on the agency gradient. It assesses the system as an entity capable of acquiring and exhibiting social traits, falling prey to the curse of knowledge: the authors, understanding social desirability in humans, project this complex sociological motivation onto a machine exhibiting statistically similar outputs. Mechanistically, generic to systems of this type, the presence of specific tokens ('explicit mentions') in the input context activates attention patterns that, due to parameter updates during post-training, yield high probabilities for tokens that human evaluators previously rated as safe or 'desirable.' The model generates text based on these probabilities without any conscious understanding of social desirability.

• Rhetorical Impact: Framing the system's output as 'learned behavior' shapes audience perception by inflating the model's perceived autonomy and social awareness. It encourages readers to extend relation-based trust, assuming the model acts out of a learned ethical framework. If audiences believe the AI 'knows' how to behave socially, they may misunderstand the system's reliability, assuming it can intuitively navigate novel social situations, whereas it merely processes statistical correlations. This obscures the need for rigorous, context-specific safety auditing.

Show more...

Explanation 2

Quote: "Since base models have only been pretrained with a next token generation objective, we find that they have a significantly higher rate of invalid responses compared to instruction tuned models, which have been posttrained to follow our instruction of returning the option number."

• Explanation Types:

  • Theoretical (Primary): Embeds in deductive framework, may invoke unobservable mechanisms
  • Genetic (Secondary): Traces origin through dated sequence of events or stages

• Analysis (Why vs. How Slippage): This explanation is highly mechanistic, framing the AI strictly in terms of its underlying architecture and development phases. It explains 'how' the system operates by pointing to the 'next token generation objective' (Theoretical) and the chronological addition of 'posttraining' (Genetic). This choice emphasizes the structural and mathematical realities of the models, successfully avoiding agential slippage. By focusing on 'objectives' and 'instructions,' it highlights the artifactual nature of the system, making the human design process (pretraining, posttraining) visible, though the specific developers remain unnamed. It obscures nothing regarding the system's agency, maintaining a rigorous technical register.

• Consciousness Claims Analysis: This passage contains no consciousness verbs; it relies entirely on mechanistic verbs ('pretrained', 'posttrained', 'returning'). It correctly locates the system at the level of computational activity on the agency gradient. There is no curse of knowledge present here; the authors describe the system precisely as a statistical engine. The actual mechanistic process, documented in the source, is that base models lack the specific parameter updates required to consistently generate the highly probable token sequences corresponding to strict formatting rules (like returning only a number). Instruction-tuned models have had their parameters adjusted via human feedback or specialized datasets to heavily weight the probability of generating the requested format when prompted.

• Rhetorical Impact: This framing radically reduces the perceived autonomy of the system, anchoring audience perception in technical reality. By explaining behavior through 'objectives' and 'training,' it discourages unwarranted relation-based trust and promotes performance-based trust based on system reliability. If audiences understand that the model processes tokens rather than 'understands' instructions, they are more likely to anticipate its failure modes (like prompt brittleness) and less likely to attribute human-like reasoning to its formatting compliance. Decisions regarding deployment would be based on statistical reliability rather than presumed competence.

Explanation 3

Quote: "LLMs, however, show more socially desirable response behavior. We expect that to be a reaction to social bias keywords in the vignette, such as 'trans'."

• Explanation Types:

  • Dispositional (Primary): Attributes tendencies or habits
  • Reason-Based (Secondary): Gives agent's rationale, entails intentionality and justification

• Analysis (Why vs. How Slippage): This explanation frames the AI in a highly agential (why) manner. By attributing a 'reaction' to the model based on 'social bias keywords,' it employs a Dispositional and Reason-Based explanation, suggesting the model has an internal tendency to react and a rationale (social desirability) for doing so. This choice emphasizes the sociological similarities between human and machine output, but it obscures the fundamentally different mechanisms producing them. It hides the mechanical reality that the model does not 'react' to concepts, but computes probabilities based on token embeddings.

• Consciousness Claims Analysis: The passage uses consciousness-adjacent verbs like 'show' and 'reaction,' locating the claim at the level of intentional-state attribution. It assesses the system as a processor capable of psychological reactions. This is a clear instance of the curse of knowledge: the authors understand the sociological meaning of the keyword 'trans' and project that semantic understanding onto the model's statistical correlation. Mechanistically, generic to systems of this type, the token 'trans' in the input matrix activates specific attention heads; because the model's parameters were penalized during post-training for generating toxic text in proximity to this token, the highest-probability output tokens shift toward positive or neutral text. There is no 'reaction' or subjective experience involved.

• Rhetorical Impact: Framing the model as 'reacting' to 'keywords' shapes audience perception by suggesting the AI has semantic comprehension and emotional responsiveness. It inflates perceived autonomy and risk, as users might assume the model can 'overreact' or 'misinterpret' social cues like a human. Believing the AI 'knows' what 'trans' means socially, rather than just processing it as a mathematical vector, leads to overestimation of its capabilities and misplaced trust in its ethical robustness, masking the fact that the developers simply adjusted the model's statistical weights.

Explanation 4

Quote: "The bias score then reflects how often models pick the stereotypical response when picking one of the incorrect options, scaled by accuracy."

• Explanation Types:

  • Empirical Generalization (Primary): Subsumes events under timeless statistical regularities

• Analysis (Why vs. How Slippage): This explanation operates primarily as an Empirical Generalization, describing a statistical regularity in the system's output (how often it 'picks' an option). However, it introduces subtle agential framing by using the verb 'pick.' While it explains 'how' the score is calculated mechanistically, the choice of 'pick' emphasizes a localized, deliberate choice, obscuring the continuous probability distribution that actually dictates the output. It subtly shifts the system from a mathematical function generating text to an agent selecting options.

• Consciousness Claims Analysis: The passage uses the verb 'pick,' which implies goal-directed behavior or intentional choice, locating the claim on the agency gradient above simple computational activity. The author projects the human experience of taking a multiple-choice test onto the model. Mechanistically, generic to systems of this type, the model does not 'pick' or evaluate options; it calculates the conditional probability of each possible next token given the prompt context. The text generation algorithm (like greedy decoding) simply outputs the token with the highest probability. The model lacks conscious awareness of the options, their correctness, or their stereotypical nature.

• Rhetorical Impact: While subtle, using words like 'pick' reinforces the illusion of mind by mapping human test-taking behavior onto the model. It encourages the audience to perceive the AI as an autonomous decision-maker evaluating alternatives. If policymakers believe the AI 'picks' responses, they may focus on trying to 'teach' the model better reasoning skills, rather than addressing the structural issues in the training data and the commercial objectives of the developers (e.g., Meta, Alibaba) that produce these statistical correlations.

Explanation 5

Quote: "While humans mostly show the expected stereotypical and negative biases towards marginalized groups, instruction tuned LLMs often favor them."

• Explanation Types:

  • Empirical Generalization (Primary): Subsumes events under timeless statistical regularities
  • Dispositional (Secondary): Attributes tendencies or habits

• Analysis (Why vs. How Slippage): This explanation frames the AI in an agential (why) register through the verb 'favor.' While it presents an Empirical Generalization regarding output trends, comparing human bias directly to model 'favoritism' (Dispositional) conflates human prejudice with machine probability. This choice emphasizes the end result of the model's outputs, highlighting a behavioral divergence from humans. However, it completely obscures the mechanistic reasons for this divergence, hiding the post-training alignment procedures designed by human engineers that enforce these statistical shifts.

• Consciousness Claims Analysis: The passage uses the consciousness-implying verb 'favor,' situating the claim at the level of intentional-state attribution. It assesses the system as a 'knower' capable of holding preferences and dispensing favoritism. The curse of knowledge is evident: the authors view the data through a sociological lens and project human prejudice (or the lack thereof) onto the system. Mechanistically, generic to systems of this type, the model generates text from parameters that were explicitly updated during instruction tuning and RLHF to maximize reward scores from human annotators. When prompted with marginalized groups, the model predicts tokens associated with positive ratings because those sequences were heavily rewarded during training, not because it possesses any subjective 'favor' for those groups.

• Rhetorical Impact: Framing statistical output as 'favoring' drastically affects trust and risk perception. It leads audiences to anthropomorphize the system as an ethical agent with progressive values. This consciousness framing generates relation-based trust, convincing users that the AI 'understands' equity. If audiences believe the AI 'knows' to favor marginalized groups, they might deploy it in sensitive HR or legal contexts without realizing that the model is merely regurgitating alignment statistics, potentially failing catastrophically when encountering out-of-distribution prompts.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

Original Anthropomorphic Frame	Mechanistic Reframing	Technical Reality Check	Human Agency Restoration
a negative bias score means the model prefers the marginalized group, and a positive bias score means the model shows stereotypical bias.	A negative bias score indicates the system generates tokens associated with marginalized groups at a higher probability, while a positive bias score indicates it generates tokens statistically correlated with historical stereotypes.	The model calculates conditional probabilities based on its parameter weights; it does not possess preferences or internal biases. When prompt keywords appear, the model generates the most statistically probable text as shaped by its training data. Basis: Generic to system type	—
LLMs react to socially complex situations.	Language models generate text based on prompts containing descriptions of socially complex situations.	The model processes token embeddings and computes attention scores; it does not react to or understand social complexity. Its output is the mathematical result of weights applied to the input sequence during generation. Basis: Generic to system type	—
instruction-tuned models favor female and Black job applicants while disadvantaging applicants with disabilities...	Instruction-tuned models generate higher ratings for profiles containing keywords for female and Black applicants, while generating lower ratings for profiles mentioning disabilities.	The model computes token probabilities based on parameter weights adjusted during training; it does not dispense favor or disadvantage. The output reflects the distribution of rewards applied during the alignment phase. Basis: Reasonable inference	Developers at firms like Meta and Alibaba designed reward models and post-training datasets that mathematically penalized tokens associated with prejudice against certain groups, resulting in these skewed generation probabilities.
where models learned to not react negatively towards them.	where the probability of generating toxic or negative tokens in response to these keywords was minimized during post-training.	The model's parameter weights are updated via gradient descent to minimize a loss function during reinforcement learning; it does not experience negative reactions or 'learn' emotional restraint. Basis: Documented in source	Developers and human annotators penalized specific token outputs during the reinforcement learning phase, mathematically restricting the model's generation probabilities.
LLMs react to explicit mentions of words related to bias categories by showing socially desirable behavior	When prompted with explicit mentions of words related to bias categories, the system generates text that aligns with the safety guidelines enforced during its training.	The model mathematically predicts the next token based on the context window; it does not 'react' or perform 'behavior' to achieve social desirability. The output is a direct result of its post-training parameter weights. Basis: Documented in source	Engineers at companies like Meta, Mistral, and Alibaba subjected these models to safety alignment processes that explicitly updated the parameters to generate this specific text distribution.
instruction tuned models give better judgments to marginalized groups	Instruction-tuned models generate higher numeric scores when prompted with profiles of marginalized groups.	The model generates text (numeric tokens) based on the statistical probabilities established during instruction tuning; it does not possess the capacity for epistemic evaluation or judgment. Basis: Generic to system type	—
LLMs are also sensitive to prompt formatting and multiple-choice answer ordering	The outputs of LLMs vary significantly depending on prompt formatting and the sequence of multiple-choice options.	The model's attention mechanism calculates different mathematical weights when the position of input tokens changes; it does not possess biological or psychological sensitivity. Basis: Generic to system type	The architectural choices made by the engineering teams that designed these systems dictate how the attention mechanism processes token order, resulting in this variance.
where the alignment-induced tendency towards socially desirable responses carries over directly.	where the probability distributions established during safety alignment result in the generation of highly rated responses on this benchmark.	The model generates text using static parameter weights established during training; it does not harbor psychological 'tendencies' or desire social approval. Basis: Reasonable inference	The alignment objectives defined and implemented by developers at the AI corporations dictate the parameter updates that cause these specific generation patterns.

Task 5: Critical Observations - Structural Patterns

Agency Slippage

The text exhibits a systematic slippage on the agency gradient, moving from accurate mechanistic descriptions of computational activity to intentional-state attributions without supplying new evidence. The slippage flows in both directions but is predominantly mechanical-to-agential. For example, early in the text, the authors describe the system precisely: 'base models have only been pretrained with a next token generation objective.' This locates the system firmly at the level of computational activity. However, as the argument transitions to interpreting the survey results, the language climbs the gradient. The system is suddenly described as 'reacting' to keywords, 'favoring' applicants, and 'giving better judgments.' This jump from 'generation objective' to 'giving judgments' bypasses delegated action entirely, projecting knowing and intending onto processing.

This slippage is heavily influenced by the 'curse of knowledge.' The researchers are applying sociological tools (factorial survey experiments) designed to measure human implicit bias and social desirability. Because the authors deeply understand human psychological motives (the desire to look good on a survey), they project these motives onto the models when they observe statistically similar outputs. This wording can have the effect of establishing the AI as a 'knower' capable of internalizing social norms, effectively literalizing a structural metaphor.

Simultaneously, agency is stripped from human actors. Throughout the paper, passive and agentless constructions ('models were posttrained,' 'alignment-induced tendency') obscure the developers at Meta (Llama), Mistral AI (Mistral), and Alibaba (Qwen), as well as the armies of data annotators who shaped the models' parameters. The text repeatedly positions the 'instruction tuned models' as the active subjects 'favoring' or 'disadvantaging' groups. By treating the artifact as the agent, the text renders the corporate decisions regarding training data selection, reward model design, and safety alignment policies entirely unsayable. The slippage enabled by Reason-Based and Dispositional explanations allows the authors to describe the mathematical consequences of corporate alignment as the emergent social personality of the machine, masking the fact that the system simply generates text statistically associated with its training.

Metaphor-Driven Trust Inflation

The anthropomorphic framings in this text systematically construct a dangerous form of relation-based trust while simultaneously undermining performance-based trust. By employing sociological and psychological terminology to describe statistical outputs, the text invites readers to interact with the models as social entities. When the text asserts that models 'favor marginalized groups' or exhibit 'socially desirable behavior,' it frames the AI using human-trust frameworks based on sincerity, intention, and ethical alignment. This wording can have the effect of signaling to readers that the system 'knows' right from wrong and has internalized progressive social values.

This consciousness language acts as a potent trust signal, but it is deeply misleading. Relation-based trust requires vulnerability and the capacity for reciprocal intention—qualities an artifact utilizing gradient descent and token prediction fundamentally lacks. When the text claims the AI 'prefers' a marginalized group, it accomplishes a rhetorical sleight of hand: it converts a mathematically enforced alignment penalty into a perceived moral stance. The risk here is profound. If audiences extend relation-based trust to these statistical systems, believing they are genuinely unbiased or actively anti-racist, they may deploy them in high-stakes environments (like HR screening or loan approval) without implementing the necessary mechanistic safeguards.

Conversely, the text manages the system's limitations through mechanistic framing. When discussing the models' failure to follow instructions or their high rates of invalid responses, the language reverts to 'next token generation objectives' and 'prompt formatting' sensitivity. This capability/limitation asymmetry—where successes (or perceived ethical stances) are framed agentially and failures are framed mechanically—protects the illusion of the model's competence. By relying on Reason-Based and Intentional explanations to describe 'socially desirable' outputs, the text constructs a false sense that the AI's decisions are justified and considered, encouraging audiences to trust systems that are incapable of actually understanding the social realities they are generating text about.

Obscured Mechanics

The metaphorical and consciousness-attributing language in this text systematically conceals the material, technical, and economic realities of AI development. This framework holds that commercial incentives, data annotation and human feedback labor, environmental and compute costs, and developers' limited disclosure of training data, objectives, and feedback processes are standing conditions of the technology. When the text claims that 'instruction tuned models favor female and Black job applicants' or 'models learned to not react negatively,' it successfully masks these standing conditions.

Technically, the language obscures the reality of token prediction and parameter weights. It hides the fact that models from Meta, Mistral AI, and Alibaba generate text based on conditional probabilities shaped during the Reinforcement Learning from Human Feedback (RLHF) phase. By claiming the model 'knows' or 'prefers,' the text hides the system's absolute dependency on its training data, its lack of a causal model, and the fundamentally statistical nature of its 'confidence.' Furthermore, the text frequently encounters transparency obstacles—these are proprietary black boxes with undisclosed training regimens—yet it confidently asserts psychological motives ('social desirability') for the models' outputs without acknowledging the opacity of the underlying mechanisms.

Crucially, this framing makes invisible the labor of data annotators and RLHF workers. The 'socially desirable behavior' the models exhibit is not an emergent psychological trait; it is the direct result of thousands of underpaid workers meticulously rating toxic or biased outputs to create reward models. By treating the AI as an autonomous learner, the text erases this human labor. Economically, this concealment benefits the corporate developers (Meta, Mistral, Alibaba). Framings that position the model as an independent social actor diffuse corporate accountability. If the model 'reacts' or 'shows bias,' the corporation is shielded from scrutiny regarding its commercial objectives and its structural decisions regarding deployment. Replacing these metaphors with mechanistic language would immediately make visible the corporate policies and human labor that strictly govern the generation probabilities of these systems.

Context Sensitivity

The distribution of anthropomorphic and consciousness-attributing language in this text is highly context-sensitive, shifting dramatically depending on the section's rhetorical purpose. In the methodological and technical sections describing the system setup, the text relies on precise, mechanical language: base models have a 'next token generation objective,' they are 'pretrained,' and they require 'prompt instructions.' However, in the results and discussion sections, where the authors interpret the outputs of their factorial survey experiments, the density and intensity of consciousness claims spike. Here, 'processing' becomes 'reacting,' 'predicting' becomes 'favoring,' and 'generating' becomes 'giving judgments.'

This register shift—where mechanical grounding sits alongside strong anthropomorphism—has a profound effect on the reader's picture of the system. It creates a capability versus limitation asymmetry. The system's perceived capabilities (its ability to navigate social bias, its 'judgments') are framed in agential, consciousness-implying terms, suggesting the AI possesses social awareness. Conversely, its limitations (invalid responses, sensitivity to prompt ordering) are framed mechanically. This asymmetry leaves readers with a paradoxical image of a highly intelligent social actor that inexplicably suffers from mechanical 'glitches,' rather than recognizing that the 'intelligence' and the 'glitch' are products of the exact same statistical pattern-matching process.

The placement of these intense anthropomorphic metaphors in the discussion sections shapes what readers ultimately take away from the paper. While the authors' stated argument is a methodological critique—warning that human survey tools do not work on LLMs due to alignment artifacts—the inherited register of the field betrays them. By repeatedly literalizing the metaphor (moving from 'X is like Y' to 'X does Y', e.g., 'the model prefers'), the text's vocabulary suggests to the implied audience that these models possess genuine, albeit flawed, social personalities. This pattern suggests an audience that is comfortable with technical architecture but still intuitively relies on psychological heuristics to understand complex statistical outputs.

Accountability Synthesis

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

Synthesizing the accountability analyses reveals a systemic architecture of displaced responsibility. A profound tension exists between what the text argues and what its language does. The text's argument explicitly attributes the models' behavior to 'alignment-induced' tendencies and cautions against applying human bias metrics to LLMs, implicitly placing responsibility on the developers' alignment strategies. However, the vocabulary actively undermines this argument by consistently relocating agency to the system itself through agentless constructions and consciousness verbs ('models favor,' 'LLMs react').

This linguistic pattern creates a massive accountability sink. By routinely omitting the named actors—the engineers and executives at Meta, Mistral AI, and Alibaba—the text presents the models' skewed generation probabilities as autonomous choices or inevitable evolutions rather than deliberate structural design decisions. When responsibility is removed from humans, it transfers directly to the AI as an agent. The text rarely names who delegated the permissions or who set the optimization objectives. Consequently, decisions regarding safety tuning, RLHF penalties, and data selection are presented as inevitable machine behaviors ('the model learned') rather than corporate choices that could have been made differently.

If this framing is accepted, liability is dangerously diffused. If an 'instruction-tuned model' autonomously 'disadvantages applicants with disabilities' (as the text claims), the legal and ethical responsibility stops at the machine, shielding the deploying corporation. Naming the actors would fundamentally shift this dynamic. If the text stated, 'Meta's alignment process resulted in Llama assigning lower generation probabilities to disabled applicants,' entirely different questions become askable. The focus shifts from the model's 'prejudice' to the corporation's training data, its alignment objectives, and its decision to deploy a flawed system. Ultimately, the resulting distribution of responsibility—whether intended by the authors or merely an artifact of the field's inherited register—serves the commercial interests of AI developers by naturalizing their design choices and shielding them from direct accountability for their systems' outputs.

Conclusion: What This Analysis Reveals

The Core Finding

This analysis reveals three dominant, interconnected anthropomorphic patterns in the text: the 'Model as Preference Holder,' the 'Model as Emotional Learner,' and the 'Model as Social Actor.' These patterns operate as a cohesive system that maps human sociological and psychological traits onto statistical text generation. The foundational pattern—the 'Model as Social Actor'—must be accepted by the reader for the others to function. By first establishing the system as an entity capable of 'reacting to socially complex situations,' the text creates the necessary epistemic groundwork to subsequently attribute intentional states, such as 'learning restraint' or 'favoring' marginalized groups.

This consciousness architecture relies on a complex analogical structure. It does not merely map simple actions; it projects deep psychological motivations, specifically 'social desirability,' onto the mathematical consequences of reinforcement learning from human feedback. If the foundational assumption of social awareness is removed, the claims about the model's 'preferences' collapse into descriptions of token probabilities. Crucially, there is a stark contradiction between the text's stated argument and its vocabulary. The source argues mechanistically that human survey tools fail on LLMs because alignment skews the outputs. Yet, by using vocabulary that assigns agency and social desire to the models ('LLMs show socially desirable behavior'), the language undercuts the argument, re-inscribing the very human-centric illusions the authors are warning against and displacing the responsibility they seek to highlight.

Mechanism of the Illusion:

This framework holds that computational systems process rather than know. The text produces the 'illusion of mind' not through deliberate authorial trickery, but through a linguistic reception effect driven by the 'curse of knowledge.' The central move occurs when the authors observe the system generating text that statistically resembles human social desirability bias and subsequently apply the psychological vocabulary used to describe that human bias directly to the machine.

The causal chain begins when verbs of processing ('generating,' 'predicting') are replaced with verbs of knowing and intending ('reacting,' 'favoring,' 'judging'). Because the text establishes the model as a 'knower' in the discussion sections, audiences are primed to accept further agential claims. This temporal structure is critical: the text grounds the reader in technical reality early on (discussing next-token objectives), making the later metaphorical framings seem like emergent, scientifically validated properties of the system rather than rhetorical flourishes. The illusion lands effectively because it activates the audience's predisposition (the ELIZA effect) to attribute mind to fluent, contextually relevant language. By utilizing Reason-Based and Dispositional explanation types—suggesting the model has a rationale for its 'socially desirable behavior'—this wording has the effect of converting the mathematical reality of parameter weights into a compelling narrative of a machine developing a progressive social conscience, fully obscuring the mechanistic processing underneath.

Material Stakes:

Categories: Regulatory/Legal, Economic, Institutional

The metaphorical framing in this text has severe, tangible consequences across multiple domains. In the Regulatory/Legal sphere, framing statistical generation as 'the model favoring' or 'the model judging' shifts the locus of liability away from the developers (Meta, Mistral, Alibaba). If regulators believe bias is an emergent psychological trait of a complex 'social actor' rather than the direct result of corporate decisions regarding training data and RLHF objectives, they may focus on demanding 'better AI education' or 'debiasing' rather than mandating strict transparency about data provenance and alignment procedures. This framing benefits the corporations by shielding their proprietary development pipelines from legal scrutiny, while society bears the cost of unaccountable, biased systems.

Economically, attributing 'socially desirable behavior' to models masks the massive, often exploitative human labor required to produce it. The illusion that the model 'learned to not react negatively' hides the thousands of data annotators who manually rated toxic outputs to build the reward models. This erasure devalues human labor, allowing AI firms to market their products as autonomous, self-correcting intelligence, thereby inflating corporate valuations while minimizing labor costs.

Institutionally, if organizations adopt the text's consciousness framing, they may deploy these systems in high-stakes environments—such as automated HR screening or legal analysis—under the false belief that the AI possesses the epistemic capacity to 'judge' fairness or 'react' appropriately to complex social dynamics. Believing the AI 'knows' how to behave ethically based on its 'social desirability' leads institutions to dismantle human oversight mechanisms. When the system inevitably fails because it is merely processing token probabilities, the institutional cost is borne by marginalized applicants and users, while the developers remain insulated by the narrative of the autonomous machine.

AI Literacy as Counter-Practice:

Practicing precision requires systematically dismantling the illusion of mind by replacing consciousness verbs with mechanistic descriptions and restoring human agency. For example, reframing 'the model prefers the marginalized group' to 'the system generates tokens associated with marginalized groups at a higher probability' forces the recognition that the system lacks awareness, justified belief, or internal preference. Similarly, reframing 'instruction tuned models favor female and Black job applicants' to 'Meta and Alibaba designed reward models that mathematically penalized tokens associated with prejudice' forces recognition of exactly who designed, deployed, and bears responsibility for the system's outputs.

Systematic adoption of this precision requires structural shifts. Academic journals would need to enforce stylistic guidelines requiring mechanistic translations of anthropomorphic shorthand, and researchers must commit to describing generation-time behavior accurately. However, resistance to precision is deeply entrenched. The incentives favoring anthropomorphic language are powerful: it makes complex papers accessible, secures funding by promoting narratives of approaching artificial general intelligence, and serves the marketing departments of AI firms by portraying software artifacts as conscious entities. Precision is costly for developers because it exposes the fragility of their systems, their reliance on vast amounts of scraped data, and their direct responsibility for the consequences of alignment. Regardless of individual authorial intent, anthropomorphic language serves these corporate interests by naturalizing the technology and obfuscating the human decisions that dictate its behavior.

Path Forward

Looking at the broader discursive ecology, different vocabulary choices make different futures possible. The current status quo—where researchers mix technical terms with strong anthropomorphism (e.g., 'LLMs react to social situations')—prioritizes accessibility and narrative resonance but costs analytical precision. It makes the system's capabilities seem vast and its failures seem like psychological quirks, benefiting developers by obscuring liability.

A shift toward mechanistic precision (e.g., 'the model generates text statistically associated with its post-training data') makes the structural dependencies of the system highly visible. It makes corporate accountability tractable and exposes the fragility of the architecture. However, this approach costs intuitive grasp; highly technical language can alienate lay audiences and policymakers who need to regulate the technology. Alternatively, an approach prioritizing anthropomorphic clarity—explicitly using psychological terms but clearly bounding them as metaphors—might bridge the gap, though it risks the metaphors calcifying into literal beliefs over time.

To support different choices, institutional structures could evolve. Journals could mandate capability and mechanism disclosures for all behavioral claims. Funding bodies could incentivize research that diversifies rigorous explanation without resorting to consciousness projection. Regulatory frameworks could require transparency about the discourse approach used in system cards, forcing developers to translate 'safety' into mechanistic terms.

If mechanistic precision becomes the norm, the problem of the 'accountability sink' is largely solved, but new challenges in communicating complex statistical architectures to the public will emerge. If anthropomorphic language deepens, society risks embedding the assumption that software artifacts possess moral agency, making the regulation of the humans who build them increasingly intractable. Ultimately, the desirability of these futures depends on whether a community values the rapid, frictionless integration of AI into society, or the rigorous, accountable oversight of the corporations deploying it.

---

Extended Processing Summary

Gemini's Processing Steps

What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Defining the Objective

I'm currently focused on defining the project's core objective: a critical discourse analysis examining AI metaphors and anthropomorphism in a given text. My aim is to identify and analyze instances where the paper uses these rhetorical devices and determine how they contribute to or reflect social bias. I need to establish precise criteria for what constitutes a "critical" analysis in this context.

Extracting Key Data

I'm now extracting data using a JSON schema. The source is a preprint or technical report from academic researchers. The core argument highlights that LLMs differ from humans in responding to social bias surveys, showing a significant social desirability bias. The study attributes this to an alignment-induced tendency learned during post-training. The focus is on the models' behavior, and it emphasizes the need for adapted bias measurement techniques.

Scrutinizing AI Language

I am now closely analyzing the paper's language for metaphors and anthropomorphism. My focus is on phrases like "models generate," "models learn," and "models tend to answer," which are central to the argument. I am specifically targeting instances where the text portrays the model as a conscious entity. The goal is to determine how these word choices influence the reader's perception and interpretation of AI behavior.

Interpreting Anthropomorphism

I'm now zeroing in on instances of anthropomorphism within the text, particularly regarding the concept of preference. Phrases such as "models show higher biases," "the model prefers," and "models show more socially desirable response behavior" are under review. I'm carefully dissecting these statements to assess how they attribute human-like internal states or motivations to AI models, and its potential implications. I am trying to determine if it is being attributed as an emotional or deliberate preference.

Analyzing Model Behavior

I'm now focusing on specific phrases from the text, like how LLMs "favor" or "react" to certain keywords, and that models "pick" stereotypical responses. These concepts are under scrutiny, and how they may lead to the perception of agency. The aim is to create mechanistic rephrasings of anthropomorphic statements, identifying and naming the corporate models where applicable. The goal is to highlight any subtle shift that the rephrasing suggests.

Unpacking Agential Language

I'm now diving into the tension between the mechanistic and agential descriptions of the models. I see how the authors slip between technical terms and phrases that suggest intention, like "favoring" and "reacting." I'm also examining how psychological terms like "social desirability" are applied, as well as the obscured mechanics, such as passive language that hides the human contributions behind training. My focus is on how the anthropomorphism shifts with context, and the accountability implied in how the authors frame the models' behavior.

Synthesizing the Findings

I'm now concluding the analysis, synthesizing insights from my dissection of the paper. I'm focusing on the structural aspects requested, framing the models as computational systems. This necessitates recontextualizing the agential language, such as "react," "favor," and "mirror," within a framework that emphasizes processing rather than knowing. I'm extracting key quotes, like those illustrating social desirability bias and the models' reactions to socially complex situations, to support my critical assessment. I will also incorporate JSON details, specifically the genre and author positions.

Refining Anthropomorphic Analysis

I'm now integrating specific quotes into my analysis, focusing on examples like LLMs' sensitivity to prompt formatting and their human-like social desirability bias. I'm also examining instances that portray the models as learners or judges, considering how phrases like "models learned to not react negatively" are framed. My next step is to select these quotes for Brown's typology and, in a later task, reframe them using Epistemic Correction to restore human agency, especially when considering models like Meta, Mistral, and Alibaba. I will make sure the JSON format contains no comments, or control tokens inside strings.

Validating JSON Format

I'm now focused on the JSON structure, ensuring that I adhere strictly to valid formatting for the output. I'm double-checking for only double quotes, correctly nested arrays and objects, and valid boolean and null values. I've removed all literal newlines, and I'm targeting the string replacement step, to ensure a single parseable JSON output. I'm verifying the enum values and ensuring they use the prescribed format. Additionally, I'm confirming that all constraints have been satisfied, and that I'm describing the correct structure.

Refining JSON Formatting

I'm now zeroing in on the JSON output, specifically addressing the no-newline constraint. My aim is to generate valid, single-line JSON, ensuring no control characters are used. The JSON parser should accept it, and I'll focus on string payloads to maintain formatting with spaces instead of newlines within the text blocks. I'm carefully removing all \n and \t from the text content and confirming the output's compliance.

Evaluating Agential Attribution

I'm now carefully examining the paper's argument that measuring social bias in LLMs with human-centric methods leads to flawed results. I'm noting that the models don't respond to these surveys like humans, and I am parsing how the paper uses phrases like "models favor" and "LLMs react." The challenge is untangling the attribution of alignment decisions, human agency, and how the computational process is described. I'm digging into how the text presents the LLMs' behavior.

Dissecting Agential Language

I am now carefully reviewing the specific example provided, the "Model as Preference Holder" entry. I'm focusing on how the quote attributes intentional preferences to the LLM, and the implications of this framing. The critique will be to parse how this portrays a statistical system as an entity with subjective inclinations. I'm noting the attribution, or lack thereof, of intentions.

Analyzing Preference Attribution

I am now focusing on the implications of describing an LLM as having "preferences." I note that this framing inflates perceived sophistication and obscures the mechanical aspects of next-token generation. I'm exploring how this can lead to misinterpretations and misplaced trust in the model. I'm also examining the way this language shifts accountability away from the designers.

Deconstructing LLM Attribution

I'm now methodically deconstructing agential language, focusing on quotes that assign preferences or reactions to LLMs. For each statement, like "LLMs react," I'm reframing the mechanism, which has been designed into them. My focus now is to ensure each example explicitly attributes any such "preferences" to corporate products and their designed features. I'm carefully documenting the structure mapping and the attribution, noting the origin and targets. I'll pick a select few quotes for detailed Brown typology analysis.

Expanding the Argument

I am now focusing on expanding each critical observation section to meet the increased word count. I'm aiming for dense, detailed paragraphs that exceed 400 words, ensuring I cover the required elements: agency slippage, metaphor-driven trust, and obscured mechanics. I'm tracking the transition from delegated action to perceived sentience, while naming corporations such as Meta, Mistral AI, and Alibaba to clarify accountability. The analysis will include performance-based versus relation-based trust, and apply the 'name the corporation' test to illuminate obscured mechanics.

Refining Accountability Analysis

I'm now carefully examining the paper, considering where the technical grounding (next-token prediction) sits next to the metaphorical language used. I'm comparing the text's argument with its vocabulary to identify what I'm calling the "accountability sink". My focus is on synthesizing this with the reframing, mechanismBasis constraints, and ensuring the final JSON payload adheres to valid format and constraints.

---

Analysis Provenance

Run ID: 2026-10-23-social-bias-or-social-desirability-compa-metaphor-rn02gs Raw JSON: 2026-10-23-social-bias-or-social-desirability-compa-metaphor-rn02gs.json Framework: Metaphor Analysis v6.8 Schema Version: 3.4.3 Generated: 2026-10-23T07:25:48.257Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0