Skip to main content

The Geometry of Metaphor: Probing and Steering Metaphor Representations in Large Language Models

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Source Reading​

Genre: Preprint or technical report · Author position: Academic or independent research

Argument: The text argues that the concept of metaphoricity is encoded in specific, interpretable linear directions within the internal activations of large language models. The authors propose that by identifying these causal subspaces using mechanistic interpretability techniques like activation patching and distributed alignment search (DAS), it is possible to steer the models' outputs toward generating either more literal or more metaphorical language across a variety of architectures and sizes.

Responsibility claims: The text primarily treats the AI models as mechanistic artifacts under investigation, making no broad responsibility claims regarding the societal impacts of AI. It does, however, briefly note in its limitations section that steering techniques carry inherent risks, as they can be used to steer models toward undesirable behaviors as easily as desirable ones, implicitly placing the responsibility for safe steering on the researchers or developers who utilize these inference-time interventions.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

1. Model as Comprehending Mind​

Quote: "Large language models such as Llama 3 have some capacity to understand and produce metaphorical language"

  • Frame: Model as an entity with cognitive comprehension
  • Projection: This framing projects human epistemic states onto computational architecture. The term 'understand' suggests conscious comprehension, semantic awareness, and the ability to grasp meaning, while 'capacity' implies a latent cognitive potential. Located at the epistemic/conscious state level of the agency gradient, this claim is made without evidence of subjective awareness; the text measures classification accuracy and generation alignment, which establish computational activity rather than conscious comprehension. Mechanistically, generic to systems of this type, Llama 3 processes sequences of tokens through transformer blocks, where self-attention mechanisms weigh input context based on parameters optimized during training to predict the next most probable token. It does not comprehend the semantics of the metaphor, but instead correlates input structures with high-dimensional distributions learned from its training data.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The text states 'have some capacity to understand' as a literal factual premise. I considered Hedged/Qualified because of the word 'some', which limits the extent of the capacity, but 'some' quantifies the ability without qualifying the framing of 'understanding' itself.)
  • Implications: By projecting conscious understanding onto a statistical pattern-matching system, this framing inflates the perceived sophistication of the model. It encourages readers to trust the system's outputs as the product of reasoned comprehension rather than probabilistic token generation. When policymakers or the public believe a model 'understands' metaphors, they may falsely assume it also understands context, nuance, or harm in the same way a human would, leading to unwarranted trust in its reliability and a misallocation of liability when the model produces biased or harmful outputs.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The agentless construction 'Large language models such as Llama 3 have some capacity' obscures the human decisions behind the model. Meta developed Llama 3; engineers selected its vast training data, set its optimization objectives, and deployed it. The text positions the 'capacity to understand' as an innate characteristic of the artifact rather than the intentional outcome of Meta's data scraping, instruction tuning, and RLHF pipeline. Naming Meta forces recognition of the commercial and engineering processes that produce the statistical correlations mischaracterized here as understanding. I considered Partial, but no human actors or generic categories like 'engineers' are mentioned in the immediate context of this claim.
Show more...

2. Model as Intentional Interpreter​

Quote: "Since metaphor is a key part of human language, we would like to understand how LLMs interpret and identify metaphor."

  • Frame: Model as an active interpreter of meaning
  • Projection: This mapping projects human analytical reasoning ('interpret' and 'identify') onto mathematical classification processes. 'Interpret' sits at the intentional-state or epistemic level of the agency gradient, implying an active, conscious effort to unpack meaning and evaluate context. The text provides evidence only for goal-directed computational behavior (sorting strings into 'literal' or 'metaphorical' categories based on prompt templates). Mechanistically, generic to systems of this type, the model calculates logits over its vocabulary conditioned on the input prompt and its learned weights. It generates the token 'metaphorical' or 'literal' based on statistical probabilities shaped by training data, not through an internal process of cognitive interpretation or semantic analysis.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The phrasing is presented as a straightforward research objective without any framing qualifications. I considered Explicitly Acknowledged because mechanistic interpretability often uses these terms loosely, but there are no scare quotes or meta-commentary redefining 'interpret' as a purely statistical operation.)
  • Implications: Using 'interpret' rather than 'classify' or 'process' obscures the mechanical reality of the operation, making the AI seem like a collaborative partner in linguistic analysis. This framing can lead readers to overestimate the model's robustness. If an LLM is seen as 'interpreting' text, a user might trust it to handle highly novel, culturally specific, or deeply context-dependent metaphors that fall outside its training distribution, mistakenly believing the system applies reasoning rather than interpolating between learned data points.

Accountability Analysis:

  • Actor Visibility: Partial (some attribution)
  • Analysis: The text identifies the researchers ('we') as the actors seeking to understand the process, but treats the LLMs as autonomous interpreters, obscuring the original developers. While the researchers are named as the investigators, the developers who built the models (Google, Meta, Mistral, Alibaba) and dictated how they handle text via training data and reward modeling remain hidden in this specific framing. I considered Named, but the actual architects of the 'interpretation' behavior are omitted in favor of highlighting the researchers studying the finished artifact.

3. Concept Possession​

Quote: "The model could have another metaphoricity concept located in a different position adapted for generation."

  • Frame: Model as a container of abstract concepts
  • Projection: This metaphor projects the human psychological capacity to possess and locate mental representations ('concepts') onto the model's parameter weights and activation vectors. It locates the AI at the epistemic/conscious state level by suggesting it 'has a concept.' The text's evidence is based on Interchange Intervention Accuracy (IIA) scores from distributed alignment search, which identifies a causal subspace, not a cognitive concept. Mechanistically, as documented in the source and generic to systems of this type, certain activation vectors across layers and token positions strongly influence the logits for specific output tokens. The 'concept' is actually a high-dimensional geometric subspace within the residual stream that correlates with the target behavior when the model generates text.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Hedged/Qualified (The phrase uses 'could have', which hedges the certainty of the claim. I considered Direct, but the modal verb 'could' introduces explicit uncertainty about whether the model possesses this secondary representation, though it still frames the representation itself as a 'concept'.)
  • Implications: Describing activation subspaces as 'concepts' encourages the audience to map human semantic organization onto machine architecture. This creates an illusion of conceptual coherence, implying that the model organizes information logically and comprehensively. This framing risks regulatory and epistemic confusion: if policymakers believe models possess discrete 'concepts' like humans do, they might mandate that developers simply 'delete' harmful concepts (like bias or danger), vastly underestimating the entangled, distributed, and statistical nature of model weights.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The sentence presents the model as the sole entity that 'could have' a concept, entirely omitting the human engineering that structures the model's internal geometry. The developers at the corporations that built these models (e.g., Meta for Llama, Alibaba for Qwen) determined the architecture, depth, width, and training regimen that give rise to these specific linear subspaces. By framing the 'concept' as something the model autonomously possesses or locates, the text obscures the deliberate design choices and vast data curation efforts of these tech companies. I considered Ambiguous, but the syntactic structure clearly positions 'The model' as the active possessor.

4. Model as Jurist​

Quote: "we use Qwen 2.5 72B Instruct as a judge. Only the first sentence of each completion is judged, and a degeneracy filter applied before judging"

  • Frame: Model as an authoritative evaluator
  • Projection: This framing projects human judicial and evaluative capacity onto a computational classification process. 'Judge' implies intentional-state attribution, suggesting the system evaluates, weighs evidence, and applies normative standards to reach a verdict. The text demonstrates goal-directed behavior (outputting a specific label based on a prompt). Mechanistically, generic to instruction-tuned systems, Qwen 2.5 takes the input strings and generates tokens that align with the statistical patterns of evaluation text in its fine-tuning data. It does not judge; it computes the maximum likelihood of the next token ('literal' or 'metaphorical') conditioned on the prompt, completely devoid of evaluative reasoning or comprehension of the text's actual meaning.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The framing of LLM-as-judge is presented without qualification, as a standard methodology. I considered Hedged/Qualified, but there is no language qualifying the system's role as a literal 'judge' in this context; it is stated declaratively.)
  • Implications: The 'LLM-as-judge' metaphor transfers the epistemic authority of human evaluation onto a statistical artifact. This creates a severe risk of automation bias, where readers and researchers accept the model's outputs as objective verdicts rather than statistical correlates. It obscures the fact that the 'judge' is merely reproducing biases, heuristics, and distributions present in Alibaba's training data. If widely adopted, this framing validates replacing human oversight with automated systems in high-stakes domains, under the false assumption that models possess actual evaluative judgment.

Accountability Analysis:

  • Actor Visibility: Partial (some attribution)
  • Analysis: The researchers ('we') are named as the actors who decided to employ the model in this role. However, the agency behind the 'judging' itself is displaced onto the system (Qwen 2.5), obscuring Alibaba, the corporation that built the model and defined the objective functions that govern its behavior. When the model 'judges,' it is actually executing actions under authority delegated by the researchers, relying on the instruction-tuning provided by Alibaba. Naming Alibaba reveals whose data and values are actually functioning as the baseline for this automated evaluation. I considered Named, because 'we' specifies the researchers, but the originators of the judging mechanism are obscured.

5. The Embodied Adversary​

Quote: "whether they are up to human performance is debated"

  • Frame: Model as a competitive agent
  • Projection: This framing maps the human concept of competitive capability and comparative skill onto the computational outputs of an artifact. 'Up to human performance' projects goal-directed behavior or intentional-state attribution onto the model, suggesting it is an autonomous entity striving to match humans. The text actually provides evidence for computational activity: the accuracy rates of the model matching human-labeled benchmarks. Mechanistically, generic to this system type, the model's generated strings are mathematically evaluated against a ground-truth dataset provided by humans; the model does not perform or compete, it merely updates its state and generates token probabilities.
  • Agency Level: Goal-directed behavior
  • Acknowledgment: Hedged/Qualified (The phrase 'is debated' acts as a hedge on the certainty of the claim regarding the capability level. I considered Direct, but the debate framing explicitly qualifies the absolute nature of the model's status relative to humans.)
  • Implications: Framing AI as 'up to human performance' anthropomorphizes the system into a rival or peer, inviting the audience to view the model holistically as a human equivalent. This obscures the narrow, brittle nature of its statistical pattern-matching, leading to a capability overestimation where users assume that competence in generating metaphor correlates implies competence in general reasoning. It stokes adversarial narratives between humans and machines while hiding the corporate actors actually driving the automation.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The agentless construction 'they are up to human performance is debated' obscures both the developers striving to build these systems and the researchers conducting the debates. The companies (Meta, Google, etc.) that dictate the training objectives and benchmark targeting are rendered invisible, replaced by a framing that treats the models ('they') as independent actors naturally evolving toward human parity. Identifying the corporations emphasizes that models do not strive for human performance; developers explicitly optimize them to score well on human-designed benchmarks to drive commercial value. I considered Partial, but no human actors are indicated in this passive construction.

6. Truth as Spatial Geography​

Quote: "Llama models exhibited a 'truth direction' at particular layers"

  • Frame: Model as a spatial landscape containing truth
  • Projection: This metaphor projects the epistemic concept of 'truth'—a property of justified human belief and external reality—onto a mathematical vector within a high-dimensional parameter space. It places the model at the epistemic/conscious state level by suggesting it inherently encodes 'truth'. The evidence provided is computational activity: specific activation layers predictably correlate with outputs labeled as true in a dataset. Mechanistically, documented in the cited literature and generic to the system type, the model learns a linear subspace where activation vectors reliably differentiate between text strings labeled 'true' and 'false' in the training distribution. It encodes a statistical feature boundary, not the epistemic concept of truth.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Explicitly Acknowledged (The authors use scare quotes around 'truth direction', explicitly recognizing it as a terminological shorthand or metaphor rather than a literal geographic heading of factual accuracy. I considered Hedged/Qualified, but the typographic marker signifies explicit metaphorical acknowledgement.)
  • Implications: Even with scare quotes, the 'truth direction' metaphor is highly deceptive. It conflates mathematical feature separation with epistemic truth, suggesting the model has an internal compass for reality. This dramatically inflates perceived reliability. If audiences believe models have a 'truth direction,' they are more likely to trust AI outputs implicitly, assuming the model can reliably separate fact from fiction. This obscures the fact that the direction merely separates strings that structurally resemble the training data's 'true' examples from its 'false' examples, offering no protection against novel hallucinations or systemic biases.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The phrasing 'Llama models exhibited' positions the model as a natural phenomenon revealing its properties, obscuring the human actors who designed the architecture and curated the dataset that necessitated this mathematical separation. Meta engineers designed Llama's transformer layers and objective functions; human annotators and dataset creators provided the labels that the model mathematically differentiates. By making the AI the active subject 'exhibiting' properties, the text hides the intensive human labor required to shape the parameter space. I considered Partial, but no human actors are mentioned; the model is the sole actor.

7. The Mechanical Steersman​

Quote: "we are able to steer the model towards more metaphorical output"

  • Frame: Model as a vehicle being driven
  • Projection: This metaphor projects the physical dynamics of navigation and driving onto the computational process of inference-time intervention. It locates the AI at the delegated action level, treating it as an object manipulated by an operator. The text provides strong evidence for computational activity: injecting activation vectors during the forward pass alters the generated output probabilities. Mechanistically, as documented in the source, researchers extract hidden states from specific layers, compute a directional vector, and patch this vector into the residual stream at generation time, mathematically shifting the probability distribution of the subsequent tokens toward vocabulary associated with the 'metaphorical' subspace.
  • Agency Level: Delegated action
  • Acknowledgment: Direct (Unacknowledged) (The term 'steer' is used literally in the context of mechanistic interpretability to describe vector intervention, without hedging or scare quotes. I considered Explicitly Acknowledged, but the term has become entirely normalized in the field as a direct description of the process.)
  • Implications: While 'steering' is less anthropomorphic than 'understanding,' it still masks the complexity of the intervention. It implies a smooth, predictable control mechanism—like turning a steering wheel—when the reality is a brittle, high-dimensional vector injection that can easily result in degenerate or nonsensical output (as the authors acknowledge elsewhere via their 'degeneracy filter'). This framing can lead stakeholders to overestimate the safety and controllability of deployed LLMs, assuming that undesirable behaviors can be simply 'steered' away, thus minimizing the perceived risk of deploying flawed systems.

Accountability Analysis:

  • Actor Visibility: Partial (some attribution)
  • Analysis: The researchers ('we') are explicitly named as the actors applying the steering mechanism, making their agency visible. However, the model is treated as a generic vehicle, obscuring the original developers (e.g., Meta, Google) whose specific architectural choices and training regimens determine the model's topology and dictate whether 'steering' is even possible or effective. While the text acknowledges who is doing the intervening, it minimizes the structural dependencies created by the corporate entities that built the artifact. I considered Named, but the developers of the 'vehicle' are completely omitted in this functional framing.

8. Model as Student​

Quote: "models have relatively strong performance at understanding metaphor, especially when fine-tuned"

  • Frame: Model as a learning entity improving through study
  • Projection: This framing projects human pedagogical development onto the statistical process of weight optimization. 'Understanding' projects epistemic state, while 'fine-tuned' (in conjunction with 'performance') evokes a student improving through practice. The text provides evidence for computational activity: accuracy metrics on classification benchmarks improve after supervised updates. Mechanistically, generic to systems of this type, fine-tuning uses gradient descent to adjust the model's internal parameter weights based on a loss function calculated against a specialized dataset of human-labeled metaphorical and literal sentences. The model does not learn to understand; its parameters are shifted to mathematically minimize errors on a specific data distribution.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The claim is presented as a straightforward statement of fact about model capabilities, without qualification. I considered Hedged/Qualified, but 'relatively strong' modifies the degree of performance, not the epistemological claim of 'understanding'.)
  • Implications: Conflating weight updates with cognitive learning encourages unwarranted trust in the model's generalizability. If users believe a model 'improves its understanding' through fine-tuning, they will expect it to handle novel, out-of-distribution metaphors intelligently. In reality, fine-tuning often results in shallow heuristic adoption or catastrophic forgetting. This framing masks the system's reliance on vast amounts of human-annotated data, making the AI appear as an autonomous learner rather than an artifact heavily dependent on continuous human labor and curation.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The construction obscures both the companies that initially built the models and the human annotators whose labor makes fine-tuning possible. Models do not spontaneously improve their 'performance at understanding.' Engineers curate datasets, thousands of underpaid annotators label the text, and developers execute the compute-heavy optimization processes. By stating that performance is strong 'especially when fine-tuned', the text erases the humans turning the dials and providing the ground truth, attributing the resulting statistical alignment solely to the model. I considered Partial, but there are no human actors mentioned in relation to the fine-tuning process.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: conscious mind / human speaker → model​

Quote: "Large language models such as Llama 3 have some capacity to understand and produce metaphorical language"

  • Source Domain: conscious mind / human speaker
  • Target Domain: model
  • Mapping: This mapping projects the cognitive architecture of human language comprehension onto the statistical architecture of a language model. In the source domain, a human speaker possesses internal semantic representations, recognizes abstract parallels between disparate concepts, and consciously formulates speech to convey meaning. Projected onto the target domain, this invites the assumption that the LLM similarly grasps the semantic depth of the input prompt, recognizes the figurative intent, and intentionally selects words to express an abstract concept. It maps conscious awareness of meaning onto the calculation of token probabilities.
  • What Is Concealed: This mapping conceals the purely mathematical, mechanistic reality of transformer architectures. It hides the fact that the model relies entirely on statistical co-occurrences learned from massive datasets during training, possessing no actual semantic grounding, causal world model, or conscious awareness. Furthermore, it obscures the proprietary opacity of Llama 3; Meta does not fully disclose the training data, so we cannot know if the model is genuinely generalizing or simply regurgitating metaphorical structures it memorized from its corpus.
Show more...

Mapping 2: interpreter / human analyst → application​

Quote: "we would like to understand how LLMs interpret and identify metaphor."

  • Source Domain: interpreter / human analyst
  • Target Domain: application
  • Mapping: This structure maps the deliberate, analytical process of a human interpreting a text onto the computational process of an LLM classifying a string. The source domain involves a subject applying contextual knowledge, cultural background, and reasoning to resolve ambiguity. The mapping invites the assumption that the AI system performs a similar cognitive evaluation, actively deciding whether a sequence of words carries literal or figurative meaning by 'thinking' about the concepts involved.
  • What Is Concealed: The framing completely hides the mathematical reality of logits, vector spaces, and softmax functions. It obscures that 'identifying' a metaphor is actually the process of the model calculating the highest probability for a specific token (e.g., 'metaphorical') based on the prompt template and the input sequence. It conceals the model's total dependence on the patterns embedded in its training data by human writers, projecting an illusion of independent analytical reasoning onto correlation.

Mapping 3: human mind / container of ideas → model​

Quote: "The model could have another metaphoricity concept located in a different position adapted for generation."

  • Source Domain: human mind / container of ideas
  • Target Domain: model
  • Mapping: This metaphor draws on the psychological domain where human minds contain discrete, locatable 'concepts' or ideas that can be accessed and utilized for different tasks. Projected onto the LLM, it maps the idea of a cohesive mental representation onto a specific geometric subspace within the model's vast parameter network. It invites the assumption that the model organizes information logically and topically, possessing a distinct, unified 'understanding' of metaphoricity that it can deploy when generating text.
  • What Is Concealed: This mapping hides the highly distributed, entangled, and polysemantic nature of neural network weights. A 'concept' in an LLM is not a distinct semantic object; it is a human-interpretable label applied post-hoc to a high-dimensional linear subspace that correlates with a specific output behavior. The metaphor obscures the reality that these 'concepts' are merely statistical feature boundaries discovered by researchers using tools like DAS, rather than innate, bounded ideas possessed by the model.

Mapping 4: jurist / human evaluator → application​

Quote: "we use Qwen 2.5 72B Instruct as a judge. Only the first sentence of each completion is judged"

  • Source Domain: jurist / human evaluator
  • Target Domain: application
  • Mapping: This powerful mapping projects the authority, reasoning, and normative judgment of a human evaluator onto a language model. In the source domain, a judge reads a text, considers criteria, weighs evidence, and delivers a reasoned verdict. Projected onto the target domain, this invites the assumption that the LLM engages in a similar evaluative process to determine whether a generated sentence is 'literal' or 'metaphorical,' applying epistemic authority and critical thought to the task.
  • What Is Concealed: The 'judge' metaphor entirely conceals the mechanism of next-token prediction based on instruction tuning. Qwen is not weighing evidence; it is computing the maximum likelihood of the tokens 'literal' or 'metaphorical' given the prompt string, heavily influenced by the reinforcement learning and fine-tuning data provided by Alibaba. This conceals the lack of ground truth, the absence of actual reasoning, and the fact that the 'judgment' is merely the statistical reflection of the preferences of human annotators and engineers.

Mapping 5: human competitor / athlete → model​

Quote: "whether they are up to human performance is debated"

  • Source Domain: human competitor / athlete
  • Target Domain: model
  • Mapping: This mapping projects human competition, striving, and capability onto an inanimate software artifact. The source domain involves conscious entities exerting effort to achieve a standard or defeat a rival. Projected onto the LLM, it positions the AI as an autonomous agent actively participating in a contest of skill against humanity. It invites the audience to measure the machine holistically against human intelligence, assuming that matching 'performance' equates to matching cognitive capability.
  • What Is Concealed: This competitive framing obscures the artificiality of the benchmarks and the immense human labor required to optimize models to pass them. It conceals the mathematical reality that 'performance' merely means generating strings that match a human-labeled evaluation set with a high degree of statistical accuracy. It completely hides the corporate developers (Meta, Google, etc.) who deliberately fine-tune the models on benchmark-like data to drive marketing narratives, erasing the human designers behind the illusion of the machine's autonomous achievement.

Mapping 6: physical landscape / compass heading → model​

Quote: "Llama models exhibited a 'truth direction' at particular layers"

  • Source Domain: physical landscape / compass heading
  • Target Domain: model
  • Mapping: This mapping projects physical spatial navigation and the epistemic concept of truth onto high-dimensional linear algebra. It maps a geographic 'direction' onto a vector within the residual stream, and maps the philosophical absolute of 'truth' onto the structural separation of data points. It invites the assumption that the model contains an internal compass pointing toward objective reality, capable of fundamentally distinguishing fact from falsehood.
  • What Is Concealed: This framing conceals the fact that the 'truth direction' is merely a linear classification boundary that separates strings labeled 'true' from strings labeled 'false' in the specific dataset used for the probe. It obscures the statistical reality that the model is tracking structural and stylistic correlations in the text, not epistemic truth. It also hides Meta's proprietary training data; without knowing what Llama was trained on, it is impossible to know if the model is genuinely generalizing a feature or simply recalling memorized correlations.

Mapping 7: vehicle / navigation → model​

Quote: "we are able to steer the model towards more metaphorical output"

  • Source Domain: vehicle / navigation
  • Target Domain: model
  • Mapping: This mapping projects the physical manipulation of a vehicle (steering) onto the process of causal intervention in a neural network. The source domain involves a driver applying continuous, predictable control to guide a machine along a path. Projected onto the LLM, it invites the assumption that researchers have smooth, deterministic control over the model's behavior, and that modifying the output is a straightforward matter of adjusting a mechanical input.
  • What Is Concealed: This conceals the fragility, complexity, and unpredictability of vector patching in high-dimensional space. It obscures the fact that 'steering' involves injecting a calculated activation vector (h_patched) into the residual stream at every generation step, which forces the model's token probabilities to shift. It hides the frequent side effects of such interventions, such as grammatical collapse or repetitive loops (which the authors mitigate using a 'degeneracy filter'), masking the fundamentally statistical and unstable nature of the generation process.

Mapping 8: student / learning process → model​

Quote: "models have relatively strong performance at understanding metaphor, especially when fine-tuned"

  • Source Domain: student / learning process
  • Target Domain: model
  • Mapping: This mapping projects the human pedagogical experience onto the machine learning process. In the source domain, a student improves their cognitive comprehension through study and practice ('fine-tuning'). Projected onto the AI, it invites the assumption that the model is an active learner that develops a deeper, more robust mental grasp of metaphorical concepts when exposed to additional data.
  • What Is Concealed: This metaphor conceals the mechanical reality of gradient descent and weight updates. 'Fine-tuning' does not impart understanding; it mathematically shifts the model's parameter weights to minimize the cross-entropy loss against a specific dataset of human-annotated examples. It obscures the massive amount of invisible human labor—the dataset curation and the annotators who labeled the metaphors—treating the resulting statistical alignment as the machine's own cognitive achievement.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "The larger the difference, the more influential the hidden state in position i and layer ℓ on the model’s prediction."

  • Explanation Types:

    • Functional (Primary): Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): This explanation relies heavily on a mechanistic (how) framing, describing the internal architecture of the model as a system of interacting parts. The use of 'Functional' explanation isolates specific components ('hidden state in position i and layer ℓ') and defines their role by how they causally affect the final output ('the model's prediction'). This choice emphasizes the mathematical, determinative nature of the transformer architecture, treating the model appropriately as a computational artifact rather than a thinking agent. By focusing on variables like 'difference' and 'influence,' it accurately portrays the system as a complex equation where changing one parameter alters the result. However, it slightly obscures human agency by attributing the 'prediction' to the model itself, rather than describing it as the output of the mathematical function designed by the developers.

  • Consciousness Claims Analysis: The passage avoids direct consciousness verbs, utilizing mechanistic terms like 'hidden state', 'layer', 'position', and 'prediction'. The term 'prediction' sits at the computational activity level on the agency gradient; it does not claim the system 'knows' the future, but rather calculates a statistical probability. There is little curse of knowledge evident here, as the authors are adhering strictly to the vocabulary of linear algebra and network architecture. Mechanistically, as documented in the source and generic to systems of this type, the researchers are using activation patching: they swap the activation vector at a specific layer and token position during the forward pass with a vector from a different prompt, and measure the change in the output logit probabilities. This is a highly precise technical description of causal intervention that correctly attributes changes in output to mathematical manipulations of the residual stream.

  • Rhetorical Impact: This functional, mechanistic framing strips away the illusion of mind, grounding the audience's perception of the AI in its reality as a software tool. It reduces perceived autonomy by showing exactly how specific mechanical interventions reliably alter the output. By avoiding consciousness framing, it prevents unwarranted trust; readers are positioned to see the 'prediction' as the result of a calculated state rather than a reasoned judgment. If audiences understand AI through this lens, they are more likely to demand rigorous technical audits and hold developers accountable for the mathematical structures they build, rather than treating the AI as an independent agent that makes 'mistakes'.

Show more...

Explanation 2​

Quote: "LLMs have relatively strong performance at understanding metaphor, especially when fine-tuned"

  • Explanation Types:

    • Intentional (Primary): Refers to goals/purposes, presupposes deliberate design
    • Genetic (Secondary): Traces origin through dated sequence of events or stages

  • Analysis (Why vs. How Slippage): This explanation utilizes an agential (why) framing through an Intentional lens, attributing the cognitive goal of 'understanding metaphor' directly to the system. The secondary Genetic explanation ('especially when fine-tuned') implies a developmental process, suggesting the model evolves or matures into better understanding over time. This choice emphasizes the model's capabilities and end-results, making it highly accessible to non-technical audiences. However, it severely obscures the mechanistic reality (how) of what fine-tuning actually entails—gradient descent, loss functions, and weight updates. It also entirely erases the human labor involved in creating the fine-tuning datasets, presenting the improvement as an autonomous cognitive leap rather than the direct result of human engineering and data annotation.

  • Consciousness Claims Analysis: The passage explicitly utilizes the consciousness verb 'understanding', making a direct epistemic claim about the model's internal state. This projects knowing onto processing, elevating the system to the epistemic/conscious state on the agency gradient. This represents a clear curse of knowledge dynamic: the authors understand the complex mathematical correlations the model uses to correctly classify metaphors, but they project their own human experience of semantic comprehension onto the system's successful output. Mechanistically, generic to systems of this type, the model does not 'understand'; during fine-tuning, its parameter weights are adjusted via backpropagation to minimize the error rate on a specific dataset of human-annotated metaphors. At generation time, it produces text by calculating token probabilities from those optimized parameters. The 'understanding' is an illusion generated by successful statistical alignment.

  • Rhetorical Impact: The intentional and genetic framing strongly encourages audiences to perceive the AI as an autonomous, capable agent. The consciousness framing ('understanding') acts as a powerful trust signal, suggesting the model can be relied upon to grasp nuance, context, and intent. This severely inflates perceived sophistication. If policymakers and users believe the AI 'understands', they are likely to deploy it in sensitive, high-stakes environments (like content moderation or legal analysis) where true semantic comprehension is required, risking catastrophic failures when the model's statistical heuristics break down outside its fine-tuning distribution.

Explanation 3​

Quote: "we save the activations for one input, and patch these into the activations for another input, examining which layers and tokens affect the output."

  • Explanation Types:

    • Functional (Primary): Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): This is a purely mechanistic (how) explanation operating through a Functional register. It explicitly outlines the methodology of the system's operation and the researchers' intervention. It emphasizes the structural reality of the AI as an artifact composed of distinct, manipulable parts ('activations', 'layers', 'tokens'). This framing makes the human actors ('we') highly visible as the agents performing the manipulation, correctly placing the AI as the object being acted upon. It obscures very little, offering a transparent look at the mechanics of interpretability research. It strips away any illusion of intentionality from the model, framing 'the output' simply as the end result of a mechanical chain of events rather than a deliberate choice by the system.

  • Consciousness Claims Analysis: There are no consciousness or epistemic verbs present in this passage. The text relies on mechanistic verbs ('save', 'patch', 'affect') and accurately assesses the system's behavior as processing rather than knowing, keeping it firmly at the computational activity level on the agency gradient. The curse of knowledge is avoided, as the authors describe the system precisely in the terms of its architecture. Mechanistically, as documented in the source, this describes the technique of causal tracing or activation patching. During the forward pass of the model, a specific high-dimensional vector (activation) at a specific transformer layer and token position is cached. On a subsequent run with a different prompt, that cached vector overwrites the naturally occurring vector, and the resulting shift in the final logit distribution (the output) is measured.

  • Rhetorical Impact: This framing severely limits the audience's ability to anthropomorphize the system. It builds performance-based trust in the researchers' methodology while preventing relation-based trust in the AI. By exposing the model as a construct whose 'thoughts' (activations) can literally be swapped out like mechanical parts to alter its 'speech' (output), it reinforces the artifactual nature of the technology. Decisions based on this understanding would correctly focus on technical audits, mathematical constraints, and the robustness of the system's architecture, rather than attempting to align the model's 'intentions' or 'beliefs'.

Explanation 4​

Quote: "The model could have another metaphoricity concept located in a different position adapted for generation."

  • Explanation Types:

    • Theoretical (Primary): Embeds in deductive framework, may invoke unobservable mechanisms
    • Dispositional (Secondary): Attributes tendencies or habits

  • Analysis (Why vs. How Slippage): This explanation utilizes a hybrid framing, blending mechanistic structures ('located in a different position', 'generation') with agential, cognitive mapping ('have another metaphoricity concept'). The Primary Theoretical explanation posits an unobservable psychological mechanism ('concept') embedded within the spatial architecture of the network. The Secondary Dispositional framing suggests the model has adapted this concept for a specific purpose ('adapted for generation'). This choice emphasizes the model's internal complexity but obscures the reality of what a 'concept' actually is in this context. It hides the fact that the 'concept' is merely a human-defined linear subspace, and masks the developers who designed the architecture that forces the model to organize statistical representations in this manner.

  • Consciousness Claims Analysis: The passage uses the consciousness-adjacent verb 'have' combined with the psychological noun 'concept', making an epistemic claim that projects knowing onto processing. It locates the system at the epistemic/conscious state level by suggesting it possesses discrete mental representations. This is a classic curse of knowledge dynamic: researchers find a dense cluster of statistical correlations that govern how the model outputs metaphorical text, and they project their own conceptual understanding onto that cluster, naming it a 'concept'. Mechanistically, generic to systems of this type and based on the authors' DAS methodology, the 'concept' is a rotation matrix trained to align a subset of the residual stream with a specific classification task. It is a mathematical feature direction that influences next-token prediction during generation, entirely devoid of semantic awareness or conceptual thought.

  • Rhetorical Impact: The theoretical framing of AI possessing 'concepts' profoundly shapes audience perception, constructing an illusion of a structured, human-like mind. It encourages audiences to believe the AI has a coherent, generalized understanding of the world that exists independently of its immediate output. This increases perceived autonomy and misplaced trust. If a user believes the AI possesses a 'metaphoricity concept,' they may assume it can reliably identify irony, sarcasm, or harmful subtext, leading to over-reliance on the system in critical analytical contexts and a misunderstanding of how easily such statistical 'concepts' can break down.

Explanation 5​

Quote: "we use Qwen 2.5 72B Instruct as a judge. Only the first sentence of each completion is judged, and a degeneracy filter applied before judging"

  • Explanation Types:

    • Functional (Primary): Explains behavior by role in self-regulating system with feedback
    • Intentional (Secondary): Refers to goals/purposes, presupposes deliberate design

  • Analysis (Why vs. How Slippage): This passage operates through a dual register. The Primary Functional explanation describes the system's role within the experimental pipeline (it acts as a filter and evaluator for generated sentences). However, the Secondary Intentional explanation ('as a judge', 'is judged') frames the mechanism agentially, attributing a deliberate, purposeful role to the software. This emphasizes the utility and automated nature of the evaluation pipeline while severely obscuring the actual mechanism of how Qwen operates. By repeatedly using the verb 'judge', it masks the reality that the model is simply predicting tokens based on Alibaba's instruction-tuning data. It displaces the agency of the original developers, treating their artifact as an objective arbiter of truth.

  • Consciousness Claims Analysis: The passage relies heavily on the consciousness verb 'judge', moving the system from computational activity to intentional-state attribution. It assesses the system as a 'knower' capable of evaluation rather than a 'processor' executing a function. The curse of knowledge is present in the assumption that the model's statistically generated label ('literal' or 'metaphorical') equates to human judicial reasoning. Mechanistically, generic to systems of this type, Qwen 2.5 72B is an autoregressive transformer. When prompted with a sentence, it processes the text through its layers and outputs the token with the highest probability. Because it has been fine-tuned by Alibaba on vast amounts of instruction-response pairs, its output probability distribution mimics human evaluation formats, but the generation itself is a statistical correlation, not an act of judging.

  • Rhetorical Impact: The rhetorical impact of framing the LLM as a 'judge' is immense; it constructs a powerful aura of authority, objectivity, and reliability around the system's outputs. It encourages relation-based trust in the machine's 'verdicts'. This framing fundamentally alters accountability: if the model is seen as a judge, errors are viewed as 'bad calls' by an autonomous agent, rather than as systemic biases embedded in Alibaba's training data or methodological flaws in the researchers' experimental design. Decisions to automate evaluation or moderation are easily justified when audiences believe the software is literally capable of 'judging'.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
Large language models such as Llama 3 have some capacity to understand and produce metaphorical languageLarge language models such as Llama 3 process input strings and generate output text that statistically correlates with metaphorical patterns found in their training data.The system does not understand language. Llama 3 generates text by calculating the probabilities of subsequent tokens based on its parameters and context, a statistical process optimized during training to recognize and reproduce patterns without conscious comprehension.

Basis: Generic to system type		Meta engineers designed Llama 3 and trained it on massive datasets; the resulting model generates text that mimics the metaphorical structures present in the data Meta selected.
we would like to understand how LLMs interpret and identify metaphor.we would like to understand how LLMs mathematically classify strings and generate text correlated with metaphor.The system does not interpret or identify. It processes the prompt through its layers and calculates the highest probability for a specific classification token (e.g., 'metaphorical') based on weight distributions learned during training.

Basis: Generic to system type		Researchers seek to understand how the models developed by corporations like Google and Meta compute outputs that align with human definitions of metaphor.
The model could have another metaphoricity concept located in a different position adapted for generation.The model's parameter weights may contain another linear subspace that correlates with generating metaphorical text, located at a different layer or token position.The system possesses no concepts. It contains high-dimensional geometric features within its residual stream—specific vectors that mathematically influence the logits for next-token prediction during generation.

Basis: Documented in source		—
we use Qwen 2.5 72B Instruct as a judge. Only the first sentence of each completion is judgedwe use Qwen 2.5 72B Instruct as an automated classifier. Only the first sentence of each completion is processed by the classifierThe system does not judge or evaluate. Qwen generates the next most probable token ('literal' or 'metaphorical') based on its context window and the statistical distributions established by its instruction-tuning data.

Basis: Generic to system type		Researchers delegated the classification task to Qwen 2.5, a model built and instruction-tuned by Alibaba; the outputs reflect Alibaba's training objectives rather than objective judgment.
whether they are up to human performance is debatedwhether the text generated by these models statistically matches human-labeled benchmarks with high accuracy is debatedThe system does not perform or compete. It mechanically generates sequences of tokens, and its accuracy is simply a mathematical measure of how closely its output distribution aligns with a pre-defined test set.

Basis: Generic to system type		Researchers debate whether the models built by corporate developers output text that aligns accurately with benchmarks created by human evaluators.
Llama models exhibited a 'truth direction' at particular layersLlama models contained a linear classification boundary at particular layers that statistically separated data labeled 'true' from data labeled 'false'.The system does not possess a direction for truth. It possesses a mathematical vector in its activation space that linearly separates structural features associated with the 'true' and 'false' labels present in the training distribution.

Basis: Generic to system type		Meta engineers designed Llama's architecture and selected its training data; the resulting parameter space contains mathematical boundaries reflecting the patterns in Meta's data.
we are able to steer the model towards more metaphorical outputwe are able to shift the model's token generation probabilities toward vocabulary associated with metaphor by intervening on its internal activations.The system is not steered like a vehicle. Researchers mechanically alter the model's generation by injecting a computed activation vector into the residual stream at every forward step, mathematically forcing a change in the output logits.

Basis: Documented in source		Researchers execute causal interventions on models built by corporations, altering the output probabilities defined by the original developers.
models have relatively strong performance at understanding metaphor, especially when fine-tunedmodels generate text that statistically aligns well with metaphorical benchmarks, especially after their weights are updated on targeted datasets.The system does not learn to understand. During fine-tuning, its parameters are mathematically adjusted via backpropagation to minimize error rates on a specific dataset; at generation, it produces text from those updated parameters.

Basis: Generic to system type		Developers update the models' parameter weights using datasets curated and labeled by human annotators, which mathematically forces the models' outputs to better match the annotators' classifications.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text exhibits a systematic oscillation between highly precise mechanical framings and deeply agential, anthropomorphic claims. This slippage primarily flows from the mechanical to the agential (mechanical→agential): the researchers isolate a specific, mathematical vector within the model's activation space using Distributed Alignment Search (a mechanical reality), but immediately elevate this finding by naming it a 'metaphoricity concept' or questioning whether the model 'understands' (agential claims). This climb on the agency gradient occurs rapidly; the delegated action of generating a string based on a prompt template becomes the intentional state of 'interpreting,' which finally becomes the epistemic state of 'possessing a concept.' The slippage is most dramatic in the introduction and discussion sections, where the 'curse of knowledge' is highly active. The authors understand the complex correlations that allow the model to pass metaphor benchmarks, and they project their own human experience of semantic comprehension onto the system, using words like 'understand' and 'interpret'. When the text shifts into the methodology section, the vocabulary reverts to mechanical precision ('activation patching,' 'hidden states,' 'matrices'). However, this mechanical language is used to validate the earlier agential claims, creating a dynamic where theoretical and reason-based explanation types enable the slippage. The text treats the AI as a mechanism when describing the experimental intervention, but treats it as an agent when describing the baseline capabilities or the results. Furthermore, human agency is frequently displaced through agentless constructions ('models have some capacity', 'whether they are up to human performance'). By default, this wording can have the effect of obscuring the corporate developers (Meta, Alibaba, Mistral) who built the architectures and curated the training data. The shorthand of referring to the model as an 'interpreter' or a 'judge' renders the massive human labor of instruction-tuning unsayable, while making the illusion of an autonomous, cognitive machine highly sayable.

Metaphor-Driven Trust Inflation​

The metaphorical framings in the text construct an architecture of authority that encourages relation-based trust in statistical systems. The most potent example is the framing of Qwen 2.5 as a 'judge'. This intentional-state attribution serves as a profound trust signal; it implies that the model's outputs are the result of objective evaluation, reasoning, and adherence to normative standards, rather than the mechanical calculation of token probabilities. When the text claims the model 'interprets' or 'understands' metaphor, it accomplishes something far different than claiming the model 'classifies' or 'predicts'. Understanding implies a holistic grasp of meaning, nuance, and context, suggesting the AI is competent to handle novel or out-of-distribution language. This encourages readers to extend relation-based trust—the kind of trust reliant on a belief in the actor's sincerity and comprehension—to a system completely incapable of reciprocating it. The text builds this perceived competence by applying human-trust frameworks to mathematical processes. For instance, citing the existence of a 'truth direction' conflates statistical feature separation with epistemic reliability. When the model inevitably fails, the mechanistic framing is briefly reintroduced (e.g., the need for a 'degeneracy filter' to catch repetitive loops), but these are framed as mechanical glitches rather than fundamental absences of reasoning. The stakes of this trust construction are significant. When audiences extend relation-based trust to these systems, they overestimate the model's reliability in high-stakes domains. If a system is believed to 'understand' metaphor, it might be trusted to analyze sensitive political speech, literature, or legal texts, masking the reality that its decisions are justified only by statistical correlation, not by actual comprehension of human meaning.

Obscured Mechanics​

The anthropomorphic language in this text actively conceals the technical, material, and labor realities of AI production. This framework holds that commercial incentives, the labor of data annotation and human feedback, the environmental and compute costs of training and deployment, and developers' limited disclosure of training data, objectives, and feedback processes are standing conditions of the technology. When the text claims that 'models such as Llama 3 have some capacity to understand' or act as a 'judge', it obscures the corporate entities—Meta for Llama, Alibaba for Qwen, Google for Gemma, and Mistral—that actually engineered these behaviors. Technically, framing the system as possessing 'concepts' or 'understanding' hides the model's absolute reliance on training data and its absence of a causal world model; the statistical nature of 'confidence' in next-token prediction is masked behind the illusion of cognitive certainty. Mechanistically, at the level the source documents, the models process prompts through transformer blocks and generate text based on learned weights, a reality obscured by claims of 'interpretation'. Economically, the wording can have the effect of hiding the commercial objectives of these firms; Meta and Alibaba develop models that perform well on benchmarks not to achieve consciousness, but to secure market dominance and drive adoption. Labor realities are entirely erased: the 'fine-tuning' that allows the model to correctly identify metaphors relies on thousands of invisible data annotators who manually labeled the literal and metaphorical sentences. If the metaphorical language were replaced with mechanistic precision—describing the model as a statistical tool that generates text correlated with Alibaba's or Meta's training data—the vast corporate infrastructure, the dependency on underpaid labor, and the fundamental lack of semantic comprehension would immediately become visible, shifting accountability back to the developers.

Context Sensitivity​

The distribution of anthropomorphic and consciousness-attributing language is highly context-sensitive, following a distinct structural pattern that shapes the reader's understanding of the system. Anthropomorphism heavily concentrates in the introduction, abstract, and conclusion (the vision-setting and summary sections), where the text makes broad claims about what the model 'understands', 'interprets', and the 'concepts' it possesses. In these sections, the climb on the agency gradient reaches epistemic and intentional states. However, when the text shifts to the methodology section (describing activation patching, DAS, and vector interventions), the language becomes rigorously mechanical. This creates a powerful rhetorical effect: the dense mechanical language in the middle of the paper serves to launder and validate the anthropomorphic claims surrounding it. The reader is left with the impression that the 'concept of metaphoricity' has been scientifically proven and located as a physical object within the machine. There is also a marked capabilities versus limitations asymmetry. When describing capabilities, the text uses agential terms ('LLMs have relatively strong performance at understanding'). Conversely, when addressing limitations or failures, the text reverts to mechanical terms, noting that intervention can lead to 'degenerate' output (single words repeating). This asymmetry—capabilities framed as conscious action, limitations framed as mechanical errors—positions the AI as an intelligent agent that occasionally suffers hardware glitches, rather than an inherently statistical system that lacks reasoning entirely. By treating the field's inherited register as standard, the text's vocabulary can have the effect of making the AI's capabilities intuitive for lay audiences reading the introduction, while satisfying technical audiences with mechanistic proof, ultimately leaving all readers with an inflated picture of the system's actual cognitive capacities.

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

An analysis of the text's accountability architecture reveals a systemic pattern of displaced responsibility, where the agency of corporate developers is erased in favor of an autonomous AI narrative. While the text's argument (sourceReading) briefly places responsibility for safe 'steering' on the researchers, its vocabulary routinely places agency onto the models themselves. The researchers ('we') are frequently named as the actors performing experiments (patching, evaluating), but the models ('Llama', 'Qwen', 'they') are presented as the sole actors possessing 'capacities,' 'concepts,' and the ability to 'understand' or 'judge.' The corporate actors—Meta, Alibaba, Google, Mistral—are completely absent from the narrative. This creates an accountability sink: responsibility for the model's baseline behavior disappears into the abstraction of the technology itself. By utilizing agentless constructions ('models are able to perform', 'whether they are up to human performance is debated'), the deliberate, structurally necessary choices made by firms (selecting data, setting optimization objectives, deploying models) are presented as natural inevitabilities. When Qwen is framed as a 'judge,' it acts as a delegate without a named delegator, transferring the authority of Alibaba's training pipeline onto a seemingly neutral artifact. This framework reads the resulting distribution of responsibility as serving the institutional and commercial interests of AI developers, whether or not the authors intended it. If the text consistently named the human actors—stating that 'Alibaba's instruction-tuning causes Qwen to generate classifications' rather than 'Qwen judges'—the illusion of autonomous machine intelligence would collapse. Questions about bias in Alibaba's data, or Meta's deployment priorities, would become askable, and accountability for model failures would shift from the 'AI' back to the corporations that engineered and profit from it.

Conclusion: What This Analysis Reveals​

The Core Finding

The text's vocabulary is structured around three interconnected anthropomorphic patterns: the Model as Comprehending Mind ('understands', 'interprets'), the Model as Container of Ideas ('possesses concepts', 'truth direction'), and the Model as Evaluator ('judge'). The foundational, load-bearing pattern is the Comprehending Mind; the assumption that the system possesses a baseline of semantic awareness must be accepted for the other patterns to function. If the model cannot 'understand', it cannot possess abstract 'concepts' or act as a 'judge'. These consciousness projections do the heavy lifting, mapping epistemic and intentional states onto purely mathematical operations. The metaphorical architecture is complex, blending spatial relationships (locating concepts in vector space) with psychological attributes. Crucially, the text's vocabulary complicates its underlying mechanistic argument. While the authors argue for a highly technical, geometric understanding of how models process text, their reliance on consciousness verbs subtly undercuts this precision, projecting mind onto the very matrices they are attempting to demystify, and displacing responsibility for the model's behavior away from the developers.

Mechanism of the Illusion:​

This framework holds that computational systems process rather than know. The rhetorical architecture of the illusion in this text relies heavily on the ELIZA effect, activated through a specific temporal sequence. The central move occurs when the authors isolate a causal mathematical subspace using Distributed Alignment Search, and immediately label that statistical boundary a 'metaphoricity concept.' By naming a high-dimensional vector after a human psychological construct, the text converts processing into knowing. The curse of knowledge drives this illusion: the authors fully understand the mechanistic reality of the intervention, but they project their own semantic understanding of the resulting output back onto the system. The temporal structure is key: the text establishes the AI as a 'knower' in the introduction ('capacity to understand'), validates this claim with dense mathematical proof in the methodology, and then expands the agential claims in the results (using the model as a 'judge'). This progression plays on audience predispositions to anthropomorphize fluent language, using theoretical and reason-based explanation types to build a convincing narrative of an autonomous, thinking machine, even as it describes linear algebra.

Material Stakes:​

Categories: Epistemic, Institutional, Regulatory/Legal

The metaphorical framing of LLMs as systems that 'understand' and 'judge' carries concrete, material stakes. Epistemically, framing a language model as a 'judge' fundamentally alters knowledge production. Researchers and institutions may begin substituting human qualitative analysis with LLM evaluation, under the false belief that the model is applying reasoned judgment rather than statistical heuristics. This degrades the epistemic rigor of academic and institutional research, as the biases embedded in the training data of firms like Alibaba are laundered into 'objective' verdicts. Institutionally, this leads to automation bias, where organizations deploy models in sensitive moderation or evaluation roles, trusting the AI's 'understanding' to handle nuance it mathematically cannot grasp. The beneficiaries are the tech firms, whose products gain unwarranted authority and market share, while the costs are borne by the public subject to automated, unreasoned decisions. In the Regulatory and Legal domains, consciousness framing creates liability ambiguity. If policymakers accept that models 'possess concepts' or 'interpret' data, they may draft regulations treating AI as autonomous agents rather than corporate products. This shields developers from accountability; if an AI 'judges' poorly, the framing suggests a flaw in the AI's 'reasoning' rather than a negligent data curation decision by the deploying firm. Removing these metaphors threatens the tech industry's narrative of building 'general intelligence,' forcing regulatory focus onto corporate data practices and objective functions rather than the behavior of an imagined digital mind.

AI Literacy as Counter-Practice:​

Practicing critical precision requires systematically dismantling the illusion of mind by replacing consciousness verbs with mechanistic descriptions and restoring human agency to agentless constructions. As demonstrated in the reframings, correcting 'LLMs understand metaphor' to 'LLMs generate text statistically correlated with metaphor' forces the reader to confront the system's absolute dependency on its training data and its lack of awareness. Re-identifying the 'LLM-as-judge' as an 'automated classifier built by Alibaba' restores the corporate actor, making it clear who designed the objective functions and who profits from their use. Systematic adoption of this precision requires structural shifts: academic journals must demand mechanistic translations of anthropomorphic shorthand, and researchers must commit to describing generation-time behavior accurately. However, resistance to this practice is deeply entrenched. The incentives favoring anthropomorphic language—marketing narratives driving corporate valuations, funding bodies seeking 'intelligent' breakthroughs, and the disciplinary habit of linguistic convenience—make precision costly. Anthropomorphic language serves the structural interests of the AI industry by obscuring the labor of annotators, the environmental costs of compute, and the proprietary nature of training data, regardless of an individual author's intent. Critical literacy threatens these interests by rendering the technology mundane, artifactual, and inherently tied to corporate accountability.

Path Forward​

The broader discursive ecology surrounding AI faces a critical juncture regarding vocabulary choices, with different approaches serving different stakeholders. Maintaining the status quo—where terms like 'understands', 'interprets', and 'judge' are used as acceptable shorthand—prioritizes accessibility and narrative resonance. This serves corporate marketing and facilitates rapid public adoption, but costs epistemic clarity, leaving audiences unable to gauge actual system capabilities and masking the corporate decision-making behind the models. Alternatively, a shift toward strict mechanistic precision (e.g., 'generates text from parameters and context') maximizes testability and transparency. This approach serves regulators, safety researchers, and civil society by making the limits of the technology visible, though it risks alienating lay audiences through dense technical jargon. A hybrid approach—anthropomorphic clarity, where metaphors are used but aggressively bounded and corrected in-text—attempts to balance intuition with accuracy, though it often collapses back into the ELIZA effect. Institutional changes could support these shifts: journals could mandate capability disclosures and mechanistic terminology guidelines; funding bodies could incentivize rigorous explanation over hype; and educational frameworks could teach students to translate between statistical reality and metaphorical interface. If mechanistic precision becomes the norm, problems of unwarranted trust and misplaced liability become highly tractable, though the technology may lose its mystique. If anthropomorphic language deepens, we risk embedding the assumption of machine autonomy into law and culture, making corporate accountability nearly impossible to enforce. The choice of vocabulary dictates what society can see, and consequently, what it can govern.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Analyzing the Discourse

I'm currently focused on dissecting the provided paper, aiming for a critical discourse analysis. The primary goal is to understand how the paper explores and represents AI, and the task requires a deep dive into the underlying arguments and methodologies. I've begun to examine how large language models are treated.

Examining Metaphors in AI

I'm now zeroing in on the preprint's argument. My analysis is geared toward the use of mechanistic interpretability to decode metaphorical language within LLMs, specifically examining activation patching and DAS. I'm taking a critical look at the researchers' claims about encoding metaphoricity in model activations and the potential for steering outputs. The preprint's nature is confirmed as a technical report.

Assessing Responsibility in AI

I'm now thinking about the distribution of responsibility in the paper, especially regarding AI harms. The text doesn't strongly anthropomorphize the models; rather, it frames them with mechanistic language. It attributes "capacities" with caveats. I've noted a nuanced assignment of responsibility to researchers and deployers for steering outputs, given the potential risks. My primary focus is identifying potential implications related to LLM control and steering mechanisms.

Framing LLM Capabilities

I'm now cataloging the varied ways the model is presented within the paper. I've recognized several distinct frames: competitor, concept-possessor, evaluator, predictor, steered vehicle, and adapting student. I'm focusing on these various frames, refining and categorizing them based on their implied implications regarding the model's capabilities and role in metaphor understanding. The key is in understanding how these models are conceptualized.

Mapping Source-Target Domains

I'm now mapping the source and target domains. Specifically, I'm examining how these models are discussed in terms of source-target relationships. I've broken down several instances: "capacity to understand," "delegated action," "having a concept," "goal-directed behavior," and "computational activity." The focus is on analyzing how each domain projects onto the other, and what is concealed by such conceptualizations.

Categorizing Brown's Framework

I've just categorized the five instances based on Brown's framework. I'm focusing on the primary and secondary Brown types. I've categorized them based on the text provided; the functional type appears most frequently. I've noted the intentional and genetic types used in the second instance, the theoretical type used in instance four, and a second instance of the functional type at the fifth instance. I will use these categories to refine my understanding of the paper's mechanistic framing.

Revising Mechanistic Descriptions

I'm now revising my approach. I'm actively rewriting select quotes from my initial analysis, shifting them toward mechanistic language to restore human agency. This process highlights a tension within the paper: a frequent oscillation between mechanistic descriptions and the application of cognitive verbs. I'm also cataloging the various points I've noted regarding agency slippage, metaphor-driven trust, and obscured mechanics. I'm noting the context sensitivity in anthropomorphic terms.

Synthesizing Findings

I'm now synthesizing the findings, particularly how the "mind as vector space" metaphor bridges mechanistic and cognitive domains. I'm focusing on the accountability discrepancies between the models as actors and the developers/authors who shaped them. The text frequently oscillates between mechanistic language (vectors, activations) and cognitive verbs, creating an imbalance. Steering is attributed to authors, while baseline model behavior is innate to the AI.

Analyzing Framing with Constraints

I'm now integrating crucial constraints into my analysis, ensuring precision and compliance. I'm focusing on specific directives, such as explicitly attributing AI systems to their creators when they're the subject and stating the structural lens in the appropriate sections. I'm also ensuring that I adhere to the opening premise of the conclusion. My analysis will thus be more complete.

Refining Structural Mapping

I'm now refining the structural mapping using Lakoff & Johnson's framework. I'm ensuring alignment and identifying source-target relationships. I'm focusing on the "capacity to understand" and "steer the model" to see what is concealed or revealed by the metaphors used. I'm using various constraints to ensure precision in my analysis. I'm striving for a balanced representation in the task.

Evaluating Model Attributes

I'm now evaluating new items. I see the potential for a new metaphoricity concept in the model. Further, I've noted direct attributions of goal-directed behavior. The use of Qwen 2.5 72B Instruct as a judge also demands analysis. I'm focusing on the accountability shifts and concealed implications across these various conceptualizations.

Generating Detailed Analyses

I'm now generating detailed text. I'm building out extensive analyses within a mental sandbox, adhering to specific word counts and addressing several key tasks. I'm focusing on the epistemic basis of claims in each section, and analyzing the rhetoric, correcting language and making critical observations. I'm analyzing Task 5.3, which requires the exact phrase concerning the framework. I'm synthesizing these elements into a comprehensive report.

Finalizing Conclusion Language

I'm now integrating the final constraint: the conclusion must begin with, "This framework holds that computational systems process rather than know." I am revising the conclusion's framing to ensure that it correctly reflects this premise and remains consistent with the paper. I've been meticulous about JSON brackets, control characters, and word counts. I've re-examined the constraints on the Epistemic Audit, especially the "knowing vs processing" element.

Analysis Provenance

Run ID: 2026-10-26-the-geometry-of-metaphor-probing-and-ste-metaphor-nkqwzf Raw JSON: 2026-10-26-the-geometry-of-metaphor-probing-and-ste-metaphor-nkqwzf.json Framework: Metaphor Analysis v6.8 Schema Version: 3.4.3 Generated: 2026-10-26T10:04:44.777Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0