Skip to main content

Cognitive Maps in Large Language Models

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.


Source Reading​

Genre: Preprint or technical report · Author position: Academic or independent research

Argument: The text argues that large language models construct internal representations analogous to cognitive maps found in animal brains to support flexible relational reasoning. By performing causal mosaic experiments on open-weight models like Gemma, Llama, and Qwen, the authors claim to demonstrate that LLMs represent abstract relational structure separately from entity identity. This structural separation allegedly allows them to infer unstated relationships, bind novel entities to structural roles, and reason systematically from limited experience. The central purpose is to prove that biological computational principles of spatial navigation and relational mapping naturally emerge in statistical sequence predictors when trained on sufficient data.

Responsibility claims: The text focuses entirely on the mechanistic and computational properties of the models and makes no responsibility claims. It attributes agency directly to the models, treating them as autonomous agents forming beliefs and memories capable of making their own inferences. It does not address developer responsibility, deployment consequences, or the human actors who design, train, and profit from these systems. The text implicitly treats the emergence of these cognitive maps as a natural phenomenon to be discovered rather than a designed artifact of specific optimization pressures chosen by corporate entities.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

1. The Model as Biological Navigator​

Quote: "It has long been proposed that animals construct cognitive maps to support flexible behavior in novel environments. We tested whether large language models form similar representations using relational reasoning tasks"

  • Frame: LLM representations mapped to biological cognitive maps
  • Projection: This metaphor projects the spatial awareness, environmental comprehension, and biological navigation capabilities of living animals onto the static matrices of language models. It implies the model possesses a unified internal world-model and goal-directed spatial awareness. This falls under intentional-state attribution, as it implies the model actively constructs maps to navigate environments. The text supplies evidence of separated vector representations, not subjective spatial awareness. Mechanistically, this behavior generic to systems of this type arises from self-attention heads attending to earlier tokens in the context window to determine the statistical distribution of the next token. The model generates text from its parameters and context; it does not navigate a novel environment.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Hedged/Qualified (The text uses the word similar representations rather than claiming they are identical, though it proceeds to treat the analogy as literal throughout the findings. Considered Direct, but the word similar qualifies the initial mapping.)
  • Implications: This framing heavily inflates perceived sophistication by suggesting models possess genuine spatial awareness and world-modeling capabilities akin to biological life. This creates unwarranted trust in the system's ability to handle out-of-distribution reasoning tasks, as readers intuitively assume an entity with a cognitive map can navigate unforeseen obstacles, masking the system's actual brittle reliance on statistical pattern matching.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text attributes the formation of representations entirely to the large language models. The developers who built the architecture and curated the data (Google for Gemma, Meta for Llama, Alibaba for Qwen) are entirely hidden. This serves to position the models' capabilities as a natural, spontaneous phenomenon rather than the result of deliberate corporate engineering decisions, displacing agency and obscuring the human labor required to train these statistical engines. Considered Partial, but no generic human actors are mentioned here.
Show more...

2. The Model as Conscious Reasoner​

Quote: "Overall, our findings reveal an internal organization analogous to a cognitive map that explains how language models reason with limited information."

  • Frame: Statistical generation as human reasoning
  • Projection: This framing projects the human capacity for logical deduction, conscious deliberation, and epistemic evaluation onto the process of vector multiplication and token prediction. It implies the model understands logical rules and applies them to reach a justified conclusion. This constitutes an epistemic/conscious state attribution. The text provides evidence for accurate token prediction, not conscious reasoning. Mechanistically, generic to systems of this type, the model calculates the probability of the next token based on learned correlations in its parameters and the provided context string; it does not engage in internal logical deduction or epistemic evaluation of facts.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Hedged/Qualified (The text uses the phrase analogous to a cognitive map, which provides a slight qualification to the framing. Considered Direct because reason is unhedged, but the first half of the sentence contextualizes it as an analogy.)
  • Implications: Attributing reasoning to a language model encourages users to rely on the system for high-stakes logical, legal, or medical decisions, assuming the system possesses an internal truth-verification mechanism. It masks the reality that the system will confidently generate logically contradictory text if statistically prompted to do so, increasing the risk of uncritical reliance on AI outputs.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The language models are presented as the sole actors engaged in reasoning. The engineers at Meta, Google, and Alibaba who optimized the models' weights using massive computational resources are entirely erased. This agentless construction serves the interests of AI developers by mystifying the technology, making it appear as an independent intellect rather than a corporate software product shaped by specific optimization targets. Considered Named, but no specific entities are named.

3. The Model as Trauma Victim​

Quote: "Binding was mediated by a sparse, distributed set of key-value pairs; by perturbing these pairs, we implanted false memories that altered model behavior in predictable ways."

  • Frame: Matrix perturbation as memory implantation
  • Projection: This metaphor maps human psychological experiences, specifically autobiographical memory and susceptibility to false memories, onto the manipulation of floating-point numbers in a key-value cache. It projects a conscious, temporal experience of past events onto a stateless computational process. This falls under intentional-state attribution. The evidence supplied is simply that altering vectors changes output probabilities. Mechanistically, documented in the source, the authors overwrite specific tensors in the key-value cache with values from a different prompt, causing the attention heads to pull information that alters the generation of subsequent tokens. The model does not experience memories.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Explicitly Acknowledged (The authors place false memories in explicit scare quotes, acknowledging the metaphorical nature of the claim. Considered Hedged, but the punctuation explicitly marks the non-literal usage.)
  • Implications: Using memory and false memory anthropomorphizes an attack vector or failure mode. It sanitizes vulnerabilities (like prompt injection or cache poisoning) by framing them as relatable human cognitive quirks rather than mechanical security flaws. This can alter policy approaches, making deterministic security failures seem like unavoidable psychological phenomena, thereby reducing the perceived liability of the developer.

Accountability Analysis:

  • Actor Visibility: Named (actors identified)
  • Analysis: The authors use the first-person we implanted to explicitly name themselves as the actors perturbing the system. This correctly identifies the human researchers as the agents altering the system's state. N/A - no agency displacement in this instance. Considered Hidden, but the explicit use of we rules this out.

4. The Model as Deductive Knower​

Quote: "We found that models could infer unstated relationships and retained this ability when familiar entity names were replaced with novel ones"

  • Frame: Pattern matching as logical inference
  • Projection: The term infer projects the epistemic state of grasping logical entailments and deriving truth onto the mechanical process of generating statistically correlated tokens. This attributes an epistemic/conscious state to the model. The evidence provided is behavioral accuracy on text generation tasks. Mechanistically, generic to models of this type, self-attention mechanisms identify structural patterns in the prompt and use parameter weights shaped during training to produce tokens statistically associated with the target relational structure, all without understanding truth or logical validity.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The claim that models could infer unstated relationships is presented as a literal, factual observation of the system's capabilities with no hedging or qualification in the immediate context. Considered Hedged, but no qualifying words are present.)
  • Implications: Describing text generation as inference conflates statistical prediction with deductive certainty. It leads audiences to believe the model possesses an underlying semantic world model that prevents it from generating logical contradictions, hiding the fundamental brittleness of language models when dealing with out-of-distribution reasoning tasks.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The models are presented as the active subjects performing inference and retaining abilities. The human researchers who designed the task, and the corporations (Meta, Google, Alibaba) who built the models, are obscured. This framing makes the system appear autonomously capable, serving the narrative of artificial general intelligence while erasing the massive data annotation labor required to make these statistical patterns emerge. Considered Partial, but no generic humans are mentioned.

5. The Model as Active Student​

Quote: "To construct such a representation, a learner must combine previously acquired knowledge about the relevant relational structure (e.g., the abstract structure of a family tree) with information specific to the current experience"

  • Frame: Parameter optimization as student learning
  • Projection: This metaphor projects the conscious, subjective experience of a human student acquiring knowledge onto the mechanical optimization of a neural network. It implies the system has an epistemic drive and a conscious capacity to experience data. This sits at the level of epistemic/conscious state attribution. The text provides no evidence of conscious experience. Mechanistically, generic to systems of this type, gradient descent algorithms adjust parameter weights to minimize a loss function during training over massive text corpora; the model does not possess acquired knowledge or current experience, only static weights and a context window.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The framing of the model as a learner acquiring knowledge and having experience is stated directly and literally without any scare quotes or qualifying adverbs. Considered Hedged, but the text treats the model identically to a biological learner.)
  • Implications: Framing an algorithm as a learner with knowledge masks the data dependencies of the system. It suggests the model discovers truths about the world autonomously, distracting from the reality that the system merely mimics the biases, structures, and factual assertions contained in its uncredited, often copyrighted, human-generated training data.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The algorithm is personified as a learner, erasing the developers at Google, Meta, and Alibaba who selected the training corpora, defined the objective functions, and executed the training runs. By framing the system as an independent student, the text shifts focus away from corporate data harvesting practices and the human decisions that shape the models parameters. Considered Partial, but no humans are mentioned.

6. The Model as Believer​

Quote: "This would explain why models treated Alice as Eve's mother; however, it is inconsistent with the models’ belief that Carol is Eve's daughter or Alice's granddaughter"

  • Frame: Output probabilities as epistemic beliefs
  • Projection: This metaphor maps the human epistemic state of holding a justified true belief onto a token probability distribution. It projects knowing and subjective commitment onto a static array of numbers. This represents a clear epistemic/conscious state attribution. The evidence supplied is simply that the model outputs certain tokens with higher probability. Mechanistically, generic to systems of this type, the modified key-value cache alters the attention weights, leading the final layer to assign higher logits to the tokens Carol and granddaughter. The model holds no beliefs and is entirely unaware of the text it generates.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Direct (Unacknowledged) (The phrase the models' belief is stated entirely literally, as a direct attribute of the mathematical system, without any scare quotes, hedging, or qualification. Considered Explicitly Acknowledged, but the scare quotes used elsewhere for false memories are missing here.)
  • Implications: Attributing beliefs to an AI system fundamentally distorts public understanding of how language models fail. If a model holds beliefs, then generating false information is seen as a simple mistake or a difference of opinion. Recognizing that the model merely predicts tokens based on statistical correlation forces the realization that hallucinations are an inherent, inescapable structural feature of the technology, not a fixable cognitive error.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text attributes beliefs directly to the models, treating them as autonomous epistemic agents. The developers (Meta, Google, Alibaba) whose specific architectural choices and training data distributions resulted in these specific token outputs are entirely absent. This language effects an accountability sink, where the model itself becomes the responsible agent for the generated falsehoods, shielding the deploying corporations from scrutiny. Considered Partial, but no human actors are implied.

7. The Model as Information Organizer​

Quote: "To study how language models use structural knowledge to organize newly presented information for inference, we designed a set of relational-reasoning tasks"

  • Frame: Self-attention as conscious organization
  • Projection: This framing maps the intentional human act of sorting, comprehending, and organizing data onto the mathematical operation of self-attention matrices. It projects goal-directed behavior and epistemic understanding onto mechanical state updates. This operates at intentional-state attribution. Mechanistically, generic to systems of this type, tokens are projected into query, key, and value vectors; the dot products of queries and keys determine attention weights, which are then used to compute weighted sums of the value vectors. The system mathematically processes the vectors; it does not organize knowledge.
  • Agency Level: Intentional-state attribution
  • Acknowledgment: Direct (Unacknowledged) (The claim that models use structural knowledge to organize information is presented as the literal premise of the study, with no hedging applied to the verbs. Considered Hedged, but the framing stands entirely unqualified.)
  • Implications: This language creates a deceptive sense of reliability. If an audience believes a system is actively organizing knowledge, they assume the system has internal consistency and cross-checking mechanisms. In reality, attention mechanisms can easily draw upon contradictory patterns in the context window, leading to confidently asserted falsehoods that an actively organizing mind would catch.

Accountability Analysis:

  • Actor Visibility: Named (actors identified)
  • Analysis: While the model is framed as the actor organizing information, the human authors explicitly identify themselves as the agents who designed a set of relational-reasoning tasks. However, the agency regarding the models capabilities is displaced from the developers (Google, Meta, Alibaba) onto the model. Considered Partial, but the explicit we designed makes the researchers visible in the experimental design. N/A - no agency displacement in this specific instance regarding the experiment.

8. The Model as Dynamic Restructurer​

Quote: "Therefore, once models are allowed to reason, the distinction between constructing a cognitive map and performing inference over it becomes blurred: models may dynamically refine and restructure the cognitive map as inference unfolds."

  • Frame: Autoregressive generation as dynamic cognitive restructuring
  • Projection: This metaphor projects the advanced human cognitive capacity to update mental models during complex problem-solving onto the simple autoregressive loop of appending newly generated tokens to the context window. It suggests high-level intentional-state attribution and epistemic evaluation. The evidence is just improved accuracy with chain-of-thought prompting. Mechanistically, generic to systems of this type, when generating intermediate reasoning tokens, the model appends each new token to the prompt, updating the key-value cache. Subsequent tokens attend to this expanded cache. The model does not dynamically refine a map; it simply performs the exact same static forward pass on a progressively longer string of text.
  • Agency Level: Epistemic/conscious state
  • Acknowledgment: Hedged/Qualified (The use of the modal may provides a slight hedge to the prediction, but the framing of allowed to reason and restructure the cognitive map remains grammatically direct. Considered Direct, but the modal may qualifies the specific action.)
  • Implications: Describing chain-of-thought generation as dynamic cognitive restructuring profoundly exaggerates the autonomy and adaptability of the system. It leads users to believe the system can self-correct logical errors through introspection. In reality, if early generated tokens contain a statistical error, the autoregressive nature of the system will compound the error, leading to systemic failures that a genuinely restructuring mind would avoid.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The passive construction once models are allowed to reason obscures who is allowing them (deployers, prompt engineers). The models are then framed as the sole agents refining and restructuring their internal maps. The developers at Google, Meta, and Alibaba who engineered the autoregressive inference loops are completely erased, making the software's execution appear as a spontaneous act of cognition. Considered Partial, but the passive voice hides the human deployer.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: Biological navigation and animal neuroscience, where living organisms create mental representations of physical space to plan routes and survive. → The internal vector states (activation tensors) of large language models like Gemma, Llama, and Qwen during the processing of a text prompt.​

Quote: "animals construct cognitive maps to support flexible behavior in novel environments. We tested whether large language models form similar representations"

  • Source Domain: Biological navigation and animal neuroscience, where living organisms create mental representations of physical space to plan routes and survive.
  • Target Domain: The internal vector states (activation tensors) of large language models like Gemma, Llama, and Qwen during the processing of a text prompt.
  • Mapping: The mapping projects the spatial, physical, and survival-oriented cognition of animals onto the linear algebra of transformer networks. It invites the assumption that intermediate hidden states inside an LLM are equivalent to the unified, intentional world-models of living beings. It maps the biological imperative to understand an environment onto the loss-minimizing token prediction objective of a statistical algorithm.
  • What Is Concealed: This mapping completely conceals the statistical, text-based nature of the system. It hides the fact that a language model has no physical environment, no sensory grounding, and no unified internal world. It obscures the proprietary training data and specific objective functions designed by Google, Meta, and Alibaba, replacing these mechanistic realities with the illusion of an autonomous, perceiving organism.
Show more...

Mapping 2: Human conscious logic and deductive reasoning, where a conscious mind evaluates evidence, applies logical rules, and reaches a justified conclusion. → The forward pass of a transformer network generating output probabilities based on a short context window.​

Quote: "explains how language models reason with limited information."

  • Source Domain: Human conscious logic and deductive reasoning, where a conscious mind evaluates evidence, applies logical rules, and reaches a justified conclusion.
  • Target Domain: The forward pass of a transformer network generating output probabilities based on a short context window.
  • Mapping: The structure projects the epistemic properties of human thought onto matrix multiplication. The prompt is mapped to limited information, the forward pass of the neural network is mapped to reasoning, and the output token with the highest logit is mapped to the logical conclusion. This invites the assumption that the system possesses a semantic understanding of the relationships it processes.
  • What Is Concealed: This mapping conceals the total absence of semantic understanding or truth-evaluation in a language model. It hides the statistical dependencies on the training data; the model is not reasoning from limited information, but rather matching the prompt against billions of parameters shaped by massive, undisclosed corpora. It obscures the mechanical reality that the model is simply computing probabilities, not deducing facts.

Mapping 3: Human psychology, trauma, and memory manipulation, where a subject's conscious recollection of their past is distorted. → The manual alteration of specific floating-point numbers within the key-value cache of a transformer model during inference.​

Quote: "we implanted 'false memories' that altered model behavior in predictable ways."

  • Source Domain: Human psychology, trauma, and memory manipulation, where a subject's conscious recollection of their past is distorted.
  • Target Domain: The manual alteration of specific floating-point numbers within the key-value cache of a transformer model during inference.
  • Mapping: The mapping projects autobiographical experience and temporal consciousness onto static data structures. The key-value cache is mapped to human memory, and the manual overwriting of vectors is mapped to the psychological implantation of a false memory. It invites the assumption that the model experiences the prompt as a lived timeline.
  • What Is Concealed: The mapping conceals the complete statelessness of the deployed model between forward passes. It hides the mechanical simplicity of the intervention: overwriting a matrix forces the self-attention heads to retrieve different values, altering the next-token prediction. Attributing this to false memories obscures the fact that the system has no memory whatsoever, only a temporary buffer of vectors that is wiped clean after every generation.

Mapping 4: Human logical deduction, where a person understands the transitive properties of relationships (e.g., A > B, B > C means A > C) regardless of the specific subjects. → The statistical generalization capabilities of attention heads inside language models when processing out-of-distribution tokens.​

Quote: "models could infer unstated relationships and retained this ability when familiar entity names were replaced with novel ones"

  • Source Domain: Human logical deduction, where a person understands the transitive properties of relationships (e.g., A > B, B > C means A > C) regardless of the specific subjects.
  • Target Domain: The statistical generalization capabilities of attention heads inside language models when processing out-of-distribution tokens.
  • Mapping: The structure projects conscious comprehension of logical rules onto statistical pattern matching. It maps the generation of a statistically likely token to the act of inference, and the accurate processing of synthetic tokens to the retention of a cognitive ability. It invites the assumption that the model possesses an abstract rule-engine.
  • What Is Concealed: This conceals the mechanistic reality that the models are relying on structural co-occurrence patterns learned from massive corporate training datasets. It hides the lack of true causal modeling; the model does not infer, it simply routes information through its layers such that the output probability aligns with patterns of text it was trained to mimic. The transparency obstacle of proprietary training data prevents verifying if similar structures were memorized.

Mapping 5: A human student or conscious learner actively acquiring knowledge and synthesizing past lessons with current conscious experience. → The process of a trained neural network applying static parameter weights (learned via gradient descent) to a new input prompt.​

Quote: "To construct such a representation, a learner must combine previously acquired knowledge about the relevant relational structure (e.g., the abstract structure of a family tree) with information specific to the current experience"

  • Source Domain: A human student or conscious learner actively acquiring knowledge and synthesizing past lessons with current conscious experience.
  • Target Domain: The process of a trained neural network applying static parameter weights (learned via gradient descent) to a new input prompt.
  • Mapping: The mapping projects subjective epistemic states onto static mathematical weights. The static parameter weights of the model are mapped to acquired knowledge, and the text prompt currently in the context window is mapped to current experience. It invites the assumption that the system has a continuous, experiencing self that actively synthesizes information.
  • What Is Concealed: This mapping conceals the static, frozen nature of a deployed language model's weights. It obscures the fact that the model is not actively learning or experiencing anything during generation; it is mechanically executing matrix multiplications. It also completely hides the human labor (data annotation, RLHF) that actually shaped the weights, attributing the resulting statistical patterns to the autonomous effort of a learner.

Mapping 6: Human epistemology and subjective states, where a person holds a justified conviction about the truth of a proposition. → The token probability distribution generated by the final layer of a language model.​

Quote: "inconsistent with the models’ belief that Carol is Eve's daughter"

  • Source Domain: Human epistemology and subjective states, where a person holds a justified conviction about the truth of a proposition.
  • Target Domain: The token probability distribution generated by the final layer of a language model.
  • Mapping: This mapping projects the conscious state of holding a truth commitment onto the highest-probability logit in an output vector. It maps the generation of the token Carol to the model holding a belief. It invites the dangerous assumption that the system has an internal mechanism for evaluating truth and distinguishing fact from fiction.
  • What Is Concealed: This mapping entirely conceals the probabilistic nature of text generation. It hides the fact that the model possesses no semantic grounding, no concept of truth, and no subjective awareness of the text it generates. It obscures the mechanistic reality that the token output is simply the result of attention weights heavily weighting certain vectors over others, not an epistemic commitment by a conscious agent.

Mapping 7: A conscious human agent actively sorting, categorizing, and making sense of new data to draw a conclusion. → The mathematical operations of self-attention layers computing attention weights and value vectors over a sequence of tokens.​

Quote: "models use structural knowledge to organize newly presented information for inference"

  • Source Domain: A conscious human agent actively sorting, categorizing, and making sense of new data to draw a conclusion.
  • Target Domain: The mathematical operations of self-attention layers computing attention weights and value vectors over a sequence of tokens.
  • Mapping: The mapping projects goal-directed cognitive effort onto deterministic linear algebra. The query-key dot products are mapped to the active organization of information, and the parameter weights are mapped to structural knowledge. It invites the assumption that the system actively curates its context window to ensure logical consistency.
  • What Is Concealed: This mapping conceals the blind, mechanical nature of self-attention. Attention heads do not organize information for a purpose; they compute weighted sums mathematically determined by their trained parameters. This hides the structural vulnerability of models to distraction and context manipulation, as the system does not consciously organize data but mathematically mixes it, often leading to hallucinated outputs if the input contains statistically confusing patterns.

Mapping 8: High-level human metacognition, where a person updates their mental model of a problem while actively thinking through it. → The autoregressive text generation loop, where each newly generated token is appended to the context window and processed in the next forward pass.​

Quote: "models may dynamically refine and restructure the cognitive map as inference unfolds."

  • Source Domain: High-level human metacognition, where a person updates their mental model of a problem while actively thinking through it.
  • Target Domain: The autoregressive text generation loop, where each newly generated token is appended to the context window and processed in the next forward pass.
  • Mapping: The structure projects metacognitive awareness and active self-correction onto a simple feedback loop. The appending of a new token to the cache is mapped to dynamically refining a cognitive map. It invites the assumption that chain-of-thought prompting literally allows the model to think and correct its internal logic.
  • What Is Concealed: This mapping conceals the extreme mechanical simplicity of autoregressive generation. The model is not restructuring anything; it is performing the exact same static forward pass over a slightly longer string of text. It obscures the fact that if an early token is statistically flawed, the model has no independent metacognitive map to correct it, and will blindly condition its subsequent generation on the flawed token, compounding the error.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "Models stored observed relationships as pairwise links that they composed retrospectively when queried, rather than prospectively positioning each entity in a global coordinate system."

  • Explanation Types:

    • Functional (Primary): Explains behavior by role in self-regulating system with feedback
    • Theoretical (Secondary): Embeds in deductive framework, may invoke unobservable mechanisms
  • Analysis (Why vs. How Slippage): This explanation frames the AI mechanistically in terms of its internal data organization (how), contrasting a graph-based storage mechanism with a global coordinate system. By using theoretical and functional explanation types, the authors describe the specific way information is routed and processed during the forward pass. This choice emphasizes the specific computational architecture underlying the model's behavior, treating the model as a mechanism that stores and composes data structures. However, it slightly slips into agential framing by stating the models composed them, subtly attributing active processing choices to the system rather than describing the deterministic execution of the attention heads.

  • Consciousness Claims Analysis: The passage avoids explicit consciousness verbs, relying on functional terms like stored, composed, and positioning. On the agency gradient, this sits at computational activity, safely below knowing. However, curse of knowledge dynamics are present: the authors project their own graph-theory understanding of relationships (pairwise links) onto the high-dimensional vectors in the key-value cache. Mechanistically, based on reasonable inference from the source's mosaic experiments, the model does not literally store distinct pairwise links like a database. Instead, generic to systems of this type, the self-attention mechanism computes attention weights between the query tokens and earlier relationship tokens. During generation, information from these dispersed tokens is aggregated mathematically in the final layers to produce the output token. The language of composed retrospectively is a human-readable metaphor for the complex, simultaneous vector additions occurring across multiple attention heads.

  • Rhetorical Impact: This framing shapes audience perception by making the AI appear highly structured and logically rigorous. By comparing the system to a graph database that composes links, it encourages performance-based trust, suggesting the model handles data reliably and deterministically. While less dangerous than consciousness framing, it still risks overestimating the model's robustness, as audiences might assume the system possesses a clean, error-free internal database rather than a noisy, statistical aggregation of vectors that is prone to hallucination when patterns conflict.

Show more...

Explanation 2​

Quote: "learning from these corpora using objectives that favor generalization across contexts (e.g., next-token prediction) may promote representational strategies that capture recurring relational structure."

  • Explanation Types:

    • Genetic (Primary): Traces origin through dated sequence of events or stages
    • Theoretical (Secondary): Embeds in deductive framework, may invoke unobservable mechanisms
  • Analysis (Why vs. How Slippage): This explanation frames the AI mechanistically, tracing its current capabilities (capturing structure) back to its training process (how it emerged). By identifying the specific optimization objective (next-token prediction) over large corpora, the explanation correctly identifies the forces shaping the model's parameters. This emphasizes the material and computational origins of the system's behavior, avoiding intentional language. It accurately portrays the representational strategies not as conscious choices by the model, but as emergent properties promoted by human-designed training objectives.

  • Consciousness Claims Analysis: The passage contains no consciousness verbs, using strictly mechanistic terms like learning (in the machine learning sense), favor, promote, and capture. This places the system at computational activity on the agency gradient. The author successfully avoids projecting subjective understanding onto the system. Mechanistically, generic to systems of this type, the training process involves passing massive text datasets through the model and using gradient descent algorithms to iteratively adjust the parameter weights to minimize the cross-entropy loss of next-token prediction. Over billions of iterations, the weights configure themselves to approximate the statistical distributions of the training data. At generation time, the deployed model produces text from these frozen parameters. The explanation aligns perfectly with this mechanistic reality.

  • Rhetorical Impact: This framing appropriately grounds the audience's understanding in the statistical and computational realities of machine learning. It reduces the perceived autonomy of the system by reminding readers that its capabilities are the direct result of a specific objective function applied to a specific dataset. This fosters appropriate skepticism, as audiences who understand that the system merely captures statistical structures driven by next-token prediction are less likely to attribute genuine reasoning or truth-seeking motives to its outputs.

Explanation 3​

Quote: "We found that models could infer unstated relationships and retained this ability when familiar entity names were replaced with novel ones"

  • Explanation Types:

    • Empirical Generalization (Primary): Subsumes events under timeless statistical regularities
    • Dispositional (Secondary): Attributes tendencies or habits
  • Analysis (Why vs. How Slippage): This explanation blends empirical observation with agential framing. While it describes a timeless statistical regularity (the model successfully outputs the correct token regardless of the specific entity name), it uses the highly agential verb infer. This shifts the explanation from describing how the model typically behaves (generating statistically appropriate tokens) to why it appears to choose them (because it deduced the relationship). This emphasizes the model's cognitive-like capabilities while obscuring the mechanical reality of attention mechanisms processing out-of-distribution embeddings.

  • Consciousness Claims Analysis: The passage explicitly uses the consciousness verb infer, elevating the claim to epistemic/conscious state attribution on the agency gradient. It crosses the line from processing to knowing. The authors succumb to the curse of knowledge, observing the model output the correct text for a logic puzzle and projecting their own human deductive reasoning (inference) onto the system. Mechanistically, generic to systems of this type, substituting novel entity names simply provides different input embeddings to the network. Because the relational tokens (e.g., 'daughter') remain the same, the attention heads route information similarly through the layers. At generation time, the model produces text from its parameters based on this structural similarity. The model does not infer; it mathematically processes the novel embeddings according to trained statistical associations.

  • Rhetorical Impact: Using the word infer profoundly shapes audience perception, migrating the system from a tool that calculates to an agent that deduces. This consciousness framing generates inappropriate relation-based trust. If audiences believe the AI genuinely infers unstated facts, they will trust it to handle complex, real-world logical deductions (e.g., in medical diagnoses or legal analysis) where statistical pattern matching is insufficient and dangerous. It masks the reality that the system is entirely dependent on training data patterns.

Explanation 4​

Quote: "This would explain why models treated Alice as Eve's mother; however, it is inconsistent with the models’ belief that Carol is Eve's daughter or Alice's granddaughter"

  • Explanation Types:

    • Intentional (Primary): Refers to goals/purposes, presupposes deliberate design
    • Reason-Based (Secondary): Gives agent's rationale, entails intentionality and justification
  • Analysis (Why vs. How Slippage): This explanation is entirely agential, attempting to explain the model's behavior by referring to its internal logic and epistemic commitments (why). By attributing a belief to the model, the explanation presupposes intentionality, justification, and a cohesive worldview. This severely obscures the mechanical reality of token probabilities. It emphasizes the model as a conscious actor whose outputs must be evaluated for logical consistency with its other internal beliefs, completely ignoring the fact that the system is a stateless statistical engine.

  • Consciousness Claims Analysis: This is a stark epistemic/conscious state attribution. The use of the noun belief asserts that the system holds justified epistemic states. The curse of knowledge is total here: the authors evaluate the logical consistency of the generated text and project that same concern for logical consistency onto the model itself. Mechanistically, generic to systems of this type, the model generated high-probability logits for 'Carol' in one context and 'Alice' in another. During generation, the specific attention weights created by the modified cache resulted in these specific outputs. The model has no internal beliefs, no concept of consistency, and no awareness of the text it is producing. The contradictions are mechanical artifacts of vector math, not inconsistent beliefs.

  • Rhetorical Impact: This framing is deeply misleading. Describing output probabilities as beliefs grants the system immense unwarranted autonomy and epistemic authority. It shifts liability away from the developers; if the model has its own beliefs, it acts as an independent moral agent. This severely damages critical literacy, as readers are encouraged to psychoanalyze the machine rather than audit the training data, optimization functions, and corporate decisions that actually caused the specific text to be generated.

Explanation 5​

Quote: "Once models are allowed to reason, the distinction between constructing a cognitive map and performing inference over it becomes blurred: models may dynamically refine and restructure the cognitive map as inference unfolds."

  • Explanation Types:

    • Functional (Primary): Explains behavior by role in self-regulating system with feedback
    • Intentional (Secondary): Refers to goals/purposes, presupposes deliberate design
  • Analysis (Why vs. How Slippage): This passage mixes functional description with extreme intentional framing. It describes the feedback loop of autoregressive generation (functional), but frames it entirely as cognitive, goal-directed behavior (why it appears to think). The choice to use allowed to reason, dynamically refine, and restructure emphasizes an illusion of metacognition. It obscures the blind, deterministic nature of appending tokens to a context window, painting a picture of an autonomous intellect actively improving its own internal architecture in real-time.

  • Consciousness Claims Analysis: The passage employs severe consciousness verbs (reason, refine, restructure) and explicit cognitive concepts (cognitive map). It asserts an epistemic/conscious state. The authors project their own experience of human problem-solving (where working memory is actively updated) onto the autoregressive loop. Mechanistically, generic to systems of this type, during chain-of-thought generation, the model predicts a token, appends it to the input string, and performs another forward pass over the slightly longer string. The key-value cache expands by one token position. The parameters remain frozen. The model does not dynamically refine a map; it generates text statistically associated with the increasingly longer context. There is no metacognitive restructuring occurring, only repeated matrix multiplications.

  • Rhetorical Impact: This language constructs a highly exaggerated perception of AI autonomy and capability. By suggesting the model can dynamically restructure its own understanding, it promises a level of reliability and self-correction that the system fundamentally lacks. This leads users to place dangerous levels of trust in chain-of-thought outputs, assuming the model has logically vetted its answers, when it has merely generated text that mimics the appearance of human reasoning. Decisions to deploy these systems in autonomous agentic roles hinge on precisely this illusion of self-correcting capability.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
Overall, our findings reveal an internal organization analogous to a cognitive map that explains how language models reason with limited information.Overall, our findings reveal an internal vector organization that explains how language models generate statistically likely tokens based on limited context.At generation time, the model does not reason; it calculates next-token probabilities by using self-attention mechanisms to weigh information from the prompt against parameters optimized during training on massive text corpora.

Basis: Generic to system type
—
by perturbing these pairs, we implanted 'false memories' that altered model behavior in predictable ways.By overwriting specific vectors in the key-value cache, we altered the attention weights, shifting the model's output token probabilities in predictable ways.The model does not experience or possess memories. During generation, replacing values in the key-value cache mechanically forces the attention heads to retrieve different vector representations, mathematically changing the logits produced at the final layer.

Basis: Documented in source
—
We found that models could infer unstated relationships and retained this ability when familiar entity names were replaced with novel onesWe found that models generated text consistent with unstated relationships and retained this statistical pattern-matching capability when familiar entity embeddings were replaced with novel ones.The model does not deduce or infer truths. At generation time, attention mechanisms route information based on the structural similarity of the relational tokens (e.g., 'daughter'), producing accurate outputs despite changes to the specific entity embeddings.

Basis: Generic to system type
—
To construct such a representation, a learner must combine previously acquired knowledge about the relevant relational structure (e.g., the abstract structure of a family tree) with information specific to the current experienceTo generate this output, the neural network processes the current prompt by applying parameter weights that were optimized during training to mimic relational structures found in the training data.The model is not a learner acquiring knowledge or having experiences. Its static parameters were shaped during training by data selected by developers. At generation time, it applies these frozen weights to the context window to produce text.

Basis: Generic to system type
To generate this output, the neural network processes the current prompt by applying parameter weights that engineers at Google, Meta, and Alibaba optimized during training to mimic relational structures found in their curated datasets.
however, it is inconsistent with the models’ belief that Carol is Eve's daughter or Alice's granddaughterhowever, this specific vector manipulation did not cause the model to generate highest-probability logits for 'Carol' when prompted about Eve's daughter or Alice's granddaughter.The model does not hold beliefs or check for logical consistency. During generation, the modified attention weights simply mathematically favored different tokens depending on the specific query structure, resulting in contradictory text outputs.

Basis: Generic to system type
—
To study how language models use structural knowledge to organize newly presented information for inference, we designed a set of relational-reasoning tasksTo study how the attention mechanisms of language models process structural patterns in newly presented prompts to generate correlated text, we designed a set of relational-reasoning tasks.The model does not organize information or possess structural knowledge. Generic to transformers, attention heads compute dot products between queries and keys to generate weighted sums of value vectors, a deterministic mathematical process executed during the forward pass.

Basis: Generic to system type
—
once models are allowed to reason, the distinction between constructing a cognitive map and performing inference over it becomes blurred: models may dynamically refine and restructure the cognitive map as inference unfolds.Once models generate chain-of-thought tokens, the context window continuously expands: the model processes an increasingly longer sequence of generated text during each subsequent forward pass, altering the statistical probabilities of future tokens.The model does not reason, refine, or restructure anything. During autoregressive generation, it simply appends each newly predicted token to the context window and performs identical, static matrix multiplications over the expanded text string.

Basis: Generic to system type
Once deployers prompt the models to generate chain-of-thought tokens, the context window continuously expands...
the models’ abilities extended immediately to novel instantiations of familiar relational structures.the models consistently generated highly probable tokens when processing prompts containing novel entity embeddings inserted into familiar relational text structures.The model does not understand novel instantiations. At generation time, the self-attention mechanisms recognize the unchanged structural tokens in the prompt, successfully routing the novel entity embeddings through the network based on parameters optimized during training.

Basis: Generic to system type
—

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text systematically moves between rigorous mechanical descriptions of vector math and expansive agential framings, demonstrating severe agency slippage. This pattern relies heavily on the curse of knowledge, where the authors project their own understanding of relational tasks onto the statistical outputs of the model.

The slippage typically flows in the mechanical-to-agential direction and concentrates dramatically at the transitions between methodological execution and result interpretation. In the methodology, the text grounds itself in linear algebra: it describes "linear projections," "activation tensors," and "dot products." However, the moment these mathematical operations produce a measurable shift in output probabilities, the text abandons mechanics for consciousness projection. For example, when the authors intervene on the key-value cache (the "mosaic" experiments), they do not merely state that the output logits shifted. Instead, they claim they "implanted 'false memories'" and that the model demonstrated a "belief that Carol is Eve's daughter." This represents a stark climb up the agency gradient, jumping abruptly from computational activity (modifying floating-point arrays) to intentional-state attribution (belief, memory).

This oscillation is structurally enabled by the authors' use of functional and reason-based explanations. By defining the model's intermediate vector states functionally as "cognitive maps," the text creates a linguistic bridge allowing mechanistic observations to masquerade as cognitive phenomena. When the authors observe that manipulating a specific layer's activations changes the generated token, this genetic/functional observation is immediately translated into a reason-based explanation: the model output 'Alice' because it 'inferred' the unstated relationship.

Furthermore, the text heavily employs agentless passive constructions that obscure the human actors. Phrases like "once models are allowed to reason" or "knowledge can be organized" systematically remove agency from the human deployers and developers. The engineers at Meta, Google, and Alibaba who designed the objective functions, collected the data, and constructed the autoregressive inference loops are erased. In their place, the model is positioned as the sole active agent constructing its own "understanding."

This wording can have the effect of conditioning readers to view statistical text generation as conscious deliberation. By projecting knowing onto processing, the human choices underlying the system's architecture become unsayable within the text's paradigm. The system is no longer a corporate artifact operating under delegated constraints; it is transformed linguistically into a naturally occurring cognitive entity. Shorthand terms like "infer" and "believe" might be convenient for researchers, but their effect is to systematically displace responsibility and mask the mechanical reality of the technology.

Metaphor-Driven Trust Inflation​

The central metaphorical architecture of this text revolves around the concept of the "cognitive map" and the language of "reasoning." By explicitly comparing large language models to animals navigating physical mazes and humans making logical deductions, the authors construct a powerful framework for perceived authority and competence. This consciousness framing functions as a profound trust signal, fundamentally altering how audiences relate to the technology.

Claiming that an AI "reasons," "infers," and possesses "beliefs" accomplishes something entirely different than claiming it "calculates," "correlates," or "predicts." The epistemic verbs suggest that the system possesses a justified relationship with truth—that its outputs are the result of internal logical verification rather than mere statistical approximation. This significantly inflates perceived sophistication. The text encourages a dangerous transfer of trust, inviting audiences to apply human-trust frameworks to a statistical system. When a text asserts that a model holds a "belief," it invites relation-based trust—the kind of trust predicated on sincerity, understanding, and intentionality—rather than performance-based trust, which merely requires statistical reliability.

The risk here is substantial: audiences extending relation-based trust to systems completely incapable of reciprocating are uniquely vulnerable to automation bias and over-reliance. When the system eventually produces a hallucination or failure, a user primed to view it as a "reasoning" agent with "cognitive maps" will likely misinterpret the failure as a minor cognitive lapse or a "glitch" rather than a fundamental limitation of next-token prediction.

The text's management of system limitations further compounds this issue. While capabilities are consistently framed in highly agential, cognitive terms (the model infers, understands, learns), the underlying mechanics that cause failures are often obscured or treated playfully. Attributing "false memories" to the system sanitizes a deep mechanical vulnerability—susceptibility to context manipulation—by framing a deterministic failure as a relatable, almost human cognitive quirk.

By heavily utilizing intentional and reason-based explanations to describe successful relational logic, the text constructs a sense that the AI's decisions are rigorously justified by an internal model of the world. In reality, the AI possesses no world model, no subjective awareness, and no capacity to verify the "truth" of the family trees it generates. It simply processes text according to the statistical distribution of its training corpus. This framing insulates the technology from appropriate skepticism, masking the brittleness of systems that rely entirely on surface-level statistical patterns rather than grounded semantic understanding.

Obscured Mechanics​

The anthropomorphic and consciousness-attributing language in this text actively conceals the technical, material, and economic realities of large language models. This framework holds that commercial incentives, the labor of data annotation and human feedback, the environmental and compute costs of training and deployment, and developers' limited disclosure of training data, objectives, and feedback processes are standing conditions of the technology, whether or not a given text mentions them.

Applying the "name the corporation" test reveals how deeply the language obscures human infrastructure. The text repeatedly states that "models learn," "models infer," or "models reason." In reality, Google (developer of Gemma), Meta (developer of Llama), and Alibaba (developer of Qwen) optimized statistical weights over billions of human-generated texts. The text claims to analyze "open-weight" models, yet it encounters immense transparency obstacles regarding the proprietary training datasets and precise RLHF pipelines these corporations used to shape the models' parameters. The text ignores these limitations, instead confidently asserting the spontaneous emergence of cognitive structures.

Technically, the metaphorical framing of "knowing" and "believing" hides the fact that the system relies entirely on statistical token co-occurrence. There is no ground truth or causal model in a transformer; there are only floating-point numbers in high-dimensional matrices. By claiming the model possesses a "cognitive map," the text obscures the reality that attention mechanisms merely compute weighted sums deterministically during a forward pass.

Materially, the illusion of an ethereal "cognitive map" distracts from the massive physical realities of the technology. Creating a system that can simulate reasoning through next-token prediction requires vast environmental costs, immense energy consumption, and massive data center infrastructure. The text erases the heavy carbon footprint required to compute the specific parameter updates that allow Gemma or Llama to accurately predict a "daughter" token.

Furthermore, the labor of thousands of invisible workers—data annotators, RLHF contractors, and content moderators—is completely erased. These workers painstakingly shape the exact text distributions the authors now celebrate as emergent "reasoning," yet their contributions are replaced in the text by the spontaneous "learning" of the autonomous algorithm.

Economically, the commercial objectives of Google, Meta, and Alibaba are hidden behind the veil of neutral scientific discovery. These corporations build LLMs to capture market share, drive user engagement, and integrate into enterprise software, not to replicate mammalian hippocampus functions. The metaphors benefit these exact corporate entities by mystifying their products, framing them as highly sophisticated cognitive agents rather than resource-intensive statistical engines designed for profit. Replacing these metaphors with precise mechanistic language would immediately make visible the corporate decisions and human labor that sustain the illusion of artificial cognition.

Context Sensitivity​

The distribution of anthropomorphic and consciousness-attributing language across the text is not uniform; it follows a distinct pattern that shapes the reader's understanding of the system. The text exhibits a stark asymmetry between its detailed methodological sections and its broader claims about capability and theory, intensifying anthropomorphism precisely when summarizing the system's significance.

In the technical sections describing the "mosaic" experiments, the language is appropriately mechanical. The authors describe "activation tensors," "linear projections," and "hidden vectors." However, the moment the text interprets the results of these vector manipulations, the language abruptly climbs the agency gradient. "Processing" becomes "understanding," and outputting a specific token becomes "inferring unstated relationships." This register shift is profound: what begins as "X is structurally analogous to Y" (acknowledged metaphor) frequently literalizes into "X does Y" (direct capability claim).

This pattern is most evident in the introduction and discussion sections, where consciousness claims concentrate heavily. Here, the text posits that the model acts as a "learner" acquiring "knowledge" and dynamically "restructuring" its cognitive map. By bracketing the rigorous mathematical methodology with highly agential vision-setting, the text creates a rhetorical sandwich. The mechanical rigor of the middle sections serves to legitimize the sweeping cognitive claims made at the beginning and end. This positioning strongly affects the reader, suggesting that the existence of internal cognitive reasoning has been mathematically proven rather than merely interpreted from statistical outputs.

Furthermore, the text exhibits a capabilities versus limitations asymmetry. When the model successfully completes a relational logic task, the capability is framed in agential, conscious terms: the model "inferred," "reasoned," or utilized its "belief." However, when addressing the model's limitations or anomalous behaviors (such as answering incorrectly after a cache manipulation), the text resorts to either mechanical language or playful anthropomorphism (like "false memories").

This distribution serves the field's inherited register, conditioning audiences—especially non-technical readers—to view the system primarily through a cognitive lens. The mechanical language sits alongside stronger anthropomorphism not to clarify the mechanism, but to provide an aura of scientific certainty to the illusion of mind. The pattern suggests an implied audience that desires a bridge between the opaque mathematics of neural networks and the intuitive understanding of biological cognition, a desire the text fulfills by linguistically transforming statistical text generation into artificial thought.

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

The accountability architecture of this text relies almost entirely on displacing responsibility from human actors onto the language models themselves. The text's central argument focuses exclusively on the representational capacities of LLMs, arguing that they naturally form cognitive maps to solve reasoning tasks. The vocabulary used perfectly mirrors this argument, framing the models as autonomous learners, reasoners, and epistemic agents. In doing so, it creates a massive accountability sink.

Throughout the text, passive voice and agentless constructions concentrate around the creation, training, and deployment of the models. Phrases like "models learn from these corpora" and "models are allowed to reason" treat the engineering of the system as an inevitability rather than a series of choices. The human actors—the developers at Google (Gemma), Meta (Llama), and Alibaba (Qwen)—are entirely unnamed regarding their role in shaping the models' parameters. The only actors clearly named are the researchers themselves, and only regarding their execution of the experimental methodology.

By locating agency entirely within the model, the text diffuses responsibility into the artifact. If a model generates false information, the text's vocabulary suggests it is because the model holds an "inconsistent belief" or suffers from a "false memory." This linguistic architecture shields the corporations who built the systems from liability. If the framing is accepted, legal and ethical responsibility for systemic failures (such as generating biased or dangerous text) is deflected away from the corporate decisions regarding training data selection and optimization objectives, and blamed instead on the "cognitive" errors of the machine.

Applying the "name the actor" test reveals what is missing. If the text stated, "Google's optimization algorithms caused Gemma to output logically contradictory tokens," rather than "the model holds inconsistent beliefs," entirely different questions become askable. We could ask about Google's data curation practices, the flaws in their RLHF pipeline, and the inherent limitations of their chosen architecture.

The resulting distribution of responsibility serves the institutional and commercial interests of the AI industry. By framing statistical artifacts as autonomous reasoners, developers are absolved of the deep, structural flaws in next-token prediction architectures. The vocabulary constructs an illusion of artificial autonomy that implicitly asks society to regulate the AI as if it were a willful agent, rather than regulating the corporations who mass-produce and profit from fundamentally brittle statistical software.

Conclusion: What This Analysis Reveals​

The Core Finding

Three dominant anthropomorphic patterns operate within this text: the spatial cognition metaphor (cognitive maps), the epistemic personification (beliefs, inferences), and the student/learner metaphor. These patterns do not operate in isolation; they interconnect to form a comprehensive architecture of artificial mind. The spatial cognition metaphor provides the foundation, establishing the premise that the model possesses a unified internal representation of the world. Once this foundation is accepted, the epistemic personification pattern maps statistical outputs onto conscious reasoning, allowing the text to claim the model "infers" and "believes." Finally, the learner metaphor explains how these capabilities arise, attributing parameter optimization to active knowledge acquisition.

The load-bearing pattern is the spatial cognition metaphor. By defining the high-dimensional vector space as a "cognitive map," the authors create a functional linguistic bridge. Without this initial mapping, the subsequent claims about the model holding "beliefs" about its location in a family tree would collapse into obvious category errors. The text employs a complex analogical structure, projecting the holistic survival mechanics of biological navigation onto deterministic matrix multiplications. Crucially, the text's vocabulary perfectly supports its argument: both aim to position the artifact as an autonomous reasoning agent, thereby obscuring the human labor and corporate decisions that actually shape the system's token outputs.

Mechanism of the Illusion:​

This framework holds that computational systems process rather than know. The illusion of mind in this text is produced primarily through a reception effect, driven by the "curse of knowledge" and activated by strategic verb choices. The central move occurs when the authors transition from describing vector manipulations in the key-value cache to describing the resulting text output. Because the model outputs fluent text containing words like "Alice" and "daughter," readers are predisposed by the ELIZA effect to attribute human intentionality to the system.

The authors provide this tendency with scientific material to work with by projecting their own understanding of family trees onto the model. When the model outputs a statistically likely token following a vector perturbation, the authors write that the model formed a "belief" or accessed a "false memory." The temporal structure of the illusion is critical: the text first establishes mathematical rigor by detailing the architecture of self-attention layers, and then abruptly substitutes consciousness verbs (infers, understands) for mechanistic ones (processes, predicts).

This causal chain—from linear algebra to "cognitive map" to "conscious inference"—allows the audience to accept the epistemic claims as scientifically proven. Brown's explanation types heavily enable this: the authors use functional explanations of vector routing to justify reason-based claims about the model's logic. By blurring the line between processing vectors and knowing facts, the vocabulary exploits the reader's desire to encounter a thinking machine, transforming a statistical text generator into an illusion of intellect.

Material Stakes:​

Categories: Regulatory/Legal, Epistemic, Economic

The metaphorical framing of language models as conscious reasoners generates severe material consequences. Epistemically, when a technical paper claims models "infer," "reason," and hold "beliefs," it degrades societal information practices. Users, including professionals in law and medicine, are conditioned to view these systems as logical truth-engines rather than statistical correlators. This leads to profound automation bias, where users trust the system to deduce facts it is structurally incapable of verifying. The cost is borne by those harmed by hallucinated outputs, while developers benefit from the unwarranted trust placed in their products.

Regulatorily and legally, attributing "false memories" and "beliefs" to a software product shifts liability away from the manufacturer. If a model is perceived as an autonomous cognitive agent that simply "makes mistakes" like a human, lawmakers are less likely to apply strict product liability frameworks appropriate for deterministic software. This framing shields corporations like Google, Meta, and Alibaba from accountability for their data curation and algorithmic design choices, effectively blaming the artifact for systemic design flaws.

Economically, the "cognitive map" metaphor serves as powerful, unpaid marketing for the AI industry. Framing statistical pattern matching as artificial reasoning justifies massive corporate valuations and the immense environmental resources consumed by data centers. By obscuring the human labor (annotators, RLHF workers) required to train the models, the narrative justifies the economic marginalization of these workers. The companies who profit from the deployment of these models are the clear winners, protected from scrutiny by a vocabulary that mystifies their technology.

AI Literacy as Counter-Practice:​

Practicing critical literacy requires replacing consciousness verbs with precise mechanistic descriptions and aggressively restoring human agency. As demonstrated in the reframings, replacing "models infer unstated relationships" with "models generate text consistent with unstated relationships" forces the recognition that the system operates statistically, not deductively. Replacing "models hold inconsistent beliefs" with "the vector manipulation mathematically favored contradictory tokens" dispels the illusion of a conscious mind making an epistemic error.

Restoring human agency is equally critical. Stating that "Google's engineers optimized parameters" rather than "the learner acquired knowledge" forces recognition of the corporate architectures and human labor underlying the system. This practice directly counters the material stakes by relocating liability and capability assessments back to reality.

However, systematic adoption of this precision faces massive resistance. The incentives of the AI industry heavily favor anthropomorphic vocabulary because it drives investment, media hype, and user engagement. Within academia, the inherited register of the field rewards metaphors that make statistical algorithms sound like groundbreaking cognitive discoveries. Precision is costly for developers whose valuations depend on the illusion of reasoning, and for researchers whose prestige relies on the perceived sophistication of their subject. Adopting this critical literacy threatens the mystification that shields the industry from both regulatory oversight and public understanding of its environmental and labor costs.

Path Forward​

The discourse surrounding artificial intelligence currently sits at a crossroads, with different communities prioritizing different vocabularies. Mapping these alternatives reveals stark trade-offs. The status quo, which freely mixes mechanical data with anthropomorphic claims (e.g., "the model thinks"), prioritizes narrative resonance and accessibility, benefiting industry marketers and media narratives but fundamentally blinding the public to the technology's actual limitations.

Alternatively, a strictly mechanistic vocabulary (e.g., "the model generates text statistically associated with the prompt") prioritizes technical precision and epistemic hygiene. This approach makes the corporate dependencies, data reliance, and statistical brittleness of the systems highly visible, serving the needs of regulators, civil society, and safety researchers. However, it costs intuitive accessibility, making it harder for lay audiences to quickly grasp the system's functional outputs.

If the anthropomorphic approach deepens, the future discourse will likely embed assumptions of AI autonomy into law and policy. This could lead to "agentic" systems being granted undue authority, making society uniquely vulnerable to catastrophic statistical failures masked as "cognitive errors." Conversely, if mechanistic precision becomes the norm, supported by structural changes such as academic journals demanding literal translations of metaphors and regulators requiring clear capability disclosures, a different future emerges. In this future, the AI industry would be forced to compete on the actual, verifiable reliability of their software rather than the imagined cognitive prowess of their "artificial minds." Which future unfolds depends entirely on whether society values the comfort of a relatable illusion over the rigorous demands of mechanical reality.


Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Analyzing the Metaphors

I'm now zeroing in on the metaphorical language used in the PDF, "Cognitive Maps in Large Language Models." My aim is to extract and analyze how metaphors shape the arguments presented, aligning with the user's provided schema for critical discourse analysis. The early focus is on identifying key metaphors and their function within the text's argumentative framework.

Deconstructing Agency Projections

I'm now focusing on how the text anthropomorphizes AI systems, especially in ways that displace human agency and project consciousness onto the machines. I'm working with the schema to systematically analyze this through reframing language and identifying specific instances where agency is subtly shifted. I'm building a metaphor audit as well, with detailed source-target mappings.

Examining the Core Argument

I'm now diving deep into the source document, "Cognitive Maps in Large Language Models." I'm identifying key claims and constructing a mental model of the central argument: LLMs form "cognitive maps" for relational reasoning, evidenced through "mosaic experiments." I'm noting the authors' academic position and the document's genre, and beginning to map the argumentative structure within.

Assessing Responsibility Claims

I'm now investigating the specific claims about open-weight models, noting that the authors propose that LLMs can abstract relational structures separately from entity identities. I'm especially focused on the attribution of agency to the models themselves. The text portrays them as autonomous agents that "infer," "reason," and "retrieve," with no mention of human accountability. I'm focusing on "responsibility claims" as they're absent and the implications. I am cataloging the language that implies autonomy.

Pinpointing Agentive Language

I am now closely examining the language used to describe the LLMs' cognitive processes. I'm focusing on the quotes, paying close attention to words like "inferred," "retained," "construct," "believe," "use," and "refine," all of which suggest agency and cognitive capacity. I'm cross-referencing these with my schema. I'm also now going to incorporate the self-correction to ensure strict adherence to the defined ENUM values.

Mapping Source-Target Domains

I am now performing the source-target mapping analysis on eight quotes, focusing on the metaphorical language employed. I'm aligning source domains like human memory and logical deduction with target domains such as vector representations and token probability distributions. My initial work on eight source and eight target domains should also meet my user's word limit expectations. I'm making sure my mapping for each domain is detailed and precise.

Categorizing Quote Attributes

I'm now cataloging the primary and secondary attributes of the quotes based on source. I'm assigning each quote a primary and secondary label to characterize its source. For example, quote two is marked as "Genetic (Primary), Theoretical (Secondary)." The goal is to classify the key aspects of these quotes according to a taxonomy that will map onto user-defined attributes. I am also making sure to adhere to the requested length constraints.

Reframing Linguistic Agency

I'm now generating reframed versions of the eight quotes, with each exceeding fifty words to meet the length requirements. I'm focusing on deflecting agentive language and specifying the mechanismBasis for each, to align with the provided ENUM. For example, I'm adjusting "models use" to be something like "processes within the model facilitated by its architecture" to address agency as it's indicated. I'm prioritizing accuracy to adhere to the source material.

Crafting Critical Arguments

I'm now drafting the critical observations section, focusing on agency slippage, metaphor-driven trust, and obscured mechanics. I'm building each section to approximately 400-500 words, ensuring a thorough deconstruction of the source material. My initial focus is on the move from mathematical processes to claims of reasoning, belief, and mapping the role of the metaphors used.

Synthesizing Pattern Recognition

I'm now consolidating the analysis to draft the conclusion's core arguments. I'm focusing on the rhetorical strategies: I'm examining the tension between the technical detail, with the anthropomorphic claims that portray the LLMs as entities that know, reason, and believe. I'm aiming to expose how the text facilitates an "accountability sink", where the model is seen as autonomous. I'm prioritizing the rhetorical architecture and material implications.

Refining Analysis Constraints

I'm now implementing self-correction protocols to meet strict formatting requirements. I'm focusing on word counts and ensuring valid JSON formatting without control tokens. I am specifically reviewing string escaping and word count minimums to achieve precise outputs. I'm also now aligning with the user schema and applying the appropriate attributes, while rigorously avoiding literal newline characters as instructed.

Validating Attribute Compliance

I'm now cross-referencing my work against the ENUM values. I've re-examined the source document to check genre and authorPosition and I have re-validated the agencyLevel, ensuring it's accurate and fits within my schema. I'm focusing on ensuring valid JSON formatting, including rigorous escape for text, to prevent any parsing issues.

Constructing JSON Payload

I'm now generating the JSON payload, prioritizing adherence to all formatting rules. I'm focusing on crafting extensive text for each attribute to meet word count requirements and avoid control tokens like newlines. I'm building a rich, detailed representation of the analysis to ensure complete compliance. I'm incorporating the reframing of the arguments to avoid agentive language and emphasize the system mechanics.

Elaborating Structural Claims

I'm now expanding on the "mosaic experiments" to clarify the structural claims. I am emphasizing that the authors' primary goal is to demonstrate that certain biological principles of spatial navigation and relational mapping naturally emerge in statistical sequence predictors. I've increased the focus on the implications of these principles.

Refining Text Formatting

I'm now implementing strategies for text formatting, specifically avoiding newline characters and focusing on word count minimums. I'm focusing on making extensive text for analysis, epistemicClaimAnalysis, and rhetoricalImpact to meet user expectations. I'm prioritizing accuracy in all strings and planning to implement a JSON flattening strategy to ensure the document is properly formatted.

Calculating Lengths for JSON

I'm now calculating lengths for JSON content, to meet the specifications, particularly in criticalObservations and conclusion. I'm pre-writing sections and ensuring that I satisfy the word counts for each respective field. I'm focusing on "Agency Slippage" (400-500 words) and how the paper moves from "vectors and attention heads" to "beliefs and cognitive maps", to ensure an appropriate length. I'm also planning the content and length of the conclusion (250-350 words).

Evaluating Agentive Shifts

I'm now zeroing in on the critical argument and its rhetorical strategies. I'm focusing on the "curse of knowledge" concept, and I'm developing a more detailed explanation of how this plays out in the analysis of the LLM. I'm also now analyzing the use of agentless constructions to obscure the actions of the developers of the LLMs.

Expanding Agency Explanations

I'm now expanding the analysis of how functional explanations contribute to agency attribution, particularly noting the move from mechanistic observations to cognitive claims. I'm focusing on "the curse of knowledge" and using the example of models that produce "Alice", and how the authors create a linguistic bridge where mechanism can be portrayed as human-like cognition. I'm prioritizing accuracy to meet the word count requirements.

Analyzing Metaphor's Function

I'm now focusing on section 5.2, specifically the "Metaphor-Driven Trust" analysis. I'm expanding on the concepts of "Cognitive Maps," "Reasoning," and "Knowledge," as they're critical. I'm examining how these metaphors contribute to the argument's persuasive power and the potential to build user trust in the LLM's capabilities. My current word count is 535 words, and I'm expanding this with relevant information.

Deconstructing Epistemic Verbs

I'm now zeroing in on epistemic verbs like "reasons," "infers," and "believes," and the resulting inflation of perceived sophistication. I'm focusing on "consciousness framing" and how comparing models to animals navigating mazes constructs a framework for trust and perceived authority. This is a profound trust signal, and it shifts the focus to relation-based trust, rather than performance-based trust. I'm analyzing how these words are a core rhetorical feature.

Refining Trust Frameworks

I'm now zeroing in on the trust framework within the analysis. I'm focusing on "relation-based trust" and its contrast with "performance-based trust." The goal is to illuminate how epistemic verbs and metaphor inflate perceived sophistication, and create a situation where users are vulnerable to automation bias. I'm aiming to fully explain how the trust system is framed and managed.

Expanding Trust's Implications

I'm now expanding the analysis of metaphor-driven trust in section 5.2. I'm focusing on the significant risks if this framing influences decision-makers and the public, potentially overestimating the system's reasoning capabilities and increasing vulnerability to adversarial manipulation. I'm noting the subtle shift of safety burdens. I am still prioritizing an accurate analysis.

Elaborating Unreliability Claims

I'm expanding the analysis of reliability, highlighting how anthropomorphic language insulates technology from scrutiny. This section, exceeding the 500-word mark, now critiques the migration of LLMs into a "quasi-human" space where "mistakes" are excused as cognitive processes. I'm focusing on the accountability implications, given the deterministic software, and expanding on regulatory concerns. I have now met all specifications and included all criteria.

Clarifying Corporate Responsibility

I'm now zeroing in on corporate responsibility, specifically concerning the framing of LLMs. My focus is on emphasizing that language like "models learn" obscures the actions of the developers. I'm building on this by analyzing how the text obscures corporate actors, relying on a technical framing. I am also planning to apply the "name the corporation" test to highlight this agency shift. I am making sure to meet length constraints.

Critiquing Erasure Mechanisms

I'm now diving deep into the material and labor components, focusing on the paper's selective erasure of environmental costs and human labor. I'm focusing on the "cognitive map" metaphor's obfuscation, which obscures the computational demands, the energy footprint, and the invisible labor that is inherent in LLM creation, including data annotation, reinforcement learning, and content moderation. I'm now also analyzing the paper to ensure it exposes these omissions.


Analysis Provenance

Run ID: 2026-10-23-cognitive-maps-in-large-language-models-metaphor-s6gygn Raw JSON: 2026-10-23-cognitive-maps-in-large-language-models-metaphor-s6gygn.json Framework: Metaphor Analysis v6.8 Schema Version: 3.4.3 Generated: 2026-10-23T07:37:24.228Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0