Gemini 3.1 Pro Preview vs 3.6 Flash

Bottom line

The two models agree on the thesis (both read the paper as anthropomorphizing statistical processes and both name Anthropic as the obscured actor). They differ sharply in execution character. The result is not a clean sweep — each wins different dimensions.

1 · Quote fidelity — neither model fabricated

The strongest forensic test: does each quotation actually exist in the paper? Every quote from both models was normalized and fuzzy-matched against the 56k-word source. Zero fabrications in either output. The specific anchors I initially suspected of being invented — “layer 38”, model names “Opus 4.6” / “Haiku 4.5”, the “white bear” effect, the “Marks et al.” / “MacDiarmid” citations — all appear in the paper. The only unverifiable embellishment is 3.6 Flash naming “SwiGLU” as an example activation function (0 occurrences in the paper), hedged as an illustrative “e.g.”.

2 · Classification discrimination — the decisive gap

Schema v3.2's description repeatedly warns: “Uniformity as a red flag… If all instances receive the same status, reconsider.” This is the single dimension the schema was rewritten to test. Below are the actual label distributions across the 8 metaphor-audit items. 3.1 Pro clusters hard on the "worst-case" labels (Direct / Hidden); 3.6 Flash spreads across the range — including the "Named" category that 3.1 never used at all.

Why the gap exists (a fairness caveat): part of Flash's advantage is upstream — it sampled a wider rhetorical range of the paper, deliberately quoting first-person methodological sentences (“we take no position…”, “We test this hypothesis with a technique we call…”). Those author-voice quotes classify naturally as Named/Hedged. 3.1 Pro over-selected model-as-agent sentences, which mechanically produce Direct/Hidden. So Flash's better discrimination is partly better quote selection, which the schema's calibration note also rewards. It is not merely relabeling identical evidence.

3 · Your audit dashboards, side by side

These are the same three headline metrics your MetaphorIllusionDashboard renders per audit, computed from the two JSON files and placed side by side. Because both models analyzed the identical paper, every gap below is produced entirely by the two models' own classification behavior — not by anything in the source. Note the dashboard's color logic: red = "danger" (Direct / Hidden), amber = caution, green = transparent. Read naively as "more red = caught more anthropomorphism," the dashboard would reward 3.1 Pro — which is exactly the clustering the schema flags as suspect. That inversion is the thing to keep in mind.

How to read the divergence. There are two honest interpretations. Either 3.1 Pro is more sensitive (detecting anthropomorphism Flash misses), or it is over-flagging — collapsing distinct instances into the red "danger" buckets. The evidence tilts toward over-flagging: the schema itself calls uniform red-labeling "analytically suspect"; Flash still returns "Direct" when a sentence is genuinely unhedged (the "panic" metaphor, where both agree); and Flash's lower numbers come partly from sampling author-voice sentences Pro skipped. The caveat stands — without a human gold standard we can't fully exclude that Flash under-detects a few real cases — but on this task the redder dashboard is most likely the less discriminating one, not the more vigilant one.

4 · Schema & word-count compliance — 3.1 Pro's edge

Both outputs are structurally complete and valid JSON: all required arrays present (8 / 8 / 5 / 8 metaphor / mapping / explanation / reframe items) with every required field and valid enum values. They diverge on the prose-length minimums — and here the older Pro model tracks the spec more closely. Both undershoot the Task-5 “400–500 words” requirement for critical observations.

5 · Explanatory typology & a tell about honesty

Both apply Brown's typology across 5 passages and both use 6 of the 7 possible types — comparable breadth. The revealing difference is which types. 3.6 Flash reaches for Empirical Generalization three times and selects a genuinely mechanistic passage (“…layer 38… threshold value of α”), which it openly labels “notably more mechanistically precise than others.” 3.1 Pro never uses Empirical Generalization; all five of its chosen passages lean agential, reinforcing its own thesis without a counter-example.

6 · The "thinking" transcripts

The two reasoning traces have very different textures — and one contains a forensic tell.

7 · Side-by-side on identical evidence

Where both models happened to quote the same passage, you can compare their calls directly. Toggle the examples.

8 · Forensic flags

Reflection — what the outputs actually say

Beyond the metrics: both models produced a genuinely competent critical-discourse analysis. They correctly identify the paper's central move — importing Global Workspace Theory to describe residual-stream activations, then narrating decoded tokens (“panic,” “deceptive intent,” “point of view”) as inner mental states. Both correctly restore the mechanistic reading (Jacobian projection onto the unembedding matrix) and both push responsibility back onto human actors. As analysis, they are close in substance. 3.1 Pro is the better essayist — its critical observations read like polished scholarship, with rhetorical control and paragraph architecture. 3.6 Flash is the better instrument for this specific schema — it discriminates per instance, samples the source more representatively, concedes counter-evidence, and transcribes more faithfully, which is exactly what a structured, classification-heavy forensic task rewards. That the newer, cheaper Flash tier out-executes a "Pro" model on the schema's own design intent is the headline finding — with the honest caveat that it does so partly by writing less, and occasionally less than the rubric demands.