We hid translator names, dates, and publishers. Readers saw up to 12 comparisons. On 668 of the 674 analyzed ballots, the opening eight formed a balanced cycle in which every available translator appeared twice. Before each of the final four comparisons, the test refit a regularized Bradley–Terry model using that reader’s earlier head-to-heads. It first covered translators with the least evidence. If skips had split the comparison network, it favored pairs that joined those fragments. The remaining score rewarded close contenders, uncertainty, and pairs the reader had not already seen. Scene assignment followed the phase-specific rules detailed below. Left and right positions were independently randomized for every comparison. Skips changed neither translator’s preference score, although the pair and scene still counted as having been shown. Detailed implementation facts:
-
Initialization: Every translator began at ability 0, equivalent to a neutral 50% predicted win probability.
-
Updating scores: After each completed head-to-head, the browser refit the model from all previous completed head-to-heads. It ran 80 gradient iterations with L2 regularization of 0.7, shrinking sparse estimates toward zero. Skips were omitted from the ability fit.
-
Eligible pairs: The starting universe was every unordered pair among the translators present in that ballot—28 pairs for eight translators.
-
Coverage and connectivity filters: If completed head-to-heads left disconnected groups, only pairs joining two groups were considered. It then retained pairs containing the largest possible number of translators tied for the fewest decisions.
-
Repeated pairs: Previously shown pairs were excluded whenever an unseen pair survived those filters. Repeats were permitted only when necessary and were penalized according to 1 / (1 + previous meetings). A skipped pair still counted as previously shown.
-
Closeness: For abilities aᵢ and aⱼ, closeness was 1 / (1 + |aᵢ − aⱼ|). This was only one part of the composite score, alongside contender strength, uncertainty, connectivity, and novelty.
-
Final selection: The algorithm kept the six highest-scoring eligible pairs and made a weighted selection among them. Higher scores were more likely, but the highest score was not automatically chosen.
-
Scene assignment: The method changed prospectively. Analyzed ballots 1–6 used the earlier four-scene cycle. Ballots 7–447 balanced scene exposure while also considering editorial strength, lexical contrast, and pair freshness. For analyzed ballots 448–674, the selector removed the immediately previous scene when alternatives existed, retained the least-shown scenes, sorted them by ID, and selected one with a ballot-specific deterministic hash. Only this final phase was fully blind to candidate identity, wording, and the reader’s selections.
-
Unavailable material: Pair selection only saw translators in the session’s candidate set; scene assignment only saw passages present in that session. Nothing unavailable was imputed or backfilled.
-
Left/right order: The logical pair was independently flipped with a 50/50 cryptographic random draw before every comparison.
-
Randomness and seeds: There was no fixed study-wide assignment seed. The server cryptographically shuffled translator and passage order and generated a random ballot UUID. That UUID became a per-ballot seed for reproducible adaptive-pair and scene assignment. Left/right flips remained separately cryptographically random and unseeded.
A JSON validation bug changed which options were available early in collection. Daniel Mendelsohn was missing for the first 6 readers and entered with reader 7. Harmonious household and The Lotus-Eaters were missing for the first 457 readers. Of those, 447 remain in the analysis. The scenes entered with reader 458, or analyzed ballot 448.
We did not rewrite or remove ballots because of the bug. The report separately excludes 11 ballots completed in under 90 seconds, including 10 collected while the two scenes were missing. Opening rounds balanced the options available at the time. The scene-adjusted model accounts for translator-by-scene and recruitment-wave effects.
The published ranking fits a regularized Bradley–Terry model:
P(i ≻ j) = σ(βᵢ − βⱼ) = 1 / (1 + e−(βᵢ−βⱼ))
β is a translation’s estimated ability. We report pairwise strength as its average probability of beating each of the other seven translations. A unit L2 penalty pulls weakly observed estimates toward the field average instead of allowing extreme scores from sparse matchups.
The compensated scene model adds partially pooled scene effects, recruitment wave effects, and a first-position term:
logit P(i ≻ j) = (αᵢ − αⱼ) + (sₖᵢ − sₖⱼ) + (wₜᵢ − wₜⱼ) + δ
k identifies the scene, t the recruitment wave, and δ the first-option effect. Partial pooling keeps small scene samples from overreacting. In five-fold grouped cross-validation, mean held-out log loss improved from 0.669 to 0.633; the final sampler had R̂ = 1.00 and no divergences.
The 95% ranges come from 50,000 whole-ballot bootstrap resamples, preserving the dependence among one reader’s choices. The initial analyzed charts use 674 ballots and 7,919 head-to-heads. Retakes and later writes from the same anonymous browser never entered the counted totals.
50,000 / 50,000
whole-ballot reruns ranked Fagles first
5.5% lower
held-out prediction error after scene adjustment
R̂ 1.00 · 0
maximum R̂ and sampler divergences; minimum effective sample size 1,230