The short version: repeating an instruction in your system prompt genuinely helps. It stops helping at around four repetitions. Everything after that is superstition, and it costs you tokens.

That’s the whole finding. It cost about a dollar to get, and I think it’s a nice little thing to know on a Tuesday.

This is the first of what I want to make a weekly habit here. Pick one claim that floats around about AI coding agents, test it in a weekend, publish the numbers whether or not they’re flattering. Not research. Just somebody actually checking.

You’ve seen the advice. Repeat the important instruction. Put it at the top and the bottom. Say it twice so the model takes it seriously. Everybody does some version of this, myself included, and I’ve never seen a number attached to any of it.

Then a paper came through my daily brief with an actual shape for it. Han-yu Wang’s When More Becomes Less: Position-Dependent Repetition Effects in Language Models (paper: arXiv 2608.04021, briefing: 6 August) tests what happens as you add more copies of a target, and finds the answer depends on where the copies sit. Copies stacked next to each other climb and then flatten out. Copies spread away from where the model reads out produce a hump, rising to an early peak and then falling.

That’s a specific, checkable claim about something I do every week, so I checked the half that matches how I actually write prompts.

I wrote my guess down first, which is a rule I’m keeping. My guess was that I’d see the hump, including the fall. I was wrong, and being wrong sent me back to read the paper properly, which is its own small lesson.

The setup is deliberately boring.

I picked one rule a model can either follow or not: use single quotes, never double quotes. Then I asked for six ordinary Python functions, the kind of thing you’d write on any given afternoon. Merge some intervals. Flatten a dictionary. Parse a version string.

The only thing that changed between runs was how many times that quote rule appeared in the system prompt: zero times, once, twice, four, eight, or sixteen. Same rule, just repeated more.

Thirty tries of each combination. 1,080 runs total, on Gemini 2.5 Flash, all of it on Vertex.

Checking the answers needed no judgment calls. I ran Python’s own tokenizer over the generated code and counted strings that opened with a double quote. Zero of them means it followed the rule. That’s it. No model grading another model, no me squinting at diffs deciding what counts.

The zero-repetition runs are the important control. That’s where I never mention quotes at all, which tells me what the model does when left alone.

One detour worth mentioning: my first three candidate rules were all duds. I tried “no comments,” “no docstring,” and “no type hints,” and the model obeyed all three about 99% of the time on the first ask. You can’t measure whether repetition helps when there’s no room left to improve. So I went looking for a rule the model actually resists, and quote style turned out to be one.

The control row is my favorite number in the table. Left to itself, the model used double quotes every single time. Not most of the time. All 171 of them. So its habit here is about as strong as a habit gets, which makes the rest of the table mean something.

Say the rule once and you’re at 74%. Say it four times and you’re at 97%. Those two are far enough apart that I’m comfortable calling it real.

Past four, the line goes flat. Eight and sixteen land inside the same range as four. My predicted decline never showed up, and to be straight with you, at this sample size I could miss a small one. What I can say is that nobody is getting paid back for repetitions five through sixteen.

Here’s the part I only understood afterward. I stacked all my copies of the rule right next to each other, which is the adjacent case in Wang’s paper, and adjacent is exactly the case that’s supposed to climb and then flatten. The hump I went looking for belongs to the other case, where the copies are spread out away from where the model is reading. So this didn’t contradict the paper. It landed on the paper’s prediction from a completely different direction, with a natural-language rule handed to a coding model instead of tokens in a probe. That’s a better outcome than the one I predicted, and I’d have missed it entirely if I hadn’t gone back to the source.

The average is hiding almost everything. Two of my six tasks hit 100% on the very first mention and never wavered. Another one, merging intervals, sat at 20% with a single mention and needed four to climb to 97%. So repetition isn’t broadly making the model more obedient. It’s rescuing the specific spots where the model’s habit is fighting your rule. If none of your work looks like those spots, you’re paying for nothing.

Most of these cells are coin flips. Between half and two thirds of my task-and-repetition combinations came back neither all-pass nor all-fail across thirty identical runs. Same prompt, same model, same settings, different answer. If you’ve ever tweaked a prompt, run it twice, and concluded the tweak worked, this is the number that should bother you. It bothers me.

The leftover violations had a shape. Once the rule appears even once, ordinary double-quoted strings disappear completely. What survives is the triple-quoted docstring at the top of the function. The model seems to file """this""" under documentation rather than under strings, so a rule about quotes never reaches it. If you’ve had a constraint that got obeyed everywhere except one stubborn place, that’s probably what’s happening. The model has the thing in a different mental drawer.

If you work somewhere with a prompt library. Cap repetition at about four in your templates and spend the leftover room on examples instead. The bigger one is the coin-flip problem: if your team evaluates a prompt change by running it once before and once after, that process is theater. Ask how many runs before you ask what the result was. Three is a floor. Ten is better.

If you’re building something on your own. When a rule isn’t landing, repeating it up to four times is the cheapest fix you have and it genuinely works. If four doesn’t do it, stop repeating and change something else, because five through sixteen bought me nothing. And when a constraint gets followed everywhere except one place, go look for the thing the model has filed under a different name, the way a docstring isn’t a string.

This is one person, one weekend, one model. It’s not state of the art and it isn’t trying to be.

I tested Gemini 2.5 Flash with thinking off, on one day. Different model, different family, or thinking switched on could all move this. I tested one rule about syntax, repeated literally, with every copy in the same place. Rules about behavior, or rephrased each time, are untested here.

The big untested one is spacing. Every copy of my rule sat in one block, and the paper says that’s the case that flattens. Spreading the copies through the prompt is the case that’s supposed to turn around and hurt you, and that’s the next experiment rather than a caveat I can hand-wave. And these were six small standalone functions, not a real repository with a real agent loop, which is exactly the kind of thing that usually doesn’t survive the jump.

One more, because it nearly cost me the whole experiment. My first real run threw away most of its samples as unreadable. Gemini’s thinking tokens count against your output limit but get reported separately, so a limit that looked generous was quietly eaten by reasoning and the actual code got cut off mid-word. Worse, it cut off more often in some conditions than others, so what survived was skewed differently in every column. Before I caught it, my headline number read 39%. After, 88%. Same code, same model, same afternoon. If your evaluation setup doesn’t record why generation stopped, it can hand you a confident wrong answer and never mention it.

It’s all public. The guess I wrote down before running, the code, the checker, and every one of the 1,080 runs including the ugly ones.

git clone https://github.com/nkhola/field-tests
cd field-tests/ft-01-say-it-four-times
python analyze.py

If you run it and get something different, I genuinely want to hear about it.

This one came out of The Post-Human Briefing, my daily AI and markets brief. Machine-built, human-audited. New Field Test most weeks.