`for x in y` 隐藏了什么——从零开始编写代码
What `for x in y` hides from you – From Scratch Code

原始链接: https://fromscratchcode.com/blog/what-for-x-in-y-hides-from-you/

Python 的 `for` 循环常被视为遍历集合的简单机制,但实际上它通过一套标准且优雅的协议运作。Python 并不是直接在对象上循环,而是将 `for x in y` 作为迭代器的一种高级封装。 在底层,循环执行三个步骤: 1. 对目标对象调用 `iter()` 以获取迭代器。 2. 反复对该迭代器调用 `next()`。 3. 当触发 `StopIteration` 异常时终止。 这种设计具有高度的灵活性;由于循环只关心迭代器协议,因此无需为每种类型编写自定义逻辑,即可无缝处理列表、字符串、区间(range)和生成器。即使是更复杂的语法,如元组解包(`for x, y in z`),也仅仅是标准迭代过程与赋值解包的结合。 理解这一机制揭示了 Python 的行为逻辑,例如为何耗尽的迭代器不会产生任何输出。归根结底,`for x in y` 并不意味着“循环这个项目”,而是“询问这个项目如何被迭代”。掌握这一协议,能将“黑盒”语法转化为构建自定义数据结构和高效生成器的强大统一工具。

Sorry.
相关文章

原文

There are certain Python features that are so straightforward to use, you forget they are doing anything at all.

Consider for x in y.

You write a list, or a string, or a range, and Python politely hands you back one item at a time. No index variable and no bounds checks. Compared to i++ from C/C++ or forEach from JavaScript, Python's version just works.

For a long time, I treated for x in y as just a syntax that meant “loop over this thing,” and that was enough. Then I started building Memphis, my Python interpreter in Rust, and eventually I had to stop hand-waving and answer an extremely rude question:

What is a for loop actually doing?

It turns out the answer is both simpler and more involved than I expected.

The illusion

Let’s start with a tiny example.

for x in [10, 20, 30]:
    print(x)

Run it and you get the familiar behavior: one value at a time, in order.

So far, no real surprises.

If you’re starting out with Python, it’s easy to come away with a mental model like this:

  • Python peers into the list
  • it walks through the elements one at a time
  • it assigns each one to x
  • then it runs the body of the loop

That’s not behaviorally wrong, but it hides the most important part:

Python is not looping over the collection directly. It is looping over an iterator.

That distinction matters because it explains why for works on so many different kinds of objects, and why iteration in Python feels so flexible.

The hidden step

When Python sees this:

for x in [10, 20, 30]:
    print(x)

The real story is closer to something like this:

it = iter([10, 20, 30])

while True:
    try:
        x = next(it)
        print(x)
    except StopIteration:
        break

This is the crux of it.

A for loop is really:

  1. ask for an iterator by calling iter(...)
  2. repeatedly ask for the next value by calling next(...)
  3. stop when next(...) raises StopIteration

The for x in y syntax is just a pretty wrapper.

Once this clicked for me, a bunch of Python behavior stopped feeling mysterious.

You can see it directly

We can prove this to ourselves without even touching interpreter internals yet.

nums = [10, 20, 30]
it = iter(nums)

print(next(it))
print(next(it))
print(next(it))

This prints the list elements one at a time.

But if we push one step too far:

nums = [10, 20, 30]
it = iter(nums)

print(next(it))
print(next(it))
print(next(it))
print(next(it))

Run that second snippet too. The last next(it) raises StopIteration, which is exactly how the loop knows it is done.

So the loop is not “reading from the list” in some special one-off way. It is using the same iterator protocol you can use yourself.

Why this matters

At first glance, this may seem like one of those facts that is technically true but not especially useful.

I don’t think that’s the case.

This one idea explains why all of these work:

for x in [1, 2, 3]:
    print(x)

for ch in "cat":
    print(ch)

for n in range(3):
    print(n)

These are completely different kinds of objects. A list is not a string is not a range. Yet Python can loop over all of them because each one can produce an iterator.

It also explains why generators work so naturally with for. A generator is just another Python type that participates in this same protocol. We'll see an example of that in a minute.

Once you see that, Python’s iteration model starts to feel unified instead of magical.

The part I had to implement

In Memphis, I couldn’t get away with “loop over this thing somehow.” I had to decide what the runtime would actually do.

At a high level, my treewalk interpreter handles a for loop by:

  1. evaluating the expression on the right-hand side
  2. calling iter(...) on the result
  3. repeatedly calling next(...)
  4. binding each returned value to the loop variable
  5. stopping when StopIteration is raised

That sounds almost boring when written out, which I mean as a credit to the Python specification. Good abstractions often become boring once you can name them clearly.

The surprise for me was not that for uses iteration. It was how little the loop itself knows.

The for loop does not need special logic for lists, tuples, ranges, strings, generators, or anything else. It just needs the iterator protocol. The specific object handles the details.

In full Python, this goes one step farther: custom objects can participate too by implementing __iter__() and __next__(). I’m still working on that part in Memphis, but it’s one of the coolest consequences of this design.

That’s a very Pythonic design.

Another example which may help this click

Consider this:

items = [1, 2, 3]
it = iter(items)

for x in it:
    print(x)

for x in it:
    print("again:", x)

Try running it yourself. The second loop prints nothing.

That’s because the iterator was already exhausted by the first loop. The for loop didn’t “rewind” anything. It just kept calling next(...) until there was nothing left.

If you want to make the distinction more concrete, inspect the types too:

items = [1, 2, 3]
it = iter(items)

print(type(items))
print(type(it))

This is a subtle but important distinction:

  • an iterable can usually give you a fresh iterator (list, str, range, etc)
  • an iterator is usually a one-way trip

A generator function can create a fresh generator each time you call it, but a generator object itself is already an iterator.

That difference hides behind the word in: it initializes a new iterator for you, unless you already gave it one.

There’s another useful variation here too:

pairs = [(1, 10), (2, 20), (3, 30)]

for x, y in pairs:
    print(x, y)

This can look like a different kind of loop, but it really isn’t. The iteration part is the same: Python still asks for an iterator and pulls one value at a time. It’s just that each value happens to be a two-item tuple, and Python then unpacks that tuple into x and y.

So for x, y in z isn't a special flavor, it's ordinary iteration plus unpacking.

An example to break your brain

If you want to make this stick, it helps to play with a generator.

def countdown():
    print("Starting")
    yield 3
    yield 2
    yield 1
    print("Done")

for x in countdown():
    print("Got", x)

Run it and watch the order carefully. What the for loop is hiding here is more than just repetition.

Each call to next(...) resumes the generator, runs it until the next yield, hands that value back to the loop, and pauses again.

Once I started thinking of iteration this way, for loops stopped feeling flat. They became a protocol between the loop and some other object that knows how to produce values over time.

That is a much more accurate mental model.

And that, more and more, is what I enjoy about building Memphis. Python has all these smooth surfaces that make it pleasant to use. But when I reimplement one of them from scratch, I get to see the levers underneath.

In this case, it was just this surprisingly small mechanism:

  • iter(...)
  • next(...)
  • StopIteration

The end

The nicest part of Python’s loop syntax is that it lets beginners be productive before they understand any of this. That’s impressive language design.

But once you hit generators, custom iterables, or bugs involving exhausted iterators, the nicer syntax can become a liability if you never learned what it was hiding.

The phrase I keep coming back to is this:

for x in y does not mean “loop over y.” It means “ask y how to be iterated.”

That may not sound like much, but it was a real shift for me.

And if you’ve mostly been using for x in y as a black box, maybe it’ll shift something for you too.

联系我们 contact @ memedata.com