Python Generators: yield, Lazy Evaluation & When to Use Them

yield vs return

A function with `yield` becomes a generator function. Calling it returns a generator object — it does not execute the body immediately. Each `next()` call resumes from where it last yielded.

def countdown(n):
    while n > 0:
        yield n       # pauses here, returns n
        n -= 1

gen = countdown(3)    # does NOT run the body yet
print(next(gen))      # 3
print(next(gen))      # 2
print(next(gen))      # 1
# next(gen) now raises StopIteration

Iterating generators

Generators implement the iterator protocol, so you can use them directly in for loops, list(), sum(), and any function that accepts an iterable.

def squares(n):
    for i in range(n):
        yield i ** 2

# for loop automatically calls next() until StopIteration
for s in squares(5):
    print(s)          # 0, 1, 4, 9, 16

# Convert to list (consumes the generator)
result = list(squares(5))  # [0, 1, 4, 9, 16]

total = sum(squares(5))    # 30

Generator expressions

Generator expressions use parentheses instead of brackets. They're the lazy equivalent of list comprehensions.

# List comprehension — builds entire list immediately
lst = [x**2 for x in range(1_000_000)]   # ~8 MB

# Generator expression — produces one value at a time
gen = (x**2 for x in range(1_000_000))   # ~200 bytes

# Both work the same way in a for loop or sum()
total = sum(x**2 for x in range(1_000_000))

Infinite generators

Because generators are lazy, they can represent infinite sequences. You control how many values you consume.

def integers_from(n):
    while True:       # infinite loop — fine in a generator
        yield n
        n += 1

gen = integers_from(1)
print(next(gen))  # 1
print(next(gen))  # 2

# Take first 5 using itertools.islice
import itertools
first_five = list(itertools.islice(integers_from(1), 5))
print(first_five)  # [1, 2, 3, 4, 5]

yield from — delegating to sub-generators

`yield from` delegates to another iterable or generator, forwarding values without a manual loop.

def chain(*iterables):
    for it in iterables:
        yield from it    # delegates each item from the iterable

result = list(chain([1, 2], [3, 4], [5]))
print(result)  # [1, 2, 3, 4, 5]

Exam tip

Key exam questions: (1) a generator can only be iterated once — calling list() on it a second time returns []; (2) the body doesn't execute until next() is called; (3) StopIteration is raised automatically when the generator function returns.