Comprehensions — Building Collections in One Expression

A comprehension is a loop that builds a collection, compressed into a single expression. The thesis: one shape — [expr for item in iterable if condition] — replaces the "create empty, loop, append" pattern (Tutorial 9), and the same shape with different brackets builds lists, sets, dicts, and even lazy generators. Learn the shape once and you've learned four constructors.

This builds on loops and dicts & sets, and the generator expressions here feed straight into Iterators & Generators. Every output below was produced by running the code.

Table of Contents

1. List comprehensions
2. The loop translation
3. Dict and set comprehensions
4. Generator expressions
5. Nesting, flattening, and when not to
6. Mental-model summary
7. Gotcha checklist

1. List comprehensions

A list comprehension builds a list from an iterable: [expr for item in iterable] transforms each item; add if condition to keep only some.

Output:

[1, 4, 9, 16, 25]
[0, 2, 4, 6, 8]

Analysis. The first comprehension applies n * n to each n in 1..5 — a transform (map). The second keeps only the n where n % 2 == 0 — a filter. Together, [expr for item in iterable if cond] does "transform the items that pass the filter," the two most common collection operations in one line.

Intuition. Mechanism. A comprehension runs its for (and optional if) just like a loop, evaluating expr for each surviving item and collecting the results into a new list. Critically, it runs in its own scope — the loop variable is local to the comprehension and does not leak into the surrounding code (a deliberate change from Python 2).

Concrete bite. So you can't read the loop variable afterward:

Traceback (most recent call last):
  File "/w/main.py", line 2, in <module>
    print(n)        # the comprehension variable does not leak
          ^
NameError: name 'n' is not defined

The n lived only inside the comprehension; once it finished, n is gone, so print(n) is a NameError. (A plain for loop does leak its variable; a comprehension does not.)

2. The loop translation

Every comprehension has an exactly equivalent for loop. Knowing the translation both ways is the key to reading and writing them fluently.

Output:

['A', 'B']
['A', 'B']

Analysis. The comprehension [x.upper() for x in ...] and the three-line loop produce the identical list. Read a comprehension by mentally expanding it: [EXPR for VAR in ITER if COND] becomes "make an empty list; for each VAR in ITER, if COND, append EXPR."

Intuition. Mechanism. The two parts of a comprehension occupy different positions and mean different things. A filter if goes at the end and selects which items survive (no else allowed — an item either passes or doesn't). A conditional expression a if cond else b goes at the front, in the expr slot, and chooses the value for every item.

Concrete bite. Putting a filter-if with an else at the end is a syntax error, because a filter can't have an "otherwise":

  File "/w/main.py", line 2
    print([x for x in nums if x % 2 == 0 else 0])   # filter-if cannot take an else
                                         ^^^^
SyntaxError: invalid syntax

The two correct forms are distinct — filter at the end (no else), transform at the front (with else):

[2, 4]
[0, 2, 0, 4]

3. Dict and set comprehensions

The same shape with {} builds a dict ({k: v for ...}) or a set ({expr for ...}).

Output (illustrative — the set's order is arbitrary and may differ for you):

{'apple': 5, 'banana': 6, 'cherry': 6}
{'b', 'a', 'c'}

Analysis. {w: len(w) for w in words} builds a dict mapping each word to its length. {w[0] for w in words} builds a set of first letters (duplicates would collapse, and order is unspecified — sets have no order). The presence of a colon (k: v) is what makes it a dict comprehension rather than a set one.

Intuition. Mechanism. Brackets choose the result type: [] → list, {k: v ...} → dict (colon present), {expr ...} → set (no colon). All four (including the generator in §4) share the identical for/if machinery; only the collector differs.

Concrete bite. The ambiguity of {} catches everyone — empty braces are a dict, not a set:

dict
set
set

{} is the empty dict (dicts predate sets in the syntax), so the empty set must be written set(). With elements, {1, 2} is unambiguously a set.

4. Generator expressions

Swap the brackets for parentheses and you get a generator expression — same syntax, but it produces items lazily, one at a time, instead of building the whole collection. This is the memory-efficient choice for large or streamed data.

Output (exact byte sizes are CPython-specific):

generator
85176 bytes (list)
200 bytes (generator)
333283335000

Analysis. The list comprehension built all 10,000 squares up front — ~85 KB. The generator expression stored almost nothing (~200 bytes); it computes each square only when asked. Passing a generator straight to sum(...) (note: no extra parentheses needed) computes the total without ever materializing the list — constant memory regardless of size.

Intuition. Mechanism. A list comprehension is eager: it computes and stores every element immediately. A generator expression is lazy: it yields one element per request and forgets it, so memory stays flat no matter how many elements flow through. (This is the iterator protocol, the subject of Tutorial 17.)

Concrete bite. Laziness has a cost: a generator is single-use — once consumed, it's empty:

[0, 1, 2]
[]

The first list(gen) pulled all three values; the second finds the generator exhausted and gets []. A list can be iterated again and again; a generator flows past once and is gone.

5. Nesting, flattening, and when not to

Comprehensions can contain multiple for clauses — most usefully to flatten a nested structure.

Output:

[1, 2, 3, 4, 5, 6]

Analysis. Two for clauses flatten the matrix: for each row in matrix, for each x in row, collect x. The clauses read left to right in the same order as nested loops — outer (for row in matrix) first, inner (for x in row) second.

Intuition. Mechanism. Multiple for clauses nest left-to-right exactly like written-out loops: the leftmost is the outermost. Each later clause can use the variables bound by earlier ones — but not the reverse.

Concrete bite. Write them in the "natural-looking" but wrong order — inner expression first — and the later-defined name isn't available yet:

Traceback (most recent call last):
  File "/w/main.py", line 2, in <module>
    flat = [x for x in row for row in matrix]   # wrong order: row used before its loop
                       ^^^
NameError: name 'row' is not defined. Did you mean: 'pow'?

for x in row comes first but row isn't defined until the second clause — so it's a NameError. The clause order must match nested-loop order: outer source first.

6. Mental-model summary

7. Gotcha checklist

Your Turn

Before you move on, check your understanding with the coach — explain the idea, apply it, weigh the trade-offs, then defend your reasoning.