Python collections Module: Counter, defaultdict and deque

Think about a small courier office in Siyaganj on Diwali week. Parcels pour in all day. One clerk keeps a tally of parcels per PIN code.

Another sorts parcels into bins by area, and a new bin appears whenever an unfamiliar area turns up. At the dispatch window, the oldest parcel goes out first, while new ones join at the back.

Each clerk has a specialised tool. The tally sheet, the self-creating bins and the first-in-first-out queue could all be built from ordinary lists and dictionaries, but you would rewrite the same bookkeeping code every time.

Python ships those tools ready-made in the collections module. The tally sheet is a Counter, the self-creating bins are a defaultdict, and the dispatch queue is a deque.

The collections module is a part of Python's standard library that provides specialised container types, including Counter, defaultdict, deque, namedtuple, OrderedDict and ChainMap, which extend the built-in list, dict and tuple for common data-handling jobs.

You have already met dictionaries and sorting in this series. Every tool here builds on them, and nothing needs installing: from collections import ... works in any Python 3.12+ environment, including the browser runner on this page.

Which collections type should you use?

Each container solves one recurring problem. Pick by the question you are asking of your data.

Match the problem on the left to the container on the right; each one replaces a few lines of manual bookkeeping.

How do you count with Counter?

A Counter is a dictionary that maps each item to how many times it appears. Pass it any iterable and it does the counting for you.

from collections import Counter

orders = ["poha", "chai", "samosa", "chai", "jalebi", "chai", "poha", "samosa", "chai"]
counts = Counter(orders)

print(counts)
print("Chai sold:", counts["chai"])
print("Kachori sold:", counts["kachori"])     # missing item: 0, not KeyError
print("Top 2:", counts.most_common(2))
print("Total items:", counts.total())

Two features set it apart from a normal dictionary. A missing key returns 0 instead of raising KeyError, and most_common(n) returns the n largest counts already sorted. total(), added in Python 3.10, sums all the counts.

Updating and combining counters

Counters can be updated with more data and added or subtracted like numbers, which is handy for comparing two periods.

from collections import Counter

october = Counter({"chai": 410, "poha": 220, "samosa": 180})
november = Counter({"chai": 455, "poha": 190, "jalebi": 95})

print("Both months:", october + november)
growth = november - october               # keeps only positive differences
print("Grew by:", growth)

november.update(["chai", "chai", "kachori"])   # add more raw items
print(november.most_common(3))

words = "late delivery cold food late delivery wrong order late".split()
print(Counter(words).most_common(1))

Note that - drops items whose result would be zero or negative. That makes it a quick way to answer "what increased?". Use november.subtract(october) if you want negative counts kept.

How does defaultdict work?

A defaultdict is a dictionary that calls a default factory whenever you access a key that does not exist yet. The factory is any function that takes no arguments, such as list, int or set. The new value is stored under the key and returned.

from collections import defaultdict

sales = [("Rajwada", "INV-1", 1200), ("Palasia", "INV-2", 800),
         ("Rajwada", "INV-3", 450), ("Vijay Nagar", "INV-4", 2300),
         ("Palasia", "INV-5", 650)]

invoices = defaultdict(list)       # missing key -> []
totals = defaultdict(int)          # missing key -> 0

for branch, inv, amount in sales:
    invoices[branch].append(inv)
    totals[branch] += amount

print(dict(invoices))
print(dict(totals))
print(totals["Sarafa"], "Sarafa" in totals)   # reading also creates the key

Compare this with the get(key, 0) and setdefault() patterns from the dictionaries article. The logic is identical, but the defaults are declared once at the top instead of repeated on every line.

The last line shows the one surprise: simply reading a missing key inserts it. If you only want to check, use in or .get() instead of square brackets.

namedtuple: tuples with field names

Rows stored as tuples are compact but cryptic: what is row[2]? A namedtuple creates a lightweight tuple class whose fields have names, so you can write row.amount instead. It is still a tuple underneath, so it is immutable, unpacks normally and uses no more memory than a plain tuple.

from collections import namedtuple

Sale = namedtuple("Sale", ["branch", "product", "qty", "price"])

rows = [Sale("Rajwada", "earphones", 3, 499),
        Sale("Palasia", "charger", 5, 349),
        Sale("Rajwada", "cover", 10, 199)]

for s in rows:
    print(f"{s.branch:<8} {s.product:<10} revenue={s.qty * s.price:>5}")

first = rows[0]
print(first.product, first[1])           # name or index
print(first._asdict())
print(first._replace(qty=4))             # a new tuple with one field changed

Plain tuples remain fine for two or three values. Reach for namedtuple once rows have four or more fields, or once you catch yourself writing comments like # index 3 is price. For records that need defaults, type hints or methods, the later lesson on Python dataclasses is a better fit.

deque: fast queues and rolling windows

A deque (pronounced "deck") is a double-ended queue. Adding or removing items at either end takes constant time. A list is fast only at its right end: list.pop(0) and list.insert(0, x) shift every other item along, which gets slow on long lists.

With maxlen set, appending on the right pushes the oldest item out of the left, which is exactly a rolling window.

from collections import deque

queue = deque(["parcel-1", "parcel-2"])
queue.append("parcel-3")              # join at the back
queue.appendleft("urgent-1")          # jump the queue
print("Dispatch:", queue.popleft())   # oldest out of the front
print("Waiting:", list(queue))

daily_sales = [12000, 15500, 9800, 17200, 14100, 20300]
window = deque(maxlen=3)
for day, amount in enumerate(daily_sales, start=1):
    window.append(amount)
    avg = sum(window) / len(window)
    print(f"Day {day}: last {len(window)} days avg = {avg:,.0f}")

The maxlen=3 version is a moving average in five lines, with no index arithmetic. pandas does the same thing for whole columns with df["sales"].rolling(3).mean(), but the deque version is handy for streaming data that arrives one value at a time.

OrderedDict and ChainMap

Since Python 3.7, normal dictionaries keep insertion order, so OrderedDict is rarely needed. It still adds move_to_end() and an order-sensitive ==, which are useful for small caches.

ChainMap groups several dictionaries into one view. A lookup searches each dictionary in turn and returns the first match, without copying anything.

from collections import ChainMap, OrderedDict

defaults = {"currency": "INR", "decimals": 2, "city": "Indore"}
user = {"decimals": 0}
settings = ChainMap(user, defaults)
print(settings["decimals"], settings["currency"])   # user value first, then default

recent = OrderedDict.fromkeys(["C101", "C102", "C103"])
recent.move_to_end("C101")                          # mark as most recently used
print(list(recent))

Common mistakes with the collections module

• Calling the factory. defaultdict(list()) passes an empty list, not the function. Write defaultdict(list).
• Accidental keys in a defaultdict. Reading d[key] creates it. Check with key in d.
• Expecting Counter subtraction to go negative. a - b drops non-positive counts; use a.subtract(b).
• Printing a defaultdict in reports. Its repr includes the factory. Convert with dict(d) first.
• Using list.pop(0) as a queue. It works but slows down on large lists. Use deque.popleft().
• Treating namedtuple as mutable. row.qty = 5 fails. Use row._replace(qty=5).

Key takeaways

• Counter counts items, returns 0 for missing keys and ranks with most_common().
• defaultdict(list) and defaultdict(int) turn grouping into one line per row.
• namedtuple gives tuple rows readable field names at no memory cost.
• deque is fast at both ends, and maxlen makes a rolling window.
• ChainMap layers dictionaries; OrderedDict is mainly for move_to_end().

That completes the data structures section. Next, start the functions section with Python functions, where you will wrap logic like this grouping into reusable pieces.