lesson.md

layout	default
title	Performance and Memory
parent	Lessons
nav_order	8
permalink	/lessons/performance-memory/
course_lesson	true
course_index	08
previous_page	/lessons/concurrency-parallelism/
previous_title	Concurrency and Parallelism
next_page	/lessons/packaging-distribution/
next_title	Packaging and Distribution

08 - Performance and Memory

Performance work is an evidence loop: define a user-visible problem, measure a baseline, find the real bottleneck, change one thing, and measure again. Fast incorrect code is still incorrect.

A simple picture

If a school bus is late, making every child run faster may not help—the bus route may be the delay. Profiling finds the actual slow stage before effort is spent.

Define the requirement

Different systems care about different measures:

• latency: time for one operation;
• throughput: operations completed per unit time;
• memory: current and peak retained data;
• startup: time until useful work begins;
• tail latency: slow requests near the worst end, not only the average.

Write the input size, environment, and target before optimizing.

Complexity before micro-optimization

def common_values(left, right):
    right_lookup = set(right)
    return [value for value in left if value in right_lookup]

Repeated list membership can compare against many right-side items for each left-side item. Building a set adds upfront time and memory but usually makes repeated membership much cheaper. For tiny inputs, the simpler list approach may still be sufficient.

Big-O describes growth, not exact time. Constants, data distribution, cache behavior, and implementation details still matter.

Measure small code with timeit

from timeit import repeat


measurements = repeat(
    "target in values",
    setup="values = list(range(10_000)); target = 9_999",
    repeat=5,
    number=1_000,
)

print(min(measurements))

Use repeated runs, realistic setup, and compare the same behavior. The minimum can approximate the least disturbed run; the distribution reveals noise. Do not hide important setup costs unless the real application also reuses that setup.

Profile a whole program

python -m cProfile -s cumulative app.py

The cumulative view helps locate call paths consuming time. A profiler adds overhead, so use it to find candidates, then benchmark the candidate separately.

For memory allocations:

import tracemalloc


tracemalloc.start()
result = build_report()
current, peak = tracemalloc.get_traced_memory()
print(f"current={current:,}, peak={peak:,}")
tracemalloc.stop()

Keep result alive if the real program keeps it alive. Otherwise the measurement answers a different question.

Object references and retention

CPython mainly reclaims objects when reference counts reach zero and also has a cyclic garbage collector for certain reference cycles. An object is not leaked merely because memory is not immediately returned to the operating system; Python's allocator may keep arenas for reuse.

Common accidental retention sources include:

• unbounded dictionaries and caches;
• global lists of completed work;
• callbacks that close over large objects;
• tasks or exceptions retaining stack frames;
• queues whose consumers cannot keep up.

Measure object growth over repeated operations, not only process size after one run.

Streaming and compact objects

Generators reduce peak result storage when consumers can process one item at a time. They do not make a final sorted() or full grouping lazy.

__slots__ can remove the usual per-instance attribute dictionary for classes with fixed fields:

class Point:
    __slots__ = ("x", "y")

    def __init__(self, x, y):
        self.x = x
        self.y = y

Slots may reduce memory for very many instances, but affect inheritance, weak references, and dynamic attributes. Measure the real model before choosing them. @dataclass(slots=True) is often clearer.

Caching

from functools import lru_cache


@lru_cache(maxsize=256)
def parse_rule(rule_text):
    return expensive_parse(rule_text)

A cache exchanges memory and freshness complexity for speed. Define:

• stable hashable key;
• maximum size or expiry;
• invalidation rule;
• behavior for failures;
• whether returned mutable objects can be safely shared.

Do not cache exceptions or user-specific data under an incomplete key.

Benchmark design

A useful benchmark records:

Python/runtime:
hardware and operating system:
input source and size:
warm-up:
number of repeats:
metric and summary:
baseline result:
changed result:
behavior-equivalence checks:

Control randomness, network variability, logging, and background load where practical. Report uncertainty instead of claiming precision the experiment cannot support.

Under the hood

Python objects usually carry type and memory-management metadata in addition to payload. Containers hold references to objects, so a list's own size does not include all referenced object memory. sys.getsizeof() reports shallow size only.

Function calls, attribute lookup, object allocation, and Python bytecode execution have costs, but algorithm and I/O choices usually dominate. Built-ins often run optimized C loops, yet replacing clear code is worthwhile only after measurement.

Bug Hunter

Bug 1: benchmark changes the behavior

fast_result = set(values)

Bug 2: unbounded cache

@cache
def report_for_user(user_id, date):
    ...

Bug 3: measures only task submission

start = perf_counter()
futures = [pool.submit(work, item) for item in items]
print(perf_counter() - start)

Show Bug Hunter fixes

1. A set removes duplicates and ordering; first prove those changes are allowed.
2. Use a bound/expiry and confirm the key includes every value affecting the result.
3. Wait for and inspect all results before stopping the timer.

Practice

1. Analyze growth of a nested membership search.
2. Replace repeated list lookup with a set while preserving required output.
3. Benchmark two equivalent implementations using repeated realistic runs.
4. Profile a slow application and identify cumulative hot paths.
5. Use tracemalloc to compare eager and streaming versions.
6. Compare measured memory of normal and slotted records.
7. Add a bounded cache and state its invalidation policy.
8. Find an optimization that changes ordering or error behavior.
9. Design a latency benchmark that includes warm-up and percentiles.
10. Write a stable performance budget check for a critical operation.

Show hints

1. Count repeated membership work. 2. Keep original iteration order. 3. Validate outputs before timing. 4. Sort by cumulative time. 5. Record peak memory while consuming results. 6. Include referenced field objects in your reasoning. 7. Define key, bound, and freshness. 8. Compare the full contract. 9. Keep every raw result. 10. Use a generous regression threshold, not a noisy microsecond equality.

Show solution ideas

1. A set changes repeated lookup growth after one construction pass. 2. Iterate left and look up in right_set. 3. Record environment and distributions. 4. Optimize the meaningful path, not necessarily the most-called tiny function. 5. Consume the same amount of input in both versions. 6. Slots help only when instance count makes the saving meaningful. 7. lru_cache(maxsize=...) handles least-recently-used eviction. 8. Test duplicates, order, exceptions, and types. 9. Report median and a high percentile. 10. Run in a controlled CI job or use the check as monitored evidence rather than a flaky unit test.

Homework

Profile one completed project. Publish a short report containing the requirement, baseline, profiler evidence, selected change, correctness checks, after measurement, and one optimization you rejected.

Checkpoint

Explain the difference between complexity, benchmarking, profiling, shallow object size, retained memory, and a true leak. Defend one optimization with evidence.