lesson.md

layout	default
title	Testing and Code Quality
parent	Lessons
nav_order	10
permalink	/lessons/testing-code-quality/
course_lesson	true
course_index	10
previous_page	/lessons/packaging-distribution/
previous_title	Packaging and Distribution
next_page	/lessons/production-apis/
next_title	Production APIs

10 - Testing and Code Quality

Tests provide evidence about behavior. Different test levels answer different questions, and automated quality checks shorten feedback without replacing thoughtful design and review.

A simple picture

A bicycle factory checks one brake, then an assembled bicycle, then a real ride. Testing only individual screws cannot prove the full bicycle works; testing only full rides makes every failure slow and difficult to locate.

Test levels

Level	Main question	Typical dependencies
Unit	Does one rule behave correctly?	Fakes or plain values
Integration	Do real boundaries work together?	Database, filesystem, serializer
Contract	Do two sides agree on a protocol?	Recorded or provider examples
End-to-end	Can a user complete a critical flow?	Running system

Use many focused fast tests, enough real integration tests to catch boundary mistakes, and a small set of critical end-to-end paths. The right mix follows risk.

Behavior-focused unit tests

def discount(price, percent):
    if price < 0:
        raise ValueError("price cannot be negative")
    if not 0 <= percent <= 100:
        raise ValueError("percent must be between 0 and 100")
    return price * (1 - percent / 100)


def test_full_discount_returns_zero():
    assert discount(100, 100) == 0

A test name should describe observable behavior. Avoid asserting private helper calls unless those calls are the actual contract.

For floating-point business values, choose a domain representation such as Decimal when exact decimal rules matter. Do not hide a model error behind a very wide approximate comparison.

Arrange, act, assert

def test_service_saves_valid_order():
    repository = MemoryOrderRepository()       # arrange
    service = OrderService(repository)

    created = service.create({"total": 150})   # act

    assert created.total == 150                 # assert
    assert repository.saved == [created]

Keep one behavior per test while allowing several assertions about that one outcome.

Parametrization and fixtures

With pytest:

import pytest


@pytest.mark.parametrize(
    ("percent", "expected"),
    [(0, 100), (25, 75), (100, 0)],
)
def test_discount_examples(percent, expected):
    assert discount(100, percent) == expected

A fixture provides setup and cleanup. Keep fixtures understandable; a deeply connected global fixture graph can hide why a test passes.

Use temporary directories and isolated databases. Tests should not depend on execution order or a developer's machine.

Fakes, stubs, and mocks

• A stub returns prepared answers.
• A fake has a lightweight working implementation, such as an in-memory repository.
• A mock records expected interactions.

Mock external boundaries where needed, not every internal function. Over-mocking couples tests to implementation and can prove that mocks agree with themselves while the real integration is broken.

class FakeClock:
    def __init__(self, current):
        self._current = current

    def now(self):
        return self._current

Inject clocks, random sources, UUID generators, and delay functions to remove nondeterminism.

Failure-path and property thinking

Test normal cases, boundaries, malformed input, dependency failures, retries, cancellation, and cleanup. Invariants often reveal better cases than examples:

sorting never changes the number of items
encode then decode returns the original supported value
adding an item then removing it restores the prior state

Property-based testing tools can generate cases, but even handwritten tests benefit from invariant thinking.

Flaky tests

A flaky test sometimes passes and sometimes fails without a relevant code change. Common causes are real time, random values, thread scheduling, shared files, network calls, unordered collections, and leaked global state.

Do not fix flakiness with a larger arbitrary sleep. Synchronize on events or observable state, inject nondeterminism, and isolate resources.

Type checking, linting, and formatting

• Type checking finds incompatible assumptions before execution.
• A linter finds suspicious constructs and consistency problems.
• A formatter removes style debate.
• Security and dependency scanners find specific known risks.

No tool proves correctness. Keep configurations version-controlled and avoid enabling hundreds of unexplained rules at once.

Continuous integration

A useful CI pipeline starts from a clean checkout and runs:

install declared dependencies
-> format check
-> lint
-> type check
-> unit tests
-> integration tests
-> build package
-> install/test artifact

Pin CI actions by trusted immutable versions according to the project's supply-chain policy, minimize token permissions, and never expose secrets to untrusted contributions.

Under the hood

assert expression is a language statement that can be removed when Python runs with optimization. It is appropriate inside tests and for internal invariants, not for validating user input or permissions.

Test collection imports modules, so import-time side effects can make tests slow or order-dependent. Coverage measures executed lines or branches, not whether assertions are meaningful. Mutation testing can reveal tests that execute code without detecting changed behavior.

Bug Hunter

Bug 1: catches its own expected failure

try:
    create_user(invalid_data)
except ValueError:
    pass

Bug 2: order-dependent shared state

repository = MemoryRepository()

at module scope for every test.

Bug 3: waits for time

time.sleep(2)
assert worker.finished

Show Bug Hunter fixes

1. Use a test helper such as pytest.raises and assert the message or error data when part of the contract.
2. Create fresh state per test or reset it through a clear fixture.
3. Wait on a synchronization event with a timeout, or test the worker rule synchronously.

Practice

1. Write normal, boundary, and invalid tests for a price rule.
2. Parametrize a parser across valid and malformed rows.
3. Build a temporary SQLite integration fixture.
4. Replace an HTTP boundary with a small fake client.
5. Write one contract test that every repository adapter must pass.
6. Inject time and randomness into a service.
7. Diagnose an order-dependent test suite.
8. Test a retry policy without sleeping.
9. Add a format, lint, type, test, and build CI sequence.
10. Review high coverage and find one important behavior it misses.

Show hints

1. Include both ends of allowed ranges. 2. Give every case an expected result or exception. 3. Isolate and close the database. 4. Return programmed responses and record calls. 5. Pass an adapter factory into shared tests. 6. Use ports such as clock.now(). 7. Randomize order and find shared resources. 8. Inject a no-op delay and scripted failures. 9. Fail early on cheap checks. 10. Coverage cannot judge assertions.

Show solution ideas

1. Test zero, maximum, out-of-range, and negative price. 2. Include blank, missing, wrong type, and Unicode data. 3. Use a temporary path and real schema. 4. Keep the fake at the transport port. 5. Exercise save/get/missing/update semantics. 6. Fixed values make tests reproducible. 7. Remove module globals and filesystem collisions. 8. Assert attempt count and final error. 9. Build and install the artifact after tests. 10. Add failure behavior, permission, concurrency, or cleanup assertions based on risk.

Homework

Take an earlier project from fragile to maintainable. Add a risk-based test plan, unit and integration tests, deterministic failure cases, typing, formatting, linting, and a CI workflow. Explain what remains untested and why.

Checkpoint

Choose an appropriate test level for five risks, explain why mocks can mislead, remove one source of flakiness, and state why coverage is evidence rather than a score of correctness.