Skip to content

Latest commit

 

History

History
479 lines (311 loc) · 15.8 KB

File metadata and controls

479 lines (311 loc) · 15.8 KB

Python Typing Patterns

A guide to Python's type system — writing type-safe code that is flexible at inputs and precise at outputs.

typing module · PEP 484 · mypy docs · pyright

The Robustness Principle

Be conservative in what you send, be liberal in what you accept.

Applied to types: accept the most abstract type that works for inputs; return the most concrete type for outputs.

from collections.abc import Iterable

# Bad: too restrictive on input — callers must pass a list, not a tuple or generator
def total(items: list[int]) -> int:
    return sum(items)

# Good: accept anything iterable — works with lists, tuples, generators, sets
def total(items: Iterable[int]) -> int:
    return sum(items)


# Bad: vague output — callers don't know what they get back
def get_names(users: Iterable[User]) -> Iterable[str]:
    return [u.name for u in users]

# Good: concrete output — callers know exactly what they receive and can index it
def get_names(users: Iterable[User]) -> list[str]:
    return [u.name for u in users]

Abstract Input Types Cheat Sheet

collections.abc

Abstract type Use when the caller should be able to pass...
Iterable[T] Anything you can loop over once (generators, files, sets, lists, tuples)
Sequence[T] Anything with indexing and len(), but without mutation (list, tuple, str)
MutableSequence[T] A sequence the function will mutate in-place
Mapping[K, V] Any dict-like you only read from
MutableMapping[K, V] A dict-like the function will modify
Set[T] / MutableSet[T] Set semantics (membership, union, intersection)
Callable[..., T] Any callable returning T

Built-in Generics (Python 3.9+)

PEP 585

Python 3.9+ allows using built-in types directly as generics. No import from typing needed.

# Python 3.9+ — use built-in types
def process(items: list[str]) -> dict[str, int]:
    return {item: len(item) for item in items}

def wrap(value: int) -> tuple[int, str]:
    return (value, str(value))

# Python 3.8 and earlier — import from typing
from typing import List, Dict, Tuple
def process(items: List[str]) -> Dict[str, int]: ...


# Python 3.10+ — union shorthand with |
def parse(value: str | int | None) -> str:
    return str(value) if value is not None else ""

# Python 3.8/3.9 equivalent
from typing import Optional, Union
def parse(value: Union[str, int, None]) -> str: ...

Optional and Union

Optional · Union

# Optional[T] is exactly equivalent to T | None
def find_user(user_id: str) -> User | None:
    return db.get(user_id)

# Be explicit about when None is a valid return — never use Optional to mean "I'm unsure"
def get_timeout(config: dict) -> int | None:
    """Returns None only when timeout is explicitly absent from the config."""
    return config.get("timeout")

# Union — when a value can be one of several unrelated types
def coerce(value: str | int | float) -> float:
    return float(value)

Generic Functions and Classes

PEP 695 · type statement

Generic functions and classes let you write code whose types are linked to their argument types — preserving type information through transformations instead of losing it to Any.

Prior to Python 3.12, generics could only be written with TypeVar + Generic (PEP 484) — importing from typing and declaring each type variable as a module-level name. Python 3.12 built generics into the language grammar via PEP 695: no imports needed, and the type parameter is scoped to the class/function/alias instead of leaking into module scope. Variance (covariant/contravariant) is also inferred automatically — no covariant=True flag needed.

import copy

def first[T](items: list[T]) -> T | None:
    """Return type mirrors the element type of items."""
    return items[0] if items else None

first([1, 2, 3])    # inferred as int | None
first(["a", "b"])   # inferred as str | None


# Bounded — T must be a subtype of the given class
class Animal:
    def speak(self) -> str: ...

def clone[A: Animal](animal: A) -> A:
    return copy.deepcopy(animal)


# Constrained — T must be exactly one of the listed types
def ensure_str[S: (str, bytes)](value: S) -> str:
    return value.decode() if isinstance(value, bytes) else value

Generic Classes

class Stack[T]:
    def __init__(self) -> None:
        self._items: list[T] = []

    def push(self, item: T) -> None:
        self._items.append(item)

    def pop(self) -> T:
        return self._items.pop()

    def peek(self) -> T | None:
        return self._items[-1] if self._items else None


stack: Stack[int] = Stack()
stack.push(1)
stack.push("hello")   # type error — Stack[int] expects int

Variance — Covariant and Contravariant

Variance · PEP 484 — variance

Variance decides whether Container[Dog] can be used where Container[Animal] is expected. With PEP 695 it is inferred from how T is used — you don't declare it:

  • CovariantT appears only in output positions (return types). A producer of Dog can stand in for a producer of Animal.
  • ContravariantT appears only in input positions (parameters). A consumer of Animal can stand in for a consumer of Dog.
  • InvariantT appears in both — no substitution allowed either way.
class Animal:
    def speak(self) -> str: ...

class Dog(Animal):
    def fetch(self) -> None: ...

class Cat(Animal): ...


# Covariant — T only ever comes out, so Box[Dog] can be passed where Box[Animal] is expected
class Box[T]:
    def __init__(self, item: T) -> None:
        self._item = item

    def get(self) -> T:
        return self._item

def print_animal(box: Box[Animal]) -> None:
    print(box.get().speak())

print_animal(Box(Dog()))   # OK — a Box[Dog] is a Box[Animal] for reading purposes

# Counter-example — the reverse direction fails
def fetch_from_box(box: Box[Dog]) -> None:
    box.get().fetch()

fetch_from_box(Box(Animal()))   # type error — an Animal isn't guaranteed to be a Dog with .fetch()


# Contravariant — T only ever goes in, so Handler[Animal] can be passed where Handler[Dog] is expected
class Handler[T]:
    def handle(self, item: T) -> None: ...

class AnimalHandler(Handler[Animal]):
    def handle(self, item: Animal) -> None:
        print(item.speak())

def register(handler: Handler[Dog]) -> None: ...

register(AnimalHandler())   # OK — a handler that accepts any Animal can accept a Dog

# Counter-example — the reverse direction fails
class DogHandler(Handler[Dog]):
    def handle(self, item: Dog) -> None:
        item.fetch()

def register_animal_handler(handler: Handler[Animal]) -> None:
    handler.handle(Cat())   # would crash — DogHandler.handle() expects a Dog, not any Animal

register_animal_handler(DogHandler())   # type error — a Handler[Dog] is not a Handler[Animal]

Pre-3.12, variance had to be declared explicitly on the TypeVar:

from typing import TypeVar

T_co = TypeVar("T_co", covariant=True)       # for Box
T_contra = TypeVar("T_contra", contravariant=True)   # for Handler

Generic Type Aliases

# The `type` statement replaces TypeAlias
type IntList = list[int]
type Pair[T] = tuple[T, T]

Pre-3.12 Codebases — TypeVar and Generic

TypeVar · Generic · PEP 484

Use this form only when the project must support Python 3.9–3.11 — it is the sole way generics were expressed before PEP 695.

import copy
from typing import Generic, TypeVar

T = TypeVar("T")

def first(items: list[T]) -> T | None:
    return items[0] if items else None


class Animal:
    def speak(self) -> str: ...

A = TypeVar("A", bound=Animal)

def clone(animal: A) -> A:
    return copy.deepcopy(animal)


AnyStr = TypeVar("AnyStr", str, bytes)

def ensure_str(value: AnyStr) -> str:
    if isinstance(value, bytes):
        return value.decode()
    return value

Protocol — Structural Subtyping

Protocol · PEP 544

Protocol expresses duck typing in the type system: if an object has the right methods and attributes, it satisfies the Protocol — regardless of what it inherits.

from typing import Protocol, runtime_checkable


class Drawable(Protocol):
    def draw(self) -> None: ...
    def resize(self, factor: float) -> None: ...


class Circle:
    def draw(self) -> None: print("○")
    def resize(self, factor: float) -> None: self.radius *= factor


class Square:
    def draw(self) -> None: print("□")
    def resize(self, factor: float) -> None: self.side *= factor


def render_all(shapes: list[Drawable]) -> None:
    for shape in shapes:
        shape.draw()  # works for Circle and Square — no shared base class needed


# @runtime_checkable enables isinstance() checks (only checks method existence, not signatures)
@runtime_checkable
class Closeable(Protocol):
    def close(self) -> None: ...


isinstance(open("file.txt"), Closeable)   # True at runtime

Protocol vs ABC: use Protocol when you cannot control the types being passed (third-party, external data); use ABC when defining your own class hierarchy and want an enforced contract.


TypedDict

TypedDict · PEP 589

TypedDict is a dict whose keys and value types are declared statically. It is better than dict[str, Any] whenever the shape of the dict is known.

from typing import TypedDict, NotRequired


class UserDict(TypedDict):
    id: str
    name: str
    email: str


class CreateUserPayload(TypedDict):
    name: str
    email: str
    role: NotRequired[str]   # key may be absent — distinct from Optional (value can't be None)


def create_user(payload: CreateUserPayload) -> UserDict:
    return {
        "id": generate_id(),
        "name": payload["name"],
        "email": payload["email"],
    }


# TypedDict catches typos and wrong value types at type-check time
payload: CreateUserPayload = {"name": "Alice", "email": "alice@example.com"}
payload["nmae"] = "Bob"   # type error — unknown key

Literal — Exact Value Types

Literal · PEP 586

Literal constrains a parameter to a fixed set of allowed values. Better than str when only a handful of strings are valid.

from typing import Literal

Direction = Literal["north", "south", "east", "west"]
HttpMethod = Literal["GET", "POST", "PUT", "DELETE", "PATCH"]


def move(direction: Direction, steps: int) -> None: ...

move("north", 3)    # OK
move("up", 3)       # type error — "up" is not a valid Direction


def request(method: HttpMethod, url: str) -> Response: ...

request("GOT", "/users")   # type error — caught before runtime

Callable

Callable · collections.abc.Callable

from collections.abc import Callable

# Callable[[arg_types, ...], return_type]
def apply(func: Callable[[int, int], int], a: int, b: int) -> int:
    return func(a, b)

apply(lambda x, y: x + y, 1, 2)   # 3


# Unknown/variable signature — use Callable[..., ReturnType]
def call_later(func: Callable[..., None], delay: float) -> None:
    import time
    time.sleep(delay)
    func()


# Functions that return functions
def make_adder(n: int) -> Callable[[int], int]:
    return lambda x: x + n

overload — Multiple Signatures

overload

@overload expresses that a function has several distinct signatures — useful when the return type depends on the argument types.

from typing import overload


@overload
def process(value: str) -> str: ...
@overload
def process(value: int) -> int: ...
@overload
def process(value: list[str]) -> list[str]: ...

def process(value):
    """Only the implementation (no @overload) has a body."""
    if isinstance(value, str):
        return value.upper()
    if isinstance(value, int):
        return value * 2
    return [s.upper() for s in value]


result = process("hello")    # inferred as str
result = process(42)         # inferred as int
result = process(["a", "b"]) # inferred as list[str]

TYPE_CHECKING — Avoiding Circular Imports

TYPE_CHECKING · PEP 563

TYPE_CHECKING is False at runtime but True when a type checker runs. Use it to break circular imports or avoid importing heavy modules at startup.

from __future__ import annotations   # all annotations become strings — evaluated lazily
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from myapp.models import User   # not imported at runtime — only when type-checking


def get_user(user_id: str) -> User:   # works because the annotation is a string (lazy)
    from myapp.models import User     # real runtime import inside the function body
    return User.get(user_id)

Other Useful Typing Constructs

These constructs are common in real codebases. The linked docs cover them in full.

Construct Purpose Docs
Final Mark a name as a constant — cannot be reassigned or overridden in subclasses PEP 591
Annotated Attach metadata to a type (validation rules, DI markers — used by FastAPI, Pydantic) PEP 593
NewType Create a distinct type from an existing one (UserId = NewType("UserId", str)) to prevent accidental mixing PEP 484
TypeGuard Mark a function as a type predicate — narrows the type inside the if block where it returns True PEP 647
ParamSpec Capture a callable's parameter spec — preserves argument names and types through decorator wrappers (see python-patterns.md) PEP 612

Core rules to remember:

  1. Accept the most abstract input type that covers your needs (Iterable, Sequence, Mapping)
  2. Return the most concrete output type you can commit to (list, dict, your domain class)
  3. Use TypeVar to link input and output types without collapsing to Any
  4. Use Protocol for duck typing with type-checker support; use ABC for enforced class hierarchies
  5. Use NewType to distinguish values that share a primitive type but must never be mixed
  6. On Python 3.12+, prefer PEP 695 syntax (class Stack[T], def first[T](...), type Alias = ...) over TypeVar/Generic