Skip to content

Latest commit

 

History

History
655 lines (505 loc) · 22.8 KB

File metadata and controls

655 lines (505 loc) · 22.8 KB

Worked Examples

Before/after pairs for the core rules, in language-agnostic pseudocode. Adapt the syntax to your stack (xUnit, JUnit, pytest, Jest, RSpec, …) and your existing conventions. The point is the shape of the test, not the language.

Examples are grouped by book. The Khorikov set covers the cross-cutting rules; each later section adds pairs that illustrate that book's distinctive ideas.


Khorikov — Unit Testing

1. Test observable behavior, not interactions

A DeliveryService validates a delivery date using a Clock to know "today."

❌ Communication-based, coupled to internals

test "validate checks the clock":
    clock = mock(Clock)
    service = DeliveryService(clock)

    service.validate(delivery)

    verify(clock.now was called once)      # asserting a call into a stub

This asserts how the service works (it consults the clock). Cache "today," call it twice, or read the date differently and the test breaks though behavior is fine.

✅ Output-based, coupled to behavior

test "delivery with a past date is invalid":
    clock   = stub(Clock, now = 2026-06-09)   # stub: feeds data, never asserted
    service = DeliveryService(clock)

    result = service.validate(delivery(date = 2026-06-01))

    assert result.isValid == false

Now any internal change that still rejects past dates keeps the test green.


2. Mock unmanaged dependencies; use the real managed one

A RegisterUser use case writes the user to your database (managed) and sends a welcome email via an SMTP gateway (unmanaged).

❌ Everything mocked, asserting DB calls

test "register saves and emails":
    repo  = mock(UserRepository)
    email = mock(EmailGateway)
    sut   = RegisterUser(repo, email)

    sut.execute("a@b.com")

    verify(repo.save was called with any User)   # mocking a MANAGED dependency
    verify(email.send was called)

The repo.save assertion pins down your data-access; changing the schema or query breaks it. It also proves nothing about whether the user is actually persisted correctly.

✅ Integration test: real DB (assert state), mocked email (assert interaction)

test "registering a user persists them and sends a welcome email":
    db    = realTestDatabase()                 # managed → use the real thing
    email = mock(EmailGateway)                 # unmanaged → mock + assert
    sut   = RegisterUser(UserRepository(db), email)

    sut.execute("a@b.com")

    saved = db.query("users", email = "a@b.com")
    assert saved.exists                        # observable STATE
    verify(email.send was called with to = "a@b.com")   # observable CONTRACT

3. Don't leak domain knowledge into the test

Testing a priceWithTax(amount, rate) function.

❌ Re-implements the formula → tests nothing

test "applies tax":
    amount = 100; rate = 0.2
    expected = amount + amount * rate          # same algorithm as production
    assert priceWithTax(amount, rate) == expected

If the production formula is wrong, expected is wrong the same way, and the test still passes.

✅ Hard-coded expected value

test "a 20% tax on 100 yields 120":
    assert priceWithTax(100, 0.2) == 120

4. Don't test private methods — extract instead

A Report class has a complex private calculateScore() you're tempted to test directly.

❌ Reaching into a private method

test "calculateScore works":
    report = Report(data)
    assert report.callPrivate("calculateScore") == 42   # testing an internal

✅ Extract the logic into its own unit with a public API

# Production: pull the calculation out into a pure, public type
class ScoreCalculator:
    function calculate(data) -> number: ...

# Test the extracted unit through its public surface, output-based
test "score of a fully-completed dataset is 100":
    assert ScoreCalculator().calculate(completedData) == 100

The need to test a private method is a signal of a missing abstraction. Extract, then the test is clean and the design improves.


5. One behavior per test; parameterize the rest

❌ Multiple Acts / branching in one test

test "validation":
    assert validate(pastDate).isValid   == false
    assert validate(today).isValid      == true
    assert validate(futureDate).isValid == true     # three behaviors in one test

✅ Parameterized, one behavior each

parameterized test "delivery validity by date" with cases:
    (date = yesterday, expected = false)
    (date = today,     expected = true)
    (date = tomorrow,  expected = true)
  →
    assert validate(delivery(date)).isValid == expected

6. Inject time as a value, not via the ambient clock

❌ Reads the clock inside the logic (non-deterministic)

function isExpired(token):
    return token.expiresAt < now()    # hidden dependency on the real clock

✅ Time passed in as a plain value → pure and output-testable

function isExpired(token, currentTime):
    return token.expiresAt < currentTime

test "a token is expired one second after its expiry":
    token = tokenExpiringAt(2026-06-09T00:00:00)
    assert isExpired(token, 2026-06-09T00:00:01) == true

7. Name tests as behavior, not as code

❌ Mechanical ✅ Behavioral
Sum_TwoPositives_ReturnsSum Sum_of_two_positive_numbers
IsValid_EmptyName_ReturnsFalse A_customer_without_a_name_is_invalid
Withdraw_Overdraft_ThrowsException Withdrawing_more_than_the_balance_fails

Read the name out loud to a non-programmer. If it sounds like a sentence about what the system does, it's good; if it sounds like a method signature, rewrite it.


The Art of Unit Testing (Osherove)

1. One test per exit point

A transfer(from, to, amount) unit of work has three exit points: it returns a receipt (value), updates two balances (state), and notifies a bus (third party).

❌ One test asserts everything, including internal calls

test "transfer works":
    bus   = mock(Bus)
    ledger = mock(Ledger)                       # mocking an in-process collaborator
    sut   = Transfer(ledger, bus)

    receipt = sut.transfer("A", "B", 30)

    assert receipt.ok == true
    verify(ledger.debit was called with "A", 30)   # internal call, not an exit point
    verify(ledger.credit was called with "B", 30)  # internal call, not an exit point
    verify(bus.publish was called)

Three behaviors tangled together, two assertions on internal calls. Any refactor of how the ledger is updated breaks this, and the name tells you nothing when it fails.

✅ One focused test per exit point; value/state preferred, mock only the third party

test "a transfer returns a confirmed receipt":          # value exit
    receipt = realTransfer().transfer("A", "B", 30)
    assert receipt.ok == true

test "a transfer moves the amount from one balance to the other":   # state exit
    ledger = Ledger(A = 100, B = 0)
    Transfer(ledger, busStub).transfer("A", "B", 30)
    assert ledger.balanceOf("A") == 70
    assert ledger.balanceOf("B") == 30

test "a completed transfer is announced on the bus":    # third-party exit
    bus = mock(Bus)                                      # the announcement IS the result
    Transfer(realLedger, bus).transfer("A", "B", 30)
    verify(bus.publish was called with TransferCompleted("A", "B", 30))

The real Ledger is an in-process collaborator — use it, don't mock it. Only the bus, an outgoing third party, is mocked.


2. Stop over-specifying: assert the exit point, not the choreography

A checkout() reads a cart from a repository (incoming) and returns a total.

❌ Asserts a call into a stub, and stacks mocks

test "checkout":
    cart = mock(CartRepo)
    when(cart.findById("c1")).thenReturn(cartWith(2 items @ 50))
    tax  = mock(TaxService)
    sut  = Checkout(cart, tax)

    sut.checkout("c1")

    verify(cart.findById was called with "c1")   # stub-as-mock: implementation detail
    verify(tax.calculate was called)             # second mock: internal choreography

✅ Stub the input, assert the returned total (the exit point)

test "checkout totals the cart with tax":
    cart = stub(CartRepo, findById("c1") = cartWith(2 items @ 50))
    sut  = Checkout(cart, TaxService(rate = 0.1))   # real, in-process

    total = sut.checkout("c1")

    assert total == 110          # 100 + 10% tax — the observable result

Changing how the total is computed internally no longer breaks the test.


3. Name by behavior (USE → sentence)

Osherove's USE naming (Unit-Scenario-Expectation) is good; this repo prefers the behavior sentence, which carries the same facts and reads aloud.

❌ Method-shaped ⚖️ USE (book's convention) ✅ Behavior sentence (house style)
testVerify1 verify_withFailedRule_returnsErrorWithReason A_password_that_fails_a_rule_is_rejected
testTransferThrows transfer_overBalance_throws Transferring_more_than_the_balance_is_rejected

The sentence form doesn't bind the test name to the method name, so renaming the method doesn't leave a misleading test name behind.


Effective Software Testing (Aniche)

1. Derive cases from partitions and boundaries, don't guess

shippingCost(weightKg, country): weight must be in (0, 30]; over 30 is rejected; country is domestic or international.

❌ Two happy-path asserts, picked by feel

test "shipping":
    assert shippingCost(5, "domestic") == 4.0
    assert shippingCost(10, "international") == 12.0

Nothing at the limits, no invalid inputs, no rejection path. The off-by-one in weight <= 30 vs < 30 ships undetected.

✅ Partitions + on/off boundary points, parameterized

parameterized test "shipping cost by weight and destination" with cases:
    (0,     "domestic",      INVALID)    # lower on point (0 is excluded)
    (0.01,  "domestic",      4.0)        # lower off point (just valid)
    (30,    "international",  ACCEPTED)   # upper on point (30 is included)
    (30.01, "international",  REJECTED)   # upper off point (just over)
    (10,    "unknown",        INVALID)    # invalid-country partition
  →
    assert shippingCost(weight, country) == expected

Every case exists for a named reason; both boundaries are pinned by an on/off pair.


2. Use coverage as a guide, not a target

A coverage report shows the else of a discount branch is never executed.

❌ Hit the line to make the number green

test "covers the else branch":
    order = orderWith(total = 50)
    applyDiscount(order)            # executes the else… but asserts nothing

100% branch coverage, zero protection — a tautological, assertion-free test.

✅ Treat the uncovered branch as a missing partition and test the behavior

# The else fires when total < the discount threshold → that's a real partition.
test "orders below the threshold get no discount":
    order = orderWith(total = 50)        # threshold is 100
    result = applyDiscount(order)
    assert result.discount == 0
    assert result.payable  == 50

The uncovered branch was the signal; the fix is a behavior test, not a line-hitter.


3. Replace a pile of examples with one property

reverse(list) tested by hand-picked examples.

❌ A few examples that miss edge cases

test "reverse": assert reverse([1,2,3]) == [3,2,1]
test "reverse2": assert reverse([1]) == [1]

✅ State the property; let the generator explore

property "reversing twice returns the original list":
    for all lists xs:
        assert reverse(reverse(xs)) == xs

property "reverse preserves length and multiset of elements":
    for all lists xs:
        assert length(reverse(xs)) == length(xs)
        assert asMultiset(reverse(xs)) == asMultiset(xs)

The framework throws hundreds of lists (empty, huge, duplicates) at the round-trip and invariants, and shrinks any failure to a minimal counterexample.


xUnit Test Patterns (Meszaros)

1. Test Code Duplication → Creation Method

Several tests each construct the same "overdue invoice" by hand.

❌ Duplicated, intent-obscuring setup (Obscure Test + Test Code Duplication)

test "overdue invoice accrues a late fee":
    inv = Invoice()
    inv.customer = Customer("Acme")
    inv.amount = 100
    inv.dueDate = 2026-01-01
    inv.status = SENT                      # 6 lines of noise; which fields matter?
    assert lateFee(inv, today = 2026-02-01) == 10

✅ Creation Method names the intent; the test shows only what matters

# A Creation Method (optionally an Object Mother / Test Data Builder) hides the mechanics:
function anOverdueInvoice(amount):
    return Invoice(customer = aCustomer(), amount = amount,
                   dueDate = 2026-01-01, status = SENT)

test "overdue invoice accrues a 10% late fee":
    assert lateFee(anOverdueInvoice(amount = 100), today = 2026-02-01) == 10

Change the Invoice constructor once, not in twenty tests; each test states only the attribute it depends on.


2. Assertion Roulette → one condition, Custom Assertion

❌ A pile of unlabeled assertions — when CI goes red, which one failed?

test "register user":
    u = register("a@b.com", "Ann")
    assert u.email == "a@b.com"
    assert u.name == "Ann"
    assert u.active == true
    assert u.id != null
    assert u.createdAt != null            # Assertion Roulette + Eager Test

✅ One behavior per test; a Custom Assertion names the concept

test "a newly registered user is active":
    assert register("a@b.com", "Ann").active == true

test "a registered user keeps the details it was given":
    assertHasIdentity(register("a@b.com", "Ann"), email = "a@b.com", name = "Ann")
    # assertHasIdentity is a Custom Assertion: one clear failure message,
    # checking only the attributes that define identity (no Sensitive Equality)

3. Shared Fixture (Erratic Test) → Fresh Fixture

Tests share one database row to save setup time, then fail intermittently.

❌ Shared Fixture → Interacting Tests / Test Run War

sharedUser = db.insert(User("a@b.com"))     # built once, reused by all tests

test "deactivate":  deactivate(sharedUser); assert sharedUser.active == false
test "rename":      rename(sharedUser, "Bo"); assert sharedUser.name == "Bo"
# Run "deactivate" first and "rename" sees an inactive user. Run them in parallel
# (two CI jobs on the same DB) and they corrupt each other → flaky.

✅ Fresh Fixture per test (transient, or rolled back); tests are independent

test "deactivating a user clears its active flag":
    user = aUser()                          # fresh per test (in-memory, or own DB row)
    deactivate(user)
    assert user.active == false

test "renaming a user changes its name":
    user = aUser()
    rename(user, "Bo")
    assert user.name == "Bo"

If real-DB speed forces sharing, isolate with a per-developer Database Sandbox + Transaction Rollback Teardown, or swap the DB for an in-memory Fake to get Fresh-Fixture independence at Shared-Fixture speed.


Working Effectively with Legacy Code (Feathers)

1. Pin current behavior with a characterization test

You must change PriceCalculator but it has no tests and no spec. Don't guess what it should return — discover what it does.

❌ Guessing the expected value (you don't actually know it)

test "price calc":
    assert calc.totalFor(order) == 42.0     # …is 42 even right? you're guessing

✅ Let the failure tell you, then lock it in

# Step 1: assert something you know is wrong, and run it:
test "characterize totalFor a standard order":
    assert calc.totalFor(standardOrder()) == -1     # FAILS: "expected -1 but was 117.5"

# Step 2: change the assertion to the value the code actually produced:
test "characterize totalFor a standard order":
    assert calc.totalFor(standardOrder()) == 117.5  # documents ACTUAL behavior

This pins the behavior (even if 117.5 is itself a bug). Now you can refactor and any change to that number turns the test red. After you fix the logic, replace this with a behavior test asserting the correct price.


2. Break a dependency at a seam (Parameterize Constructor)

InvoiceSender can't be put in a test harness because its constructor opens a real SMTP connection.

❌ Hard-wired dependency — untestable without sending real email

class InvoiceSender:
    constructor():
        this.mailer = new SmtpMailer("smtp.prod:25")   # no seam; runs on construction
    function send(invoice): this.mailer.deliver(render(invoice))

✅ Introduce an object seam by parameterizing the constructor

class InvoiceSender:
    constructor(mailer):                # enabling point: the constructor parameter
        this.mailer = mailer
    function send(invoice): this.mailer.deliver(render(invoice))

# Production wires the real one; the test passes a Test Spy / Mock:
test "sending an invoice delivers a rendered message":
    spy = mailerSpy()
    InvoiceSender(spy).send(anInvoice(total = 100))
    assert spy.delivered.contains("Total: 100")

The smallest behavior-preserving edit that makes the class instantiable in a harness. (If you can't change the constructor, use Extract and Override Factory Method or Supersede Instance Variable instead.)


3. Add behavior under pressure with Sprout Method

A 300-line untested process() needs a new audit-logging step now; you can't get the whole method under test today.

❌ Editing into the untested monster

function process(batch):
    ... 150 lines ...
    auditLog.record(batch.id, now())     # new code buried in untested code, untested
    ... 150 lines ...

✅ Sprout a tested method and call it

# New behavior lives in its own method, developed test-first:
function auditEntryFor(batch, currentTime):     # pure, fully unit-tested
    return AuditEntry(id = batch.id, at = currentTime, items = batch.size)

function process(batch):
    ... 150 lines ...
    auditLog.record(auditEntryFor(batch, now()))   # one-line call into tested code
    ... 150 lines ...

test "an audit entry records the batch id and item count":
    e = auditEntryFor(batchOf(3, id = "B1"), currentTime = 2026-06-09T00:00:00)
    assert e.id == "B1" and e.items == 3

The new logic is tested even though process() isn't yet. Schedule getting process() under characterization tests as a follow-up.


Agile Testing (Crispin & Gregory)

This skill is strategy-level, so its "before/after" pairs are planning artifacts, not test code.

1. From "we'll test it" to a quadrant plan

Planning testing for a new checkout feature.

❌ Vague intent

"We'll write unit tests and do some manual testing before release."

No view of usability, performance, or security — gaps get discovered in production.

✅ Walk the four quadrants and decide per context

Feature: Checkout & payment

Q1 (tech-facing, support):     unit tests for pricing/tax/discount rules;
                               integration test for the payment-gateway adapter.   [auto]
Q2 (business-facing, support): acceptance examples — "gold customer gets free
                               shipping over $50"; "declined card shows an error".  [auto]
Q3 (business-facing, critique):exploratory charters on the checkout flow;
                               usability test of the card form on mobile.           [manual]
Q4 (tech-facing, critique):    load test at 10× peak; security review of payment
                               handling (PII, OWASP); failure/timeout behavior.     [tools]

Risk-weighted: heavy Q4 (it's payments) and real Q3; Q1 is the foundation.

The plan makes the team consciously decide each dimension instead of forgetting one.


2. From a vague "done" to an explicit definition

❌ "Done" means whatever the author thinks

Story moved to Done when the developer says it works.

✅ An explicit, shared Definition of Done (story level)

A story is DONE when:
  □ code complete and peer-reviewed
  □ Q1 unit + integration tests written and green
  □ Q2 acceptance examples automated and passing
  □ explored (Q3) — charter run, issues triaged
  □ relevant Q4 checks pass (perf budget / security checklist if applicable)
  □ meets the customer's examples (demoed/accepted)
  □ docs & deploy scripts updated; CI green

Now "done" means the same thing to everyone, and nobody finds untested work at release.


Lessons Learned in Software Testing (Kaner, Bach & Pettichord)

This skill is investigative, so its pairs are a testing charter and a bug report, not code.

1. From "test the feature" to an exploratory charter

❌ A vague tasking that produces aimless clicking

"Test the file import."

✅ A charter: a mission, a risk focus, a time box

Charter: Explore CSV import with malformed, huge, and adversarial files
         to discover data-loss, corruption, and crash risks.   (~90 min)
Ideas to try (heuristics, not a script):
  - empty file; header only; 1M rows; wrong delimiter; mixed encodings (UTF-8/BOM)
  - duplicate keys; embedded newlines/quotes; a cell that's a formula (=cmd)
  - interrupt the import midway; import the same file twice
Oracles: consistency with previous release; no silent row drops; no partial commit.
Debrief: bugs found, areas still risky, new charters suggested.

Disciplined and accountable, but the tester still designs each test from what the last one revealed.


2. From a bug report that gets ignored to one that gets fixed

❌ Vague, unmotivating, easy to dismiss

Title: Import is broken
Body:  I imported a file and it didn't work. Please fix.

No repro, no impact, no expected/actual — a programmer can't act on it and a manager won't prioritize it.

✅ An advocacy document: reproducible, impact-framed, worst-case found

Title: Importing a CSV with embedded newlines silently DROPS rows (data loss)

Impact: Any customer importing exported spreadsheets hits this — embedded newlines are
        common. Rows are lost with no error, so users won't notice until data is gone.
        Worked correctly in v3.1; regressed in v3.2.

Repro (minimal):
  1. import the attached orders.csv (3 rows; row 2 has a newline inside the "notes" cell)
  2. open Orders → only 2 rows appear; no warning shown
Expected: 3 rows imported, or an explicit error.
Actual:   2 rows imported; row 2 silently discarded.
Follow-up: with 1,000 such rows, ~30% are dropped — not an edge case. Logs attached.

States the benefit of fixing, pre-empts "edge case," gives exact repro, and reports the worst version found through follow-up testing.