Skip to content

test-plan-webhook.md

Latest commit

 

History

History
75 lines (58 loc) · 8.16 KB

test-plan-webhook.md

File metadata and controls

75 lines (58 loc) · 8.16 KB
Raw

Test Plan — Webhook Endpoint

Coverage Summary

8 unit tests · 4 integration tests · 5 E2E tests · 18 UX visual tests = 35 total covering the Webhook endpoint area (POST /webhooks/shopify/product-update, Pydantic validation, structlog output, error handling, recovery flows).

Webhook Endpoint — Core POST /webhooks/shopify/product-update

Validates incoming Shopify product update payloads, logs them via structlog, and returns 200 OK or 422 on validation failure.

E2E (full request-response lifecycle via TestClient, multi-action chains)

  • WH-E2E-001 (e2e) — Smoke test: send a valid product update, verify 200 response, then send an invalid request, verify 422, then send another valid request to confirm recovery.
  • WH-E2E-002 (e2e) — State traversal: idle → valid webhook received → validation passes → logging fires → 200 returned → connection closed. Verifies the full happy-path state machine with log output capture.
  • WH-E2E-003 (e2e) — Error path: send payload missing required id, verify 422 with error detail mentioning id, verify structlog does NOT emit a product_update log line for the failed request.
  • WH-E2E-004 (e2e) — Recovery: send an invalid payload (missing title), get 422, then immediately send a valid payload, get 200 with correct log output. Confirms service recovers cleanly from errors.
  • WH-E2E-005 (e2e) — Large payload: send a webhook with 1000 variant objects in the extra fields, verify 200 response and that the full payload appears in the log output. Tests the extra="allow" behaviour under load.

Integration (multi-component interactions)

  • WH-INT-001 (integration) — Valid payload flows through FastAPI router → Pydantic model → structlog → response. Captures actual structlog output and verifies event, product_id, title, payload fields match the input.
  • WH-INT-002 (integration) — Invalid payload (missing id) flows through FastAPI router → Pydantic validation failure → 422 response with FastAPI's standard error detail structure. Verifies error schema matches spec §4.
  • WH-INT-003 (integration) — Minimal payload (only id + title) flows through full pipeline. Verifies optional fields default to None and the log output contains only the provided fields plus None defaults.
  • WH-INT-004 (integration) — Concurrent requests: send 10 valid payloads simultaneously (using asyncio), verify all return 200 and all 10 log lines are emitted. Tests stateless handling under concurrency.

Unit (single function/component in isolation)

  • WH-UNIT-001 (unit) — POST with valid top-level fields returns 200 and body {"status": "received"}. (AC: spec §15 AC-1)
  • WH-UNIT-002 (unit) — Valid payload causes structlog to emit a log line with event="shopify.product_update", correct product_id, title, and full payload. (AC: spec §15 AC-2)
  • WH-UNIT-003 (unit) — POST without id field returns 422; error detail mentions id. (AC: spec §15 AC-3)
  • WH-UNIT-004 (unit) — POST without title field returns 422; error detail mentions title. (AC: spec §15 AC-4)
  • WH-UNIT-005 (unit) — POST with extra nested fields (variants, images) returns 200; extra fields not rejected. (AC: spec §15 AC-5)
  • WH-UNIT-006 (unit) — POST with malformed JSON body returns 422. (AC: spec §15 AC-6)
  • WH-UNIT-007 (unit) — POST with null for id returns 422; Pydantic rejects null for required int field.
  • WH-UNIT-008 (unit) — POST with string "abc" for id returns 422; Pydantic rejects type mismatch.
  • WH-UNIT-009 (unit) — POST with only id and title (no optional fields) returns 200; optional fields default to None.
  • WH-UNIT-010 (unit) — POST with empty body returns 422.
  • WH-UNIT-011 (unit) — POST without Content-Type: application/json header but with valid JSON body still returns 200 (FastAPI parses body regardless of Content-Type).
  • WH-UNIT-012 (unit) — structlog output is valid JSON and a single line per event (no duplicate lines, no non-JSON output). (Risk: implementation-plan Risk #1 — structlog + stdlib bridge misconfiguration)

UX (visual/interaction verification per mockup state)

Mockup A — Idle State

  • WH-UX-A-001 (ux) — Mockup A: On app startup, Uvicorn is listening on port 8080 and GET /health returns 200, confirming the idle/service-running state.
  • WH-UX-A-002 (ux) — Mockup A: No active connections exist on startup — the service accepts connections but has not processed any requests yet.
  • WH-UX-A-003 (ux) — Mockup A: Routes are registered (POST /webhooks/shopify/product-update and GET /health) — verify via OpenAPI schema or FastAPI router inspection that both routes exist.

Mockup B — Webhook Received

  • WH-UX-B-001 (ux) — Mockup B: Sending a POST to /webhooks/shopify/product-update with Content-Type application/json is accepted (connection established, request received).
  • WH-UX-B-002 (ux) — Mockup B: POST without Content-Type header is still accepted and routed to the handler (FastAPI parses body regardless).
  • WH-UX-B-003 (ux) — Mockup B: POST with an unsupported HTTP method (e.g., PUT) to /webhooks/shopify/product-update returns 405 Method Not Allowed.

Mockup C — Validation State

  • WH-UX-C-001 (ux) — Mockup C: Pydantic model validates required fields id (int) and title (str) — valid types accepted, invalid types rejected with 422.
  • WH-UX-C-002 (ux) — Mockup C: HMAC verification is skipped — requests without X-Shopify-Hmac-Sha256 header are accepted (per spec §16 out-of-scope).
  • WH-UX-C-003 (ux) — Mockup C: Extra nested fields (variants, images, metafields) pass through without validation — extra="allow" is honoured.

Mockup D — Active Logging

  • WH-UX-D-001 (ux) — Mockup D: structlog emits a single JSON line per webhook with event="shopify.product_update", product_id, title, and payload fields matching the spec §5 Log Output Schema.
  • WH-UX-D-002 (ux) — Mockup D: Log output includes ISO 8601 UTC timestamp and level: "info" fields.
  • WH-UX-D-003 (ux) — Mockup D: Failed validation requests (422) do NOT produce a shopify.product_update info log line — only valid webhooks are logged at info level.

Mockup E — Success/Completion

  • WH-UX-E-001 (ux) — Mockup E: After valid webhook processing, HTTP 200 OK is returned with body exactly {"status": "received"}.
  • WH-UX-E-002 (ux) — Mockup E: Response Content-Type is application/json.
  • WH-UX-E-003 (ux) — Mockup E: After response is sent, the service is ready for the next request (no lingering state or connection issues).

Mockup G — Error State

  • WH-UX-G-001 (ux) — Mockup G: Missing required field returns 422 with FastAPI standard error detail containing loc, msg, and type fields.
  • WH-UX-G-002 (ux) — Mockup G: Malformed JSON returns 422 with error detail indicating JSON decode failure.
  • WH-UX-G-003 (ux) — Mockup G: After a 422 error, the service accepts and correctly processes a subsequent valid request (error state is not sticky).

Gaps & Open Questions

  • GAP-1 Question: Should E2E tests use a real Uvicorn server or FastAPI TestClient? Recommendation: Use TestClient for all tests — the spec §12 explicitly states "All testing is done via HTTP requests using httpx/TestClient" and marks E2E browser tests as N/A. TestClient exercises the full ASGI stack without network overhead.
  • GAP-2 Question: Mockup F (Health Check) is excluded from this area-scoped run — should we still include health-check-adjacent tests like verifying the endpoint exists on app startup? Recommendation: Include WH-UX-A-001 and WH-UX-A-003 which verify health endpoint registration as part of idle-state verification, but defer dedicated health-endpoint tests to the health-check area run.
  • GAP-3 Question: The spec's §14 edge case "Very large payload (>1 MB)" has no explicit limit — should WH-E2E-005 test a specific size? Recommendation: Test with 1000 variant objects (~50-100KB) to exercise extra="allow" without hitting Uvicorn defaults. A separate performance/load test is out of scope.