Expand documentation with comprehensive guides addressing #134 by chrisguidry · Pull Request #145 · chrisguidry/docket

chrisguidry · 2025-06-25T00:04:54Z

Summary

Closes #134 by adding detailed documentation that covers all the advanced features demonstrated in the test suite.

Added four new comprehensive guides:

Dependencies Guide - dependency injection, retry patterns, timeouts, custom dependencies
Testing with Docket - pytest fixtures, testing utilities, integration testing
Advanced Task Patterns - perpetual tasks, striking/restoring, logging, task chains
Docket in Production - Redis architecture, monitoring, deployment strategies

Enhanced Getting Started with better explanations of task keys and idempotency. Updated navigation structure for better learning flow. Added FastAPI/Typer context to README.

Fixed factual errors in dependency examples and clarified task cancellation behavior. Reformatted code blocks and softened overly confident language.

🤖 Generated with Claude Code

## Summary - Closes #132 by adding PEP 740 attestations to PyPI releases - Switches from `uv publish` to `pypa/gh-action-pypi-publish` for built-in attestation support - Adds PyPI environment configuration for proper trusted publishing setup ## Security Benefits - Automatic generation of cryptographically signed attestations for all releases - Provides verifiable link between published packages and source repository - Uses GitHub's OIDC token for trusted publishing and attestation signing ## Test Plan - [ ] Create a test release to verify attestations are generated correctly - [ ] Confirm attestations appear on PyPI package page - [ ] Validate workflow runs successfully with new configuration 🤖 Generated with [Claude Code](https://claude.ai/code) Co-authored-by: Claude <noreply@anthropic.com>

github-actions · 2025-06-25T00:05:17Z

📚 Documentation has been built for this PR!

You can download the documentation directly here:
https://github.com/chrisguidry/docket/actions/runs/15865124653/artifacts/3397327439

codecov-commenter · 2025-06-25T00:05:51Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (8892251) to head (7182b98).

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #145   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           28        28           
  Lines         3570      3570           
  Branches       191       191           
=========================================
  Hits          3570      3570

Flag	Coverage Δ
python-3.12	`100.00% <ø> (ø)`
python-3.13	`100.00% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

chrisguidry · 2025-06-25T00:09:36Z

docs/getting-started.md

+Ready for more? Check out:

-Docket provides _at-least-once_ delivery semantics. When a worker picks up a
-task, if it crashes or fails to acknowledge within `redelivery_timeout`, the
-task will be considered unacknowledged and redelivered to another available
-worker. This ensures tasks are not lost but may be delivered more than once. To
-achieve exactly-once processing, design your tasks to be idempotent.
+- **Dependencies Guide** - Access current docket, advanced retry patterns, timeouts, and custom dependencies
+- **Testing with Docket** - Ergonomic testing utilities for unit and integration tests
+- **Advanced Task Patterns** - Perpetual tasks, striking/restoring, logging, and task chains
+- **Docket in Production** - Redis architecture, monitoring, and deployment best practices
+- **[API Reference](api-reference.md)** - Complete documentation of all classes and methods


These should be links, right?

chrisguidry · 2025-06-25T00:13:12Z

docs/testing.md

+
+## Controlling Perpetual Tasks
+
+Use `run_at_most()` to limit how many times specific tasks run, which is essential for testing perpetual tasks:


Can we make all function references like this links to the appropriate API reference?

chrisguidry · 2025-06-25T00:13:53Z

docs/testing.md

+
+You can use task keys in `run_at_most()` to control specific task instances rather than all tasks of a given type.
+
+## Testing with Built-in Utility Tasks


Cut this section, it's kinda nonsense

chrisguidry · 2025-06-25T00:14:27Z

docs/testing.md

+- Debugging task execution order
+- Creating synthetic failures for error handling tests
+
+## Testing Retry Logic


This section is also nonsense, cut it

chrisguidry · 2025-06-25T00:16:56Z

docs/testing.md

+        assert service.attempts == 3
+```
+
+## Testing Dependencies


Unclear if this section makes sense. LEt's rewrite it with the goal of explaining simple patterns for testing custom dependencies, which won't themselves be mocks like the weird example shows here.

chrisguidry · 2025-06-25T00:19:29Z

docs/testing.md

@@ -0,0 +1,454 @@
+# Testing with Docket


One thing I think we're missing here is the simple fact that you can often test your tasks without running a worker! they're simple python functions and you can pass your own test values for all the dependency parameters as kwargs! We should show more examples of doing that before we jump into complex testing examples where we're setting up dockets and workers. There is absolutely a place for using dockets in tests to verify future scheduling (great example below) and running workers, so let's keep these in as well.

chrisguidry · 2025-06-25T00:20:15Z

docs/testing.md

+@pytest.fixture
+async def test_docket() -> AsyncGenerator[Docket, None]:
+    """Create a test docket with a unique name for each test."""
+    async with Docket(
+        name=f"test-{uuid4()}",
+        url="redis://localhost:6379/0"
+    ) as docket:
+        yield docket
+
+@pytest.fixture
+async def test_worker(test_docket: Docket) -> AsyncGenerator[Worker, None]:
+    """Create a test worker with fast polling for quick tests."""
+    async with Worker(
+        test_docket,
+        minimum_check_interval=timedelta(milliseconds=5),
+        scheduling_resolution=timedelta(milliseconds=5)
+    ) as worker:
+        yield worker


We set up these great fixtures, but then many of the examples below set up their own dockets and workers, let's just have as many examples as possible use these fixtures

chrisguidry · 2025-06-25T00:20:42Z

docs/testing.md

+# Mock task fixture for testing
+@pytest.fixture
+def mock_task() -> AsyncMock:
+    task = AsyncMock()
+    task.__name__ = "mock_task"
+    return task
+
+@pytest.fixture
+def now() -> Callable[[], datetime]:
+    """Consistent time source for tests."""
+    from functools import partial
+    from datetime import timezone
+    return partial(datetime.now, timezone.utc)


Let's drop these

chrisguidry · 2025-06-25T00:21:08Z

docs/testing.md

+async def test_task_execution(
+    test_docket: Docket, test_worker: Worker, mock_task: AsyncMock
+):


We shouldn't be using mock_tasks in these examples, the user would use their own real tasks!

chrisguidry · 2025-06-25T00:21:28Z

docs/testing.md

+Test that tasks are properly registered and can be called by name:
+
+```python
+async def test_task_registration_by_name(test_docket, test_worker, mock_task):


Missing types

chrisguidry · 2025-06-25T00:29:30Z

docs/production.md

+## Redis Streams Architecture
+
+Docket uses Redis streams and sorted sets to provide reliable task delivery with at-least-once semantics.
+
+### Data Storage Model
+
+Docket creates several Redis data structures for each docket:
+
+- **Stream (`{docket}:stream`)**: Ready-to-execute tasks using Redis consumer groups
+- **Sorted Set (`{docket}:queue`)**: Future tasks ordered by scheduled execution time
+- **Hashes (`{docket}:{key}`)**: Serialized task data for scheduled tasks
+- **Set (`{docket}:workers`)**: Active worker heartbeats with timestamps
+- **Set (`{docket}:worker-tasks:{worker}`)**: Tasks each worker can execute
+- **Stream (`{docket}:strikes`)**: Strike/restore commands for operational control


Let's move these down lower, they are implementation details

chrisguidry · 2025-06-25T00:30:36Z

docs/production.md

+**Task Counters:**
+- `docket_tasks_added` - Tasks scheduled
+- `docket_tasks_started` - Tasks begun execution
+- `docket_tasks_succeeded` - Successfully completed tasks
+- `docket_tasks_failed` - Failed tasks
+- `docket_tasks_retried` - Retry attempts
+- `docket_tasks_stricken` - Tasks blocked by strikes
+
+**Task Timing:**
+- `docket_task_duration` - Histogram of task execution times
+- `docket_task_punctuality` - How close tasks run to their scheduled time
+
+**System Health:**
+- `docket_queue_depth` - Tasks ready for immediate execution
+- `docket_schedule_depth` - Tasks scheduled for future execution
+- `docket_tasks_running` - Currently executing tasks
+- `docket_redis_disruptions` - Redis connection failures
+- `docket_strikes_in_effect` - Active strike rules
+


these got formatted oddly in the resulting page:

chrisguidry · 2025-06-25T00:32:01Z

docs/production.md

+Configure your OpenTelemetry exporter to send traces to your observability platform:
+
+```python
+from opentelemetry import trace
+from opentelemetry.exporter.jaeger.thrift import JaegerExporter
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import BatchSpanProcessor
+
+# Configure tracing before creating workers
+trace.set_tracer_provider(TracerProvider())
+jaeger_exporter = JaegerExporter(
+    agent_host_name="jaeger",
+    agent_port=6831,
+)
+span_processor = BatchSpanProcessor(jaeger_exporter)
+trace.get_tracer_provider().add_span_processor(span_processor)
+```


Rather than showing them in our docs, let's just link them out to https://opentelemetry.io/docs/languages/python/ and such

chrisguidry · 2025-06-25T00:33:02Z

docs/production.md

+### Deployment Strategies
+
+**Blue-green deployments:**
+```bash
+# Deploy new workers with different name
+docket worker --name orders-worker-v2 --tasks myapp.tasks:v2_tasks
+
+# Gradually strike old task versions
+docket strike old_task_function
+
+# Scale down old workers after tasks drain
+```
+
+**Rolling updates:**
+```bash
+# Update worker configuration gradually
+# Workers automatically reconnect and pick up new tasks
+```


I think we can skip this stuff

- Move Redis data structures section lower as implementation details - Fix metrics formatting from bold to proper headings - Replace OpenTelemetry code example with documentation link - Remove rolling updates section

- Fix testing.md to use fixtures consistently and add missing type annotations - Replace all instances of 'best practices' with better terminology - Simplify Redis connection pool example by removing socket options - Correct Redis requirements - remove cluster/sentinel support claims - Remove confusing Testing Dependencies section

chrisguidry commented Jun 25, 2025

View reviewed changes

chrisguidry added 5 commits June 24, 2025 21:07

Address PR review comments on production.md

a24a7fc

- Move Redis data structures section lower as implementation details - Fix metrics formatting from bold to proper headings - Replace OpenTelemetry code example with documentation link - Remove rolling updates section

Merge branch 'main' into docs-boost

5ed5a06

Update advanced-patterns.md

d2c250c

fixes

7182b98

chrisguidry merged commit 977b99e into main Jun 25, 2025
16 checks passed

chrisguidry deleted the docs-boost branch June 25, 2025 01:27


		## Controlling Perpetual Tasks

		Use `run_at_most()` to limit how many times specific tasks run, which is essential for testing perpetual tasks:


		You can use task keys in `run_at_most()` to control specific task instances rather than all tasks of a given type.

		## Testing with Built-in Utility Tasks

Conversation

chrisguidry commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

github-actions bot commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

chrisguidry commented Jun 25, 2025 •

edited

Loading

github-actions bot commented Jun 25, 2025 •

edited

Loading

codecov-commenter commented Jun 25, 2025 •

edited

Loading