Skip to content

Commit 26a895f

Browse files
authored
docs: add CLAUDE.md guide for repository contributors (#214)
This file provides comprehensive documentation for Claude.ai/code about this repository's structure, development workflows, and integration architecture patterns.
1 parent 5d20d4d commit 26a895f

1 file changed

Lines changed: 115 additions & 0 deletions

File tree

CLAUDE.md

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Repository Overview
6+
7+
This is the Dagster Community Integrations repository - a collection of community-built and maintained integrations for the Dagster data orchestration platform. The repository provides a centralized place for custom integrations without requiring individual maintainers to manage their own build and release processes.
8+
9+
## Repository Structure
10+
11+
- `libraries/` - Contains all integration packages (e.g., dagster-anthropic, dagster-polars, etc.)
12+
- `libraries/_template/` - Template for creating new integrations
13+
- `pipes/implementations/` - Dagster Pipes implementations for Java, Rust, and TypeScript
14+
15+
## Development Commands
16+
17+
All integrations use standardized Makefiles with these commands:
18+
19+
```bash
20+
# Install dependencies
21+
make install # or: uv sync
22+
23+
# Build package
24+
make build # or: uv build
25+
26+
# Run tests
27+
make test # or: uv run pytest
28+
29+
# Format and lint code
30+
make ruff # Runs: uv run ruff check --fix && uv run ruff format
31+
32+
# Type checking
33+
make check # Runs: uv run pyright
34+
```
35+
36+
### Running Individual Tests
37+
38+
```bash
39+
# Run a specific test file
40+
uv run pytest dagster_<package>_tests/test_specific.py
41+
42+
# Run a specific test function
43+
uv run pytest dagster_<package>_tests/test_specific.py::test_function_name
44+
45+
# Run tests with verbose output
46+
uv run pytest -v
47+
```
48+
49+
## Creating New Integrations
50+
51+
1. Navigate to `libraries/` directory
52+
2. Copy the template: `cp -r _template dagster-<my-package>`
53+
3. Replace all instances of `example-integration` with your package name
54+
4. Update version in `dagster_<package>/__init__.py`
55+
5. Create GitHub Actions workflows from templates in `.github/workflows/template-*`
56+
57+
## Code Quality Standards
58+
59+
- **Python Version**: Default to Python 3.11
60+
- **Package Management**: Use `uv` exclusively (never pip, poetry, or conda)
61+
- **Formatting**: Enforced with Ruff (`ruff format`)
62+
- **Linting**: Enforced with Ruff (`ruff check --fix`)
63+
- **Type Checking**: Required with Pyright (`pyright`)
64+
- **Testing**: Required with pytest
65+
66+
## Release Process
67+
68+
```bash
69+
# Create a release tag
70+
./release.sh dagster-<package> X.X.X
71+
72+
# Tags follow pattern: <integration-name>-X.X.X
73+
```
74+
75+
## Integration Architecture
76+
77+
Each integration follows this structure:
78+
```
79+
dagster-<package>/
80+
├── Makefile # Standard development commands
81+
├── README.md # Documentation with usage examples
82+
├── pyproject.toml # Package configuration and dependencies
83+
├── uv.lock # Locked dependencies
84+
├── dagster_<package>/ # Main package code
85+
│ ├── __init__.py # Contains __version__
86+
│ └── ... # Integration modules
87+
└── dagster_<package>_tests/
88+
└── test_*.py # Test files
89+
```
90+
91+
### Key Architectural Patterns
92+
93+
1. **Dagster Resources**: Integrations typically implement Dagster resources for external services
94+
2. **IO Managers**: For data storage integrations, implement custom IO managers
95+
3. **Ops and Assets**: Provide pre-built ops and assets for common operations
96+
4. **Type Annotations**: Use type hints throughout the codebase
97+
5. **Error Handling**: Wrap external API calls with appropriate error handling
98+
99+
## Testing Local Changes
100+
101+
```bash
102+
# Run GitHub Actions locally with act
103+
act -W .github/workflows/quality-check-dagster-<package>.yml
104+
105+
# For testcontainers support
106+
act --env TESTCONTAINERS_HOST_OVERRIDE=`ipconfig getifaddr en0` -W .github/workflows/quality-check-dagster-<package>.yml
107+
```
108+
109+
## Important Notes
110+
111+
- All tests, formatting, and type checking must pass before merging
112+
- Follow existing patterns in similar integrations
113+
- Include comprehensive documentation with usage examples
114+
- Version is managed in `__init__.py` as `__version__ = "X.X.X"`
115+
- Dependencies are specified in `pyproject.toml` and locked with `uv.lock`

0 commit comments

Comments
 (0)