Skip to content

Latest commit

 

History

History
610 lines (482 loc) · 22.8 KB

File metadata and controls

610 lines (482 loc) · 22.8 KB

Contributing to djust

Thank you for your interest in contributing! We welcome contributions from everyone.

Getting Started

  1. Fork the repository
  2. Clone your fork: git clone https://github.com/YOUR_USERNAME/djust.git
  3. Create a branch: git checkout -b feature/your-feature-name
  4. Make your changes
  5. Run tests: cargo test && pytest
  6. Commit: git commit -m "Add your feature"
  7. Push: git push origin feature/your-feature-name
  8. Open a Pull Request

Development Setup

Prerequisites

  • Python 3.11+
  • Rust 1.70+
  • Django 4.2+ (see pyproject.toml for the exact pin)

Environment Setup

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Install uv (fast Python package manager)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install dependencies and build Rust extension
uv sync --extra dev

# Install pre-commit and pre-push hooks (required for contributions)
uvx pre-commit install
uvx pre-commit install --hook-type pre-push

Optional: commit wrapper that auto-restages reformats

Pre-commit hooks like ruff-format reformat staged files but do not auto-restage the rewrite — so git commit exits non-zero and you have to re-git add + re-commit by hand. scripts/git-commit-with-precommit.sh (or make commit MSG="...") wraps that loop:

# instead of: git add foo.py && git commit -m "feat: bar"
make commit MSG="feat: bar"

# or directly, forwarding any git commit args:
scripts/git-commit-with-precommit.sh -m "feat: bar" --signoff

The wrapper runs pre-commit against your staged files first; if hooks rewrote anything it re-stages and commits. The bare git commit path still works — the wrapper is opt-in. Closes #1464.

Working in a git worktree

A linked git worktree has no .venv of its own and the editable maturin develop install binds Python imports to the main checkout's python/ (via a plain djust.pth). Two helpers make worktree work behave:

  • Interpreter resolution (#1796). The pre-push hook and make targets resolve the interpreter from the main checkout via scripts/run-with-venv-python.sh, so they no longer fail with exit-127 in a worktree — no --no-verify needed for that reason alone.
  • Python-source gating (#1810). The pre-push pytest hook prepends the worktree's python/ to PYTHONPATH (it wins over djust.pth) and symlinks the matching compiled _rust.*.so from the main checkout, so a worktree git push runs the suite against the worktree's Python changes — not the main tree's. (PYTHONPATH is inserted before .pth processing, so the worktree source wins; the symlinked .so is gitignored and never shows in git status.)

Caveat — Rust changes. This shadows only Python source. If your worktree changes Rust (crates/, anything compiled into djust._rust), the symlinked .so is still the main checkout's build. Run maturin develop against the worktree before relying on the pre-push gate for Rust changes, or build + verify in the worktree manually. CI is the authoritative gate either way.

To reproduce the gated PYTHONPATH manually from a worktree:

WT="$(bash scripts/run-with-venv-python.sh --worktree-pythonpath)"
PYTHONPATH="${WT:+$WT:}." bash scripts/run-with-venv-python.sh -m pytest tests/ python/tests/ -q

--worktree-pythonpath prints the path to prepend (empty in the main checkout, so the same command is a no-op there).

Shared-config corruption: core.bare = true (#1938)

A linked worktree shares one .git/config with the main checkout (the worktree's .git is a file pointing at <main>/.git/worktrees/<name>, and core.* is read from the shared [core] section). If anything flips core.bare to true in that shared config — a build/PyO3-repoint step that runs git config core.bare true to repoint the compiled extension (the #1804 pattern), an IDE/GitKraken integration, or a stray manual command — then git status / git push break in both the worktree and the main checkout: every tracked file shows as deleted, because git now thinks the work tree has no working directory.

No djust pre-push hook, test, or script writes core.bare (every in-repo git operation is read-only or scoped to an isolated tmp dir — verified in #1938), so this is an external corruption, not a framework bug. Two mitigations:

  • Push --no-verify from worktrees. A --no-verify push does not run the pre-push hook chain and does not leak core.bare. Run the gates manually first (the gated-PYTHONPATH command above) and rely on CI as the authoritative gate. This is the established worktree-subagent pattern.

  • Detect + recover with the helper. Run the check before and after a worktree git push; --fix performs the documented recovery (core.bare false). It reads the shared config (works from any worktree) and never writes core.bare true:

    bash scripts/check-shared-git-config.sh         # exit 1 if leaked
    bash scripts/check-shared-git-config.sh --fix   # auto-recover a leak

    Manual recovery is equivalent: git config core.bare false.

Code Style

Python

  • Follow PEP 8
  • Use type hints where possible
  • Use ruff format for formatting and ruff check for linting
ruff format python/
ruff check python/
mypy python/
bandit -r python/djust/ -ll

JavaScript

  • Run eslint for security linting (runs automatically via pre-commit hook)
npm run lint

Rust

  • Follow standard Rust conventions
  • Run cargo fmt before committing
  • Use cargo clippy for linting
cargo fmt
cargo clippy -- -D warnings

Security

IMPORTANT: All contributors must follow the security guidelines in docs/SECURITY_GUIDELINES.md.

Key requirements:

  • Use safe_setattr() instead of raw setattr() with untrusted keys
  • Use sanitize_for_log() before logging user input
  • Use create_safe_error_response() for error responses
  • Never include stack traces or params in production error responses
  • Template tags: Always use format_html() or escape() — never mark_safe(f'...') with user-controlled values
  • Multi-tenant: New features touching data storage or queries must respect tenant isolation (key prefixing, queryset scoping)
  • CSRF: New endpoints must maintain Django CSRF protection — do not add @csrf_exempt without documented justification and equivalent protection
from djust.security import safe_setattr, sanitize_for_log, create_safe_error_response

See docs/SECURITY_GUIDELINES.md for complete details on template tag security, multi-tenant isolation, and PWA/offline sync hardening.

Testing

Rust Tests

cargo test
cargo test --release  # Run with optimizations

Python Tests

pytest
pytest --cov=djust  # With coverage

Integration Tests

cd examples/demo_project
python manage.py test

CI Mirror — catch coverage / xdist surprises pre-push

make test runs Python, Rust, and JS in parallel for speed but uses a different invocation than CI. Before pushing a branch, run:

make ci-mirror

This executes the EXACT pytest commands from .github/workflows/test.yml:

  • Full Python suite with pytest-xdist (-n auto, as CI runs it)
  • Security-tests with --cov-fail-under=75 coverage threshold

Catches the two classes of bugs make test can miss:

  • Coverage-threshold regressions (e.g. PR #959 shipped with 64.72% security coverage; only caught by CI)
  • xdist-ordering issues that pass under sequential runs

How many times the suite runs, and where the time goes

The full Python suite is ~27,000 tests. On the CI runner that is 3,105 recorded seconds — a mean of 114 ms; the same suite recorded on a 12-core Mac is 1,094s / 40 ms, which is the 2.8x that made the shards unbalanced below. Neither mean is slow. The cost is how often the suite runs and how unevenly that time is distributed.

  • The pre-push hook is not the gate. Since #2526 it runs only the tests the pushed range can affect (scripts/select-tests.py), and falls back to the whole suite for anything with whole-suite blast radius: any conftest.py, pyproject.toml, pytest.ini, the pre-commit config, the hook or the selector itself, the package roots, anything under crates/djust_core|djust_templates|djust_vdom/src/, a branch named flip/routing/convergence, or a selection that came out empty. It also runs the whole suite whenever the selector cannot run at all.

    That is a heuristic, not a proof: the selection is by changed test file, by name or import of a changed module, and by basename mentions (which is what catches the source-pin tests). A test that exercises a changed module without naming or importing it can be missed. CI runs every root on every PR and is the authoritative run — treat a green hook as "no obvious breakage before the round-trip", never as a substitute. DJUST_PREPUSH_FULL=1 forces the whole suite.

  • The tail is where the wall-clock is. Measured from the committed (runner-recorded) .test_durations: the slowest 50 of 27,263 tests are 59% of the recorded time, and one file — python/tests/test_differential_reachability_manifest_2345.py — is 35% on its own. Then test_refusal_collapsed_agreement_2454.py (7%), test_checks.py (5%) and test_make_doctor_2061.py (4%). The slowest single test is 205s.

    That last number is the ceiling on shard balancing: xdist cannot divide one test, so the longest test is a floor for whatever shard holds it. With the shards bin-packed on these numbers the recorded balance is exactly 1.00x while the real steps still spread 1.45x — that gap is this tail, not the split. Optimising broadly across 27,000 tests averaging 114 ms would buy little; the top of this list is the lever. Tracked in #2723.

Collected-count floor — .test_collected_floor

CI runs pytest with DJUST_COLLECTED_FLOOR=1, which arms the second check in tests/lost_items_guard.py (#2746): the run must collect at least the number in the committed .test_collected_floor, or it exits red with the delta named. This catches the run that silently loses a slice of the suite — fewer tests reported, still green — which nothing else distinguishes from a clean run. The first check (every selected item produced a report, with the missing node ids listed) is always on, serial or -n auto.

  • Adding tests never trips it. It is a floor, not an equality.
  • Removing tests does. Run make test-collected-floor in the same PR and commit the regenerated file — it is derived from a real collection by the guard itself, never hand-typed.
  • Local runs are unaffected unless you export DJUST_COLLECTED_FLOOR=1; a -k or single-file run has no floor to meet.

CI shards — .test_durations

CI runs the Python suite as four pytest-split shards (--splits 4 --group N), each a separate python-tests (pyX.Y, shard N/4) job with -n auto inside it. The shards are balanced by the per-test durations in the committed .test_durations file, not by test count. Every shard collects the full suite and deselects the other groups, so the union of the four is exactly one unsharded run (tests/test_ci_python_test_shards.py pins this).

  • A missing or stale entry never drops a test. pytest-split falls back to splitting by count for tests it has no timing for, so staleness only unbalances the shards (one runs longer than the others). You do not need to regenerate the file for every PR.

  • Regenerate it from a CI run, not from your machine (#2584). Every python-tests shard records what it measured (--store-durations --clean-durations) and uploads it as test-durations-shard-N; the four are disjoint and their union is the whole suite, timed on the runner:

    make test-durations-from-ci            # newest successful main run
    make test-durations-from-ci RUN=<id>   # a specific run
    git add .test_durations

    make test-durations still exists and still works, but it records this machine. That is what the file used to hold, and it does not transfer: on run 34173511325 the py3.12 shards took 182/235/584/204s of pytest against 280/278/328/193s recorded on a 12-core Mac — per-shard slowdown factors of 2.6x to 7.1x, so no single scale factor maps one to the other. The recorded imbalance read 1.70x while the runner's was 3.21x, which is why balancing on local numbers looked fine and was not.

  • Regenerate when the suite grows or shifts by more than ~10% (a large new test module, a big deletion, a change that makes many tests much slower/faster), or when one shard is visibly the straggler in CI. The two @pytest.mark.slow guards in tests/test_ci_python_test_shards.py fail on either condition, so you normally find out from a red test rather than from reading a chart.

  • The shards are bin-packed (--splitting-algorithm least_duration), not cut into four contiguous runs. The default, duration_based_chunks, cannot separate two adjacent heavyweight files: it dealt one shard 204 tests of which 186s was half of a single file. Measured on the same durations, contiguous chunks give 280/278/328/193s (1.70x) and bin-packing gives 270/270/270/270s (1.00x).

  • make test-shard GROUP=2 runs a single shard exactly as CI does.

  • The lint/type/doc checks (ruff, mypy, ADR/doc-snippet/lockfile checks) run on shard 1 only; they are per-checkout, not per-shard.

Benchmarks

cd benchmarks
python benchmark.py

Pull Request Guidelines

  • Keep PRs focused on a single feature or fix
  • Write clear commit messages
  • Add tests for new features
  • Update documentation as needed
  • Ensure all tests pass

Changelog fragments

Do not edit CHANGELOG.md's ## [Unreleased] section in a PR — pre-commit refuses it. Add one file per PR instead:

changelog.d/<issue-or-slug>.<section>.md   # section: added | changed | fixed | security | documentation | removed | deprecated

The body is exactly the bullet you would have written under that heading (- **…**, multi-paragraph allowed). The release cut runs make changelog-compile, which folds every fragment into [Unreleased] in canonical section order and deletes it. make changelog-preview shows the result without writing. Shape and rules: changelog.d/README.md.

Documentation

  • Code comments for complex logic
  • Docstrings for public APIs
  • Update README.md for new features
  • Add examples when appropriate

Performance

  • Profile before optimizing
  • Run benchmarks to verify improvements
  • Consider memory usage
  • Document performance characteristics

Dual Implementation Maintenance

djust uses a hybrid Python/Rust architecture where some functionality exists in both languages:

  • Python: Public API, business logic, Django integration
  • Rust: Performance-critical operations (template rendering, VDOM diffing)

When to Implement in Both Languages

You need to maintain dual implementations when:

  1. UI Components with Rust Optimization: Components that support both Python rendering (for flexibility) and Rust rendering (for performance)

    • Example: Form field components can render in Python or be optimized with Rust
    • Python provides the developer-facing API
    • Rust provides optional performance optimization
  2. Core Abstractions: Backend interfaces that support multiple implementations

    • Example: StateBackend (InMemory vs Redis)
    • Python defines the abstract interface
    • Each implementation must follow the contract
  3. Serialization: Data structures that cross the Python/Rust boundary

    • Example: LiveView state serialization
    • Both sides must agree on format (MessagePack)

Guidelines for Maintaining Consistency

When working on dual implementations:

1. Define Contracts Clearly

# Python: Define abstract interface
class StateBackend(ABC):
    @abstractmethod
    def health_check(self) -> Dict[str, Any]:
        """
        Returns:
            - status: 'healthy' or 'unhealthy'
            - latency_ms: Response time
            - error: Error message if unhealthy
        """
        pass

2. Test Both Implementations

  • Write tests for each implementation path
  • Verify consistent behavior and response format
  • Use integration tests to ensure they work together
# Test each backend implementation
def test_inmemory_health_check():
    backend = InMemoryStateBackend()
    result = backend.health_check()
    assert result["status"] == "healthy"

def test_redis_health_check():
    backend = RedisStateBackend(...)
    result = backend.health_check()
    assert result["status"] == "healthy"

3. Document Differences

  • Note any behavioral differences in docstrings
  • Document performance characteristics
  • Explain when to use each implementation
class InMemoryStateBackend(StateBackend):
    """
    In-memory state backend for development and testing.

    Fast and simple, but:
    - Does not scale horizontally
    - Data lost on server restart
    """

4. Keep APIs Synchronized

  • When adding methods to abstract base classes, implement in all subclasses
  • Maintain consistent return types and error handling
  • Use type hints to enforce contracts

5. Version Compatibility

  • When changing serialization formats, maintain backward compatibility
  • Document breaking changes clearly
  • Provide migration paths

Common Patterns

Pattern 1: Abstract Base Class with Multiple Implementations

# Python defines interface
class StateBackend(ABC):
    @abstractmethod
    def get(self, key: str) -> Optional[Tuple[RustLiveView, float]]:
        pass

# Each implementation follows contract
class InMemoryStateBackend(StateBackend):
    def get(self, key: str) -> Optional[Tuple[RustLiveView, float]]:
        return self._cache.get(key)

class RedisStateBackend(StateBackend):
    def get(self, key: str) -> Optional[Tuple[RustLiveView, float]]:
        data = self._client.get(self._make_key(key))
        if not data:
            return None
        view = RustLiveView.deserialize_msgpack(data)
        return (view, view.get_timestamp())

Pattern 2: Python API with Rust Acceleration

# Python provides public API
class LiveView:
    def render(self):
        # Use Rust for heavy lifting
        return self._rust_view.render()

# Rust handles performance-critical work
#[pyclass]
struct RustLiveView {
    template: String,
    vdom: VirtualDom,
}

Pattern 3: Client JavaScript — bundled ES modules, plus a few documented standalone assets

The BUNDLED client is built from the ES modules under python/djust/static/djust/src/ (e.g. debounce/throttle logic in src/09-event-binding.js). scripts/build-client.sh concatenates src/[0-9]*.js (in filename order) into static/djust/client.js and the minified+gzipped client.min.js(.gz) that actually ships. Do not hand-edit client.js / client.min.js; they are build artifacts.

The live_view.py-embedded copy is gone, and so are the two files that used to shadow the bundle (#2659): static/djust/decorators.js (a tested-but-never- shipped duplicate of the src/ logic) and static/djust/js/pwa.js (loaded by no template tag). What remains outside src/ is either a build OUTPUT (client.js, client.min.js, debug-panel*.js) or a deliberately standalone asset with its own loader (service-worker.js, ext/dj-chart.js, bug_capture_replay.js, client-dev.js). security.js was the one exception — documented as a global with no loader, so djustSecurity was undefined in every browser — and was deleted for that reason (#2679); the allowlist now carries no loader-less rows. tests/js/non-bundle-importers-2659.test.js fails when a test imports a static/djust/ file that is neither a src/ module nor on that documented list — so a new never-shipped-but-tested file cannot reappear silently.

Workflow:

  • Update the logic in the relevant src/<NN>-*.js module
  • Add/adjust tests in tests/js/ (every src/ feature file needs one)
  • Run JavaScript tests: npm test
  • Rebuild the bundle (make build-js / scripts/build-client.sh) if you need to exercise the runtime locally
  • Mind the client size budget (see CLAUDE.md → JavaScript)

Testing Strategy

For dual implementations:

  1. Unit Tests: Test each implementation independently
  2. Interface Tests: Verify all implementations satisfy the contract
  3. Integration Tests: Test Python and Rust working together
  4. Benchmark Tests: Compare performance when relevant

Example:

class TestStateBackendInterface:
    """Test all backends follow the same contract."""

    @pytest.fixture(params=['memory', 'redis'])
    def backend(self, request):
        if request.param == 'memory':
            return InMemoryStateBackend()
        elif request.param == 'redis':
            return RedisStateBackend(...)

    def test_health_check_returns_correct_fields(self, backend):
        """All backends must return same fields."""
        result = backend.health_check()
        assert "status" in result
        assert "backend" in result
        assert "latency_ms" in result

Checklist for Dual Implementation Changes

When modifying code with dual implementations:

  • Update Python interface/abstract base class
  • Update all implementations (InMemory, Redis, etc.)
  • Update type hints and docstrings
  • Add/update tests for each implementation
  • Verify interface tests pass for all implementations
  • Update relevant documentation
  • Check for breaking changes
  • Run benchmarks if performance-critical

Areas for Contribution

High Priority

  • Additional Django template tags (custom tags beyond built-ins)
  • More comprehensive test coverage
  • Performance optimizations
  • Documentation improvements

Medium Priority

  • Additional example applications
  • Browser compatibility testing
  • Error message improvements
  • Accessibility features

Future Features

  • Template inheritance support
  • Redis session backend

Supporting the Project

Beyond code contributions, there are many ways to support djust:

Financial Support

Non-Financial Support

  • ⭐ Star the repository on GitHub
  • 📢 Share djust on social media and with your network
  • 📝 Write blog posts or tutorials about djust
  • 🎤 Give talks about djust at conferences or meetups
  • 💬 Help answer questions in Discord and GitHub Discussions
  • 📚 Improve documentation
  • 🐛 Report bugs and suggest features

Every contribution, big or small, helps make djust better for everyone!

Questions?

Code of Conduct

Be respectful, inclusive, and professional. We're all here to build great software together.

Thank you for contributing! 🚀