You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Postgres integration tests run on Linux when a Postgres instance is available.
6
+
7
+
### Start Postgres and init schema
8
+
9
+
From the repo root:
10
+
11
+
```bash
12
+
make init-services
13
+
```
14
+
15
+
This starts the Postgres container (from this `docker-compose.yml`), waits for it, and applies the schema (`schema/sql/types.sql`, `tables.sql`, `functions.sql`, `partitions.sql`).
16
+
17
+
Credentials (default): user `nautilus`, password `pass`, database `nautilus`, port `5432`.
18
+
19
+
### Run Postgres tests
20
+
21
+
**Python:**
22
+
23
+
```bash
24
+
make test-postgres
25
+
```
26
+
27
+
Requires `make init-services` (or at least `make start-services` then `make init-db`) to have been run first.
-**common-test-data**: caches large test data under `tests/test_data/large`.
14
14
-**common-wheel-build**: builds and installs Python wheels across Linux, macOS, and Windows for multiple Python versions.
15
15
-**install-capnp**: installs the Cap'n Proto compiler with caching across Linux, macOS, and Windows.
@@ -18,8 +18,8 @@ CI/CD, testing, publishing, and automation within the NautilusTrader repository.
18
18
19
19
## Workflows (`.github/workflows`)
20
20
21
-
-**build.yml**: main CI pipeline - pre-commit, cargo-deny, Rust tests, Python tests, wheel builds, and artifact uploads.
22
-
-**build-v2.yml**: CI pipeline for the v2 Rust-native system.
21
+
-**build.yml**: main CI pipeline - plan, pre-commit, cargo-deny, Rust tests, Python tests, wheel builds, and artifact uploads. Uses Depot 8-core runners for Linux and Windows builds. Includes a plan step that skips builds on docs-only changes and skips Rust tests on Python-only changes.
22
+
-**build-v2.yml**: CI pipeline for the v2 Rust-native system. Uses Depot 8-core runners for Linux builds.
23
23
-**build-docs.yml**: dispatches documentation build on `master` and `nightly` pushes.
24
24
-**cli-binaries.yml**: builds and publishes CLI binaries for multiple platforms.
25
25
-**codeql-analysis.yml**: CodeQL security scans for Python and Rust on PRs and via cron.
@@ -45,24 +45,80 @@ CI/CD, testing, publishing, and automation within the NautilusTrader repository.
45
45
### Dependency security
46
46
47
47
-**cargo-deny**: Rust dependency auditing for security advisories (RUSTSEC/GHSA), license compliance, banned crates, and supply chain integrity. Configuration in `deny.toml`.
48
-
-**Dependency pinning**: Key tools (pre-commit, Python versions, Rust toolchain, cargo-nextest) are locked to fixed versions or SHAs.
48
+
-**Dependency pinning**: Key tools (pre-commit, Python versions, Rust toolchain, cargo-nextest, uv) are locked to fixed versions or SHAs. The uv version is pinned via `required-version` in `pyproject.toml` and extracted by `scripts/uv-version.sh` for CI, Docker, and local builds.
49
+
-**Dependency cooldown**: Python dependency resolution excludes packages published within the last 3 days (`exclude-newer = "3 days"` in `[tool.uv]`). This gives the community time to detect and quarantine compromised releases before they enter the lockfile.
49
50
-**Code scanning**: CodeQL is enabled for continuous security analysis of Python and Rust code on all PRs and weekly via cron.
50
51
51
52
### Build integrity
52
53
53
54
-**Build attestations**: All published artifacts include cryptographic SLSA build provenance attestations, linking each artifact to a specific commit SHA. Verify via `gh attestation verify`.
54
55
-**Immutable action pinning**: All third-party GitHub Actions are pinned to specific commit SHAs.
55
56
-**Docker image pinning**: Base images in Dockerfiles and service containers in workflows are pinned to SHA256 digests to prevent supply-chain attacks via tag mutation.
56
-
-**Caching**: Caches for sccache, pip/site-packages, pre-commit, and test data speed up workflows while preserving hermetic (reproducible) builds.
57
+
-**Caching**: Rust target directory cache (`Swatinem/rust-cache`), pip/site-packages, pre-commit, and test data caches speed up workflows while preserving hermetic (reproducible) builds. Rust cache saves are restricted to push events to prevent PR cache pollution.
58
+
-**Concurrency**: PR CI runs are cancelled when a new push arrives to the same PR. Push events to mainline branches are never cancelled.
59
+
-**Runners**: Linux and Windows builds use Depot 8-core runners (32 GB RAM, 150 GB SSD). macOS builds use GitHub free runners. Lightweight jobs (plan, cargo-deny, cargo-vet, publish) use GitHub free runners. Custom runner labels are declared in `.github/actionlint.yaml`.
57
60
58
61
### Runtime hardening
59
62
60
-
-**Hardened runners**: Most workflows employ `step-security/harden-runner` with `egress-policy: audit` to reduce attack surface and monitor outbound traffic.
63
+
-**Hardened runners**: All workflows employ `step-security/harden-runner` to reduce attack surface and
64
+
monitor outbound traffic. All workflows default `egress-policy` to `block`. Set
65
+
`STEP_SECURITY_EGRESS_POLICY=audit` only as a temporary rollback while expanding an allow list. Jobs that
66
+
declare a GitHub Environment can override the repo or org value with an environment-scoped variable. The
67
+
publish environments (`r2-develop`, `r2-nightly`, `release`) can use this override too. The
68
+
`security-audit.yml` workflow also reads its allow list from GitHub Environments so it can validate
69
+
branch changes before promoting the same settings to scheduled runs on the default branch.
70
+
-**Fork PR handling**: `build.yml` falls back to `egress-policy: audit` for fork PRs. Forks cannot
71
+
access repo or org variables, so the allow lists would be empty and block all network access. Fork PRs
72
+
run with read-only permissions and no access to secrets, so audit mode is safe.
73
+
74
+
### Security gate override
75
+
76
+
The `security-gate-nightly` job runs `cargo audit` and `osv-scanner` to catch vulnerabilities
77
+
before publishing. Occasionally, upstream events outside our control (transitive dependency
78
+
advisories, crate yanks for non-security reasons) can block the nightly pipeline with no
79
+
actionable fix on our side.
80
+
81
+
The repo-scoped variable `SECURITY_GATE_OVERRIDE` holds an ISO 8601 UTC timestamp
82
+
(e.g. `2026-03-28T02:00:00Z`). When the current time is before the timestamp, the security
83
+
gate is skipped. When the timestamp passes, the gate re-enables automatically with no manual
84
+
reset. The variable will be left unset for normal operations.
85
+
86
+
A repo admin will thoroughly assess all flagged items before setting the timestamp, and will
87
+
scope it to the minimum window needed for the blocked build to complete:
88
+
89
+
```
90
+
date -u -d '+2 hours' --iso-8601=seconds # e.g. 2 hour window
91
+
```
92
+
93
+
Modifying repo variables requires admin access. An attacker with that level of access can
94
+
already disable workflows or push directly, so the override does not widen the attack surface.
95
+
96
+
`cargo audit` catches CVEs and unsound code advisories independent of yank status. A crate
0 commit comments