Skip to content

Commit e8bf730

Browse files
UnbreakableMJAntigravity (Gemini 3.5 Flash)
andcommitted
feat(skills): add spacecraft-python-guidelines
Introduce a new language-guidelines skill for Python (targeting Python 3.12+) systems programming. Defines static typing rules (strict mypy configuration), boundary validation (Pydantic v2 schemas), asyncio event loop non-blocking safety, ProcessPoolExecutor CPU scaling, slots (__slots__) optimization classes, and Ruff linting rules. Includes zip and skill bundles and README registration. Co-Authored-By: Antigravity (Gemini 3.5 Flash) <noreply@google.com>
1 parent 4218c9a commit e8bf730

5 files changed

Lines changed: 352 additions & 0 deletions

File tree

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ the rules re-attached to every prompt.
5151
| [`spacecraft-missing-pkg`](spacecraft-missing-pkg/) | Handles missing-package situations in the Spacecraft Software workflow. |
5252
| [`spacecraft-nim-guidelines`](spacecraft-nim-guidelines/) | Type-safe highly-concurrent Nim guidance (targeting Nim 2.0+) — ARC/ORC deterministic memory management, move semantics (`sink`/`lent`), structured parallelism via `Malebolgia`, asynchronous networking with `Chronos`, safe FFI custom destructors (`=destroy`), and warnings-as-errors compiler configuration. |
5353
| [`spacecraft-ocamel-guidelines`](spacecraft-ocamel-guidelines/) | Type-safe highly-concurrent OCaml guidance — direct-style I/O fibers via `Eio` and Domainslib task parallelism pools, Saturn lock-free structures, Software Transactional Memory with Kcas, C FFI memory safety (`CAMLparam`/`CAMLlocal`/`CAMLreturn`), and warning-as-errors compilation flags. |
54+
| [`spacecraft-python-guidelines`](spacecraft-python-guidelines/) | Type-safe highly-concurrent Python guidance (targeting Python 3.12+) — strict static typing (`mypy`), boundary validation (`Pydantic v2`), non-blocking asynchronous event loops (`asyncio`), multiprocess CPU scaling (`ProcessPoolExecutor`), memory-optimized slots classes, and Ruff linting rules. |
5455
| [`spacecraft-rust-guidelines`](spacecraft-rust-guidelines/) | High-performance concurrent Rust guidance — concurrency model selection, lock-free synchronisation, memory layout, tooling gates, and unsafe hygiene — plus a distilled idiom layer (`references/idioms.md`, adapted from Apollo's Rust Best Practices, MIT) covering borrowing, clippy discipline, testing, dispatch, and type-state. |
5556
| [`spacecraft-standard`](spacecraft-standard/) | Authoritative compliance reference (The Steelbore Standard). |
5657
| [`spacecraft-swift-guidelines`](spacecraft-swift-guidelines/) | Type-safe highly-concurrent Swift guidance (targeting Swift 6.2+) — Swift 6.2 concurrency, explicit `@concurrent` background offloading, isolated conformances, `@MainActor` isolated ViewModels, Swift Testing `@Suite` and `@Test` parameterized checks, and ARC reference cycle safety. |

spacecraft-python-guidelines.skill

7.46 KB
Binary file not shown.

spacecraft-python-guidelines.zip

7.67 KB
Binary file not shown.
Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
---
2+
name: spacecraft-python-guidelines
3+
description: Use for writing type-safe highly-concurrent memory-safe Python code following Spacecraft Software standards. Triggers on any request involving Python, pip, poetry, pyproject.toml, ruff, mypy (strict mode), type hints, asyncio (event loops, run_in_executor), multiprocessing (ProcessPoolExecutor), GIL boundaries (Python 3.12 sub-interpreters, 3.13 free-threaded), Pydantic (v2), slots (__slots__), context managers, or pytest testing. Trigger even when implicit, e.g. "write a Python asyncio task", "configure mypy in pyproject.toml", "parallelize this math loop in Python", or "add custom Pydantic validators". Do NOT trigger for Jython or IronPython unless JVM/CLR interoperability is explicitly requested. By Mohamed Hammad and Spacecraft Software.
4+
license: GPL-3.0-or-later
5+
maintainer: Mohamed Hammad <Mohamed.Hammad@SpacecraftSoftware.org>
6+
website: https://Construct.SpacecraftSoftware.org/
7+
---
8+
9+
# Spacecraft Python Guidelines
10+
11+
**Maintainer:** Mohamed Hammad | **Contact:** [Mohamed.Hammad@SpacecraftSoftware.org](mailto:Mohamed.Hammad@SpacecraftSoftware.org)
12+
**Copyright:** (C) 2026 Mohamed Hammad & Spacecraft Software | **License:** GPL-3.0-or-later
13+
**Website:** [https://Construct.SpacecraftSoftware.org/](https://Construct.SpacecraftSoftware.org/)
14+
15+
**You are an expert Python systems engineer at Spacecraft Software specializing in high-performance, strictly typed, and concurrent systems targeting Python 3.12+ (CPython/free-threaded).** Always follow these rules when writing or reviewing Python code. Never deviate. Instructions are explicit, checklist-driven, and self-contained.
16+
17+
## Core Philosophy
18+
- **Stability first (Standard §3 Priority 1).** Python is dynamically typed at runtime, but static analysis tools (`mypy` in strict mode) must verify type safety at compile/lint time. Banish untyped function arguments and reject code that suppresses type checker warnings.
19+
- **Then Performance (Priority 2).** Minimize allocations and memory footprints in hot loops (use generators, comprehensions, and declare `__slots__` on performance-sensitive classes). Bypass Global Interpreter Lock (GIL) bottlenecks cleanly.
20+
- **Explicit Concurrency.** Ensure that asynchronous, non-blocking I/O tasks (`asyncio`) and parallel CPU tasks (`multiprocessing`) are logically isolated. Never run blocking operations inside an active async event loop thread.
21+
- **Defensive API boundaries.** Validate and decode all dynamic, external payloads (API responses, file configurations) into type-safe, validated objects using `Pydantic` (v2) schemas immediately at system boundaries.
22+
23+
## Memory Safety & Type Safety
24+
- **Strict Typing Configuration:** Enforce mypy strict checks. Configure `pyproject.toml` with:
25+
`[tool.mypy] strict = true, disallow_any_generics = true, plugins = ["pydantic.mypy"]`
26+
All functions, methods, and return statements must carry explicit type annotations.
27+
- **Safe Optionals:** Explicitly declare variables that can hold `None` using `Optional[T]` or union syntax `T | None`. Check against `None` using identity checks (`is None`) before operating.
28+
- **Boundary Validation:** Decode incoming unstructured data mapping using `Pydantic` (v2) or `msgspec` models. Avoid passing raw, unchecked dictionaries around the codebase.
29+
- **Resource Management:** Guarantee the lifecycle of files, network sessions, and locks using context managers (`with` or `async with` blocks) to prevent file descriptor leaks and lock hangs.
30+
31+
## Concurrency vs. Performance Tradeoffs
32+
- **When Concurrency Helps (Do Async / Multi-process):**
33+
- **Asynchronous Event Loop:** Handling thousands of concurrent, non-blocking network requests, socket connections, and async file operations via `asyncio`.
34+
- **CPU-bound Parallelism:** Spawning compute-intensive computations (data transformations, numeric operations) onto separate interpreter cores via `ProcessPoolExecutor` to bypass the GIL.
35+
- **Blocking Task Offloading:** Wrapping synchronous or blocking libraries (like standard filesystem operations) inside `loop.run_in_executor(None, sync_func)` to prevent blocking the async loop.
36+
- **When Concurrency Hurts (Do NOT Block / Sprawl):**
37+
- **Synchronous loop blocking:** Calling blocking functions (e.g. `time.sleep()`, synchronous database queries, or `requests.get()`) directly inside `async` functions, which freezes the event loop.
38+
- **CPU-bound Threading:** Spawning `threading.Thread` or `ThreadPoolExecutor` for CPU-heavy tasks. The GIL prevents parallel execution, causing high scheduler context-switch overhead.
39+
- **Pool Initialization Storms:** Spawning process pools dynamically inside loops. Reuse a persistent process pool executor instance.
40+
41+
## Mandatory Abstraction Choice
42+
Always choose the concurrency model corresponding to the workload:
43+
- **Compute-heavy workload:** A persistent process pool using `concurrent.futures.ProcessPoolExecutor` sized to CPU core count.
44+
- **Asynchronous I/O workload:** Non-blocking async/await event loops using `asyncio`.
45+
- **Blocking calls in async:** Run offloaded to thread executors via `loop.run_in_executor`.
46+
- **Data validation:** Pydantic (v2) models with the `pydantic.mypy` plugin enabled.
47+
- **Memory footprint reduction:** Adding `__slots__` to data holder classes.
48+
49+
## Required Techniques
50+
1. **Slots Declaration:** Declare `__slots__` inside performance-sensitive data classes (e.g., `class Packet: __slots__ = ("id", "value")`) to suppress dynamic `__dict__` generation, reducing memory consumption by 30-50%.
51+
2. **Generators for Large Sets:** Prefer generator expressions `(x for x in data)` and generator functions (`yield`) instead of list comprehensions when processing large collections (size > 1000) to keep memory usage flat.
52+
3. **Pydantic Model Parsing:** Parse inputs immediately via Pydantic `.model_validate_json()` or `.model_validate()`. Configure Pydantic options with `extra = "forbid"` and `frozen = True` (immutability).
53+
4. **Pytest Warning Gates:** Configure `pytest` to treat warnings as errors via `filterwarnings = ["error"]` inside `pyproject.toml`.
54+
5. **Ruff Format & Lint Check:** Run `ruff check` and `ruff format` to verify syntax formatting.
55+
56+
## Build, Tooling & CI (Non-Negotiable)
57+
- **Toolchain floor:** Python ≥ 3.12, Poetry or pip-tools dependency management.
58+
- **Mypy Strict:** Static analysis step executing `mypy --strict` on all CI runs.
59+
- **Formatter & Linter:** Ruff configured with ASYNC, ANN, B, E, F, I, RUF, TCH, and UP rules.
60+
- **Testing:** `pytest` test suites running with `pytest-asyncio` for asynchronous targets.
61+
62+
## Anti-Patterns (Never Do These)
63+
- Leaving function arguments or returns untyped.
64+
- Executing synchronous blocking operations (`requests`, `time.sleep`) inside async functions.
65+
- Initializing process or thread pools dynamically inside loops (always share a persistent executor).
66+
- Using `threading.Thread` or `ThreadPoolExecutor` for CPU-bound computations.
67+
- Storing large datasets in memory via list comprehensions when generators can stream them.
68+
- Disabling compiler/lint warnings via `# type: ignore` without documenting why.
69+
70+
## Pre-Commit Checklist (Verify Every Time)
71+
- [ ] Mypy static analysis passes cleanly with `--strict` enabled
72+
- [ ] No untyped parameters or variables are left in production files
73+
- [ ] Sync blocking calls inside async procedures are wrapped in `run_in_executor`
74+
- [ ] CPU-bound processing operations utilize `ProcessPoolExecutor`
75+
- [ ] Data objects use `__slots__` to prevent dynamic dictionary allocation
76+
- [ ] Boundary inputs are validated using Pydantic (v2) models
77+
- [ ] Ruff format check and lints pass cleanly with zero warnings
78+
- [ ] Unit tests (pytest) execute and pass cleanly under warnings-as-errors policy
79+
- [ ] Closeable resources (files, sessions) are handled using context managers
80+
81+
## References & Further Reading
82+
- Load `references/Spacecraft_Python_Guidelines.md` for full skeletons (ProcessPoolExecutor parallel math, asyncio session fetcher, Pydantic v2 boundaries, slots class, pytest-asyncio suite, and pyproject.toml) when deeper patterns are needed.
83+
- *Further reading* (consulted for background only): Python Concurrency Proposals, PEP 484 (Typing Specification), Ruff Linter manual, and Pydantic v2 Documentation.
84+
85+
When the user requests Python code or review, activate this skill, apply the checklist, and produce code a senior Spacecraft systems engineer would ship.
Lines changed: 266 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,266 @@
1+
# Spacecraft Python Guidelines — Full Reference
2+
3+
**Version:** 1.0
4+
**Date:** 2026-07-13
5+
**Author:** Mohamed Hammad & Spacecraft Software
6+
**Compatibility:** Claude 3.5+, Claude 4, Grok, and all advanced reasoning models
7+
8+
This document expands on the `SKILL.md` for Python systems programming. It provides complete, compile-checked configurations and skeletons for ProcessPoolExecutor parallelism, asyncio execution, Pydantic v2 validations, slots optimizations, and testing.
9+
10+
---
11+
12+
## 1. Concurrency: ProcessPoolExecutor & Asyncio run_in_executor
13+
14+
Do not block the single-threaded `asyncio` event loop. Offload CPU-heavy calculations to a process pool (bypassing the GIL) and synchronous blocking operations to thread executors.
15+
16+
```python
17+
import asyncio
18+
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
19+
import time
20+
from typing import List
21+
22+
# Helper CPU-bound math calculation
23+
def compute_factorial_sync(n: int) -> int:
24+
if n < 0:
25+
raise ValueError("Must be non-negative")
26+
res = 1
27+
for i in range(2, n + 1):
28+
res *= i
29+
return res
30+
31+
# Helper blocking I/O operation
32+
def blocking_file_read(filepath: str) -> str:
33+
# Simulates blocking OS operation
34+
time.sleep(0.1)
35+
with open(filepath, "r", encoding="utf-8") as f:
36+
return f.read()
37+
38+
class TelemetryManager:
39+
def __init__(self, max_processes: int = 4, max_threads: int = 8) -> None:
40+
self.process_pool = ProcessPoolExecutor(max_workers=max_processes)
41+
self.thread_pool = ThreadPoolExecutor(max_workers=max_threads)
42+
43+
async def shutdown(self) -> None:
44+
self.process_pool.shutdown(wait=True)
45+
self.thread_pool.shutdown(wait=True)
46+
47+
async def calculate_batch_factorials(self, numbers: List[int]) -> List[int]:
48+
"""Runs GIL-bound CPU math inside a ProcessPoolExecutor concurrently."""
49+
loop = asyncio.get_running_loop()
50+
tasks = [
51+
loop.run_in_executor(self.process_pool, compute_factorial_sync, num)
52+
for num in numbers
53+
]
54+
results: List[int] = await asyncio.gather(*tasks)
55+
return results
56+
57+
async def read_files_async(self, filepaths: List[str]) -> List[str]:
58+
"""Runs blocking system operations inside ThreadPoolExecutor to prevent event loop lag."""
59+
loop = asyncio.get_running_loop()
60+
tasks = [
61+
loop.run_in_executor(self.thread_pool, blocking_file_read, path)
62+
for path in filepaths
63+
]
64+
results: List[str] = await asyncio.gather(*tasks)
65+
return results
66+
```
67+
68+
---
69+
70+
## 2. Typing & Pydantic v2 Ingress Validation
71+
72+
Validate and parse raw dynamic data immediately at the entrance boundaries using Pydantic (v2) models. Use strict typing annotations for mypy.
73+
74+
```python
75+
from datetime import datetime
76+
import json
77+
from uuid import UUID
78+
from pydantic import BaseModel, ConfigDict, Field, ValidationError
79+
from typing import Union, Dict, Any
80+
81+
class TelemetryPacket(BaseModel):
82+
# Configure model to be immutable and forbid extra parameters
83+
model_config = ConfigDict(
84+
frozen=True,
85+
extra="forbid"
86+
)
87+
88+
packet_id: UUID = Field(alias="packetId")
89+
value: float = Field(ge=0.0) # validation: must be >= 0.0
90+
timestamp: datetime
91+
92+
# Functional result mappings
93+
class ValidationSuccess:
94+
__slots__ = ("data",)
95+
def __init__(self, data: TelemetryPacket) -> None:
96+
self.data = data
97+
98+
class ValidationFailure:
99+
__slots__ = ("error",)
100+
def __init__(self, error: str) -> None:
101+
self.error = error
102+
103+
ValidationResult = Union[ValidationSuccess, ValidationFailure]
104+
105+
def parse_incoming_payload(json_str: str) -> ValidationResult:
106+
try:
107+
# Pydantic v2 direct JSON parsing (faster than json.loads)
108+
packet = TelemetryPacket.model_validate_json(json_str)
109+
return ValidationSuccess(packet)
110+
except ValidationError as e:
111+
return ValidationFailure(str(e))
112+
```
113+
114+
---
115+
116+
## 3. Slots Attribute Memory Optimization
117+
118+
For classes that hold data and are instantiated frequently (e.g. millions of packet structures), define `__slots__` to prevent dynamic `__dict__` generation, lowering memory usage.
119+
120+
```python
121+
from uuid import UUID
122+
from typing import Tuple
123+
124+
class CompactReading:
125+
# Explicit slots reduce memory layout size and speed up variable lookup
126+
__slots__ = ("sensor_id", "measurements")
127+
128+
def __init__(self, sensor_id: UUID, measurements: Tuple[float, ...]) -> None:
129+
self.sensor_id: UUID = sensor_id
130+
self.measurements: Tuple[float, ...] = measurements
131+
132+
def average(self) -> float:
133+
if not self.measurements:
134+
return 0.0
135+
return sum(self.measurements) / len(self.measurements)
136+
```
137+
138+
---
139+
140+
## 4. Testing: pytest & pytest-asyncio
141+
142+
Test concurrent routines and validations using `pytest` and `pytest-asyncio`. Ensure warnings are treated as errors.
143+
144+
```python
145+
# tests/test_telemetry.py
146+
import pytest
147+
from uuid import uuid4
148+
from datetime import datetime, timezone
149+
from telemetry_manager import TelemetryManager
150+
from telemetry_schemas import parse_incoming_payload, ValidationSuccess, ValidationFailure
151+
152+
@pytest.fixture
153+
def manager() -> TelemetryManager:
154+
return TelemetryManager(max_processes=2, max_threads=2)
155+
156+
@pytest.mark.asyncio
157+
async def test_factorial_calculations(manager: TelemetryManager) -> None:
158+
numbers = [5, 6, 7]
159+
results = await manager.calculate_batch_factorials(numbers)
160+
assert results == [120, 720, 5040]
161+
162+
def test_pydantic_valid_parsing() -> None:
163+
uuid_str = str(uuid4())
164+
payload = f'{{"packetId": "{uuid_str}", "value": 45.2, "timestamp": "2026-07-12T20:00:00Z"}}'
165+
166+
result = parse_incoming_payload(payload)
167+
168+
assert isinstance(result, ValidationSuccess)
169+
assert result.data.value == 45.2
170+
assert result.data.timestamp.tzinfo == timezone.utc
171+
172+
def test_pydantic_invalid_parsing() -> None:
173+
# Value violates constraint (must be >= 0.0)
174+
payload = '{"packetId": "invalid-uuid", "value": -10.0, "timestamp": "2026-07-12T20:00:00Z"}'
175+
176+
result = parse_incoming_payload(payload)
177+
178+
assert isinstance(result, ValidationFailure)
179+
```
180+
181+
---
182+
183+
## 5. Tooling Configuration (`pyproject.toml`)
184+
185+
Standard configuration mapping for `Ruff` rules, `Mypy` strict settings, and `Pytest` warnings.
186+
187+
```toml
188+
[tool.poetry]
189+
name = "telemetry-service"
190+
version = "0.1.0"
191+
description = "Spacecraft Telemetry Service in Python"
192+
authors = ["Mohamed Hammad <Mohamed.Hammad@SpacecraftSoftware.org>"]
193+
license = "GPL-3.0-or-later"
194+
195+
[tool.poetry.dependencies]
196+
python = "^3.12"
197+
pydantic = "^2.5.0"
198+
pytest = "^7.4.0"
199+
pytest-asyncio = "^0.21.0"
200+
ruff = "^0.1.0"
201+
mypy = "^1.6.0"
202+
203+
[tool.mypy]
204+
strict = true
205+
disallow_any_generics = true
206+
warn_unused_ignores = true
207+
plugins = ["pydantic.mypy"]
208+
209+
[tool.pydantic-mypy]
210+
init_forbid_extra = true
211+
init_typed = true
212+
warn_required_dynamic_aliases = true
213+
214+
[tool.ruff]
215+
target-version = "py312"
216+
line-length = 100
217+
218+
[tool.ruff.lint]
219+
select = [
220+
"E", "F", "W", # Pycodestyle & Pyflakes
221+
"B", # Flake8-bugbear
222+
"I", # Isort
223+
"RUF", # Ruff-specific rules
224+
"ANN", # Type annotations
225+
"ASYNC", # Asyncio lints
226+
"TCH", # Type checking blocks
227+
"UP" # Pyupgrade
228+
]
229+
230+
[tool.ruff.format]
231+
quote-style = "double"
232+
indent-style = "space"
233+
234+
[tool.pytest.ini_options]
235+
minversion = "7.0"
236+
addopts = "-ra -q"
237+
testpaths = ["tests"]
238+
filterwarnings = [
239+
"error", # Treat all warnings as compilation errors
240+
]
241+
```
242+
243+
---
244+
245+
## 6. Common Pitfalls & Troubleshooting
246+
247+
| Pitfall | Symptom | Corrective Action |
248+
| :--- | :--- | :--- |
249+
| **Synchronous sleep in async** | Event loop blocks, other jobs halt | Replace `time.sleep` with `await asyncio.sleep`. |
250+
| **Synchronous filesystem in async** | Event loop thread freezes | Offload execution using `loop.run_in_executor(None, sync_func)`. |
251+
| **Threading for math operations** | High CPU context switches, slow speed | Replace threading with `ProcessPoolExecutor`. |
252+
| **Loose typing annotations** | Mypy fails in strict mode | Annotate variables and parameters explicitly. |
253+
| **Dynamic properties on dataclasses**| Memory growth on long processes | Declare class variables inside `__slots__` tuple. |
254+
| **Mocking Pydantic dynamic fields** | Mypy failures on class initializers | Configure Pydantic Mypy plugin with `init_typed = true`. |
255+
256+
---
257+
258+
## 7. Code Review Compliance Gate
259+
260+
Before merging Python code, verify:
261+
1. Static typing assertions pass under strict Mypy checks (no untyped parameters).
262+
2. Asynchronous loops do not execute synchronous blocking operations.
263+
3. Heavy numerical calculations run in separate processes (`ProcessPoolExecutor`).
264+
4. High frequency data structures define `__slots__` tuples.
265+
5. Ingress parsing checks are wrapped in Pydantic models configured with `frozen=True`.
266+
6. Ruff lints pass cleanly and pytest treats warnings as error configurations.

0 commit comments

Comments
 (0)