Skip to content

Commit e930700

Browse files
edenoclaude
andauthored
Add Safe Scientific Development System for Claude Code (#10)
* feat: add hook utility libraries for env and numerical validation - env_check.sh: Conda environment detection and validation - numerical_validation.sh: Snapshot and invariant checking utilities - Fix .gitignore to allow .claude/hooks/lib/ (was blocked by /lib/ pattern) Part of safe scientific development system. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * feat: add pre-tool-use hook for environment and commit safety - Warns when Python commands run outside conda environment - Reminds to run tests before commits - Blocks snapshot updates without approval Part of safe scientific development system. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * Fix test cache detection in pre-tool-use hook The find command on line 42 was looking for files with a "*.pytest_cache" pattern, which doesn't exist. The .pytest_cache directory contains files with various names (like .gitignore, CACHEDIR.TAG, README.md, v/cache/, etc.), not files ending in .pytest_cache. Changed from: find .pytest_cache -name "*.pytest_cache" -mmin -5 To: find .pytest_cache -type f -mmin -5 This correctly finds any files in the .pytest_cache directory that were modified within the last 5 minutes, properly detecting recent test runs. Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * feat: add user-prompt-submit hook for snapshot detection - Detects when snapshot files have changed - Reminds Claude to provide full analysis before updates Part of safe scientific development system. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * feat: add scientific-tdd skill for test-driven development Pragmatic TDD workflow for scientific code with numerical validation. Part of safe scientific development system. * feat: add numerical-validation skill for mathematical correctness Comprehensive numerical validation workflow with tolerance guidelines. Part of safe scientific development system. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * feat: add safe-refactoring skill for structure changes Zero-tolerance workflow for behavior-preserving refactoring. Part of safe scientific development system. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * docs: enhance CLAUDE.md with operational rules, numerical standards, and workflow guide Add three critical sections to CLAUDE.md for safe scientific development: **Task 7 - Critical Operational Rules:** - Mandatory skills usage (scientific-tdd, numerical-validation, safe-refactoring, jax) - Environment enforcement rules (conda activation required) - Guided autonomy boundaries (what Claude can do vs. must ask permission) - Snapshot update approval process with required 4-part analysis format **Task 8 - Numerical Accuracy Standards:** - When numerical validation is required (which files/components) - Tolerance specifications table (1e-14 for refactoring, 1e-10 for algorithms) - Mathematical invariants that must always hold (probabilities, stochastic matrices, etc.) - Validation commands (property tests, golden regression, snapshots) **Task 9 - Workflow Selection Guide:** - Decision tree for selecting appropriate workflow/skill - Task-based guidance (new features, bugs, refactoring, JAX, etc.) - JAX code requirements and best practices - JAX-specific validation checklist These enhancements provide Claude with clear operational guidelines, numerical accuracy requirements, and workflow selection criteria for scientific development. Before: 132 lines After: 357 lines Added: 225 lines Part of safe scientific development system implementation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * docs: add README files for skills and hooks directories Task 10: Create .claude/skills/README.md - Overview of all 3 skills (scientific-tdd, numerical-validation, safe-refactoring) - Usage guidance for each skill - Workflow summaries - Integration and maintenance information Task 11: Create .claude/hooks/README.md - Overview of both hooks (pre-tool-use.sh, user-prompt-submit.sh) - Utility library documentation (env_check.sh, numerical_validation.sh) - Testing procedures and expected behavior - Debugging guidance - Integration with skills explanation Both READMEs provide clear documentation of the safe scientific development system components. Part of safe scientific development system implementation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * feat: add integration test and user guide for safe development system Task 12: Integration test script (tests/test_safe_dev_system.sh) - Tests all 8 categories: hook utilities, hooks, skills, CLAUDE.md, functional tests, snapshot protection, git config, documentation - Validates complete system installation and functionality - All tests passing Task 13: Comprehensive user guide (docs/SAFE_DEVELOPMENT_GUIDE.md) - Quick start for users and Claude - Three-layer system explanation - Common workflow examples with checkpoints - Approval gate processes with checklists - Numerical tolerance guidelines - Troubleshooting guide - Customization instructions - Best practices and FAQ Both are final documentation/testing deliverables for the safe scientific development system. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * docs: add system verification report Complete verification of all components, functional testing results, and maintenance procedures. Final deliverable for safe scientific development system. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * Add XML headers to skill files - Add proper XML metadata headers to scientific-tdd skill - Add proper XML metadata headers to numerical-validation skill - Add proper XML metadata headers to safe-refactoring skill Headers follow Claude Code skill format with name, description, tags, and version. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * fix: resolve ruff linting errors - Fix import sorting (ruff I001) - Remove unused variable assignment (ruff F841) - Remove trailing whitespace (ruff W291) - Remove whitespace from blank lines (ruff W293) - Remove unused import (ruff F401) All auto-fixed with ruff check --fix and ruff format. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * fix: apply black formatting for CI compliance Black and ruff have slightly different formatting preferences. Applying black formatting to match CI requirements. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * refactor: switch from black to ruff formatting - Remove black formatting check from CI workflow - Use ruff format exclusively for code formatting - Reformat all files with ruff format Ruff provides equivalent formatting to black with additional linting capabilities, simplifying the toolchain. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * fix: actually remove black formatting check from CI Previous commit included the workflow file but the edit didn't apply correctly. This commit properly removes the black formatting check step. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> --------- Co-authored-by: Claude <[email protected]>
1 parent 2197d48 commit e930700

28 files changed

+6179
-221
lines changed

.claude/hooks/README.md

Lines changed: 195 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,195 @@
1+
# Claude Code Hooks for Safe Scientific Development
2+
3+
This directory contains hooks that provide technical enforcement of safety requirements for the `non_local_detector` project.
4+
5+
## Hook Overview
6+
7+
Hooks automatically intercept commands and verify safety requirements before execution. They provide the first layer of defense in the three-layer system (hooks + skills + documentation).
8+
9+
## Available Hooks
10+
11+
### pre-tool-use.sh
12+
**Triggers:** Before every Bash tool execution
13+
14+
**Enforces:**
15+
1. **Conda environment validation**
16+
- Warns if Python commands run outside `non_local_detector` environment
17+
- Suggests activation command
18+
- Auto-prepends activation to commands
19+
20+
2. **Test-before-commit**
21+
- Checks if tests have run recently before allowing commits
22+
- Warns if no test cache found
23+
- Recommends running test suite
24+
25+
3. **Snapshot update protection**
26+
- Blocks `--snapshot-update` without approval flag
27+
- Requires full analysis before approval
28+
- Clears approval flag after use
29+
30+
**Exit codes:**
31+
- `0`: Command allowed (may include warnings)
32+
- `1`: Command blocked (e.g., snapshot update without approval)
33+
34+
---
35+
36+
### user-prompt-submit.sh
37+
**Triggers:** After user submits a prompt
38+
39+
**Detects:**
40+
- Snapshot file changes (`.ambr` files)
41+
- Recent snapshot test runs
42+
43+
**Actions:**
44+
- Reports modified snapshot files
45+
- Reminds Claude to provide full analysis
46+
- Lists requirements for snapshot update approval
47+
48+
**Exit codes:**
49+
- Always `0` (informational only, never blocks)
50+
51+
---
52+
53+
## Utility Libraries
54+
55+
### lib/env_check.sh
56+
**Functions:**
57+
- `check_conda_env()`: Verify conda environment active
58+
- `get_activation_cmd()`: Return activation command string
59+
- `prepend_activation()`: Auto-prepend activation to command
60+
- `needs_conda()`: Check if command requires conda environment
61+
62+
### lib/numerical_validation.sh
63+
**Functions:**
64+
- `check_invariants()`: Scan test output for NaN/Inf
65+
- `has_snapshot_changes()`: Detect modified snapshot files
66+
- `has_snapshot_approval()`: Check if approval flag set
67+
- `set_snapshot_approval()`: Create approval flag file
68+
- `clear_snapshot_approval()`: Remove approval flag file
69+
70+
---
71+
72+
## Hook Environment Variables
73+
74+
Hooks receive these environment variables from Claude Code:
75+
76+
- `CLAUDE_TOOL_NAME`: Name of tool being invoked (e.g., "Bash")
77+
- `CLAUDE_TOOL_COMMAND`: Full command string being executed
78+
- `CONDA_DEFAULT_ENV`: Current conda environment name
79+
80+
---
81+
82+
## Testing Hooks
83+
84+
### Manual testing
85+
86+
```bash
87+
# Test environment check
88+
export CLAUDE_TOOL_NAME="Bash"
89+
export CLAUDE_TOOL_COMMAND="pytest --version"
90+
.claude/hooks/pre-tool-use.sh
91+
echo "Exit code: $?"
92+
93+
# Test snapshot protection (should block)
94+
export CLAUDE_TOOL_COMMAND="pytest --snapshot-update"
95+
.claude/hooks/pre-tool-use.sh
96+
echo "Exit code: $?" # Should be 1
97+
98+
# Test with approval
99+
source .claude/hooks/lib/numerical_validation.sh
100+
set_snapshot_approval
101+
.claude/hooks/pre-tool-use.sh
102+
echo "Exit code: $?" # Should be 0
103+
```
104+
105+
### Expected behavior
106+
107+
| Scenario | Hook | Expected Result |
108+
|----------|------|-----------------|
109+
| Python command, wrong env | pre-tool-use | Warning (exit 0) |
110+
| Commit without recent tests | pre-tool-use | Warning (exit 0) |
111+
| Snapshot update, no approval | pre-tool-use | Blocked (exit 1) |
112+
| Snapshot update, with approval | pre-tool-use | Allowed, flag cleared (exit 0) |
113+
| Snapshot files modified | user-prompt-submit | Info message (exit 0) |
114+
115+
---
116+
117+
## Integration with Skills
118+
119+
Hooks work together with skills:
120+
121+
1. **Skills** (layer 2) guide Claude through workflows
122+
2. **Hooks** (layer 1) enforce critical requirements
123+
3. **Documentation** (layer 3) explains the "why"
124+
125+
Example flow:
126+
```
127+
Claude uses scientific-tdd skill
128+
129+
Skill says: "Run tests"
130+
131+
Claude executes: pytest command
132+
133+
pre-tool-use hook checks conda env
134+
135+
Hook passes or warns
136+
137+
Command executes
138+
```
139+
140+
---
141+
142+
## Debugging Hooks
143+
144+
If hooks misbehave:
145+
146+
1. **Check hook permissions:**
147+
```bash
148+
ls -l .claude/hooks/*.sh
149+
# Should show -rwxr-xr-x
150+
```
151+
152+
2. **Test utilities directly:**
153+
```bash
154+
source .claude/hooks/lib/env_check.sh
155+
check_conda_env && echo "OK" || echo "FAIL"
156+
```
157+
158+
3. **Run hook with debug output:**
159+
```bash
160+
bash -x .claude/hooks/pre-tool-use.sh
161+
```
162+
163+
4. **Check environment variables:**
164+
```bash
165+
echo "Tool: $CLAUDE_TOOL_NAME"
166+
echo "Command: $CLAUDE_TOOL_COMMAND"
167+
echo "Conda: $CONDA_DEFAULT_ENV"
168+
```
169+
170+
---
171+
172+
## Maintenance
173+
174+
### When to update hooks:
175+
176+
- Conda environment name changes
177+
- New tools need environment checking
178+
- Validation requirements change
179+
- New safety checks needed
180+
181+
### Best practices:
182+
183+
- Keep hooks fast (< 100ms)
184+
- Always exit with appropriate code (0 or 1)
185+
- Provide helpful error messages
186+
- Source utilities from `lib/` directory
187+
- Test thoroughly before deployment
188+
189+
---
190+
191+
## Related Documentation
192+
193+
- **CLAUDE.md**: Project operational rules
194+
- **Skills**: Workflow enforcement
195+
- **Testing Plan**: Coverage improvement strategy

.claude/hooks/lib/env_check.sh

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
#!/bin/bash
2+
# Environment validation utilities for non_local_detector project
3+
4+
# Check if conda environment is activated
5+
check_conda_env() {
6+
local required_env="non_local_detector"
7+
8+
# Check CONDA_DEFAULT_ENV
9+
if [ "$CONDA_DEFAULT_ENV" = "$required_env" ]; then
10+
return 0
11+
fi
12+
13+
# Check python path
14+
local python_path=$(which python 2>/dev/null)
15+
if [[ "$python_path" == *"/envs/$required_env/"* ]]; then
16+
return 0
17+
fi
18+
19+
return 1
20+
}
21+
22+
# Get activation command
23+
get_activation_cmd() {
24+
echo "conda activate non_local_detector"
25+
}
26+
27+
# Auto-prepend conda activation to command
28+
prepend_activation() {
29+
local cmd="$1"
30+
echo "conda activate non_local_detector && $cmd"
31+
}
32+
33+
# Check if command needs conda environment
34+
needs_conda() {
35+
local cmd="$1"
36+
37+
# Check for Python-related commands
38+
if [[ "$cmd" =~ ^(python|pytest|pip|black|ruff|mypy) ]]; then
39+
return 0
40+
fi
41+
42+
# Check for full paths to conda binaries
43+
if [[ "$cmd" =~ /envs/[^/]+/bin/ ]]; then
44+
return 0
45+
fi
46+
47+
return 1
48+
}
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
#!/bin/bash
2+
# Numerical validation utilities for scientific code
3+
4+
# Check if mathematical invariants hold
5+
check_invariants() {
6+
local test_output="$1"
7+
8+
# Look for common numerical issues in test output
9+
if echo "$test_output" | grep -q "NaN\|Inf\|-Inf"; then
10+
echo "❌ Numerical issue detected: NaN or Inf in outputs"
11+
return 1
12+
fi
13+
14+
return 0
15+
}
16+
17+
# Detect snapshot changes
18+
has_snapshot_changes() {
19+
# Check if any snapshot files have been modified
20+
if git status --porcelain | grep -q "\.ambr$"; then
21+
return 0
22+
fi
23+
24+
# Check syrupy snapshot directory
25+
if [ -d ".pytest_cache/v/cache/snapshot" ]; then
26+
if [ -n "$(find .pytest_cache/v/cache/snapshot -mmin -5 2>/dev/null)" ]; then
27+
return 0
28+
fi
29+
fi
30+
31+
return 1
32+
}
33+
34+
# Check if approval flag is set
35+
has_snapshot_approval() {
36+
[ -f ".claude/snapshot_update_approved" ]
37+
}
38+
39+
# Set approval flag
40+
set_snapshot_approval() {
41+
mkdir -p .claude
42+
touch .claude/snapshot_update_approved
43+
echo "Snapshot update approved at $(date)" > .claude/snapshot_update_approved
44+
}
45+
46+
# Clear approval flag
47+
clear_snapshot_approval() {
48+
rm -f .claude/snapshot_update_approved
49+
}

.claude/hooks/pre-tool-use.sh

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
#!/bin/bash
2+
# Pre-tool-use hook for Claude Code
3+
# Enforces conda environment and test-before-commit requirements
4+
5+
# Source utilities
6+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
7+
source "$SCRIPT_DIR/lib/env_check.sh"
8+
source "$SCRIPT_DIR/lib/numerical_validation.sh"
9+
10+
# Get the tool name and command from arguments
11+
TOOL_NAME="${CLAUDE_TOOL_NAME:-unknown}"
12+
TOOL_COMMAND="${CLAUDE_TOOL_COMMAND:-}"
13+
14+
# Only process Bash tool invocations
15+
if [ "$TOOL_NAME" != "Bash" ]; then
16+
exit 0
17+
fi
18+
19+
# Extract base command (first word)
20+
BASE_CMD=$(echo "$TOOL_COMMAND" | awk '{print $1}')
21+
22+
# Check 1: Environment enforcement for Python commands
23+
if needs_conda "$BASE_CMD"; then
24+
if ! check_conda_env; then
25+
echo "❌ Wrong conda environment detected!"
26+
echo "Required: non_local_detector"
27+
echo "Current: ${CONDA_DEFAULT_ENV:-none}"
28+
echo ""
29+
echo "Run: $(get_activation_cmd)"
30+
echo ""
31+
echo "Or commands will be auto-prepended with activation."
32+
33+
# Don't block - just warn (auto-prepend will handle it)
34+
exit 0
35+
fi
36+
fi
37+
38+
# Check 2: Test-before-commit enforcement
39+
if [[ "$TOOL_COMMAND" =~ ^git[[:space:]]+commit ]]; then
40+
# Check if tests have run recently (within last 5 minutes)
41+
if [ -d ".pytest_cache" ]; then
42+
CACHE_AGE=$(find .pytest_cache -type f -mmin -5 2>/dev/null | wc -l)
43+
if [ "$CACHE_AGE" -eq 0 ]; then
44+
echo "⚠️ No recent test runs detected"
45+
echo "Recommendation: Run tests before committing"
46+
echo ""
47+
echo "Run: /Users/edeno/miniconda3/envs/non_local_detector/bin/pytest -v"
48+
echo ""
49+
echo "Proceeding with commit anyway (warning only)..."
50+
fi
51+
fi
52+
fi
53+
54+
# Check 3: Snapshot update protection
55+
if [[ "$TOOL_COMMAND" =~ --snapshot-update ]]; then
56+
if ! has_snapshot_approval; then
57+
echo "❌ Snapshot update attempted without approval!"
58+
echo ""
59+
echo "Snapshot updates require explicit approval."
60+
echo "Claude must provide full analysis first:"
61+
echo " 1. Diff: What changed in snapshots"
62+
echo " 2. Explanation: Why the change occurred"
63+
echo " 3. Validation: Mathematical properties still hold"
64+
echo " 4. Test case: Demonstrate correctness"
65+
echo ""
66+
echo "After approval, user must set approval flag."
67+
68+
# Block this command
69+
exit 1
70+
fi
71+
72+
# Clear approval flag after use
73+
clear_snapshot_approval
74+
fi
75+
76+
# All checks passed
77+
exit 0

0 commit comments

Comments
 (0)