Skip to content

Commit a12c256

Browse files
committed
test(core): quality audit — 18 new tests, fix stale docs, harden .gitignore
Add 18 new tests covering previously untested paths: - Config validation edge cases (9 should_panic tests for all validate() branches) - Block boundary matrix tests across 3 configurations (block/full AttnRes) - Block accumulation count verification through full forward pass - Numerical stability: large magnitude, near-zero, and zero inputs - RMSNorm zero-input safety (eps prevents NaN) - Two-phase stress tests: deep model (24 sublayers) and Full AttnRes mode - Property tests: identical-source identity, output finiteness Fix stale documentation: - README: update from v0.1.0/57 tests to v0.2.0/84 tests, remove false "no serialization" claim, add actual current limitations - AGENTS.md: fix incorrect "SwiGLU-style MLP" → "GELU activation" - ROADMAP.md: update current phase from v0.1.0 to v0.2.0 Harden .gitignore with IDE files, OS files, model checkpoint artifacts. 84 tests passing. Clippy clean. Fmt clean. https://claude.ai/code/session_017JX3N9cDig1WYJG5P1rdVy
1 parent a744d1b commit a12c256

6 files changed

Lines changed: 340 additions & 14 deletions

File tree

.gitignore

Lines changed: 19 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,22 @@ target
1717
# Contains mutation testing data
1818
**/mutants.out*/
1919

20-
# RustRover
21-
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
22-
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
23-
# and can be added to the global gitignore or merged into this file. For a more nuclear
24-
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
25-
#.idea/
20+
# IDE / Editor files
21+
.idea/
22+
.vscode/
23+
*.swp
24+
*.swo
25+
*~
26+
27+
# OS files
28+
.DS_Store
29+
Thumbs.db
30+
31+
# Model checkpoint artifacts
32+
*.mpk
33+
*.bin
34+
*.onnx
35+
*.safetensors
36+
37+
# Criterion benchmark output
38+
target/criterion/

AGENTS.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ src/
2929
├── serialization.rs # Model weight save/load (NamedMpk, binary, compact formats)
3030
├── two_phase.rs # Two-phase inference primitives (phase1_batched, online_softmax_merge)
3131
├── attention.rs # Multi-head self-attention
32-
├── feed_forward.rs # SwiGLU-style MLP
32+
├── feed_forward.rs # Two-layer MLP with GELU activation
3333
└── utils.rs # Causal mask generation helpers
3434
3535
tests/
@@ -55,7 +55,7 @@ fixtures/ # Reference outputs from PyTorch
5555

5656
```bash
5757
cargo build # Build the project
58-
cargo test --all-features # Run all 66 tests
58+
cargo test --all-features # Run all 84 tests
5959
cargo test test_name # Run specific test
6060
cargo clippy -- -D warnings # Lint (warnings = errors)
6161
cargo fmt # Format code

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -83,13 +83,13 @@ cargo bench # Benchmarks
8383

8484
## Current Status
8585

86-
**Alpha** (v0.1.0). Core algorithm implemented and tested with 57 passing tests (unit, differential, property-based, integration). Built on burn 0.20. Suitable for research and experimentation. Not yet suitable for production training at scale.
86+
**Alpha** (v0.2.0). Core algorithm implemented and tested with 84 passing tests (unit, differential, property-based, integration, doctest). Built on burn 0.20. Serialization (NamedMpk, binary, compact/half-precision) and two-phase inference integrated. Suitable for research and experimentation. Not yet suitable for production training at scale.
8787

8888
Known limitations:
89-
- No weight serialization/loading (safetensors support planned)
90-
- Two-phase inference optimization is implemented but not integrated into the main forward pass
89+
- No PyTorch checkpoint import (safetensors format)
9190
- NdArray backend only tested; GPU backends (wgpu, CUDA, Metal) untested
9291
- No distributed training support
92+
- No KV-cache for autoregressive generation
9393

9494
See [ROADMAP.md](ROADMAP.md) for planned features and progress.
9595

ROADMAP.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# attnres-rs Roadmap
22

3-
## Current Phase: Alpha (v0.1.0)
3+
## Current Phase: Alpha (v0.2.0)
44

5-
Core algorithm implemented and tested. Suitable for research and experimentation.
5+
Core algorithm, serialization, and two-phase inference implemented. 84 tests passing. Suitable for research and experimentation.
66

77
---
88

@@ -29,7 +29,7 @@ Core algorithm implemented and tested. Suitable for research and experimentation
2929
- [x] Config save/load (JSON via burn's Config trait)
3030
- [x] Integrate two-phase inference into main `forward_two_phase` method
3131
- [x] Layer accessor methods for two-phase inference components
32-
- [x] 66 tests passing (unit, differential, property-based, integration, doctest)
32+
- [x] 84 tests passing (unit, differential, property-based, integration, doctest)
3333
- [ ] Pre-trained weight loading from PyTorch checkpoints
3434
- [ ] Model export utilities
3535

tests/property_tests.rs

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,4 +102,68 @@ proptest! {
102102
let output = op.forward(&blocks, &partial);
103103
prop_assert_eq!(output.dims(), [batch, seq_len, d_model]);
104104
}
105+
106+
/// Zero-init AttnRes with identical sources should return that source.
107+
/// This is the identity property: if all inputs are the same, the output
108+
/// must equal that input regardless of num_blocks.
109+
#[test]
110+
fn identical_sources_produce_identical_output(
111+
num_blocks in 1_usize..5,
112+
batch in 1_usize..3,
113+
seq_len in 1_usize..5,
114+
) {
115+
let d_model = 16;
116+
let device = Default::default();
117+
let config = AttnResConfig::new(d_model, 12, num_blocks);
118+
let op = config.init_op::<TestBackend>(&device);
119+
120+
// All sources are the same tensor
121+
let source = Tensor::random(
122+
[batch, seq_len, d_model],
123+
Distribution::Uniform(-1.0, 1.0),
124+
&device,
125+
);
126+
let blocks: Vec<_> = (0..num_blocks).map(|_| source.clone()).collect();
127+
128+
let output = op.forward(&blocks, &source);
129+
let diff: f32 = (output - source).abs().max().into_scalar();
130+
prop_assert!(
131+
diff < 1e-3,
132+
"Identical sources should produce that source, diff={diff}"
133+
);
134+
}
135+
136+
/// AttnRes output should be finite (no NaN/Inf) for any reasonable input.
137+
#[test]
138+
fn output_is_always_finite(
139+
num_blocks in 1_usize..4,
140+
batch in 1_usize..3,
141+
seq_len in 1_usize..5,
142+
) {
143+
let d_model = 16;
144+
let device = Default::default();
145+
let config = AttnResConfig::new(d_model, 12, num_blocks);
146+
let op = config.init_op::<TestBackend>(&device);
147+
148+
let blocks: Vec<_> = (0..num_blocks)
149+
.map(|_| {
150+
Tensor::random(
151+
[batch, seq_len, d_model],
152+
Distribution::Uniform(-10.0, 10.0),
153+
&device,
154+
)
155+
})
156+
.collect();
157+
let partial = Tensor::random(
158+
[batch, seq_len, d_model],
159+
Distribution::Uniform(-10.0, 10.0),
160+
&device,
161+
);
162+
163+
let output = op.forward(&blocks, &partial);
164+
let has_nan: bool = output.clone().is_nan().any().into_scalar();
165+
let has_inf: bool = output.is_inf().any().into_scalar();
166+
prop_assert!(!has_nan, "Output contains NaN");
167+
prop_assert!(!has_inf, "Output contains Inf");
168+
}
105169
}

0 commit comments

Comments
 (0)