|
| 1 | +# Testing in AGNI |
| 2 | + |
| 3 | +This document describes the testing infrastructure and how to run tests for AGNI. |
| 4 | + |
| 5 | +## Running Tests |
| 6 | + |
| 7 | +### Prerequisites |
| 8 | +Before running tests, ensure that the `RAD_DIR` environment variable is set to your SOCRATES installation: |
| 9 | + |
| 10 | +```bash |
| 11 | +export RAD_DIR=/path/to/SOCRATES |
| 12 | +``` |
| 13 | + |
| 14 | +### Run All Tests |
| 15 | +From the AGNI root directory: |
| 16 | + |
| 17 | +```bash |
| 18 | +julia --project=. test/runtests.jl |
| 19 | +``` |
| 20 | + |
| 21 | +### Run Fast Tests Only |
| 22 | +The fast test suite excludes expensive integration tests: |
| 23 | + |
| 24 | +```bash |
| 25 | +julia --project=. test/runtests.jl fast |
| 26 | +``` |
| 27 | + |
| 28 | +## Test Coverage |
| 29 | + |
| 30 | +### Generating Coverage Reports |
| 31 | + |
| 32 | +#### Run Tests with Coverage |
| 33 | +From the AGNI root directory: |
| 34 | + |
| 35 | +```bash |
| 36 | +julia --project=. --code-coverage test/runtests.jl |
| 37 | +``` |
| 38 | + |
| 39 | +#### Process Coverage Data |
| 40 | +From the AGNI root directory: |
| 41 | + |
| 42 | +```bash |
| 43 | +julia --project=. test/get_coverage.jl |
| 44 | +``` |
| 45 | + |
| 46 | +This generates: |
| 47 | +- `coverage.info` - LCOV format coverage data |
| 48 | +- `coverage.json` - Codecov JSON format |
| 49 | +- `coverage.total` - Single number with overall coverage percentage |
| 50 | +- `coverage.md` - Markdown formatted file with report on coverage |
| 51 | + |
| 52 | +The `coverage.md` file outlines: |
| 53 | +- Overall coverage percentage |
| 54 | +- Per-file coverage breakdown with color coding (🔴 <50%, 🟡 50-80%, 🟢 >80%) |
| 55 | +- Files needing attention (<50% coverage) |
| 56 | +- Quick wins (small files with 0% coverage) |
| 57 | +- Lists of uncovered line numbers |
| 58 | + |
| 59 | +### Test Design Principles |
| 60 | + |
| 61 | +1. **Fast by default**: Most tests avoid expensive operations like full atmosphere allocation |
| 62 | +2. **Unit-focused**: Each test targets specific functions or small groups of related functions |
| 63 | +3. **Self-contained**: Tests don't depend on external files when possible (except SOCRATES spectral files) |
| 64 | +4. **Comprehensive**: Tests cover: |
| 65 | + - Normal operation (happy path) |
| 66 | + - Edge cases (boundary conditions, empty inputs) |
| 67 | + - Error conditions (missing files, invalid inputs) |
| 68 | + - Type stability and return value validation |
| 69 | + |
| 70 | +### Example Test Pattern |
| 71 | + |
| 72 | +```julia |
| 73 | +using Test |
| 74 | +using AGNI |
| 75 | + |
| 76 | +@testset "module_name" begin |
| 77 | + @testset "function_name" begin |
| 78 | + # Test normal case |
| 79 | + result = AGNI.module.function(valid_input) |
| 80 | + @test result > 0 |
| 81 | + @test typeof(result) == Float64 |
| 82 | + |
| 83 | + # Test edge case |
| 84 | + result_edge = AGNI.module.function(edge_case_input) |
| 85 | + @test isfinite(result_edge) |
| 86 | + |
| 87 | + # Test error handling |
| 88 | + @test_throws ErrorException AGNI.module.function(invalid_input) |
| 89 | + end |
| 90 | +end |
| 91 | +``` |
| 92 | + |
| 93 | +## Adding New Tests |
| 94 | + |
| 95 | +When adding new tests: |
| 96 | + |
| 97 | +1. Create a new test file in `test/` following the naming convention `test_<module>.jl` |
| 98 | +2. Any `test_*.jl` file in `test/` will be picked up automatically by `test/runtests.jl`, so no manual edit to `test/runtests.jl` is needed |
| 99 | +3. Follow the existing test patterns (see above) |
| 100 | +4. Avoid expensive operations: |
| 101 | + - Don't call `allocate!`, `setup!`, or `deallocate!` unless absolutely necessary |
| 102 | + - Use minimal atmosphere configurations when needed |
| 103 | + - Keep test execution time under 5 seconds per file |
| 104 | +5. Clean up temporary files created during tests |
| 105 | +6. Test both success and failure paths |
| 106 | +7. Use descriptive test names in `@testset` blocks |
| 107 | + |
| 108 | +### Coverage Goals |
| 109 | + |
| 110 | +- **Target**: >80% line coverage for utility modules (`phys`, `consts`, `ocean`, `guillot`, `blake`) |
| 111 | +- **Acceptable**: >50% for core simulation modules (`atmosphere`, `energy`, `solver`) |
| 112 | +- **Document**: Any intentionally untested code (e.g., platform-specific branches) |
| 113 | + |
| 114 | +## Troubleshooting the tests |
| 115 | + |
| 116 | +### Tests fail with "RAD_DIR not set" |
| 117 | +Ensure `RAD_DIR` points to a valid SOCRATES installation: |
| 118 | +```bash |
| 119 | +export RAD_DIR=/path/to/SOCRATES |
| 120 | +``` |
| 121 | + |
| 122 | +### Tests fail with "Spectral file not found" |
| 123 | +The test suite expects spectral files in `res/spectral_files/`. Ensure AGNI's resource directory is complete. |
| 124 | + |
| 125 | +### Coverage is 0% or unexpectedly low |
| 126 | +- Make sure you ran tests with `--code-coverage` flag |
| 127 | +- Run `get_coverage.jl` from the AGNI root directory (not `test/`) |
| 128 | +- Check that `.cov` files were generated in `src/` and `test/` |
| 129 | + |
| 130 | +## Test Maintenance |
| 131 | + |
| 132 | +- Run tests before committing changes |
| 133 | +- Update tests when modifying function signatures or behavior |
| 134 | +- Keep test execution time reasonable (fast suite <30 seconds) |
| 135 | +- Regularly review coverage reports to identify untested code paths |
0 commit comments