Skip to content

Commit ff022ec

Browse files
committed
Implement XTCE validation
- Ruff formatted and ruff lint fixes - Remove namespace on test_xtce_no_namespace.xml and make namespace handling on definitions.py clearer - Fix XML structure for XTCE test files - Fix CTIM XTCE for validity - Update changelog - Tighten up warnings for no namespace for XtcePacketDefinition - Add tests for XSD schema validity - Add tests for different inputs to validation function - Add test for different inputs to definition from_xtce - Add structural validation and tests - Add validate_xtce to top level module - Add simple CLI to validation tool - Add codecov.yaml - Mock out XSD retrieval from omg.org to prevent DOSing the schema server when testing - Add caching mechanism for XSD retrievals during validation - Clean up CLI handling of common error
1 parent b42e7ae commit ff022ec

26 files changed

Lines changed: 7554 additions & 180 deletions

.devcontainer/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,4 +92,4 @@ USER $USERNAME
9292

9393
# Copy CLAUDE.md and settings to Claude config directory
9494
COPY CLAUDE.md /home/$USERNAME/.claude/
95-
COPY claude-settings.json /home/$USERNAME/.claude/.claude.json
95+
COPY claude-settings.json /home/$USERNAME/.claude/settings.json

.devcontainer/claude-settings.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
{
2+
"model": "sonnet",
23
"autoUpdates": true,
34
"hasCompletedOnboarding": false,
45
"shiftEnterKeyBindingInstalled": true,

.devcontainer/devcontainer.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,13 +27,15 @@
2727
"github.copilot",
2828
"github.copilot-chat",
2929
"anthropic.claude-code",
30-
"GitHub.vscode-pull-request-github"
30+
"GitHub.vscode-pull-request-github",
31+
"github.vscode-github-actions"
3132
],
3233
"settings": {
3334
"dev.containers.copyGitConfig": true,
3435
"python.terminal.activateEnvironment": false,
3536
"python-envs.terminal.autoActivationType": "off",
3637
"python.defaultInterpreterPath": "python",
38+
"python.REPL.enableREPLSmartSend": false,
3739
"python.testing.pytestEnabled": true,
3840
"python.testing.unittestEnabled": false,
3941
"python.testing.pytestArgs": [

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,9 @@ Thumbs.db
6060
.idea
6161
.project
6262
.run
63+
.vscode
64+
core
65+
# Sometimes devcontainers create a core dump file when restarting
6366

6467
# Virtual environment #
6568
#######################

.pre-commit-config.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,3 +35,7 @@ repos:
3535
additional_dependencies:
3636
- pyyaml
3737
- tomli
38+
- repo: https://github.com/mashi/codecov-validator
39+
rev: v1.0.0
40+
hooks:
41+
- id: ccv

codecov.yml

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Codecov configuration for space_packet_parser
2+
# https://docs.codecov.com/docs/codecovyml-reference
3+
4+
coverage:
5+
status:
6+
project:
7+
default:
8+
target: 95% # Target coverage percentage
9+
threshold: 5% # Allow 5% drop without failing
10+
if_no_uploads: error # Fail if no coverage data
11+
only_pulls: false # Check coverage on all commits
12+
patch:
13+
default:
14+
target: 85% # Lower threshold for new code
15+
threshold: 5%
16+
only_pulls: true # Only check patch coverage on PRs
17+
18+
ignore:
19+
- "scripts/**/*" # Don't include utility scripts
20+
- "docs/**/*" # Don't include documentation

docs/source/changelog.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ Release notes for the `space_packet_parser` library
1414
- *BREAKING*: Removed mid-level abstraction methods `packet_generator()` and `ccsds_packet_generator()`
1515
from `XtcePacketDefinition`. Use low-level `parse_bytes()` with bytes generators directly, or high-level
1616
`space_packet_parser.xarr.create_dataset()` for xarray integration.
17+
- Add validation support for XTCE documents.
1718
- Add support for creating a packet definition from Python objects and serializing it as XML.
1819
- BUGFIX: Fix kbps calculation in packet generator for showing progress.
1920
- Add support for string and float encoded enumerated lookup parameters.

docs/source/users.md

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -376,6 +376,62 @@ This equation can be implemented in XTCE by referencing the packet length field
376376
</xtce:BinaryParameterType>
377377
```
378378

379+
## XTCE Document Validation
380+
381+
Space Packet Parser provides comprehensive validation capabilities for XTCE documents to help ensure they are correct and will work properly for parsing packets. The validation system operates in three modes: "schema", "structure", and a default mode of "all" (both schema and structure validation).
382+
383+
- **Schema Validation**: Validates the XML document against the in-document referenced XTCE XSD schema
384+
- **Structural Validation**: Validates XTCE-specific structure and reference integrity
385+
386+
Schema validation requires correct namespacing declarations at the top of your XTCE document.
387+
e.g.
388+
389+
```xml
390+
<xtce:SpaceSystem name="SpacePacketParser"
391+
xmlns:xtce="http://www.omg.org/spec/XTCE/20180204"
392+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
393+
xsi:schemaLocation="http://www.omg.org/spec/XTCE/20180204
394+
https://www.omg.org/spec/XTCE/20180204/SpaceSystem.xsd">
395+
```
396+
397+
### CLI Validation
398+
399+
```shell
400+
spp --log-level=DEBUG validate my_xtce.xml --local-schema my_xsd.xml --level all
401+
```
402+
403+
### Programmatic Validation
404+
405+
```python
406+
from space_packet_parser import validate_xtce
407+
408+
# Validate an XTCE file against the referenced schema
409+
result = validate_xtce("my_xtce.xml", level="schema")
410+
if result.errors:
411+
for error in result.errors:
412+
print(f"Error: {error}")
413+
else:
414+
print("Document is valid")
415+
416+
# Validate an XTCE document structure to check for
417+
# unused Parameters ParameterTypes and nonexistent references
418+
result = validate_xtce("my_xtce.xml", level="structure")
419+
if result.errors:
420+
for error in result.errors:
421+
print(f"Error: {error}")
422+
else:
423+
print("Document is valid")
424+
425+
# Comprehensive validation (both schema and structure)
426+
result = validate_xtce("my_xtce.xml", level="all")
427+
print(f"Validation completed in {result.validation_time_ms:.1f}ms")
428+
if result.errors:
429+
for error in result.errors:
430+
print(f"Error: {error}")
431+
else:
432+
print("Document is valid")
433+
```
434+
379435
## Troubleshooting Packet Parsing
380436
Parsing binary packets is error-prone and getting the XTCE definition correct can be a challenge at first.
381437
Most flight software teams can export XTCE from their command and telemetry database but these exports usually require

pyproject.toml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -150,4 +150,7 @@ select = [
150150
"UP" # pyupgrade syntax upgrader
151151
]
152152
per-file-ignores = { "tests/*" = ["S"] }
153-
ignore = ["E501"] # Ignore line length errors, as we trust ruff format to handle this.
153+
ignore = [
154+
"E501", # Ignore line length errors, as we trust ruff format to handle this.
155+
"F541" # Allow f strings as docstrings
156+
]

space_packet_parser/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,14 @@
66
from space_packet_parser.ccsds import ccsds_generator
77
from space_packet_parser.common import SpacePacket
88
from space_packet_parser.xtce.definitions import XtcePacketDefinition
9+
from space_packet_parser.xtce.validation import validate_xtce
910

1011
__all__ = [
1112
"ccsds_generator",
1213
"SpacePacket",
1314
"XtcePacketDefinition",
1415
"load_xtce",
16+
"validate_xtce",
1517
]
1618

1719

0 commit comments

Comments
 (0)