Skip to content

Commit 3debe1e

Browse files
Config file improvements (drasi-project#55)
* Remove platform examples Signed-off-by: Agent of Reality <agentx@agentofreality.com> * Fix trading example configuration schema to match API naming conventions The API uses a mix of snake_case and camelCase depending on the struct: - SourceConfig/ReactionConfig top-level fields: snake_case (auto_start, bootstrap_provider) - Nested config DTOs (QueryConfigDto, SseReactionConfigDto, etc.): camelCase (sourceId, ssePath) Changes: - YAML: auto_start, bootstrap_provider for SourceConfig fields - DrasiClient.ts: sourceId for query sources, ssePath/heartbeatIntervalMs for SSE config - DrasiClient.ts: autoStart for QueryConfigDto, auto_start for ReactionConfig Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Remove deleted platform example tests from validation suite Removes test references to examples that were deleted in commit 7ffee84: - debug-platform-building-comfort - drasi-platform Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Update trading example readme file. Signed-off-by: Agent of Reality <agentx@agentofreality.com> * Add CONTRIBUTING.md with learning exercises and contribution ideas Includes: - 9 learning exercises (beginner to advanced) for understanding Drasi - Prioritized feature contribution suggestions - Documentation improvement ideas - Getting started guide for contributors Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * remove unused plugin config models Signed-off-by: Agent of Reality <agentx@agentofreality.com> * Add wide range of config file examples Signed-off-by: Agent of Reality <agentx@agentofreality.com> * Add two-pass config validation to catch unknown and snake_case fields Implements field validation that runs before typed deserialization to catch typos and snake_case field names that would otherwise be silently ignored. This prevents configuration errors from going unnoticed. Key changes: - Add src/config/validation.rs with field validation for all config types - Integrate validation into load_config_file() as first-pass check - Add rename_all_fields="camelCase" to SourceConfig/ReactionConfig enums - Fix test files and example configs to use camelCase consistently - Add comprehensive test coverage for parsing failures Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix mock source dataType and query label mismatches in examples The mock source only supports "sensor", "counter", and "generic" data types. The examples incorrectly used "sensor_live" which falls through to generic. Also fixed query labels to match actual generated data: - dataType "sensor" generates SensorReading nodes (not Sensor) - dataType "counter" generates Counter nodes - dataType "generic" generates Generic nodes Added unit tests to verify YAML deserialization of mock source config. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix templates in example config files Signed-off-by: Agent of Reality <agentx@agentofreality.com> * Add Handlebars template syntax validation to config validation Uses Handlebars' built-in register_template_string() method to validate template syntax at config parse time, catching errors early rather than at runtime when reactions try to execute templates. Validates templates in: - Log reaction: defaultTemplate and routes templates - SSE reaction: defaultTemplate and routes templates - HTTP reaction: body and url fields in routes Catches common template errors: - Unclosed braces: {{after.Name - Empty expressions: {{}} - Unclosed blocks: {{#if condition}}... - Mismatched helpers: {{#each}}...{{/if}} Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Replace manual field validation with custom serde deserializers - Add custom Deserialize implementations for SourceConfig and ReactionConfig that extract common fields (kind, id, autoStart, queries) manually and delegate remaining fields to inner DTOs with deny_unknown_fields - Add #[serde(deny_unknown_fields)] to all source and reaction inner DTOs to reject unknown/snake_case fields at deserialization time - Inline adaptive batch fields in HttpAdaptiveReactionConfigDto and GrpcAdaptiveReactionConfigDto to avoid flatten + deny_unknown_fields conflict - Remove ~1100 lines of hardcoded field validation from validation.rs (SOURCE_COMMON_FIELDS, source_fields(), reaction_fields(), etc.) - Add 44 unit tests for custom deserializers covering valid configs, missing fields, unknown fields, snake_case rejection, and error messages - Fix trading example client to use camelCase field names (autoStart, ssePath, heartbeatIntervalMs) instead of snake_case Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fixing Getting Started integration test Signed-off-by: Agent of Reality <agentx@agentofreality.com> * refactor: unify config validation using deny_unknown_fields - Add deny_unknown_fields to DrasiServerConfig, DrasiLibInstanceConfig, QueryConfigDto, and SourceSubscriptionConfigDto - Implement custom deserializer for StateStoreConfig matching the pattern used by SourceConfig and ReactionConfig - Create RedbStateStoreConfigDto with deny_unknown_fields for strict field validation - Simplify validation.rs to only handle Handlebars template syntax validation (remove manual field arrays) - Remove SERVER_FIELDS, INSTANCE_FIELDS, QUERY_FIELDS arrays - Add comprehensive tests for StateStoreConfig validation This change makes all config types use consistent validation: - Simple structs use #[serde(deny_unknown_fields)] - Tagged enums (Source, Reaction, StateStore) use custom deserializers with inner DTOs that have deny_unknown_fields Unknown fields are now rejected during deserialization instead of requiring separate manual validation. * refactor: move BootstrapProviderConfig to drasi-server and standardize on 'kind' discriminator Changes: - Created src/api/models/bootstrap.rs with local BootstrapProviderConfig - Custom deserializer with deny_unknown_fields on all inner DTOs - Conversion implementations to drasi-lib types - Comprehensive test coverage (20 tests) - Changed discriminator field from 'type' to 'kind' for consistency with SourceConfig, ReactionConfig, and StateStoreConfig - Removed unused AdaptiveBatchConfigDto struct (dead code) - Updated factories.rs to use local BootstrapProviderConfig - Updated init/prompts.rs to use local types - Updated all example configs and tests This completes the config validation consistency work: - All tagged enums now use 'kind' as discriminator - All inner DTOs have deny_unknown_fields - All custom deserializers provide contextual error messages * fix: rename bootstrapProvider type to kind in config * docs: update config examples to use camelCase and correct field names - Change bootstrap provider discriminator from 'type' to 'kind' - Convert all snake_case field names to camelCase in YAML examples - Update README.md, CLAUDE.md, and examples/getting-started/README.md - Ensures documentation matches current config schema validation * test: cleanup obsolete tests and add CLI test coverage - Add validate_command_test.rs (19 tests) for CLI validate command - Add init_output_test.rs (18 tests) for init output validation - Rename test files to use _test.rs suffix convention - Delete obsolete manual tests (grpc/, http/, sse-console/) - Delete broken shell scripts referencing missing files - Update tests/README.md with comprehensive documentation - Fix config/server-docker.yaml to use current schema format Total test count: 486 automated tests --------- Signed-off-by: Agent of Reality <agentx@agentofreality.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 9909926 commit 3debe1e

102 files changed

Lines changed: 10122 additions & 3198 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CLAUDE.md

Lines changed: 20 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -89,70 +89,70 @@ id: "my-server" # Unique server ID (defaults to UUID if not specified)
8989
# Server settings
9090
host: "0.0.0.0"
9191
port: 8080
92-
log_level: "info"
93-
persist_config: true # Enable persistence (default)
94-
persist_index: false # Use RocksDB for persistent indexing (default: false, uses in-memory)
92+
logLevel: "info"
93+
persistConfig: true # Enable persistence (default)
94+
persistIndex: false # Use RocksDB for persistent indexing (default: false, uses in-memory)
9595

9696
# Optional state store for plugin state persistence
97-
# state_store:
97+
# stateStore:
9898
# kind: redb
9999
# path: ./data/state.redb
100100

101101
# Optional capacity defaults (cascades to queries/reactions)
102102
# Supports environment variables like other fields
103-
# default_priority_queue_capacity: 10000
104-
# default_priority_queue_capacity: "${PRIORITY_QUEUE_CAPACITY:-10000}"
105-
# default_dispatch_buffer_capacity: 1000
106-
# default_dispatch_buffer_capacity: "${DISPATCH_BUFFER_CAPACITY:-1000}"
103+
# defaultPriorityQueueCapacity: 10000
104+
# defaultPriorityQueueCapacity: "${PRIORITY_QUEUE_CAPACITY:-10000}"
105+
# defaultDispatchBufferCapacity: 1000
106+
# defaultDispatchBufferCapacity: "${DISPATCH_BUFFER_CAPACITY:-1000}"
107107

108108
# Sources (parsed into plugin instances)
109109
sources:
110110
- kind: mock
111111
id: "sensors"
112-
auto_start: true
112+
autoStart: true
113113

114114
# Queries
115115
queries:
116116
- id: "high-temp"
117117
query: "MATCH (s:Sensor) WHERE s.temperature > 75 RETURN s"
118118
queryLanguage: Cypher
119119
sources:
120-
- source_id: "sensors"
121-
auto_start: true
120+
- sourceId: "sensors"
121+
autoStart: true
122122

123123
# Reactions
124124
reactions:
125125
- kind: log
126126
id: "log-temps"
127127
queries:
128128
- "high-temp"
129-
auto_start: true
129+
autoStart: true
130130
```
131131
132132
For multiple DrasiLib instances, use the `instances` array (legacy single-instance fields continue to work and map to the first instance):
133133

134134
```yaml
135135
host: "0.0.0.0"
136136
port: 8080
137-
log_level: "info"
138-
persist_config: true
137+
logLevel: "info"
138+
persistConfig: true
139139
140140
instances:
141141
- id: "analytics"
142-
persist_index: true
143-
state_store:
142+
persistIndex: true
143+
stateStore:
144144
kind: redb
145145
path: ./data/analytics-state.redb
146146
sources:
147147
- kind: mock
148148
id: "sensors"
149-
auto_start: true
149+
autoStart: true
150150
queries:
151151
- id: "high-temp"
152152
query: "MATCH (s:Sensor) WHERE s.temperature > 75 RETURN s"
153153
queryLanguage: Cypher
154154
sources:
155-
- source_id: "sensors"
155+
- sourceId: "sensors"
156156
- id: "monitoring"
157157
sources: []
158158
queries: []
@@ -173,12 +173,12 @@ DrasiServer separates two independent concepts:
173173
**Persistence is enabled when:**
174174
- Config file is provided on startup (`--config path/to/config.yaml`)
175175
- Config file is writable
176-
- `persist_config: true` in server settings (default)
176+
- `persistConfig: true` in server settings (default)
177177

178178
**Persistence is disabled when:**
179179
- No config file provided (server starts with empty configuration)
180180
- Config file is read-only
181-
- `persist_config: false` in server settings
181+
- `persistConfig: false` in server settings
182182

183183
**Read-Only mode is enabled ONLY when:**
184184
- Config file is not writable (file permissions prevent writing)

Cargo.lock

Lines changed: 67 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@ indexmap = "2.0"
8787
thiserror = "1.0"
8888
dotenvy = "0.15"
8989
inquire = "0.7"
90+
handlebars = "6"
9091

9192
[dev-dependencies]
9293
# Testing utilities

0 commit comments

Comments
 (0)