Validate JSON Schemas Against Real Sample Data

[pethers]

🎯 Objective

Validate 5 JSON export schemas against 142 real sample data CSV files to ensure correctness

📋 Background

JSON export specifications define schemas for politician, party, committee, ministry, and intelligence data per json-export-specs/. These schemas need validation against actual sample data to ensure accurate data model representation.

📊 Current State (Measured Metrics)

• JSON Schemas: 5 defined (politician-schema.md, party-schema.md, committee-schema.md, ministry-schema.md, intelligence-schema.md)
• Sample Data Files: 142 CSV files in service.data.impl/sample-data/
• Schema Validation: Not performed - schemas based on documentation only
• Data Model Accuracy: Unknown - requires verification

✅ Acceptance Criteria

• Automated validation script comparing JSON schemas to sample data structure
• All 5 schemas validated against corresponding CSV files
• Mismatches between schemas and data documented
• Schemas updated to reflect actual data structure
• Continuous validation integrated into CI/CD pipeline

🤖 Recommended Agent

Agent: @hack23-performance-engineer
Rationale: Schema validation and data pipeline optimization expertise

📚 Related Documentation

• JSON Export Specs README
• Sample Data Directory
• politician-schema.md