Skip to content

Persistence Fixes, Observability & Cleanup#49

Merged
crazyrokr merged 4 commits into
developfrom
feature/review-fixes
Jun 17, 2026
Merged

Persistence Fixes, Observability & Cleanup#49
crazyrokr merged 4 commits into
developfrom
feature/review-fixes

Conversation

@crazyrokr

Copy link
Copy Markdown
Owner

Overview

This PR addresses critical persistence layer fixes, enhances system observability, and performs a significant cleanup of repository bloat.

Changes

🗄️ Persistence Layer Refactoring

  • Primary Key Fix: Implemented a fix for strike numeric types in the persistence layer.
  • Migration: Added V39__fix_strike_numeric_pk.sql to rectify the primary key structure.
  • KeyHolderUtils: Introduced KeyHolderUtils and corresponding unit tests to provide a robust, type-safe mechanism for ID management across repositories.

🔭 Ingestion & Computation Observability

  • Ingestion Metrics: Introduced IngestionMetrics to enable granular tracing and monitoring of the ingestion pipeline.
  • Resilience Monitoring: Enhanced the SystemicResilienceMonitor and associated resilience health probes to better monitor system stability.
  • Support: Added configuration and dedicated unit tests (IngestionMetricsTest, HttpClientConfigTest) for new observability components.

🧹 Repository Cleanup

  • Knowledge Graph: Removed the auto-generated graphify-out/ content from the repository, adhering to cleaner dependency management practices.

📝 Documentation

  • Progress Report: Updated docs/reviews/00-progress.md to reflect all completed refactoring and feature work in this cycle.

Impact

  • Persistence Reliability: Standardized ID management prevents potential primary key collisions and type errors.
  • Observability: Improved visibility into ingestion performance and systemic resilience enables faster issue diagnosis.
  • Repository Health: Substantial reduction in repository size by removing auto-generated binary/data artifacts.

Verification

  • Unit Tests: All new tests (including KeyHolderUtilsTest, IngestionMetricsTest) and refactored resilience tests pass.
  • Database: SQL migration validated for consistency.

@github-actions

Copy link
Copy Markdown

Benchmark Regression Report

Engine: same-runner A/B (base d537c9e vs head 8cf949a on a single job), gated by compare_benchmarks.py at >10% mean regression. See ADR-030.

benchmark base mean (us) head mean (us) delta status
test_fama_french_capm 191.453 195.850 +2.30% ok

Runner: base = AMD EPYC 7763 64-Core Processor; head = AMD EPYC 7763 64-Core Processor; same runner = True.
Result: NO REGRESSION (0 of 1 benchmark(s) exceeded +10.00% on mean).

- Add V39 migration to fix strike numeric primary key
- Introduce KeyHolderUtils for robust ID management
- Update affected repositories to utilize improved PK handling
- Introduce IngestionMetrics for tracing and monitoring
- Improve systemic resilience monitoring in the computation module
- Add necessary configuration and test support for new metrics
- Reflect completed refactoring and feature work in progress review
@crazyrokr
crazyrokr merged commit 78f2ddb into develop Jun 17, 2026
12 checks passed
@crazyrokr
crazyrokr deleted the feature/review-fixes branch June 17, 2026 21:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant