Skip to content

Commit f082ebc

Browse files
jenniferjiangkellsjenniferjiangkells
authored
Interop engine (#116)
* Interop engine WIP * FHIR -> CDA wip * Organize configs and refactor * stuff happened and i think it's more modular now or something * Tidy * Refactoring to make core interop logic clearer * config tidy * Move configs to project-level * configure document type * Add Cda validation and medication entry templates * AllergyIntolerance template * allergy entry template * Clean up configs * Added config pydantic validation * Refactor ConfigManager * Unify validation behaviour and added tests * Clean up default config behaviour * Fixed some templates * Tidied, refactored, and added tests * Lil test fix * Added docs and some changes to config structure and base class interfaces no time to explain its too late for me * Added notes <-> DocumentReference * Update the new InteropEngine usage in CdaConnector and update tests * Remove legacy CcdAnnotator and update docs * Removed unfinished hl7v2 implementations and fixed tests
1 parent dd0d314 commit f082ebc

File tree

91 files changed

+9917
-2398
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

91 files changed

+9917
-2398
lines changed

‎README.md

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ Came here from NHS RPySOC 2024 ✨?
2626
## Features
2727
- [x] 🔥 Build FHIR-native pipelines or use [pre-built ones](https://dotimplement.github.io/HealthChain/reference/pipeline/pipeline/#prebuilt) for your healthcare NLP and ML tasks
2828
- [x] 🔌 Connect pipelines to any EHR system with built-in [CDA and FHIR Connectors](https://dotimplement.github.io/HealthChain/reference/pipeline/connectors/connectors/)
29+
- [x] 🔄 Convert between FHIR, CDA, and HL7v2 with the [InteropEngine](https://dotimplement.github.io/HealthChain/reference/interop/interop/)
2930
- [x] 🧪 Test your pipelines in full healthcare-context aware [sandbox](https://dotimplement.github.io/HealthChain/reference/sandbox/sandbox/) environments
3031
- [x] 🗃️ Generate [synthetic healthcare data](https://dotimplement.github.io/HealthChain/reference/utilities/data_generator/) for testing and development
3132
- [x] 🚀 Deploy sandbox servers locally with [FastAPI](https://fastapi.tiangolo.com/)
@@ -117,6 +118,24 @@ cda_data = CdaRequest(document="<CDA XML content>")
117118
output = pipeline(cda_data)
118119
```
119120

121+
## Interoperability
122+
123+
The InteropEngine is a template-based system that allows you to convert between FHIR, CDA, and HL7v2.
124+
125+
```python
126+
from healthchain.interop import create_engine, FormatType
127+
128+
engine = create_engine()
129+
130+
with open("tests/data/test_cda.xml", "r") as f:
131+
cda_data = f.read()
132+
133+
# Convert CDA to FHIR
134+
fhir_resources = engine.to_fhir(cda_data, src_format=FormatType.CDA)
135+
136+
# Convert FHIR to CDA
137+
cda_data = engine.from_fhir(fhir_resources, dest_format=FormatType.CDA)
138+
```
120139

121140
## Sandbox
122141

@@ -219,7 +238,7 @@ healthchain run mycds.py
219238
By default, the server runs at `http://127.0.0.1:8000`, and you can interact with the exposed endpoints at `/docs`.
220239

221240
## Road Map
222-
- [ ] 🔄 Transform and validate healthcare HL7v2, CDA to FHIR with template-based interop engine
241+
- [x] 🔄 Transform and validate healthcare HL7v2, CDA to FHIR with template-based interop engine
223242
- [ ] 🏥 Runtime connection health and EHR integration management - connect to FHIR APIs and legacy systems
224243
- [ ] 📊 Track configurations, data provenance, and monitor model performance with MLFlow integration
225244
- [ ] 🚀 Compliance monitoring, auditing at deployment as a sidecar service

‎configs/defaults.yaml

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# HealthChain Interoperability Engine Default Configuration
2+
# This file contains default values used throughout the engine
3+
4+
defaults:
5+
# Common defaults for all resources
6+
common:
7+
id_prefix: "hc-"
8+
timestamp: "%Y%m%d"
9+
reference_name: "#{uuid}name"
10+
subject:
11+
reference: "Patient/example"
12+
13+
# Mapping directory configuration
14+
mappings_dir: "cda_default"
15+
16+
# Resource-specific defaults
17+
resources:
18+
Condition:
19+
clinicalStatus:
20+
coding:
21+
- system: "http://terminology.hl7.org/CodeSystem/condition-clinical"
22+
code: "unknown"
23+
display: "Unknown"
24+
MedicationStatement:
25+
status: "unknown"
26+
medication:
27+
concept:
28+
coding:
29+
- system: "http://terminology.hl7.org/CodeSystem/v3-NullFlavor"
30+
code: "UNK"
31+
display: "Unknown"
32+
33+
# TODO: More control over settings
34+
# # Validation settings
35+
# validation:
36+
# strict_mode: true
37+
# warn_on_missing: true
38+
# ignore_unknown_fields: true
39+
40+
# # Parser settings
41+
# parser:
42+
# max_entries: 1000
43+
# skip_empty_sections: true
44+
45+
# # Logging settings
46+
# logging:
47+
# level: "INFO"
48+
# include_timestamps: true
49+
50+
# # Error handling
51+
# errors:
52+
# retry_count: 3
53+
# fail_on_critical: true
54+
55+
# # Performance settings
56+
# performance:
57+
# cache_templates: true
58+
# cache_mappings: true
59+
# batch_size: 100

‎configs/environments/development.yaml

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Development Environment Configuration
2+
# This file contains settings specific to the development environment
3+
# TODO: Implement
4+
5+
# Logging settings for development
6+
logging:
7+
level: "DEBUG"
8+
include_timestamps: true
9+
console_output: true
10+
file_output: false
11+
12+
# Error handling for development
13+
errors:
14+
retry_count: 1
15+
fail_on_critical: true
16+
verbose_errors: true
17+
18+
# Performance settings for development
19+
performance:
20+
cache_templates: false # Disable caching for easier template development
21+
cache_mappings: false # Disable caching for easier mapping development
22+
batch_size: 10 # Smaller batch size for easier debugging
23+
24+
# Default resource fields for development
25+
defaults:
26+
common:
27+
id_prefix: "dev-" # Development-specific ID prefix
28+
subject:
29+
reference: "Patient/Foo"
30+
31+
# Template settings for development
32+
templates:
33+
reload_on_change: true # Automatically reload templates when they change

‎configs/environments/production.yaml

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Production Environment Configuration
2+
# This file contains settings specific to the production environment
3+
# TODO: Implement
4+
5+
# Logging settings for production
6+
logging:
7+
level: "WARNING"
8+
include_timestamps: true
9+
console_output: false
10+
file_output: true
11+
file_path: "/var/log/healthchain/interop.log"
12+
rotate_logs: true
13+
max_log_size_mb: 10
14+
backup_count: 5
15+
16+
# Error handling for production
17+
errors:
18+
retry_count: 3
19+
fail_on_critical: true
20+
verbose_errors: false
21+
22+
# Performance settings for production
23+
performance:
24+
cache_templates: true # Enable caching for better performance
25+
cache_mappings: true # Enable caching for better performance
26+
batch_size: 100 # Larger batch size for better throughput
27+
28+
# Default resource fields for production
29+
defaults:
30+
common:
31+
id_prefix: "hc-" # Production ID prefix
32+
subject:
33+
reference: "Patient/example"
34+
35+
# Template settings for production
36+
templates:
37+
reload_on_change: false # Don't reload templates in production

‎configs/environments/testing.yaml

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Testing Environment Configuration
2+
# This file contains settings specific to the testing environment
3+
# TODO: Implement
4+
# Logging settings for testing
5+
logging:
6+
level: "INFO"
7+
include_timestamps: true
8+
console_output: true
9+
file_output: true
10+
file_path: "./logs/test-interop.log"
11+
12+
# Error handling for testing
13+
errors:
14+
retry_count: 2
15+
fail_on_critical: true
16+
verbose_errors: true
17+
18+
# Performance settings for testing
19+
performance:
20+
cache_templates: true # Enable caching for realistic testing
21+
cache_mappings: true # Enable caching for realistic testing
22+
batch_size: 50 # Medium batch size for testing
23+
24+
# Default resource fields for testing
25+
defaults:
26+
common:
27+
id_prefix: "test-" # Testing-specific ID prefix
28+
subject:
29+
reference: "Patient/test-example"
30+
31+
# Template settings for testing
32+
templates:
33+
reload_on_change: false # Don't reload templates in testing
34+
35+
# Validation settings for testing
36+
validation:
37+
strict_mode: true
38+
warn_on_missing: true
39+
ignore_unknown_fields: false # Stricter validation for testing

‎configs/interop/cda/document/ccd.yaml

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# CCD Document Configuration
2+
# This file contains configuration for CCD documents
3+
4+
# Document templates (required)
5+
templates:
6+
document: "fhir_cda/document"
7+
section: "fhir_cda/section"
8+
9+
# Basic document information
10+
code:
11+
code: "34133-9"
12+
code_system: "2.16.840.1.113883.6.1"
13+
code_system_name: "LOINC"
14+
display: "Summarization of Episode Note"
15+
confidentiality_code:
16+
code: "N"
17+
code_system: "2.16.840.1.113883.5.25"
18+
language_code: "en-US"
19+
realm_code: "GB"
20+
type_id:
21+
extension: "POCD_HD000040"
22+
root: "2.16.840.1.113883.1.3"
23+
template_id:
24+
root: "1.2.840.114350.1.72.1.51693"
25+
26+
# Document structure
27+
structure:
28+
# Header configuration
29+
header:
30+
include_patient: false
31+
include_author: false
32+
include_custodian: false
33+
include_legal_authenticator: false
34+
35+
# Body configuration
36+
body:
37+
structured_body: true
38+
non_xml_body: false
39+
include_sections:
40+
- "allergies"
41+
- "medications"
42+
- "problems"
43+
- "notes"
44+
45+
# Rendering configuration
46+
rendering:
47+
# XML formatting
48+
xml:
49+
pretty_print: true
50+
encoding: "UTF-8"
51+
52+
# Narrative generation
53+
narrative:
54+
include: true
55+
generate_if_missing: true

‎configs/interop/cda/sections/allergies.yaml

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Allergies Section Configuration
2+
# ========================
3+
4+
# Metadata for both extraction and rendering processes
5+
resource: "AllergyIntolerance"
6+
resource_template: "cda_fhir/allergy_intolerance"
7+
entry_template: "fhir_cda/allergy_entry"
8+
9+
# Section identifiers (used for extraction)
10+
identifiers:
11+
template_id: "2.16.840.1.113883.10.20.1.2"
12+
code: "48765-2"
13+
code_system: "2.16.840.1.113883.6.1"
14+
code_system_name: "LOINC"
15+
display: "Allergies"
16+
reaction:
17+
template_id: "1.3.6.1.4.1.19376.1.5.3.1.4.5"
18+
severity:
19+
template_id: "1.3.6.1.4.1.19376.1.5.3.1.4.1"
20+
21+
# Template configuration (used for rendering/generation)
22+
template:
23+
# Act element configuration
24+
act:
25+
template_id:
26+
- "1.3.6.1.4.1.19376.1.5.3.1.4.5.1"
27+
- "1.3.6.1.4.1.19376.1.5.3.1.4.5.3"
28+
- "2.16.840.1.113883.3.88.11.32.6"
29+
- "2.16.840.1.113883.3.88.11.83.6"
30+
status_code: "active"
31+
32+
# Allergy observation configuration
33+
allergy_obs:
34+
type_code: "SUBJ"
35+
inversion_ind: false
36+
template_id:
37+
- "1.3.6.1.4.1.19376.1.5.3.1.4.5"
38+
- "1.3.6.1.4.1.19376.1.5.3.1.4.6"
39+
- "2.16.840.1.113883.10.20.1.18"
40+
- "1.3.6.1.4.1.19376.1.5.3.1"
41+
- "2.16.840.1.113883.10.20.1.28"
42+
code: "420134006"
43+
code_system: "2.16.840.1.113883.6.96"
44+
code_system_name: "SNOMED CT"
45+
display_name: "Propensity to adverse reactions"
46+
status_code: "completed"
47+
48+
# Reaction observation configuration
49+
reaction_obs:
50+
template_id:
51+
- "2.16.840.1.113883.10.20.1.54"
52+
- "1.3.6.1.4.1.19376.1.5.3.1.4.5"
53+
code: "RXNASSESS"
54+
status_code: "completed"
55+
56+
# Severity observation configuration
57+
severity_obs:
58+
template_id:
59+
- "2.16.840.1.113883.10.20.1.55"
60+
- "1.3.6.1.4.1.19376.1.5.3.1.4.1"
61+
code: "SEV"
62+
code_system: "2.16.840.1.113883.5.4"
63+
code_system_name: "ActCode"
64+
display_name: "Severity"
65+
status_code: "completed"
66+
value:
67+
code_system: "2.16.840.1.113883.5.1063"
68+
code_system_name: "SeverityObservation"
69+
70+
# Clinical status observation configuration
71+
clinical_status_obs:
72+
template_id: "2.16.840.1.113883.10.20.1.39"
73+
code: "33999-4"
74+
code_system: "2.16.840.1.113883.6.1"
75+
code_system_name: "LOINC"
76+
display_name: "Status"
77+
status_code: "completed"
78+
79+
# Rendering configuration (not used)
80+
rendering:
81+
narrative:
82+
include: true
83+
template: "narratives/allergy_narrative"
84+
entry:
85+
include_status: true
86+
include_reaction: true
87+
include_severity: true
88+
include_dates: true

0 commit comments

Comments
 (0)