Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
124 commits
Select commit Hold shift + click to select a range
4ca4b0a
Improve fhirtimer
brendankowitz Feb 29, 2024
4331d99
Subscription infra
brendankowitz Apr 16, 2024
1e7eb6d
Fixes wiring up of Transaction Watchdog => Orchestrator
brendankowitz Apr 16, 2024
2c47f35
Adding code for writing to storage
brendankowitz Apr 16, 2024
85f8ba0
Allow resourceKey to deserialize
brendankowitz Apr 16, 2024
70578fd
Implement basic subscription filtering
brendankowitz Apr 16, 2024
82f5c9d
Implements Channel Interface
brendankowitz Apr 16, 2024
35954c7
Add example subscription
brendankowitz Apr 17, 2024
cb40eb2
DataLakeChannel.
fhibf Apr 17, 2024
1eb5264
Changes in DataLakeChannel and the project config.
fhibf Apr 18, 2024
03977d3
Load from DB
brendankowitz Apr 18, 2024
fcd87c0
EventGrid WIP
brendankowitz Apr 22, 2024
4fde508
Improve fhirtimer
brendankowitz Feb 29, 2024
c1a084e
Fixes for subscriptionwatchdog
brendankowitz Jul 16, 2024
b56fc82
Aligns dotnet sdk version for build
brendankowitz Jul 16, 2024
1e5540b
Adds subscription to docker build
brendankowitz Jul 17, 2024
b0cf111
Adds SearchQueryInterpreter
brendankowitz Jul 25, 2024
e1fc436
Cleanup of SearchQueryInterpreter
brendankowitz Jul 26, 2024
b9eb3cf
fix sql retry service merge
aponakampalli Aug 21, 2024
f018f15
fix sql command extensions merge
aponakampalli Aug 21, 2024
94eb190
Improve fhirtimer
brendankowitz Feb 29, 2024
be52627
Subscription infra
brendankowitz Apr 16, 2024
22e1ce2
Fixes wiring up of Transaction Watchdog => Orchestrator
brendankowitz Apr 16, 2024
186ecaf
Adding code for writing to storage
brendankowitz Apr 16, 2024
afeab31
Allow resourceKey to deserialize
brendankowitz Apr 16, 2024
adf381a
Implement basic subscription filtering
brendankowitz Apr 16, 2024
3c49e3e
Implements Channel Interface
brendankowitz Apr 16, 2024
7c4a3c7
Add example subscription
brendankowitz Apr 17, 2024
eaeccd7
DataLakeChannel.
fhibf Apr 17, 2024
35c8371
Changes in DataLakeChannel and the project config.
fhibf Apr 18, 2024
5ed896e
Load from DB
brendankowitz Apr 18, 2024
66fba51
EventGrid WIP
brendankowitz Apr 22, 2024
d51bde3
Improve fhirtimer
brendankowitz Feb 29, 2024
656d4d6
Fixes for subscriptionwatchdog
brendankowitz Jul 16, 2024
09acb1e
Aligns dotnet sdk version for build
brendankowitz Jul 16, 2024
eb3b2bf
Adds subscription to docker build
brendankowitz Jul 17, 2024
c9cb566
Adds SearchQueryInterpreter
brendankowitz Jul 25, 2024
6971bec
Cleanup of SearchQueryInterpreter
brendankowitz Jul 26, 2024
b83c201
Remove <TLogger>
feordin Aug 22, 2024
d0eb1dd
In Memory Search Filter For Subscriptions (#3971)
aponakampalli Aug 22, 2024
082e105
Rest Hook Channel for Subscriptions (#4008)
aponakampalli Aug 28, 2024
bb71c57
Merge branch 'feature/subscription-engine' of https://github.com/micr…
aponakampalli Aug 28, 2024
bd063b8
remove duplicate tests
aponakampalli Aug 28, 2024
42438f2
remove duplicates in share core project items
aponakampalli Aug 29, 2024
4e4f3d7
add feature flag to subscription module
aponakampalli Aug 29, 2024
08ff8b2
set optional http context on publish notification and heartbeat
aponakampalli Aug 30, 2024
43a2b19
Merge remote-tracking branch 'origin/main' into feature/subscription-…
brendankowitz Sep 16, 2024
dc0d6c9
Small cleanup
brendankowitz Sep 19, 2024
3798370
Merge remote-tracking branch 'origin/main' into feature/subscription-…
brendankowitz Sep 24, 2024
af3adef
Merge remote-tracking branch 'origin/main' into feature/subscription-…
brendankowitz Oct 9, 2024
e048ce0
Fixes from merge
brendankowitz Oct 9, 2024
5291b37
Merge origin/main into feature/subscription-engine
feordin Mar 16, 2026
c01a808
Add Microsoft.Health.Fhir.SqlOnFhir project with Ignixa packages
feordin Mar 16, 2026
db6027e
Add IElement adapter layer and Ignixa smoke tests (Tasks 3 & 4)
feordin Mar 23, 2026
aaa5804
Design decision: ViewDefinition (1) to (N) Subscriptions mapping
feordin Mar 24, 2026
d345446
Add SQL Table Schema Manager and Incremental Row Updater (Tasks 5 & 6)
feordin Mar 28, 2026
3cb6a80
Add full population background job for ViewDefinition tables (Task 7)
feordin Mar 28, 2026
904cd8c
Add materialization pipeline integration tests (Task 8)
feordin Mar 29, 2026
862be02
Add ViewDefinition Refresh subscription channel (Task 9)
feordin Mar 29, 2026
b932c9d
Add auto-subscription registration for ViewDefinition materialization…
feordin Mar 29, 2026
f90cc3e
Add end-to-end flow integration tests (Task 11)
feordin Mar 29, 2026
7a00ace
Add Parquet materializer and multi-target support (Task 12)
feordin Mar 29, 2026
3c9b2ac
Add \-run operation per SQL on FHIR v2 spec (Task 13)
feordin Mar 29, 2026
ed45efc
Add \-export operation with fast-path optimization (Task 13b)
feordin Mar 29, 2026
77cd9d5
Add materialization status tracking and ADR documentation (Tasks 14 &…
feordin Mar 29, 2026
cb0b677
Add HEDIS CBP demo app with spec ViewDefinitions and Synthea modules
feordin Mar 30, 2026
9662de8
Add future optimizations section to ADR
feordin Mar 30, 2026
0d686d5
Syntha module update
feordin Mar 30, 2026
521c5b6
Add crisis patients and intervention demo data bundles
feordin Mar 30, 2026
5c51087
Add parallel data loading, crisis patients, and interventions to demo…
feordin Mar 30, 2026
082d46d
feat: Delta Lake materializer for Fabric, Library-based ViewDef persi…
feordin Mar 31, 2026
3f72a11
fix: tag demo resources for targeted bulk delete, rewrite reset flow
feordin Mar 31, 2026
b1b866c
fix: add Prefer: respond-async header for bulk-delete call
feordin Mar 31, 2026
fcf0664
feat: responsive layout, add Run button with results table for ViewDe…
feordin Mar 31, 2026
4c43de2
fix: NullReferenceException in CreateResourceHandler.IsBundleParallel…
feordin Mar 31, 2026
d5891d0
fix: simplify Synthea sanitizer — keep all resources, use batch mode
feordin Mar 31, 2026
2a7b619
fix: load Synthea practitioner/hospital bundles before patient data
feordin Mar 31, 2026
bd10f5f
Merge from main
feordin Mar 31, 2026
578b511
Chnage global.json
feordin Mar 31, 2026
6cc603a
feat: session-based skip for Synthea data loading
feordin Mar 31, 2026
9f08dcb
Fix merge error
feordin Mar 31, 2026
40b10cd
fix: explicitly set TargetFrameworks to net9.0 only for SqlOnFhir pro…
feordin Mar 31, 2026
9e76703
fix: support net8.0 CI build step with empty assembly fallback
feordin Mar 31, 2026
ec41674
feat: register ViewDefinitions by POSTing Library resources
feordin Mar 31, 2026
3b5c5da
feat: ViewDefinition status endpoint with real-time polling
feordin Mar 31, 2026
1b03a43
perf: enable parallel bundle processing via x-bundle-processing-logic…
feordin Mar 31, 2026
16723b7
fix: wire up SqlOnFhir module in Startup.cs and R4.Web project
feordin Mar 31, 2026
47b31ec
fix: add SqlOnFhir reference to R5.Web and Stu3.Web projects
feordin Apr 1, 2026
49176d6
fix: add SqlOnFhir reference to R4B.Web project
feordin Apr 1, 2026
468343a
fix: use reflection to load SqlOnFhir module, avoid R4 model conflicts
feordin Apr 1, 2026
8555c6b
fix: add SqlOnFhir csproj to Docker restore layer
feordin Apr 1, 2026
09fd3b8
fix: use IServiceScopeFactory for MediatR calls from singleton manager
feordin Apr 1, 2026
e3ae983
fix: register ViewDefinitionRunController in Shared.Api projitems
feordin Apr 1, 2026
0093256
fix: register SqlOnFhir MediatR handlers via AddMediatR
feordin Apr 1, 2026
44b0982
fix: bind route parameter {idParameter} and return 404 instead of 500
feordin Apr 1, 2026
8ffc04e
feat: add GET /ViewDefinition list endpoint, revert status route to o…
feordin Apr 1, 2026
67c451f
feat: return FHIR Parameters resources from ViewDefinition status end…
feordin Apr 1, 2026
d7e87ac
fix: allow concurrent population jobs and make enqueue non-blocking
feordin Apr 1, 2026
5911469
diag: add logging to subscription channel registration and refresh
feordin Apr 1, 2026
b63ce49
fix: defer channel factory registration to first request via middleware
feordin Apr 1, 2026
58ca14e
fix: improve cleanup behavior logging and diagnostics
feordin Apr 1, 2026
81da90a
fix: prevent duplicate Libraries, improve cleanup matching, add evict…
feordin Apr 1, 2026
495fdfc
feat: use PUT for Library registration, handle both POST and PUT
feordin Apr 1, 2026
114d09a
fix: replace underscores with hyphens in Library resource IDs
feordin Apr 1, 2026
60b5b2a
fix: add ViewDefinitionPopulation queue to HostingBackgroundServiceQu…
feordin Apr 1, 2026
2530e2e
feat: population job signals completion via MediatR notification
feordin Apr 1, 2026
b3f2bfd
fix: register ViewDefinitionPopulation queue with job hosting framework
feordin Apr 1, 2026
0352e12
fix: double comma in crisis patient bundle JSON
feordin Apr 1, 2026
a4869e7
fix: skip re-registration when ViewDefinition content is unchanged
feordin Apr 1, 2026
11e9016
FIx mediatr registration
feordin Apr 1, 2026
a6966d9
remove unnecessary create library resource
feordin Apr 1, 2026
128e97e
Updates to wire up subscriptions properly
feordin Apr 2, 2026
f85e3c0
Load ViewDefinition status on startup
feordin Apr 2, 2026
cb72c21
Fix search continuation token
feordin Apr 3, 2026
7b521a2
Persisit state to library resources
feordin Apr 3, 2026
7df5a9b
Limit blood pressure readings
feordin Apr 3, 2026
0b34dbe
Improve logging
feordin Apr 3, 2026
c5a4f6a
Fix re-registration bug
feordin Apr 3, 2026
235fd10
Merge branch 'feature/subscription-sqlonfhir' of https://github.com/m…
feordin Apr 3, 2026
30f8bbf
More logging on viewdefinition population
feordin Apr 3, 2026
6fb43a9
Update ViewDefinition before setting status
feordin Apr 3, 2026
da9b2fa
Update Library resource
feordin Apr 3, 2026
b928e29
feat: add per-ViewDefinition materialization target support
feordin Apr 3, 2026
8b2f5d4
feat: add materialization target selector to Blazor demo app
feordin Apr 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .claude/settings.local.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"permissions": {
"allow": [
"Bash(dotnet build:*)",
"Bash(dotnet test:*)"
]
}
}
12 changes: 11 additions & 1 deletion Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
<HealthcareSharedPackageVersion>10.0.68</HealthcareSharedPackageVersion>
<Hl7FhirVersion>5.11.4</Hl7FhirVersion>
<Hl7FhirLegacyVersion>5.11.0</Hl7FhirLegacyVersion>
<IgnixaVersion>0.0.163</IgnixaVersion>
<DotNetSdkPackageVersion>9.0.6</DotNetSdkPackageVersion>
<OpenIddictPackageVersion>6.2.0</OpenIddictPackageVersion>
</PropertyGroup>
Expand Down Expand Up @@ -34,6 +35,7 @@
</ItemGroup>
<ItemGroup>
<PackageVersion Include="AngleSharp" Version="1.2.0" />
<PackageVersion Include="Azure.Messaging.EventGrid" Version="4.24.0" />
<PackageVersion Include="Azure.Identity" Version="1.16.0" />
<PackageVersion Include="Azure.ResourceManager.CosmosDB" Version="1.3.2" />
<PackageVersion Include="Azure.Extensions.AspNetCore.Configuration.Secrets" Version="1.4.0" />
Expand All @@ -45,6 +47,12 @@
<PackageVersion Include="Ensure.That" Version="10.1.0" />
<PackageVersion Include="FluentValidation" Version="11.11.0" />
<PackageVersion Include="Hl7.Fhir.Base" Version="$(Hl7FhirVersion)" />
<!-- Ignixa packages for SQL on FHIR ViewDefinition evaluation -->
<PackageVersion Include="Ignixa.SqlOnFhir" Version="$(IgnixaVersion)" />
<PackageVersion Include="Ignixa.SqlOnFhir.Writers" Version="$(IgnixaVersion)" />
<PackageVersion Include="Ignixa.Extensions.FirelySdk5" Version="$(IgnixaVersion)" />
<!-- Delta Lake for Fabric materialization target -->
<PackageVersion Include="DeltaLake.Net" Version="0.31.1" />
<PackageVersion Include="Hl7.Fhir.STU3" Version="$(Hl7FhirVersion)" />
<PackageVersion Include="Hl7.Fhir.R4" Version="$(Hl7FhirVersion)" />
<PackageVersion Include="Hl7.Fhir.R4B" Version="$(Hl7FhirVersion)" />
Expand All @@ -68,6 +76,8 @@
<PackageVersion Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" Version="$(AspNetPackageVersion)" />
<PackageVersion Include="Microsoft.AspNetCore.Mvc.Razor.RuntimeCompilation" Version="$(AspNetPackageVersion)" />
<PackageVersion Include="Microsoft.AspNetCore.Mvc.Testing" Version="$(AspNetPackageVersion)" />
<PackageVersion Include="Microsoft.AspNetCore.Http.Abstractions" Version="2.2.0" />
<PackageVersion Include="Microsoft.AspNetCore.Mvc.Core" Version="2.2.5" />
<PackageVersion Include="Microsoft.AspNetCore.TestHost" Version="$(AspNetPackageVersion)" />
<PackageVersion Include="Microsoft.Azure.ContainerRegistry" Version="1.0.0-preview.2" />
<PackageVersion Include="Microsoft.Azure.Cosmos" Version="3.48.0" />
Expand Down Expand Up @@ -143,4 +153,4 @@
<PackageVersion Include="Moq" Version="4.20.69" />
<PackageVersion Include="System.Text.RegularExpressions" Version="4.3.1" />
</ItemGroup>
</Project>
</Project>
601 changes: 601 additions & 0 deletions Microsoft.Health.Fhir.sln

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions R4.slnf
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,12 @@
"src\\Microsoft.Health.Fhir.Shared.Web\\Microsoft.Health.Fhir.Shared.Web.shproj",
"src\\Microsoft.Health.Fhir.SqlServer.UnitTests\\Microsoft.Health.Fhir.SqlServer.UnitTests.csproj",
"src\\Microsoft.Health.Fhir.SqlServer\\Microsoft.Health.Fhir.SqlServer.csproj",
"src\\Microsoft.Health.Fhir.Subscriptions.Tests\\Microsoft.Health.Fhir.Subscriptions.Tests.csproj",
"src\\Microsoft.Health.Fhir.Subscriptions\\Microsoft.Health.Fhir.Subscriptions.csproj",
"src\\Microsoft.Health.Fhir.Tests.Common\\Microsoft.Health.Fhir.Tests.Common.csproj",
"src\\Microsoft.Health.TaskManagement\\Microsoft.Health.TaskManagement.csproj",
"src\\Microsoft.Health.TaskManagement.UnitTests\\Microsoft.Health.TaskManagement.UnitTests.csproj",
"src\\Microsoft.Health.Fhir.ValueSets\\Microsoft.Health.Fhir.ValueSets.csproj",
"src\\Microsoft.Health.TaskManagement.UnitTests\\Microsoft.Health.TaskManagement.UnitTests.csproj",
"src\\Microsoft.Health.TaskManagement\\Microsoft.Health.TaskManagement.csproj",
"test\\Microsoft.Health.Fhir.R4.Tests.E2E\\Microsoft.Health.Fhir.R4.Tests.E2E.csproj",
"test\\Microsoft.Health.Fhir.R4.Tests.Integration\\Microsoft.Health.Fhir.R4.Tests.Integration.csproj",
"test\\Microsoft.Health.Fhir.Shared.Tests.Crucible\\Microsoft.Health.Fhir.Shared.Tests.Crucible.shproj",
Expand All @@ -44,4 +44,4 @@
"test\\Microsoft.Health.Fhir.Shared.Tests.Integration\\Microsoft.Health.Fhir.Shared.Tests.Integration.shproj"
]
}
}
}
533 changes: 533 additions & 0 deletions SQL-on-FHIR-Subscriptions-Plan.md

Large diffs are not rendered by default.

6 changes: 6 additions & 0 deletions build/docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,12 @@ COPY ./src/Microsoft.Health.Fhir.CosmosDb.Core/Microsoft.Health.Fhir.CosmosDb.Co
COPY ./src/Microsoft.Health.Fhir.CosmosDb.Initialization/Microsoft.Health.Fhir.CosmosDb.Initialization.csproj \
./src/Microsoft.Health.Fhir.CosmosDb.Initialization/Microsoft.Health.Fhir.CosmosDb.Initialization.csproj

COPY ./src/Microsoft.Health.Fhir.Subscriptions/Microsoft.Health.Fhir.Subscriptions.csproj \
./src/Microsoft.Health.Fhir.Subscriptions/Microsoft.Health.Fhir.Subscriptions.csproj

COPY ./src/Microsoft.Health.Fhir.SqlOnFhir/Microsoft.Health.Fhir.SqlOnFhir.csproj \
./src/Microsoft.Health.Fhir.SqlOnFhir/Microsoft.Health.Fhir.SqlOnFhir.csproj

COPY ./src/Microsoft.Health.Fhir.${FHIR_VERSION}.Core/Microsoft.Health.Fhir.${FHIR_VERSION}.Core.csproj \
./src/Microsoft.Health.Fhir.${FHIR_VERSION}.Core/Microsoft.Health.Fhir.${FHIR_VERSION}.Core.csproj

Expand Down
39 changes: 39 additions & 0 deletions docs/SqlOnFHir/viewdefinition-diagram.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# ViewDefinition Resource Relationships

```mermaid
graph TD
subgraph Library["📦 Library Resource"]
ViewDef["📄 ViewDefinition (contained)"]
end

Subscription["🔔 Subscription Resource"]

Library -.->|relatedArtifact / link| Subscription
```

## ViewDefinition Lifecycle

```mermaid
sequenceDiagram
actor Client
participant FHIR as FHIR Server
participant Sync as ViewDefinition SyncService
participant Sub as Subscription
participant Table as Materialized Table

Note over Client, Table: Initial Setup & Materialization
Client->>FHIR: POST Library (with contained ViewDefinition)
FHIR-->>Client: 201 Created
Sync->>FHIR: Read Library & extract ViewDefinition
Sync->>Table: Create / materialize table from existing data
Sync->>FHIR: Create Subscription for target resource type
FHIR-->>Sync: Subscription active

Note over Client, Table: Ongoing Updates
Client->>FHIR: POST new Resource (e.g., Observation)
FHIR-->>Client: 201 Created
FHIR->>Sub: Subscription notification triggered
Sub->>Sync: Notify of new/updated resource
Sync->>FHIR: Read resource data
Sync->>Table: Upsert row in materialized table
```
Loading
Loading