Add Optimized Single Column Date Rewriter #5562
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Refs ADR docs/arch/Proposals/adr-2604-date-only-search-param-sql-optimization.md Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
GetMapping("DATE") was coalescing with "DATETIME" and returning
typeof(FhirDateTime), masking custom search parameters whose
expressions use the .as(date) / .ofType(date) string-cast syntax.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Extends ISearchParameterSupportResolver.IsSearchParameterSupported to return an IsDateOnly flag derived from FhirPath type resolution. The flag is propagated to SearchParameterInfo at startup (SearchParameterStatusManager.EnsureInitializedAsync) and at runtime custom-parameter registration (SearchParameterOperations.AddSearchParameterAsync, UpdateSearchParameterAsync). Refs ADR docs/arch/Proposals/adr-2604-date-only-search-param-sql-optimization.md Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collapses Core's two-column overlap predicate (StartDateTime >= startOfDay) AND (EndDateTime <= endOfDay) into a single-column equality EndDateTime = endOfDay(d) when SearchParameterInfo.IsDateOnly is true. The rewriter is gated on the metadata flag and is therefore opt-in per parameter. Composite params and range operators pass through unchanged. Refs ADR docs/arch/Proposals/adr-2604-date-only-search-param-sql-optimization.md Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Inserted before DateTimeEqualityRewriter so the date-only equality pattern is collapsed into a single-column equality before the generic equality rewriter would otherwise add the four-predicate overlap. After the new rewriter fires, the equality rewriter no-ops because the pattern it looks for is gone. Refs ADR docs/arch/Proposals/adr-2604-date-only-search-param-sql-optimization.md Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ange) Adds GivenPatientsWithDateOnlyBirthdate_WhenSearchedByEqualityAndRange_ThenCorrectPatientsAreReturned to DateSearchTests to verify that Patient?birthdate=YYYY-MM-DD (equality) and Patient?birthdate=gt YYYY-MM-DD (range) return correct results across all data stores. This serves as a regression guard for the DateOnlyEqualityRewriter introduced in ADR-2604. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the same FhirClientException/Exception catch pattern used by all other search tests in DateSearchTests.cs, so CI failures surface the server BaseAddress and Activity Id in the failure message. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Promote the Date-Only Search Parameter SQL Optimization ADR from Proposals to Accepted now that implementation and testing are complete. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…riter SearchComparator.Ap emits the identical two-predicate AST shape as Eq but with constants shifted by an approximate delta. The rewriter previously matched on structure alone, so it would silently collapse an Ap query into End = approxEnd, a value that never exists in any stored row, returning zero results. Add a value-shape guard before the collapse: * startValue.TimeOfDay must be TimeSpan.Zero (Eq always starts at midnight) * endValue must equal startValue.AddDays(1).AddTicks(-1) (exactly one day) Ap constants fail the second check, so the expression passes through unchanged. Eq constants satisfy both conditions and are still collapsed. Also adds GivenApproximateDateOnlyExpression_WhenRewritten_PassesThrough unit test to assert the guard rejects the Ap-shaped expression. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
GetAndApplySearchParameterUpdates previously called AddNewSearchParameters without invoking the support resolver, leaving IsDateOnly=false (the default) on any custom date-only SearchParameter picked up from another instance. The optimization was therefore dead for that param until process restart. Fix: after each AddNewSearchParameters call (both the chunk-flush branch and the final-remainder branch) iterate the URLs just added, look up the SearchParameterInfo via TryGetSearchParameter, call IsSearchParameterSupported, and assign IsDateOnly from the result. Each URL is wrapped in try/catch with a logged warning so that a single resolution failure does not abort the cache refresh. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…emporal rewriter Month-precision equality must not be rewritten to a DateTimeEnd range because a stored scalar temporal value with only year precision (EndDateTime at year-end) would be excluded by a month-scoped DateTimeEnd predicate, producing false negatives. Day and year collapses are safe and are retained unchanged. IsExactMonth helper and its call site are removed. XML doc updated to explain the month pass-through rationale. A new test covers TryMatchEqualityPattern's reversed predicate order (DateTimeEnd <= end first, DateTimeStart >= start second) for the exact-day case. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Limit the single-column temporal rewrite to allow-listed FHIR date parameters and keep only Patient.birthDate for now. Remove scalar temporal diagnostics and derived metadata plumbing so future date parameters can be added through the SQL rewriter allow-list. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Revert the DATE cast mapping change and remove its dedicated tests because the simplified birthdate rewrite no longer relies on derived date-only metadata. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This reverts commit df42882. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Restructure VisitSearchParameter as a top-to-bottom pipeline (allow-list check, pattern match, value extraction, precision dispatch). Replace the cascading IsExactDay/IsExactYear boolean checks with a Precision enum and ClassifyPrecision, and give the two rewrite shapes names via BuildExactDayRewrite/BuildExactYearRewrite. Public surface unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Restore ISearchParameterSupportResolver, SearchParameterOperations, and SearchParameterStatusManager to match main. These were leftover stylistic tweaks (var vs explicit tuple type, doc comment) plus an unused urlsToAdd list that was populated but never consumed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ar-temporal-birthdate-impl # Conflicts: # test/Microsoft.Health.Fhir.Shared.Tests.E2E/Rest/Search/DateSearchTests.cs
Drop the branch's extra 2000-12 overlap assertion in favor of main's broader coverage. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The branch's versions still passed a 4-tuple to IsSearchParameterSupported, which no longer matches the interface (restored to a 2-tuple earlier in this branch). Aligning these mocks with main fixes the compile mismatch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
| (MultiaryExpression)Expression.And( | ||
| Expression.GreaterThanOrEqual(FieldName.DateTimeStart, null, start), | ||
| Expression.LessThanOrEqual(FieldName.DateTimeEnd, null, end)); |
| var reversedPattern = (MultiaryExpression)Expression.And( | ||
| Expression.LessThanOrEqual(FieldName.DateTimeEnd, null, EndOfDay), | ||
| Expression.GreaterThanOrEqual(FieldName.DateTimeStart, null, StartOfDay)); |
| (MultiaryExpression)Expression.And( | ||
| new BinaryExpression(BinaryOperator.Equal, FieldName.DateTimeEnd, endPredicate.ComponentIndex, endPredicate.Value), | ||
| Expression.Equals(SqlFieldName.DateTimeIsLongerThanADay, endPredicate.ComponentIndex, false)); |
| (MultiaryExpression)Expression.And( | ||
| Expression.GreaterThanOrEqual(FieldName.DateTimeEnd, endPredicate.ComponentIndex, yearStart), | ||
| Expression.LessThanOrEqual(FieldName.DateTimeEnd, endPredicate.ComponentIndex, yearEnd)); |
ScalarTemporalEqualityRewriter collapses day-precision birthdate equality to a single-day containment match (DateTimeEnd = endOfDay AND IsLongerThanADay = false), which deliberately excludes Patients whose stored range spans more than the searched day. Update the day-precision assertions to expect only the exact-day Patient; year and month searches still use range overlap and are unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
🤖 Automated Pipeline Failure Analysis — Build #49384❌ Failed Jobs
Root CauseThe compartmentSearchRewriter,
smartCompartmentSearchRewriter,
DateTimeEqualityRewriter.Instance,The shared E2E test Suggested FixSince the test class is decorated with Bundle dayBundle = await Client.SearchAsync(ResourceType.Patient, $"birthdate=2000-03-03&_tag={tag}");
if (Fixture.DataStore == DataStore.SqlServer)
{
// SQL path: ScalarTemporalEqualityRewriter narrows to exact-day containment
ValidateBundle(dayBundle, patient2000March03);
}
else
{
// Cosmos path: still uses range overlap (tracked by AB#191826)
ValidateBundle(dayBundle, patient2000, patient2000March, patient2000March03);
}Apply the same pattern for the Alternatively, the rewriter could be added to the Cosmos DB pipeline too, but Analysis by Azure SRE Agent |
Description
Patient.birthdate.DateTimeEndpredicates while preserving existing handling for month precision, ranges, composites, and non-allow-listed date parameters.Related issues
AB#190492
Testing
Describe how this change was tested.
FHIR Team Checklist
Semver Change (docs)
Patch|Skip|Feature|Breaking (reason)