[Internal] Change Feed: Adds Code to Read Globally Committed Data During PPAF Failovers#5379
[Internal] Change Feed: Adds Code to Read Globally Committed Data During PPAF Failovers#5379kundadebdatta wants to merge 13 commits intomasterfrom
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR implements support for reading globally committed data during partition-level automatic failover (PPAF) scenarios for Change Feed operations in less than strong consistency accounts. The key enhancement adds a new request header x-ms-cosmos-read-global-committed-data to prevent data loss during failovers.
- Adds logic to detect PPAF scenarios and automatically include the global committed data header
- Refactors consistency level validation to return the selected level for downstream use
- Implements comprehensive test coverage for PPAF scenarios with different consistency levels
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| RequestInvokerHandler.cs | Core logic to detect PPAF scenarios and add the global committed data header |
| ChangeFeedHelper.cs | New utility method to determine when Change Feed can handle missing primes safely |
| MockDocumentClient.cs | Mock enhancement to support consistency level testing |
| ChangeFeedIteratorCoreTests.cs | Comprehensive test coverage for PPAF scenarios |
| EndToEndTraceWriterBaselineTests.MiscellanousAsync.xml | Updated baseline removing client initialization trace |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
FabianMeiswinkel
left a comment
There was a problem hiding this comment.
LGTM - except for the feedback around having the consistency level condition.
|
For short term, the new header is no longer required from the SDK. BE will go with the GCLSN based approach for change feeds, and the goal is for PPAF enabled accounts (irrespective of consistency levels), false progress changes will not be reported through CF. Hence closing this PR. |
Pull Request Template
Background
Today, during a partition level automatic failover, there is a gap in the backend where for a change feed operation (incremental or full-fidelity) in a less than (<) strong consistency account, a false progress can cause data loss, since the change feed operation reads relies on the
GLSN.In a nut-shell a
GLSNis unique for a givenGCN, however, for multipleGCNs, there could be duplicateGLSNs (e.g. there could be a sameGLSN0forGCN1andGCN2). In order to fix this, the SDK need to send thex-ms-cosmos-read-global-committed-dataheader with change feed operation so that the change feed can always read the globally committed data.Description
This PR adds a new request header for Change Feed requests to read the globally committed data for less than strong consistency accounts.
x-ms-cosmos-read-global-committed-datais the header to read the globally committed data.Type of change
Please delete options that are not relevant.
Closing issues
To automatically close an issue: closes #5376