[Internal]Msdata/direct: Refactors msdata/direct branch to refresh with V3 master 3.52.1 and Direct code version 3.39.1 by ananth7592 · Pull Request #5321 · Azure/azure-cosmos-dotnet-v3

ananth7592 · 2025-07-29T00:07:35Z

Description

Refactors msdata/direct branch with changes from V3 master branch of version 3.52.1 and direct version 3.39.1

Type of change

Please delete options that are not relevant.

[] Bug fix (non-breaking change which fixes an issue)
[] New feature (non-breaking change which adds functionality)
[] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[] This change requires a documentation update

Closing issues

To automatically close an issue: closes #IssueNumber

# Pull Request Template ## Description ### Public contract changes: ![API_3 44 0](https://github.com/user-attachments/assets/835270ad-9935-4665-83e8-d4bc373824ad) ### Preview contract changes: ![API_3 45 0-preview 0](https://github.com/user-attachments/assets/8f1c2528-f47b-48bb-bdba-035b7f801ffa)

## Description Removing dependency on `Task.Delay() ` ## Type of change - [] Bug fix (non-breaking change which fixes an issue) ## Closing issues To automatically close an issue: closes #4731

## Description Removing dependency on `Task.Delay() ` ## Type of change - [] Bug fix (non-breaking change which fixes an issue) ## Closing issues To automatically close an issue: closes #4732

…on failure status codes (#4762) ## Description Added a check on Cosmos Exception to make sure events are generated only for the allowed status and substatuscode ## Type of change - [] Bug fix (non-breaking change which fixes an issue) ## Closing issues To automatically close an issue: closes #4735

[INTERNA] CI: Fixes disabling Binskim as nuget repo failing

[INTERNAL] Build: Fix warnings in encryption project

[INTERNAL]STJ: Upgrade STJ to 8.0.5 as per dependa bot

[INTERNAL] CI: Fixes styling and documentation issues

# [Internal] Usage: Add README.md for ApplicationInsights usage sample ## Description Add README.md for ApplicationInsights usage sample ## Type of change Please delete options that are not relevant. - [] Bug fix (non-breaking change which fixes an issue) - [] New feature (non-breaking change which adds functionality) - [] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [] This change requires a documentation update ## Closing issues To automatically close an issue: closes #IssueNumber --------- Co-authored-by: David Chaava <v-dchaava@microsoft.com> Co-authored-by: Ruben Bartelink <ruben@bartelink.com> Co-authored-by: Matias Quaranta <ealsur@users.noreply.github.com> Co-authored-by: Kiran Kumar Kolli <kirankk@microsoft.com>

…4725) # Pull Request Template ## Description This PR makes several regions available for public usage. Austria East France Central France South Indonesia Central Southeast US Southwest US Malaysia West Germany Central Germany North Chile Central South Central US 2 ## Type of change Please delete options that are not relevant. - [] New feature (non-breaking change which adds functionality) ## Closing issues To automatically close an issue: closes #IssueNumber --------- Co-authored-by: Kiran Kumar Kolli <kirankk@microsoft.com>

[INTERNAL] CI: Fixes timeout for tasks to 60M

…nsights sdk (#4781) ## Description As part of this PR, adding back attributes required by appinsights sdk. It broke the customer experience with appinsight as Appinsight SDK supports very specific set of attributes. This implementation will change in future release, where open telemetry attributes will be controlled by Env Variable as mentioned here. https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/database-spans.md#semantic-conventions-for-database-client-calls **4.42.3** ![image](https://github.com/user-attachments/assets/6c19eab0-4884-4648-b6d9-c941d58e8e0b) **v4.43.0** ![image](https://github.com/user-attachments/assets/fd2a70b4-ed37-4618-b1a5-f654587bf6ab) **v4.43.1** ![image](https://github.com/user-attachments/assets/5bbe54ec-34f3-4538-a7b4-abff1b1853ea) After this PR: ![image](https://github.com/user-attachments/assets/2ea64a34-4e57-4dec-a29e-32ffcd77919a) After this change, customer has to set `OTEL_SEMCONV_STABILITY_OPT_IN` to `database/dup`, in order to see latest attributes. otherwise SDK will emit only classic attributes which would be compatible with appinsights sdk also. ## Type of change - [] Bug fix (non-breaking change which fixes an issue) ## Closing issues To automatically close an issue: closes #IssueNumber

# Pull Request Template ## Description Refactor EncryptionProcessor - split code paths for deprecated local implementation of Aes, split individual transformers to improve maintainability and testability, split preview/stable To be processed after #4753 ## Type of change Please delete options that are not relevant. - [] New feature (non-breaking change which adds functionality) ## Closing issues Contributes to #4678 --------- Co-authored-by: Juraj Blazek <jublazek@microsoft.com> Co-authored-by: juraj-blazek <53177060+juraj-blazek@users.noreply.github.com> Co-authored-by: Santosh Kulkarni <66682828+kr-santosh@users.noreply.github.com> Co-authored-by: Kiran Kumar Kolli <kirankk@microsoft.com>

[INTERNAL] CI: Adds .NET8 SDK support RISK: not all pipelines are exercised part of CI gates and might be surprises later.

# Pull Request Template ## Description - This is preliminary step to enable Brotli compression on Cosmos.Encryption.Custom. - .NET8.0 target was added to the project - New compiler complaints were addressed To be processed after #4757 ## Type of change Please delete options that are not relevant. - [] New feature (non-breaking change which adds functionality) ## Closing issues Contributes to #4678 --------- Co-authored-by: Juraj Blazek <jublazek@microsoft.com> Co-authored-by: juraj-blazek <53177060+juraj-blazek@users.noreply.github.com> Co-authored-by: Santosh Kulkarni <66682828+kr-santosh@users.noreply.github.com>

Regions: Fixes decommissioned regions Two decommissioned regions which are removed ```c# public const string GermanyCentral = "Germany Central"; public const string GermanyNortheast = "Germany Northeast"; ``` --------- Co-authored-by: Debdatta Kunda <87335885+kundadebdatta@users.noreply.github.com>

…4785) This reverts commit 8e718b1.

# Pull Request Template ## Description Adds hedging documentation ## Type of change Please delete options that are not relevant. - [] Bug fix (non-breaking change which fixes an issue) - [] New feature (non-breaking change which adds functionality) - [] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [] This change requires a documentation update ## Closing issues To automatically close an issue: closes #IssueNumber --------- Co-authored-by: Justine Cocchi <jucocchi@microsoft.com> Co-authored-by: Kevin Pilch <kevinpi@microsoft.com> Co-authored-by: Kiran Kumar Kolli <kirankk@microsoft.com>

) # Pull Request Template ## Description Current OpenTelemetry sample gives runtime error due to mismatched OpenTelemetry version. This is tracked in the Otel repo here: open-telemetry/opentelemetry-dotnet#5741 There is no GA Azure Monitor Otel exporter version that uses OpenTelemetry 1.9.0, so we need to manually lift the version to avoid runtime errors. ## Type of change Please delete options that are not relevant. - [x] Bug fix (non-breaking change which fixes an issue)

# Pull Request Template ## Description Initial commit for JsonNode on Encryption path. This is currently not executed on any production/preview path. Depends on JsonNode features available from System.Text.Json 8.0+. - Adds JsonNodeSqlSerializer - JObjectSqlSerializer now doesn't format inner serialized JArrays/JObjects (with line breaks/indentations) To be processed after #4766 ## Type of change Please delete options that are not relevant. - [] New feature (non-breaking change which adds functionality) ## Closing issues Contributes to #4678 --------- Co-authored-by: Juraj Blazek <jublazek@microsoft.com> Co-authored-by: juraj-blazek <53177060+juraj-blazek@users.noreply.github.com> Co-authored-by: Santosh Kulkarni <66682828+kr-santosh@users.noreply.github.com>

# Pull Request Template ## Description Please include a summary of the change and which issue is fixed. Include samples if adding new API, and include relevant motivation and context. List any dependencies that are required for this change. ## Type of change Please delete options that are not relevant. - [] Bug fix (non-breaking change which fixes an issue) - [] New feature (non-breaking change which adds functionality) - [] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [] This change requires a documentation update ## Closing issues To automatically close an issue: closes #IssueNumber

… latest versions (#4790) [INTERNAL] FaultInjection: Fixes Newtonsoft and SystemTextJson to use latest versions

# Pull Request Template ## Description - Add Brotli compression for .NET8.0 target - To stabilize performance tests duration results, GC was reconfigured to server mode (reduces StdDev 42% -> 5%) Brotli compression is now available for .NET8.0 target. All encrypted fields matching criteria are compressed before encryption. The option is configurable via `Microsoft.Azure.Cosmos.Encryption.Custom.EncryptionOptions` property `CompressionOptions` `CompressionOptions` structure: - `Algorithm` : `CompressionOptions.CompressionAlgoritm` suports `None` and `Brotli` (for .NET8.0+), defaults to None. - `CompressionLevel` : `System.IO.Compression.CompressionLevel` - defaults to CompressionLevel.Fastest - `MinimalCompressedLength` : determines minimal size of encrypted serialized Json text for compression. Defaults to 128 bytes. It is suggested to benchmark your use cases, different settings can lead to different ratio of compute vs storage costs. Default settings in general should save both cpu time and memory consumption. **Warn: make sure all your instances are upgraded to version supporting compression BEFORE enabling it.** To be processed after #4766 ## Type of change Please delete options that are not relevant. - [] New feature (non-breaking change which adds functionality) ## Closing issues Contributes to #4678 --------- Co-authored-by: Juraj Blazek <jublazek@microsoft.com> Co-authored-by: juraj-blazek <53177060+juraj-blazek@users.noreply.github.com> Co-authored-by: Santosh Kulkarni <66682828+kr-santosh@users.noreply.github.com> Co-authored-by: Kiran Kumar Kolli <kirankk@microsoft.com>

# Pull Request Template ## Description - Added support for System.Text.JsonNode DOM on encryption path - Configuration option to select Json Processor (defaults to Newtonsoft) - Custom byte[] coverter supporting offset+length for JsonNode - Perf tests expanded to cover both json processors (JsonNode decryption still uses Newtonsoft, this will be upgraded with further PRs) To be processed after #4779 ## Type of change Please delete options that are not relevant. - [] New feature (non-breaking change which adds functionality) - [] This change requires a documentation update ## Closing issues Contributes to #4678 --------- Co-authored-by: Juraj Blazek <jublazek@microsoft.com> Co-authored-by: juraj-blazek <53177060+juraj-blazek@users.noreply.github.com> Co-authored-by: Santosh Kulkarni <66682828+kr-santosh@users.noreply.github.com>

## Description ### What is the change? `db.cosmosdb.operation_type `--> removed (moved to classic sdk support) `db.cosmosdb.machine_id` --> removed (moved to classic sdk support) `db.cosmosdb.client_id` --> ~~`db.cosmosdb.client.id`~~ it will remain same ~~(aligns with OTel messaging conventions)~~ `db.cosmosdb.request_content_length` --> ~~`db.cosmosdb.request.content_length`~~ it will remain same ~~(aligns with OTel HTTP conventions)~~ `db.cosmosdb.response_content_length` --> ~~`db.cosmosdb.request.content_length`~~ it will remain same ~~(aligns with OTel HTTP conventions)~~ `db.cosmosdb.status_code` --> `db.response.status_code` (moved `db.cosmosdb.status_code` to classic sdk support) `db.cosmosdb.consistency_level` -> added `db.cosmosdb.request_content_length_bytes` -> added for classic sdk support ### Why? Lot of discussions are going on with Azure Core and Open Telemetry Community teams. As part of that discussion, we are making these changes as our goal is to be Open Telemetry and Azure compliant, ## Type of change Please delete options that are not relevant. - [] Bug fix (non-breaking change which fixes an issue) ## Closing issues To automatically close an issue: closes #4553 --------- Co-authored-by: Kiran Kumar Kolli <kirankk@microsoft.com>

…amIterator (#4798) ## Description This change fixes an issue with GetItemQueryStreamIterator for distributed query where containerRid was not getting set in the responseHeaders, leading to an exception when building the response. We must set the container resource id in the CosmosQueryContext, prior to building the DistributedQueryPipelineStage. ## Type of change - [] Bug fix (non-breaking change which fixes an issue) --------- Co-authored-by: neildsh <35383880+neildsh@users.noreply.github.com>

…regions when preferred regions is not set on client. (#4709) # Pull Request Template ## Problem Statement As of today, customers who do not configure `ApplicationPreferredRegions` or `ApplicationRegion` are pinned to either the hub region (in multi-write accounts) or the primary region (in single-write accounts). In outage scenarios, availability is scoped to just a single region and is not ideal (both reads/writes in multi-write accounts and reads in single-write multi-region accounts have their availability curbed to just the one region). Setting `ApplicationPreferredRegions` or `ApplicationRegion` as empty is not an opt-out of availability decision from the customer's perspective unless of course on the client a regional endpoint has been set. This PR aims to fix this issue when preferred regions is not set, and a global endpoint is set on the client. ## Approach taken in this PR The idea is to construct an _effective preferred region list_ and to rely on account-level read and account-level write regions for it. If there is client-perceived unavailability or account-level topology changes - then this _effective preferred regions list_ is reordered accordingly or reflects the account-level regions post a cached `DatabaseAccount` refresh in the SDK. There are also changes made to the flow which decides when `DatabaseAccount` refresh is triggered. The decision depends on a check whether the SDK has a different effective first preferred read / write region from the first account-level read / write region. ## Closing issues closes #4665 --------- Co-authored-by: Kiran Kumar Kolli <kirankk@microsoft.com> Co-authored-by: Nalu Tripician <27316859+NaluTripician@users.noreply.github.com> Co-authored-by: Debdatta Kunda <87335885+kundadebdatta@users.noreply.github.com>

…4787) # Pull Request Template ## Description - Added support for System.Text.JsonNode DOM on deserializer and decryption path - Drops custom byte[] converter as JsonNode can take Memory<byte> directly To be processed after #4780 ## Type of change Please delete options that are not relevant. - [] New feature (non-breaking change which adds functionality) - [] This change requires a documentation update ## Closing issues Contributes to #4678 --------- Co-authored-by: Juraj Blazek <jublazek@microsoft.com> Co-authored-by: juraj-blazek <53177060+juraj-blazek@users.noreply.github.com> Co-authored-by: Santosh Kulkarni <66682828+kr-santosh@users.noreply.github.com> Co-authored-by: Kiran Kumar Kolli <kirankk@microsoft.com>

…4811) ## Description Updated changelog and versioning information for 3.44.1 hotfix release --------- Co-authored-by: Kiran Kumar Kolli <kirankk@microsoft.com>

…rom 6.0.1 to 8.0.1 (#4810) ## Description ### Why Upgrade? The new `DiagnosticSource` library includes additional Meter API features that we plan to utilize moving forward. ### Is this a Breaking Change? No, this upgrade is not expected to cause breaking changes. After reviewing this [documentation](https://learn.microsoft.com/en-us/dotnet/core/compatibility/core-libraries/8.0/parameterinfo-hasdefaultvalue), no issues related to `DiagnosticSource` or `ActivitySource` were found. ### Potential Impact on Existing Customers? Upon upgrading to this version, customers may need to update some dependencies that rely on the older version of the `DiagnosticSource` library. ## Type of change - [] New feature (non-breaking change which adds functionality)

…y reference This helps us keep the .targts file active and hence able to obtain other assets inside DLL while disabling the references

kundadebdatta · 2025-08-06T16:35:24Z

Can we please add a PR description to mention which Direct release we are syncing with ? This will help us to backtrack in the future.

# Pull Request Template ## Description Please include a summary of the change and which issue is fixed. Include samples if adding new API, and include relevant motivation and context. List any dependencies that are required for this change. ## Type of change Please delete options that are not relevant. - [] Bug fix (non-breaking change which fixes an issue) - [] New feature (non-breaking change which adds functionality) - [] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [] This change requires a documentation update ## Closing issues To automatically close an issue: closes #IssueNumber

NaluTripician · 2025-08-06T17:11:26Z

/azp run

azure-pipelines · 2025-08-06T17:11:33Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

NaluTripician · 2025-08-06T17:29:43Z

/azp run

azure-pipelines · 2025-08-06T17:29:49Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

# Pull Request Template ## Description Due to flakiness with the encryption pipelines with the msdata/direct pipelines we are removing them for a better pipeline success rate. ## Type of change Please delete options that are not relevant. - [] Bug fix (non-breaking change which fixes an issue) ## Closing issues To automatically close an issue: closes #IssueNumber Co-authored-by: Kiran Kumar Kolli <kirankk@microsoft.com>

#5341) Removing Microsoft.Azure.Cosmos.Encryption.Custom Project Ref build pipeline from msdata-preview list ## Type of change Please delete options that are not relevant. - [] Bug fix (non-breaking change which fixes an issue) - [] New feature (non-breaking change which adds functionality) - [] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [] This change requires a documentation update ## Closing issues To automatically close an issue: closes #IssueNumber

philipthomas-MSFT and others added 30 commits October 8, 2024 09:37

[Internal] Flaky Test: Fixes Client Telemetry Flaky tests (#4760)

5aee50c

## Description Removing dependency on `Task.Delay() ` ## Type of change - [] Bug fix (non-breaking change which fixes an issue) ## Closing issues To automatically close an issue: closes #4731

[Internal] Flaky Test: Fixes PartitionKeyRangeCacheTests (#4761)

f3e343c

## Description Removing dependency on `Task.Delay() ` ## Type of change - [] Bug fix (non-breaking change which fixes an issue) ## Closing issues To automatically close an issue: closes #4732

[INTERNA] CI: Fixes disabling Binskim as nuget repo failing (#4770)

8e718b1

[INTERNA] CI: Fixes disabling Binskim as nuget repo failing

[INTERNAL] Build: Fixes warnings in encryption project (#4756)

24c2ce1

[INTERNAL] Build: Fix warnings in encryption project

[INTERNAL]STJ: Fixes STJ to 8.0.5 as per dependa bot (#4775)

c54bcd9

[INTERNAL]STJ: Upgrade STJ to 8.0.5 as per dependa bot

[INTERNAL] CI: Fixes styling and documentation issues (#4774)

d61814f

[INTERNAL] CI: Fixes styling and documentation issues

[INTERNAL] CI: Fixes timeout for tasks to 60M (#4778)

8471136

[INTERNAL] CI: Fixes timeout for tasks to 60M

[INTERNAL] CI: Adds .NET8 SDK support (#4776)

2b0a021

[INTERNAL] CI: Adds .NET8 SDK support RISK: not all pipelines are exercised part of CI gates and might be surprises later.

[INTERNAL] CI: Fixes Revert disabling Binskim as nuget repo failing (#…

941efcc

…4785) This reverts commit 8e718b1.

[INTERNAL] FaultInjection: Fixes Newtonsoft and SystemTextJson to use…

43be549

… latest versions (#4790) [INTERNAL] FaultInjection: Fixes Newtonsoft and SystemTextJson to use latest versions

3.44.1 and 3.45.0-preview.1: Adds latest change and version updates (#…

05a9688

…4811) ## Description Updated changelog and versioning information for 3.44.1 hotfix release --------- Co-authored-by: Kiran Kumar Kolli <kirankk@microsoft.com>

ananth7592 force-pushed the ananth/msdata-direct-2025-7-18 branch 2 times, most recently from 0e959a1 to 1d18f1e Compare July 29, 2025 17:33

Direct file cosmetic and other fixes

aa33a33

ananth7592 force-pushed the ananth/msdata-direct-2025-7-18 branch from 1d18f1e to aa33a33 Compare July 30, 2025 18:18

CHanged the direct package dependency to disable compile-time assembl…

6446d41

…y reference This helps us keep the .targts file active and hence able to obtain other assets inside DLL while disabling the references

ananth7592 force-pushed the ananth/msdata-direct-2025-7-18 branch 2 times, most recently from 6446d41 to 6743379 Compare July 31, 2025 20:44

Removing a bunch of flaky tests

5405931

ananth7592 force-pushed the ananth/msdata-direct-2025-7-18 branch from bb892aa to 5405931 Compare August 4, 2025 23:56

ananth7592 force-pushed the ananth/msdata-direct-2025-7-18 branch from 29b6af2 to 766d938 Compare August 6, 2025 17:07

ananth7592 changed the title ~~[Internal]Msdata/direct: Refactors msdata/direct refresh July 28 2025~~ [Internal]Msdata/direct: Refreshes msdata/direct branch with latest V3 master and Direct code versioned Aug 6, 2025

NaluTripician changed the title ~~[Internal]Msdata/direct: Refreshes msdata/direct branch with latest V3 master and Direct code versioned~~ [Internal]Msdata/direct: Refactors msdata/direct branch with latest V3 master and Direct code versioned Aug 6, 2025

ananth7592 changed the title ~~[Internal]Msdata/direct: Refactors msdata/direct branch with latest V3 master and Direct code versioned~~ [Internal]Msdata/direct: Refactors msdata/direct branch to refresh with V3 master 3.52.1 and Direct code version 3.39.1 Aug 6, 2025

Remove unrelated and flaky tests

1ebeaca

ananth7592 force-pushed the ananth/msdata-direct-2025-7-18 branch from bca22e4 to 1ebeaca Compare August 7, 2025 18:09

NaluTripician and others added 2 commits August 7, 2025 12:24

ananth7592 force-pushed the ananth/msdata-direct-2025-7-18 branch 2 times, most recently from e8be443 to 8b3fd1f Compare August 11, 2025 22:16

Flaky test fixes

f459e22

ananth7592 force-pushed the ananth/msdata-direct-2025-7-18 branch from 8b3fd1f to f459e22 Compare August 11, 2025 23:47

kundadebdatta approved these changes Aug 12, 2025

View reviewed changes

NaluTripician merged commit 4f85095 into msdata/direct Aug 12, 2025
20 checks passed

NaluTripician deleted the ananth/msdata-direct-2025-7-18 branch August 12, 2025 01:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Internal]Msdata/direct: Refactors msdata/direct branch to refresh with V3 master 3.52.1 and Direct code version 3.39.1#5321

[Internal]Msdata/direct: Refactors msdata/direct branch to refresh with V3 master 3.52.1 and Direct code version 3.39.1#5321
NaluTripician merged 699 commits intomsdata/directfrom
ananth/msdata-direct-2025-7-18

ananth7592 commented Jul 29, 2025 •

edited

Loading

Uh oh!

kundadebdatta commented Aug 6, 2025

Uh oh!

NaluTripician commented Aug 6, 2025

Uh oh!

azure-pipelines Bot commented Aug 6, 2025

Uh oh!

NaluTripician commented Aug 6, 2025

Uh oh!

azure-pipelines Bot commented Aug 6, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

ananth7592 commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Closing issues

Uh oh!

kundadebdatta commented Aug 6, 2025

Uh oh!

NaluTripician commented Aug 6, 2025

Uh oh!

azure-pipelines Bot commented Aug 6, 2025

Uh oh!

NaluTripician commented Aug 6, 2025

Uh oh!

azure-pipelines Bot commented Aug 6, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

ananth7592 commented Jul 29, 2025 •

edited

Loading