[Internal] HPK: Fixes EPK Normalization problem of Hierarchical Partition Keys by consuming new Range Comparators

[ananth7592]

Fix EPK normalization for HPK containers and introduce length-aware range comparators

Design reference: Method 3 Smart Range Comparer
from https://microsoft-my.sharepoint.com/:w:/p/amudumba/EbtYRzMDKHZPkqV_Q4VVTqgBVjIhgnSG8X_CGbqu21W_bQ?e=KJwxwy

Pull Request Template

Description

- Addressed the issue of unnormalized Effective Partition Key (EPK) ranges for containers with Hierarchical Partition Keys (HPK) enabled.
- Integrated LengthAwareMinComparator and LengthAwareMaxComparator from the Direct package, enabling advanced, length-aware comparison logic for partition key range overlap detection.
- The LengthAwareMin/Maxcomparators will be enabled by default
- An env variable called AZURE_COSMOS_USE_LENGTH_AWARE_RANGE_COMPARATOR is created for emergency disablement of the LengthAWareMin/MAx comparators.
- Added comprehensive tests to validate the new CollectionRoutingMap logic, including edge cases for HPK and EPK normalization.

Safety:

- All changes are covered by new and existing unit tests to ensure backward compatibility and correctness for both HPK and non-HPK containers.
- An env variable called AZURE_COSMOS_USE_LENGTH_AWARE_RANGE_COMPARATOR is introduced which can be set to False in case of emergencies for falling back to older simple Min/max Range comparators.

Testing:

• Tested combinations of:

Partial and full partition key values.
Two- and three-level partition key hierarchies.
Backend range configurations including hybrid and fully-specified EPKs.

• Covered edge and boundary scenarios to ensure correctness and resilience.

Impact:

Reduced operational complexity by not requiring changes to how EPKs are stored.
Backward compatibility for clients, ensuring a seamless experience for both new and legacy clients.
Minimal disruption during rollout, used a feature flag for emergency rollback.

Type of change

Please delete options that are not relevant.

• [*] Bug fix (non-breaking change which fixes an issue)
• [] New feature (non-breaking change which adds functionality)
• [] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [] This change requires a documentation update

Closing issues

To automatically close an issue: closes #IssueNumber

[github-actions]

All good!

[kirankumarkolli]

Few more to follow-up

• Can the comparer be made length aware might help to not make any state changes
• PKDefinition passing: Ideal to not pass it (fragile) and can be rooted to CollectionRoutingMap for simplicity