Disable transformation of non-ascii characters on storing to CosmosDB #41144
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR is to update the
json.dumps()
function to accept theensure_ascii=False
parameter so we can properly send unicode characters to CosmosDB, which it does support storing. If this parameter is disabled, trying to send strings with a unicode character results in the string being converted to an escaped unicode sequence to comply with being an ASCII character, and that results in transformations like\\ud83d\\udf1f
for 🌟, which results in an error in Cosmos when trying to store these values:I'm assuming it's because of the escape sequence and Python interpreter do not play nice. I stumbled upon this issue when looking for answers, and decided to try implementing the solution proposed by @lidingshan in my local run, and it seems to work. To avoid an indefinite monkeypatch for my team, I am creating this PR to get this investigated/fixed.
Not sure if this is the best/long term solution for properly ensuring the unicode characters are stored in Cosmos, but it seems that Cosmos supports that storage, so idk why we would try and force the ASCII conversion here.
If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
All SDK Contribution checklist:
General Guidelines and Best Practices
Testing Guidelines