Bulk Operations #1961
Unanswered
Gamingdevil
asked this question in
Q&A
Bulk Operations
#1961
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I have an issue when importing (relatively) large graphs; awaiting each action individually can take a long time.
So I was looking into batching; basically I fire off all the tasks and then await them all together at the end.
Sample code:
This works fine for relatively small batches, but when I increase the number of concurrent calls, the database will eventually respond with a 429:
I tried to look at the documentation, and found Bulk Executor, which is also what is recommended in this discussion from a few years ago, however, looking further into the documentation teaches us that this approach is deprecated. In addition, it is also not entirely clear to me how I should use this in conjunction with Gremlinq.
According to the Microsoft Documentation, the latest version of the SDK should support this with minimal effort, by basically just setting
AllowBulkExecution = true
in the CosmosClientOptions. If I understand it correctly, this will automatically batch and/or throttle the requests for me, which would of course be vastly preferable to having to do it myself.I've tried looking for a way to do this within the Gremlinq configuration, but I haven't been able to find anything in neither
ConfigureClientFactory
, norConfigureQuerySource
.So my question is: is there a good (and easy) way to do this using Gremlinq and am I simply missing the proper setting, or is it currently not supported? And if I should use one of the Microsoft implementations, is there a good code sample available somewhere that shows how to use this together with Gremlinq?
Beta Was this translation helpful? Give feedback.
All reactions