Refinement Of Maximum Chunk Size Handling #49
Description
Is your feature request related to a problem? Please describe.
Previously we had issues with maximum chunk size being reached, and subsequently downloading things from MinIO failed. @xiegeo introduced a fix that serves as an immediate resolution to this problem.
Describe the solution you'd like
While the problem is solved for now, there is room for improvement that allows a configurable chunk size. However with a configurable chunk size comes the problem that someone may configure a chunk size too large, thus we would need some method of detecting too large chunk sizes and either automatically falling back, or erroring out.
Additionally the "maximum chunk size" is actually determined by a value during the initialization of a TemporalX server. In our configuration package I have an upcoming PR to allow controlling the two types of this size:
- Max receive message size (default to 4MB)
- Max send message size (default to
math.MaxInt32).
Additional Context
As the label says, this is pretty low priority since the issue at hand is solved, and there are other improvements to the codebase that are more desirable.