[DRAFT] Align read window with part boundaries#1671
Closed
vladem wants to merge 5 commits into
Closed
Conversation
vladem
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
vladem
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
vladem
changed the title
vladem
force-pushed
the
aligned-read-window-end-2
branch
from
dfdb17e to
a043121
Compare
vladem
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
vladem
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
vladem
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
vladem
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
vladem
force-pushed
the
aligned-read-window-end-2
branch
from
570b57c to
f44c247
Compare
vladem
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
vladem
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
vladem
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
vladem
changed the title
Signed-off-by: Vlad Volodkin <vlaad@amazon.co.uk>
Signed-off-by: Vlad Volodkin <vlaad@amazon.co.uk>
Signed-off-by: Vlad Volodkin <vlaad@amazon.co.uk>
Signed-off-by: Vlad Volodkin <vlaad@amazon.co.uk>
vladem
force-pushed
the
aligned-read-window-end-2
branch
from
f861d7c to
ef5de01
Compare
vladem
had a problem deploying
to
PR integration tests
GitHub Actions
Failure
vladem
had a problem deploying
to
PR benchmarks
GitHub Actions
Failure
vladem
had a problem deploying
to
PR benchmarks
GitHub Actions
Failure
Signed-off-by: Vlad Volodkin <vlaad@amazon.co.uk>
vladem
temporarily deployed
to
PR integration tests
GitHub Actions
Inactive
Contributor
Author
|
closing in favour of #1707 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Memory limiter currently doesn’t account for the “rounded up” memory by CRT, which means up to 8MiB of unaccounted memory per file handle. Based on this flaw in the memory model, limiter may choose larger windows, while there is actually no free memory for those. We should “round up” the read window to the next part boundary on our side.
While working on #1618 we've faced the issue that
request_start_offsetis not always available to the code picking the newread_window_end_offset. Specifically when cache is usedBackpressureControllerhas no information about the S3 request start offset.This PR attempts to move the code picking
read_window_end_offsetclose to the client and thus makerequest_start_offsetavailable to it.Does this change impact existing behavior?
No.
Does this change need a changelog entry? Does it require a version change?
No.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).