-
Notifications
You must be signed in to change notification settings - Fork 11.1k
Fix resource leak in FileBackedOutputStream to prevent file handle exhaustion #7986
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
…aks during threshold crossing. Added tests to verify proper handling of `FileOutputStream` in case of exceptions. This addresses resource leak issues identified in Issue google#5756.
} finally { | ||
if (!success && transfer != null) { | ||
try { | ||
transfer.close(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it makes sense to perform the cleanup in the existing catch (IOException e)
block? That should be the error case that we need to worry about. (And if we're worried about other cases, we could generalize it to also catch RuntimeException
.) That would also let us attach any exception from close
as a suppressed exception on e
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks @cpovirk I've put cleanup in the catch block now:
} catch (IOException e) {
if (transfer != null) {
try {
transfer.close();
} catch (IOException closeException) {
e.addSuppressed(closeException);
}
}
temp.delete();
throw e;
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
re. RuntimeException
I get what you mean, may i suggest documenting as a new issue to keep the scope clean.
Once merged can add something like:
FileBackedOutputStream: extend resource leak fix to RuntimeException
Current fix addresses IOException resource leaks during memory-to-file transition. Should extend to catch RuntimeException (SecurityException, system errors) for complete resource cleanup.
Simple change: catch (IOException | RuntimeException e)
RuntimeException scenarios are less common than IOException cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, I'm actually happy to ignore the RuntimeException
case here.
…per resource closure. Removed unnecessary success flag and streamlined the try-catch structure for better readability and reliability. This change enhances the robustness of file operations during threshold crossing.
* the FileOutputStream is properly managed even if an exception occurs. | ||
* | ||
* Note: Direct testing of the IOException scenario during write/flush is | ||
* challenging without mocking. This test verifies that normal operation |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a short summary of what the new tests cover beyond what the existing testThreshold
tests cover? I'm sure I can work it out, but you may know offhand :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The new tests are regression tests that were added alongside the resource management fix to verify normal operation works correctly.
testThresholdCrossing_ResourceManagement()
tests normal threshold crossing behavior through the memory-to-file transitiontestMultipleThresholdCrossings()
tests normal operation across repeated threshold crossings
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that threshold crossing in some sense is covered by testThreshold()
, but it looks like that covers only the case in which we have written exactly the number of bytes that fit before and then write some new bytes that immediately trigger the use of the file. Your new test writes a smaller number of bytes and then writes a large number at once, enough to take us from "not at the threshold" to "over the threshold." So that seems like a new thing.
For "multiple threshold crossings," I don't think that's the phrasing I'd use, but your test is covering a write that occurs after the write that crosses the threshold. I don't think we cover that yet, except in the singleByte
case, which doesn't take the same code path as you're covering.
Does that all sound right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that all sounds right!
For testThresholdCrossing_ResourceManagement()
: You've understood correctly. The existing testThreshold()
writes exactly the number of bytes that fit (reaching the threshold), then immediately writes more bytes. My new test covers a different scenario: writing a smaller amount first (40 bytes), then writing a large amount at once (30 bytes) that takes us from "not at threshold" to "over the threshold."
For testMultipleThresholdCrossings()
: This is indeed misleading phrasing. The test is actually covering writes that occur after the initial write that crosses the threshold. Once we've crossed the threshold and transitioned to file storage, this test verifies that continued writes to the file work correctly. As you noted, this scenario isn't covered except in the singleByte
case, which takes a different code path.
I'll update the PR to:
- Rename
testMultipleThresholdCrossings()
totestWriteAfterThresholdCrossing()
- Clarify the JavaDoc for both tests to accurately describe what they're testing
- Update the documentation to emphasize how
testThresholdCrossing_ResourceManagement()
differs from the existingtestThreshold()
Thanks for the careful review and clarification!
…y is `/sdcard`. This handling was added in feb83a1. It became unnecessary once [our minSdkVersion hit 16](#4011 (comment)), as it did years ago. (I was reminded of this while reviewing #7986 for #5756.) RELNOTES=n/a PiperOrigin-RevId: 811432723
…y is `/sdcard`. This handling was added in feb83a1. It became unnecessary once [our minSdkVersion hit 16](#4011 (comment)), as it did years ago. (I was reminded of this while reviewing #7986 for #5756.) RELNOTES=n/a PiperOrigin-RevId: 811449148
…hreshold crossing scenarios. Refactor existing tests to clarify resource management behavior when transitioning from memory to file storage. This includes renaming tests for better clarity and ensuring proper handling of writes after crossing the threshold.
Problem
FileBackedOutputStream leaks file handles when an IOException occurs during the transition from memory to file storage. The FileOutputStream created in the
update()
method is never closed ifwrite()
orflush()
fails, leading to resource exhaustion.Solution
Use proper exception handling with try-catch to ensure the FileOutputStream is closed if an exception occurs during the transition, while preserving it on success when it becomes the active output stream.
Changes
FileBackedOutputStream.update()
methodTesting
All existing tests pass. Added regression tests (
testThresholdCrossing_ResourceManagement
andtestMultipleThresholdCrossings
) to verify normal threshold-crossing behavior works correctly with the resource management improvements.Fixes #5756