Skip to content

Conversation

lmcrean
Copy link
Contributor

@lmcrean lmcrean commented Sep 14, 2025

Problem

FileBackedOutputStream leaks file handles when an IOException occurs during the transition from memory to file storage. The FileOutputStream created in the update() method is never closed if write() or flush() fails, leading to resource exhaustion.

Solution

Use proper exception handling with try-catch to ensure the FileOutputStream is closed if an exception occurs during the transition, while preserving it on success when it becomes the active output stream.

Changes

  • Added proper resource management in FileBackedOutputStream.update() method
  • FileOutputStream declared outside try block for proper cleanup scope
  • Stream is closed only when an exception occurs (with suppressed exceptions)
  • Exception handling preserves original IOException while cleaning up resources
  • No changes to public API or behavior

Testing

# Run specific tests
./mvnw test -Dtest=FileBackedOutputStreamTest -pl guava-tests

# Run with coverage for the affected class
./mvnw test -Dtest=FileBackedOutputStream* -pl guava-tests

All existing tests pass. Added regression tests (testThresholdCrossing_ResourceManagement and testMultipleThresholdCrossings) to verify normal threshold-crossing behavior works correctly with the resource management improvements.

Fixes #5756

…aks during threshold crossing. Added tests to verify proper handling of `FileOutputStream` in case of exceptions. This addresses resource leak issues identified in Issue google#5756.
} finally {
if (!success && transfer != null) {
try {
transfer.close();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it makes sense to perform the cleanup in the existing catch (IOException e) block? That should be the error case that we need to worry about. (And if we're worried about other cases, we could generalize it to also catch RuntimeException.) That would also let us attach any exception from close as a suppressed exception on e.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @cpovirk I've put cleanup in the catch block now:

        } catch (IOException e) {
        if (transfer != null) {
          try {
            transfer.close();
          } catch (IOException closeException) {
            e.addSuppressed(closeException);
          }
        }
        temp.delete();
        throw e;
      }

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re. RuntimeException I get what you mean, may i suggest documenting as a new issue to keep the scope clean.

Once merged can add something like:

FileBackedOutputStream: extend resource leak fix to RuntimeException

Current fix addresses IOException resource leaks during memory-to-file transition. Should extend to catch RuntimeException (SecurityException, system errors) for complete resource cleanup.

Simple change: catch (IOException | RuntimeException e)

RuntimeException scenarios are less common than IOException cases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I'm actually happy to ignore the RuntimeException case here.

@cpovirk cpovirk added type=defect Bug, not working as expected package=io P3 no SLO labels Sep 14, 2025
@cpovirk cpovirk self-assigned this Sep 14, 2025
…per resource closure. Removed unnecessary success flag and streamlined the try-catch structure for better readability and reliability. This change enhances the robustness of file operations during threshold crossing.
* the FileOutputStream is properly managed even if an exception occurs.
*
* Note: Direct testing of the IOException scenario during write/flush is
* challenging without mocking. This test verifies that normal operation
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a short summary of what the new tests cover beyond what the existing testThreshold tests cover? I'm sure I can work it out, but you may know offhand :)

Copy link
Contributor Author

@lmcrean lmcrean Sep 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new tests are regression tests that were added alongside the resource management fix to verify normal operation works correctly.

  • testThresholdCrossing_ResourceManagement() tests normal threshold crossing behavior through the memory-to-file transition
  • testMultipleThresholdCrossings() tests normal operation across repeated threshold crossings

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that threshold crossing in some sense is covered by testThreshold(), but it looks like that covers only the case in which we have written exactly the number of bytes that fit before and then write some new bytes that immediately trigger the use of the file. Your new test writes a smaller number of bytes and then writes a large number at once, enough to take us from "not at the threshold" to "over the threshold." So that seems like a new thing.

For "multiple threshold crossings," I don't think that's the phrasing I'd use, but your test is covering a write that occurs after the write that crosses the threshold. I don't think we cover that yet, except in the singleByte case, which doesn't take the same code path as you're covering.

Does that all sound right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that all sounds right!

For testThresholdCrossing_ResourceManagement(): You've understood correctly. The existing testThreshold() writes exactly the number of bytes that fit (reaching the threshold), then immediately writes more bytes. My new test covers a different scenario: writing a smaller amount first (40 bytes), then writing a large amount at once (30 bytes) that takes us from "not at threshold" to "over the threshold."

For testMultipleThresholdCrossings(): This is indeed misleading phrasing. The test is actually covering writes that occur after the initial write that crosses the threshold. Once we've crossed the threshold and transitioned to file storage, this test verifies that continued writes to the file work correctly. As you noted, this scenario isn't covered except in the singleByte case, which takes a different code path.

I'll update the PR to:

  1. Rename testMultipleThresholdCrossings() to testWriteAfterThresholdCrossing()
  2. Clarify the JavaDoc for both tests to accurately describe what they're testing
  3. Update the documentation to emphasize how testThresholdCrossing_ResourceManagement() differs from the existing testThreshold()

Thanks for the careful review and clarification!

copybara-service bot pushed a commit that referenced this pull request Sep 25, 2025
…y is `/sdcard`.

This handling was added in feb83a1. It became unnecessary once [our minSdkVersion hit 16](#4011 (comment)), as it did years ago.

(I was reminded of this while reviewing #7986 for #5756.)

RELNOTES=n/a
PiperOrigin-RevId: 811432723
copybara-service bot pushed a commit that referenced this pull request Sep 25, 2025
…y is `/sdcard`.

This handling was added in feb83a1. It became unnecessary once [our minSdkVersion hit 16](#4011 (comment)), as it did years ago.

(I was reminded of this while reviewing #7986 for #5756.)

RELNOTES=n/a
PiperOrigin-RevId: 811449148
…hreshold crossing scenarios. Refactor existing tests to clarify resource management behavior when transitioning from memory to file storage. This includes renaming tests for better clarity and ensuring proper handling of writes after crossing the threshold.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P3 no SLO package=io type=defect Bug, not working as expected

Projects

None yet

Development

Successfully merging this pull request may close these issues.

unreleased resource : streams

2 participants