Skip to content

Commit 9bacfe3

Browse files
authored
Use os.fdatasync before replacing temp files (#261)
* Use `os.fdatasync` before replacing temp files According to https://stackoverflow.com/a/67905617, flushing/closing may not be enough. * changelog
1 parent ef5fde8 commit 9bacfe3

2 files changed

Lines changed: 18 additions & 5 deletions

File tree

CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## Unreleased
99

10+
### Fixed
11+
12+
- Ensure filesystem syncs data from temp file before replacing with permanent file.
13+
1014
## [v1.8.1](https://github.com/allenai/cached_path/releases/tag/v1.8.1) - 2025-12-15
1115

1216
- Allow `google-cloud-storage` version 3.x.

cached_path/cache_file.py

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -31,16 +31,25 @@ def __enter__(self):
3131
return self.temp_file
3232

3333
def __exit__(self, exc_type, exc_value, traceback):
34-
self.temp_file.close()
34+
del exc_type, traceback
35+
3536
if exc_value is None:
37+
# Ensure all data is written to disk.
38+
self.temp_file.flush()
39+
if hasattr(os, "fdatasync"): # only available on linux
40+
os.fdatasync(self.temp_file) # type: ignore
41+
self.temp_file.close()
42+
3643
# Success.
3744
logger.debug(
3845
"Renaming temp file %s to cache at %s", self.temp_file.name, self.cache_filename
3946
)
4047
# Rename the temp file to the actual cache filename.
4148
os.replace(self.temp_file.name, self.cache_filename)
4249
return True
43-
# Something went wrong, remove the temp file.
44-
logger.debug("removing temp file %s", self.temp_file.name)
45-
os.remove(self.temp_file.name)
46-
return False
50+
else:
51+
# Something went wrong, remove the temp file.
52+
logger.debug("removing temp file %s", self.temp_file.name)
53+
self.temp_file.close()
54+
os.remove(self.temp_file.name)
55+
return False

0 commit comments

Comments
 (0)