Skip to content

Commit f6999e0

Browse files
committed
timeseries db and fsync
1 parent d0330d1 commit f6999e0

File tree

2 files changed

+27
-0
lines changed

2 files changed

+27
-0
lines changed

prometheus/tips.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
- [By/Without and Ignoring/On](#bywithout-and-ignoringon)
1313
- [Authentication and encryption for Prometheus and its exporters](#authentication-and-encryption-for-prometheus-and-its-exporters)
1414
- [Prometheus and fsync](#prometheus-and-fsync)
15+
- [`min-block-duration` and `max-block-duration`](#min-block-duration-and-max-block-duration)
1516

1617
## `avg_over_time`
1718

@@ -173,6 +174,21 @@ What fsync means in Prometheus:
173174

174175
I found a link: <https://groups.google.com/g/prometheus-users/c/Oy1qI3Og9ww> but the logic seems to be changed in the current source code (v3.8.0).
175176

177+
I use [deepwiki](https://deepwiki.com/search/when-fsync-is-performed_cbf97034-616c-403a-8ff3-de92da849479) to check how fsync is performed in Prometheus.
178+
- The WAL implementation performs fsync in several scenarios:
179+
- During segment rotation: When creating a new segment, the previous segment is fsynced asynchronously via `w.fsync(prev)`.
180+
- During close operation: The active segment is fsynced before closing.
181+
- Manual sync: Available via `Sync()` method for testing.
182+
- During compaction, temporary block directories are fsynced before being made visible.
183+
- Both the chunk writer and head chunk disk mapper perform fsync when finalizing files:
184+
- Chunk writer calls `tf.Sync()` during `finalizeTail()`.
185+
- Head chunk mapper calls `cdm.curFile.Sync()` during `finalizeCurFile()`.
186+
187+
According to the documentation, WAL syncs typically happen every two hours during normal operation, though the actual fsync frequency is higher during active writes due to the WAL's design.
188+
189+
> [!note]
190+
> The WAL implementation uses asynchronous fsync for segment rotation to avoid blocking writes, while compaction and chunk operations use synchronous fsync to ensure data integrity before making files visible. The WAL also tracks fsync duration as a metric for monitoring performance impact.
191+
176192
## `min-block-duration` and `max-block-duration`
177193

178194
In Prometheus, the `--storage.tsdb.min-block-duration` and `--storage.tsdb.max-block-duration` flags control the time range of data stored in individual, on-disk data blocks

victoriametrics/notes.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Notes
2+
3+
## fsync
4+
5+
I use [deepwiki](https://deepwiki.com/search/when-fsync-is-performed_093b7c38-943b-4285-bd4e-641a4279f5c8).
6+
7+
In Victoriametrics, fsync is commonly used during:
8+
- Persistent queue operations when finalizing chunks.
9+
- Backup operations when copying parts between filesystems.
10+
- Storage partitions.
11+
- MergeSet, part header.

0 commit comments

Comments
 (0)