feat: add time-based rotation to RotatingWriter (#136) (#179)

* feat: add time-based rotation to RotatingWriter (#136)

RotatingWriter now rotates segments based on wall-clock time in addition
to file size. Rotation triggers when *either* condition is met:

- File exceeds `max_file_size` bytes (existing behavior)
- A wall-clock-aligned time boundary is crossed (new)

The default rotation period is 60 seconds, aligned to round minute
boundaries (e.g. if the writer starts at 14:03:22, the first rotation
fires at 14:04:00). This is configurable via the builder:

    RotatingWriter::builder()
    .rotation_period(Duration::from_secs(300))  // every 5 min

Time-aligned segments produce clean S3 key paths when using the
`worker-s3` feature (keys already use `{YYYY-MM-DD}/{HHMM}` bucketing).

Edge cases:
- Time rotation is skipped when no real events have been written
  (avoids empty sealed segments)
- `single_file()` disables time rotation (Duration::MAX)
- Eviction budget is respected with frequent time rotations

Closes #136

* feat: add time-based rotation to RotatingWriter (#136)

RotatingWriter now rotates segments based on wall-clock time in addition
to file size. Rotation triggers when *either* condition is met:

- File exceeds `max_file_size` bytes (existing behavior)
- A wall-clock-aligned time boundary is crossed (new)

The default rotation period is 60 seconds, aligned to round minute
boundaries (e.g. if the writer starts at 14:03:22, the first rotation
fires at 14:04:00). This is configurable via the builder:

    RotatingWriter::builder()
    .rotation_period(Duration::from_secs(300))  // every 5 min

Time-aligned segments produce clean S3 key paths when using the
`worker-s3` feature (keys already use `{YYYY-MM-DD}/{HHMM}` bucketing).

Edge cases:
- Time rotation is skipped when no real events have been written
  (avoids empty sealed segments)
- `single_file()` disables time rotation (Duration::MAX)
- Eviction budget is respected with frequent time rotations

Closes #136

* PR feedback

Author: rcoh

Cargo.lock

@@ -38,6 +38,7 @@ flate2 = { version = "1" }
 time = { version = "0.3", features = ["formatting", "macros"], optional = true }
 metrique-writer = "0.1"
 metrique = { version = "0.1.23", features = ["local-format"] }
+metrique-timesource = "0.1"
 
 [features]
 analysis = []
@@ -50,7 +51,9 @@ assert2 = { workspace = true }
 criterion = "0.5"
 clap = { version = "4", features = ["derive"] }
 hdrhistogram = "7"
+metrique-timesource = { version = "0.1", features = ["custom-timesource", "tokio"] }
 metrique-writer = { version = "0.1", features = ["test-util"] }
+tokio = { version = "1.51.0", features = ["test-util"] }
 proptest = "1"
 tempfile = "3"
 tracing-subscriber = { version = "0.3", features = ["env-filter"] }

dial9-tokio-telemetry/Cargo.toml

@@ -32,6 +32,7 @@ fn main() -> std::io::Result<()> {
         .base_path("/tmp/my_traces/trace.bin")
         .max_file_size(1024 * 1024)      // rotate after 1 MiB per file
         .max_total_size(5 * 1024 * 1024) // keep at most 5 MiB on disk
+        // .rotation_period(std::time::Duration::from_secs(300)) // optional: rotate every 5 min (default: 60 s)
         .build()?;
 
     let mut builder = tokio::runtime::Builder::new_multi_thread();
@@ -49,6 +50,8 @@ fn main() -> std::io::Result<()> {
 
 Events are 6–16 bytes on the wire, and a typical request generates ~20–35 bytes of trace data (a few poll events plus park/unpark). At 10k requests/sec that's well under 1 MB/s — `RotatingWriter` caps total disk usage so you can leave it running indefinitely. Typical CPU overhead is under 5%.
 
+Segments rotate on size *or* time, whichever comes first. Time boundaries are wall-clock-aligned (e.g. a 60 s period rotates at the top of each minute), which produces clean S3 key paths when using the `worker-s3` feature.
+
 ## Can I use this in prod?
 dial9-tokio-telemetry is designed for always-on production use, but it's still early software. Measure overhead and validate behavior in your environment before deploying to production.
 
@@ -202,7 +205,7 @@ handle.disable();
 
 ### Writers
 
-`RotatingWriter` rotates files and evicts old ones to stay within a total size budget. For quick experiments, `RotatingWriter::single_file(path)` writes a single file with no rotation.
+`RotatingWriter` rotates files based on size and time, and evicts old ones to stay within a total size budget. By default, segments rotate every 60 seconds (wall-clock-aligned) or when they exceed `max_file_size`, whichever comes first. For quick experiments, `RotatingWriter::single_file(path)` writes a single file with no rotation.
 
 ### Analyzing traces