v0.4.0

Added

• Add latency stats (histogram and quantile) to glob and getting file size IO operation (#53)

• Record access for duckdb internal external file cache, and corresponding table function to query and emit status (#50, #52)

Fixed

• Fixed missing stats report for multiple IO operations; previously only open operation is correctly reported (#49)

v0.3.1

0.3.1

Added

• Add a SQL function to list all registered filesystems (#38)

Fixed

• Fix bug that httpfs filesystems are registered twice, one as unwrapped raw filesystem instance, one wrapped in observefs

v0.3.0

0.3.0

Added

• Add read request size metrics (#30)

Fixed

• Fix HTTP request stats report (#35)

• Fix quantile calculation when there're small number of data points (#36)

v0.2.0

Added

• Support wrapping ANY duckdb compatible filesystems (#24)

Improved

• Rewrite quantile estimation for better accuracy (#17, #18, #19)

• Refine display output if there's no interested IO operations happening in the current filesystem (#25)

What's Changed

• Basic observability filesystem setup by @dentiny in #1
• Wrap filesystem by @dentiny in #2
• Add histogram and rename extension by @dentiny in #3
• Basic metrics collection implementation by @dentiny in #4
• Fix config yaml by @dentiny in #5
• Integrate metrics for file open and fix by @dentiny in #6
• Implement a RAII latency guard by @dentiny in #7
• [Utils] Get bucket name from filepath by @dentiny in #8
• Config devcontainer by @dentiny in #9
• Collect bucket-wise latency metrics by @dentiny in #10
• Rename extension to observefs by @dentiny in #11
• Add metrics for read, compatible with httpfs by @dentiny in #12
• Fix segfault by @dentiny in #13
• Upgrade to duckdb v1.4 by @dentiny in #15
• Add support for List IO operation and corresponding latency metrics by @peterxcli in #14
• Upgrade duckdb submodule to v1.4 by @dentiny in #16
• Implement quantile by @dentiny in #17
• Implement small scale quantiles by @dentiny in #18
• Integrate quantile into output string by @dentiny in #19
• Refactor latency initialization in OperationLatencyCollector to use a structured approach with LatencyHeuristic by @peterxcli in #20
• Add contributing doc by @dentiny in #22
• Add README.md for ObserveFS extension by @peterxcli in #21
• Add readme page by @dentiny in #23
• Sql to wrap filesystem by @dentiny in #24
• Refine no interested IO operations output by @dentiny in #25
• Validate no repeated registration for one filesystem by @dentiny in #26
• Add changelog for v0.2 by @dentiny in #27

New Contributors

• @dentiny made their first contribution in #1
• @peterxcli made their first contribution in #14

Full Changelog: https://github.com/dentiny/duckdb-filesystem-observability/commits/v0.2.0