This directory contains working examples demonstrating HDF5 B-tree rebalancing strategies.
Use Case: Append-only workloads, small files, maximum write performance
go run ./defaultWhat it demonstrates:
- Default behavior (no rebalancing, like HDF5 C library)
- Fastest deletion (0% overhead)
- B-tree may become sparse after many deletions
Use Case: Batch deletion workloads, medium/large files (100-500MB)
go run ./lazyWhat it demonstrates:
- Lazy (batch) rebalancing mode
- 10-100x faster than immediate rebalancing
- Occasional pauses (100-500ms) during batch processing
- Tuning parameters: threshold, max delay, batch size
Use Case: Large files (>500MB), continuous operations, zero-pause requirement
go run ./incrementalWhat it demonstrates:
- Incremental (background) rebalancing mode
- ZERO user-visible pause (all rebalancing in background)
- Progress monitoring via callback
- Tuning parameters: budget, interval
Use Case: Unknown workloads, mixed operations, want auto-pilot mode
go run ./smartWhat it demonstrates:
- Smart (auto-tuning) rebalancing mode
- Automatic workload pattern detection
- Automatic mode selection and switching
- Explainability (confidence scores, reasoning)
| Example | Mode | Overhead | Pause Time | Use Case |
|---|---|---|---|---|
| default/ | None | 0% | None | Append-only, small files |
| lazy/ | Lazy (batch) | ~2% | 100-500ms batches | Batch deletions |
| incremental/ | Incremental (background) | ~4% | None (background) | Large files, continuous ops |
| smart/ | Smart (auto) | ~6% | Varies | Unknown workloads |
# Run each example individually
go run ./default
go run ./lazy
go run ./incremental
go run ./smart
# Or build all examples
go build ./...
# Or run all examples at once
for dir in default lazy incremental smart; do
echo "Running $dir..."
go run ./$dir
echo
doneOutput Files:
default-output.h5- File with no rebalancinglazy-output.h5- File with lazy rebalancingincremental-output.h5- File with incremental rebalancingsmart-output.h5- File with smart rebalancing
All files are valid HDF5 and can be opened with:
h5dump(C library tool)- Python
h5py - This library's
hdf5.Open()
- Start with default (
./default) unless you know you need rebalancing - Use lazy (
./lazy) for batch deletion workloads (10-100x faster) - Use incremental (
./incremental) for large files where pauses are unacceptable - Use smart (
./smart) only if workload is truly unknown
- Performance Tuning Guide: Comprehensive guide with benchmarks, recommendations, troubleshooting
- Rebalancing API Reference: Complete API documentation
- FAQ: Common questions
Version: v0.11.3-beta Last Updated: 2025-11-02