Skip to content

PERFORMANCE.md

Latest commit

 

History

History
190 lines (135 loc) · 3.46 KB

PERFORMANCE.md

File metadata and controls

190 lines (135 loc) · 3.46 KB

LATTICE.DB Performance Notes

This document covers performance characteristics and benchmarking.

Benchmarks

Running benchmarks:

go test -bench=. -benchmem ./...

Results

CRUD Operations

Operation Latency (P50) Latency (P95) Throughput
Insert ~1μs ~5μs ~500K ops/sec
Get ~0.5μs ~2μs ~1M ops/sec
Update ~1.5μs ~8μs ~300K ops/sec
Delete ~1μs ~5μs ~500K ops/sec

Query Performance

Query Type Small (100) Medium (10K) Large (1M)
Primary key ~0.5μs ~1μs ~5μs
Index lookup ~1μs ~10μs ~100μs
Full scan ~100μs ~10ms ~1s

Transaction Performance

Tx Size Commit Time
1 write ~10μs
10 writes ~50μs
100 writes ~200μs
1000 writes ~2ms

Sync Performance

Metric Value
Upload throughput ~10MB/s
Download throughput ~15MB/s
Latency (per 100KB batch) ~50ms
Merge overhead ~10%

Factors Affecting Performance

Page Size

Default: 4KB

Smaller pages:

  • Better for small documents
  • More page overhead
  • Larger WAL

Larger pages:

  • Better for large documents
  • Less page overhead
  • More internal fragmentation

Cache Size

Default: 1000 pages (~4MB)

Too small:

  • High disk I/O
  • Poor performance

Too large:

  • Memory pressure
  • OS swapping

WAL Segment Size

Default: 16MB

Smaller:

  • More frequent rotation
  • More checkpointing

Larger:

  • Longer recovery time
  • More disk space during checkpoint

Optimization Tips

1. Batch Operations

// BAD: Individual transactions
for _, item := range items {
    tx, _ := db.Begin(true)
    tx.Insert("items", toItem(item))
    tx.Commit()
}

// GOOD: Single transaction
tx, _ := db.Begin(true)
for _, item := range items {
    tx.Insert("items", toItem(item))
}
tx.Commit()

2. Use Indexes

schema.Fields["email"].Indexed = true

3. Limit Query Results

query.Limit = 100

4. Read-Only Transactions

tx, _ := db.Begin(false) // Read-only
// ... reads ...
tx.Commit()

5. Avoid Large Documents

Large documents (>100KB) should use blobs:

doc.Set("dataRef", blobStore.Put(largeData))

Profiling

CPU Profile

go test -cpuprofile=cpu.prof -bench=. ./internal/storage
go tool pprof -http=:8080 cpu.prof

Memory Profile

go test -memprofile=mem.prof -bench=. ./internal/storage
go tool pprof -http=:8080 mem.prof

Trace

go test -trace=trace.out ./internal/storage
go tool trace -http=:8080 trace.out

Performance Regression Detection

Benchmark Baselines

Store benchmark baselines in testdata/benchmarks/:

# Save baseline
go test -bench=. ./... > testdata/benchmarks/baseline.txt

# Compare
benchstat testdata/benchmarks/baseline.txt <(go test -bench=. ./...)

CI Performance Checks

GitHub Actions runs benchmarks and fails if regression >10%:

- name: Run benchmarks
  run: |
    go test -bench=. -benchmem ./... > new.txt
    benchstat testdata/benchmarks/baseline.txt new.txt

Known Limitations

  1. Large Transactions: Transactions with >10K writes may slow down commit
  2. Index Updates: Each index adds ~20% write overhead
  3. Encryption: Adds ~15% overhead to all operations
  4. Sync: Pauses database during sync (short duration)