Skip to content

Commit c6791d1

Browse files
danielwlzdanielwlz
committed
Add QMDB code
1 parent a660beb commit c6791d1

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

120 files changed

+23496
-8
lines changed

‎.github/workflows/benchmarks.yml‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
name: Benchmark
2+
3+
on:
4+
push:
5+
pull_request:
6+
7+
env:
8+
CARGO_TERM_COLOR: always
9+
RUST_BACKTRACE: 1
10+
ENTRY_COUNT: 4000000
11+
12+
jobs:
13+
benchmark_qmdb:
14+
name: QMDB Benchmark
15+
runs-on: ubuntu-latest
16+
17+
steps:
18+
- uses: actions/checkout@v4
19+
20+
- uses: actions/cache@v4
21+
with:
22+
path: |
23+
~/.cargo/bin/
24+
~/.cargo/registry/index/
25+
~/.cargo/registry/cache/
26+
~/.cargo/git/db/
27+
target/
28+
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.toml', '**/Cargo.lock') }}-${{ github.sha }}-bench
29+
restore-keys: |
30+
${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.toml', '**/Cargo.lock') }}-${{ github.sha }}
31+
${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.toml', '**/Cargo.lock') }}
32+
${{ runner.os }}-cargo-
33+
34+
- name: Install dependencies
35+
run: |
36+
sudo apt-get install g++ linux-libc-dev libclang-dev unzip libjemalloc-dev make -y
37+
sudo apt-get install -y time
38+
39+
- name: Build Release (QMDB)
40+
run: cargo build --bin speed --release
41+
42+
- name: Generate randsrc.dat
43+
run: head -c 10M </dev/urandom > randsrc.dat
44+
45+
- name: Log runner stats
46+
run: |
47+
echo "Runner stats:"
48+
echo "CPU: $(nproc)"
49+
echo "RAM: $(echo "scale=2; $(free -m | awk '/^Mem:/{print $2}') / 1024" | bc) GB"
50+
echo "Disk space: $(df -h . | awk 'NR==2{print $4}')"
51+
52+
- name: Run QMDB benchmark
53+
run: |
54+
set -o pipefail
55+
ulimit -n 65535
56+
/usr/bin/time -v cargo run --bin speed --release -- --entry-count $ENTRY_COUNT --hover-interval 1 --hover-recreate-block 1 --hover-write-block 1 --tps-blocks 1 --output-filename qmdb_results.json 2> >(tee qmdb_results.txt)
57+
58+
- name: Parse and store QMDB benchmark results
59+
run: |
60+
parse_results() {
61+
local results_file="qmdb_results.txt"
62+
local json_file="qmdb_benchmark.json"
63+
64+
local peak_mem_kb=$(grep "Maximum resident set size" $results_file | awk '{print $6}')
65+
local peak_mem_gb=$(echo "scale=2; $peak_mem_kb / 1048576" | bc)
66+
local user_time=$(grep "User time" $results_file | awk '{print $4}')
67+
local sys_time=$(grep "System time" $results_file | awk '{print $4}')
68+
69+
echo "{
70+
\"peak_memory_gb\": $peak_mem_gb,
71+
\"user_time_seconds\": $user_time,
72+
\"system_time_seconds\": $sys_time
73+
}" > $json_file
74+
}
75+
76+
parse_results
77+
78+
- name: Upload QMDB benchmark results as artifact
79+
uses: actions/upload-artifact@v4
80+
with:
81+
name: qmdb-benchmark-results
82+
path: qmdb_results.json
83+
84+
- name: Parse results.json for benchmark-action
85+
run: |
86+
python3 bench/results2benchmark.py qmdb_results.json benchmark-data.json
87+
88+
- name: Store benchmark result
89+
uses: benchmark-action/github-action-benchmark@v1
90+
if: github.event_name != 'pull_request'
91+
with:
92+
tool: 'customBiggerIsBetter'
93+
output-file-path: benchmark-data.json
94+
fail-on-alert: true
95+
# GitHub API token to make a commit comment
96+
github-token: ${{ secrets.GITHUB_TOKEN }}
97+
# Enable alert commit comment
98+
comment-on-alert: true
99+
# alert-comment-cc-users: '@USER'

‎.github/workflows/build.yml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
name: Build
2+
3+
on:
4+
push:
5+
pull_request:
6+
7+
env:
8+
CARGO_TERM_COLOR: always
9+
RUST_BACKTRACE: 1
10+
11+
jobs:
12+
benchmark:
13+
name: Build debug & release
14+
runs-on: ubuntu-latest
15+
16+
steps:
17+
- uses: actions/checkout@v4
18+
19+
- uses: actions/cache@v4
20+
with:
21+
path: |
22+
~/.cargo/bin/
23+
~/.cargo/registry/index/
24+
~/.cargo/registry/cache/
25+
~/.cargo/git/db/
26+
target/
27+
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.toml', '**/Cargo.lock') }}-${{ github.sha }}
28+
restore-keys: |
29+
${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.toml', '**/Cargo.lock') }}-
30+
${{ runner.os }}-cargo-
31+
32+
- name: Install dependencies
33+
run: |
34+
sudo apt-get install g++ linux-libc-dev libclang-dev unzip libjemalloc-dev make -y
35+
sudo apt-get install -y time
36+
37+
- name: Build Debug
38+
run: cargo build --verbose
39+
40+
- name: Build Release
41+
run: cargo build --release --verbose

‎.github/workflows/tests.yml‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
name: Tests
2+
3+
on:
4+
push:
5+
pull_request:
6+
7+
env:
8+
CARGO_TERM_COLOR: always
9+
RUST_BACKTRACE: 1
10+
jobs:
11+
benchmark:
12+
name: Unit tests
13+
runs-on: ubuntu-latest
14+
15+
steps:
16+
- uses: actions/checkout@v4
17+
18+
- uses: actions/cache@v4
19+
with:
20+
path: |
21+
~/.cargo/bin/
22+
~/.cargo/registry/index/
23+
~/.cargo/registry/cache/
24+
~/.cargo/git/db/
25+
target/
26+
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.toml', '**/Cargo.lock') }}-${{ github.sha }}-tests
27+
restore-keys: |
28+
${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.toml', '**/Cargo.lock') }}-${{ github.sha }}
29+
${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.toml', '**/Cargo.lock') }}-
30+
${{ runner.os }}-cargo-
31+
32+
- name: Install dependencies
33+
run: |
34+
sudo apt-get install g++ linux-libc-dev libclang-dev unzip libjemalloc-dev make -y
35+
sudo apt-get install -y time
36+
37+
- name: Build Debug
38+
run: cargo build --verbose
39+
40+
- name: Run unit tests
41+
run:
42+
RUSTFLAGS="-Awarnings" cargo test --workspace
43+
continue-on-error: true

‎.gitignore‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Generated by Cargo
2+
# will have compiled files and executables
3+
debug/
4+
target/
5+
6+
# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
7+
# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
8+
Cargo.lock
9+
10+
# These are backup files generated by rustfmt
11+
**/*.rs.bk
12+
13+
# MSVC Windows builds of rustc generate these, which store debugging information
14+
*.pdb
15+
16+
# RustRover
17+
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
18+
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
19+
# and can be added to the global gitignore or merged into this file. For a more nuclear
20+
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
21+
#.idea/
22+
*.swp
23+
24+
25+
/qmdb/DataTree
26+
/TestGenV1
27+
/ADS
28+
.DS_Store
29+
.vscode
30+
**/ADS
31+
**/ADS0
32+
**/TestGenV1
33+
**/RocksDB
34+
**/MDBX
35+
randsrc.dat
36+
37+
results.json
38+
qmdb_benchmark.json
39+
qmdb_results.txt
40+
qmdb_results.json

‎CONTRIBUTING.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Contributing to QMDB
2+
3+
## Setting Expectations
4+
5+
External contributors are encouraged to submit issues and pull requests to this repository. That being said, not all issues will be addressed nor pull requests merged (even if well-intentioned).
6+
7+
The QMDB Library provides reliable and high-performance verifiable database and contributions that do not advance these goals may not be accepted. This could include (but is not limited to) replacing code with external dependencies, implementing optional functionality, and/or introducing algorithms that substantially increase complexity to provide marginal performance improvements. On the other hand, adding more tests and benchmarks is almost always welcome!
8+
9+
## Style
10+
11+
This repository uses the default cargo and clippy formatting rules for `.rs` files, treating warnings as errors. To check linting, run:
12+
13+
```bash
14+
cargo clippy --all-targets --all-features -- -D warnings
15+
cargo fmt --all -- --check
16+
```
17+
18+
To fix linting automatically, run:
19+
20+
```bash
21+
cargo fmt --all
22+
```
23+
24+
## Releases
25+
26+
Releases are automatically published to `cargo` by [GitHub Actions](.github/workflows/publish.yml) whenever a version update is merged into the `main` branch.
27+
28+
To increment the patch version of all crates (and update the corresponding minimum required version in `workspace.dependencies`), run:
29+
30+
```bash
31+
./scripts/bump_versions.sh
32+
```
33+
34+
## Licensing and Copyright
35+
36+
You agree that any work submitted to this repository shall be dual-licensed under the included [Apache 2.0](./LICENSE-APACHE) and [MIT](./LICENSE-MIT) licenses, without any additional terms or conditions. Additionally, you agree to release your copyright interest in said work to the public domain, such that anyone is free to use, modify, and distribute your contributions without restriction.
37+
38+
## Support
39+
40+
Looking to discuss a potential contribution or get feedback? Reach out on [GitHub Discussions](https://github.com/LayerZero-Labs/qmdb/discussions)!

‎Cargo.toml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
[workspace]
2+
resolver = "2"
3+
members = [
4+
"hpfile",
5+
"qmdb",
6+
"bench",
7+
]
8+

0 commit comments

Comments
 (0)