Skip to content

Commit e6099aa

Browse files
authored
Merge pull request #1 from marcomq/dev
add sub-ms counter and python wrapper
2 parents 70aa90e + aa9898e commit e6099aa

16 files changed

Lines changed: 821 additions & 95 deletions

File tree

.github/workflows/benchmark-python.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@ jobs:
6666
run: |
6767
pip install uuid6 || true
6868
pip install uuid7 || true
69+
pip install uuid-utils || true
6970
pip install fastuuid7 || true
7071
7172
- name: Display system information

.github/workflows/ci.yml

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,11 @@ on:
55
branches: [ "main", "master" ]
66
pull_request:
77
branches: [ "main", "master" ]
8+
workflow_dispatch:
89

910
env:
1011
CARGO_TERM_COLOR: always
12+
SUB_MS_DIAGNOSTIC_SAMPLES: "5000"
1113

1214
jobs:
1315
test:
@@ -47,3 +49,39 @@ jobs:
4749
run: cargo test --verbose --target i686-unknown-linux-gnu
4850
- name: Run bench
4951
run: cargo bench
52+
53+
sub-ms-diagnostic:
54+
name: Sub-ms Diagnostic ${{ matrix.os }}
55+
runs-on: ${{ matrix.os }}
56+
strategy:
57+
fail-fast: false
58+
matrix:
59+
os: [ubuntu-latest, macos-latest, windows-latest]
60+
steps:
61+
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
62+
with:
63+
persist-credentials: false
64+
- name: Install Rust toolchain
65+
uses: dtolnay/rust-toolchain@3c5f7ea28cd621ae0bf5283f0e981fb97b8a7af9
66+
with:
67+
toolchain: stable
68+
- name: Run sub-ms diagnostic
69+
shell: bash
70+
run: |
71+
mkdir -p diagnostics
72+
cargo run --example sub_ms_diagnostic -- "${SUB_MS_DIAGNOSTIC_SAMPLES}" | tee "diagnostics/sub-ms-${{ matrix.os }}.txt"
73+
- name: Add diagnostic summary
74+
shell: bash
75+
run: |
76+
{
77+
echo "## Sub-ms Diagnostic (${{ matrix.os }})"
78+
echo
79+
echo '```text'
80+
cat "diagnostics/sub-ms-${{ matrix.os }}.txt"
81+
echo '```'
82+
} >> "$GITHUB_STEP_SUMMARY"
83+
- name: Upload diagnostic artifact
84+
uses: actions/upload-artifact@v4
85+
with:
86+
name: sub-ms-diagnostic-${{ matrix.os }}
87+
path: diagnostics/sub-ms-${{ matrix.os }}.txt

Cargo.lock

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
members = ["python"]
33

44
[workspace.package]
5-
version = "0.1.4"
5+
version = "0.1.5"
66
edition = "2021"
77
license = "MIT"
88

README.md

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ This implementation focuses on speed. It uses thread-local storage and a seeded
88

99
* **UUID v7**: Time-ordered, 128-bit unique identifiers.
1010
* **Fast**: Minimal overhead using thread-local state.
11-
* **Flexible**: Choose between maximum randomness (`gen_id`) or per-thread monotonicity (`gen_id_with_count`).
11+
* **Flexible**: Choose between maximum randomness (`gen_id`), optional sub-millisecond sort locality (`gen_id_with_sub_ms_4`, `gen_id_with_sub_ms_8`, `gen_id_with_sub_ms_12`), or per-thread monotonicity (`gen_id_with_count`).
1212

1313
## Why?
1414

@@ -40,6 +40,11 @@ The `gen_id_with_count` function uses an 18-bit counter and 56 bits of randomnes
4040
* **Pros**: Guarantees monotonicity per thread (up to ~262k IDs/ms).
4141
* **Cons**: Reduced randomness (56 bits) increases collision risk in massive distributed systems (approx. 50% chance after 4.5 billion IDs/ms globally).
4242

43+
### `gen_id_with_sub_ms_4`, `gen_id_with_sub_ms_8`, `gen_id_with_sub_ms_12`
44+
These functions keep the standard 48-bit millisecond timestamp, then place a scaled sub-millisecond fraction into the high bits of `rand_a` as described by RFC 9562. The remaining bits of `rand_a` stay random, and `rand_b` stays fully random. The millisecond timestamp comes from wall-clock time, but on supported counter backends the sub-millisecond fraction is often estimated between wall-clock refreshes instead of being freshly measured on every call.
45+
* **Pros**: Improves sort locality for IDs created within the same millisecond without adding counters or shared state.
46+
* **Cons**: This is not true nanosecond ordering, and it does not provide distributed monotonicity.
47+
4348
## Bit Layout
4449

4550
The 128-bit ID is fully compatible with UUID v7. It is composed of:
@@ -52,12 +57,17 @@ The 128-bit ID is fully compatible with UUID v7. It is composed of:
5257

5358
**Total Randomness:**
5459
* `gen_id`: **74 bits**
60+
* `gen_id_with_sub_ms_4`: **70 bits**
61+
* `gen_id_with_sub_ms_8`: **66 bits**
62+
* `gen_id_with_sub_ms_12`: **62 bits**
5563
* `gen_id_with_count`: **56 bits**
5664

5765
## Usage
5866

5967
```rust
60-
use fast_uuid_v7::{gen_id, gen_id_string, gen_id_str, gen_id_with_count};
68+
use fast_uuid_v7::{
69+
gen_id, gen_id_str, gen_id_string, gen_id_with_count, gen_id_with_sub_ms_8,
70+
};
6171

6272
fn main() {
6373
// Get ID as u128 (74 bits random), takes about 8-50ns
@@ -68,6 +78,10 @@ fn main() {
6878
let ordered_id = gen_id_with_count();
6979
println!("Ordered ID: {:032x}", ordered_id);
7080

81+
// Get an ID with 8 bits of sub-millisecond time fraction in rand_a
82+
let local_order_id = gen_id_with_sub_ms_8();
83+
println!("Locally ordered ID: {:032x}", local_order_id);
84+
7185
// Get ID as canonical string (allocates String, takes about 85-130ns)
7286
let id_string = gen_id_string();
7387
println!("Generated ID string: {}", id_string);
@@ -100,6 +114,7 @@ Generating 10 million IDs takes approximately **95ms** on a single core.
100114

101115
* **Not Cryptographically Secure**: The randomness is optimized for speed, not unpredictability. Do not use for session tokens or secrets. If you don't need speed, use the original `uuid` crate.
102116
* **Monotonicity**: Only guaranteed per-thread if using `gen_id_with_count`. Otherwise, IDs within the same millisecond are random.
117+
* **Sub-millisecond fractions are approximate**: the `gen_id_with_sub_ms_*` variants improve intra-millisecond sort locality, but the extra bits are often estimated rather than freshly measured and are not a true higher-resolution timestamp.
103118
* **Clock Drift Risk**: The batched timestamp check assumes the CPU counter frequency is stable. While we include safety checks, extreme edge cases (e.g., VM migration) might cause a 1ms timestamp lag.
104119
* **Still needs wall-clock reads**: The fastest path only happens while we can reuse the last millisecond timestamp. Once the next millisecond boundary is due, we still need to refresh from `SystemTime::now()`, so actual throughput depends on workload and platform details.
105120

benches/benchmark.rs

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
use criterion::{criterion_group, criterion_main, Criterion};
22
use fast_uuid_v7::{
33
gen_id_str, gen_id_string, gen_id_u128, gen_id_with_count, gen_id_with_count_str,
4+
gen_id_with_sub_ms_8,
45
};
56
use uuid::Uuid;
67

@@ -26,6 +27,12 @@ fn benchmark_gen_id_with_count_str(c: &mut Criterion) {
2627
});
2728
}
2829

30+
fn benchmark_gen_id_with_sub_ms_8(c: &mut Criterion) {
31+
c.bench_function("gen_id_with_sub_ms_8", |b| {
32+
b.iter(|| gen_id_with_sub_ms_8())
33+
});
34+
}
35+
2936
fn benchmark_uuid_now_v7(c: &mut Criterion) {
3037
c.bench_function("uuid_now_v7", |b| b.iter(|| Uuid::now_v7()));
3138
}
@@ -41,6 +48,7 @@ criterion_group!(
4148
benchmark_gen_id_str,
4249
benchmark_gen_id_with_count,
4350
benchmark_gen_id_with_count_str,
51+
benchmark_gen_id_with_sub_ms_8,
4452
benchmark_uuid_now_v7,
4553
benchmark_uuid_now_v7_str
4654
);

examples/sub_ms_diagnostic.rs

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
use fast_uuid_v7::{gen_id_with_sub_ms_12, gen_id_with_sub_ms_4, gen_id_with_sub_ms_8};
2+
use std::time::{SystemTime, UNIX_EPOCH};
3+
4+
const DEFAULT_SAMPLES: usize = 20_000;
5+
6+
fn main() {
7+
let samples = std::env::args()
8+
.nth(1)
9+
.map(|arg| {
10+
arg.parse::<usize>()
11+
.expect("sample count must be a positive integer")
12+
})
13+
.unwrap_or(DEFAULT_SAMPLES);
14+
15+
println!("Sub-ms diagnostic using SystemTime before/after windows");
16+
println!("Samples per variant: {samples}");
17+
println!();
18+
19+
run_variant("gen_id_with_sub_ms_4", 4, gen_id_with_sub_ms_4, samples);
20+
run_variant("gen_id_with_sub_ms_8", 8, gen_id_with_sub_ms_8, samples);
21+
run_variant("gen_id_with_sub_ms_12", 12, gen_id_with_sub_ms_12, samples);
22+
}
23+
24+
fn run_variant(name: &str, bits: u8, gen_id: fn() -> u128, samples: usize) {
25+
let mut interval_errors_ns = Vec::with_capacity(samples);
26+
let mut midpoint_errors_ns = Vec::with_capacity(samples);
27+
let mut signed_midpoint_errors_ns = Vec::with_capacity(samples);
28+
let mut call_windows_ns = Vec::with_capacity(samples);
29+
let mut overlap_count = 0usize;
30+
let mut lagging_count = 0usize;
31+
let mut leading_count = 0usize;
32+
33+
for _ in 0..samples {
34+
let before = unix_time_ns();
35+
let id = gen_id();
36+
let after = unix_time_ns();
37+
38+
let (id_start_ns, id_end_ns) = decode_sub_ms_window_ns(id, bits);
39+
let interval_error_ns = interval_gap_ns(id_start_ns, id_end_ns, before, after);
40+
let observed_mid_ns = before + (after.saturating_sub(before) / 2);
41+
let id_mid_ns = id_start_ns + ((id_end_ns.saturating_sub(id_start_ns)) / 2);
42+
let midpoint_error_ns = id_mid_ns.abs_diff(observed_mid_ns);
43+
let signed_midpoint_error_ns = id_mid_ns as i128 - observed_mid_ns as i128;
44+
45+
if interval_error_ns == 0 {
46+
overlap_count += 1;
47+
}
48+
if signed_midpoint_error_ns < 0 {
49+
lagging_count += 1;
50+
} else if signed_midpoint_error_ns > 0 {
51+
leading_count += 1;
52+
}
53+
54+
interval_errors_ns.push(interval_error_ns);
55+
midpoint_errors_ns.push(midpoint_error_ns);
56+
signed_midpoint_errors_ns.push(signed_midpoint_error_ns);
57+
call_windows_ns.push(after.saturating_sub(before));
58+
}
59+
60+
interval_errors_ns.sort_unstable();
61+
midpoint_errors_ns.sort_unstable();
62+
signed_midpoint_errors_ns.sort_unstable();
63+
call_windows_ns.sort_unstable();
64+
65+
println!("{name} ({bits} bits)");
66+
print_stats("call window", &call_windows_ns);
67+
print_stats("interval gap", &interval_errors_ns);
68+
print_stats("midpoint error", &midpoint_errors_ns);
69+
print_signed_stats("signed drift", &signed_midpoint_errors_ns);
70+
println!(
71+
" overlap rate : {:>6.2}%\n lagging rate : {:>6.2}%\n leading rate : {:>6.2}%",
72+
percentage(overlap_count, samples),
73+
percentage(lagging_count, samples),
74+
percentage(leading_count, samples),
75+
);
76+
println!(
77+
" <=100us gap : {:>6.2}%\n <=250us gap : {:>6.2}%\n <=500us gap : {:>6.2}%\n <=1ms gap : {:>6.2}%",
78+
percentage_within(&interval_errors_ns, 100_000),
79+
percentage_within(&interval_errors_ns, 250_000),
80+
percentage_within(&interval_errors_ns, 500_000),
81+
percentage_within(&interval_errors_ns, 1_000_000),
82+
);
83+
println!();
84+
}
85+
86+
fn unix_time_ns() -> u128 {
87+
SystemTime::now()
88+
.duration_since(UNIX_EPOCH)
89+
.expect("system clock should be after UNIX_EPOCH")
90+
.as_nanos()
91+
}
92+
93+
fn decode_sub_ms_window_ns(id: u128, bits: u8) -> (u128, u128) {
94+
let ms = id >> 80;
95+
let rand_a = (id >> 64) & 0x0FFF;
96+
let fraction = rand_a >> (12 - bits);
97+
let slots = 1u128 << bits;
98+
let base_ns = ms * 1_000_000;
99+
let start_ns = base_ns + (fraction * 1_000_000) / slots;
100+
let end_ns = base_ns + (((fraction + 1) * 1_000_000).saturating_sub(1)) / slots;
101+
(start_ns, end_ns)
102+
}
103+
104+
fn interval_gap_ns(a_start: u128, a_end: u128, b_start: u128, b_end: u128) -> u128 {
105+
if a_end < b_start {
106+
b_start - a_end
107+
} else if b_end < a_start {
108+
a_start - b_end
109+
} else {
110+
0
111+
}
112+
}
113+
114+
fn print_stats(label: &str, values: &[u128]) {
115+
println!(
116+
" {label:<14} p50={:>8}us p90={:>8}us p99={:>8}us max={:>8}us",
117+
ns_to_us(percentile(values, 50)),
118+
ns_to_us(percentile(values, 90)),
119+
ns_to_us(percentile(values, 99)),
120+
ns_to_us(*values.last().unwrap_or(&0)),
121+
);
122+
}
123+
124+
fn print_signed_stats(label: &str, values: &[i128]) {
125+
println!(
126+
" {label:<14} p10={:>8}us p50={:>8}us p90={:>8}us",
127+
signed_ns_to_us(percentile_signed(values, 10)),
128+
signed_ns_to_us(percentile_signed(values, 50)),
129+
signed_ns_to_us(percentile_signed(values, 90)),
130+
);
131+
}
132+
133+
fn percentile(sorted: &[u128], percentile: usize) -> u128 {
134+
if sorted.is_empty() {
135+
return 0;
136+
}
137+
138+
let rank = ((sorted.len() - 1) * percentile) / 100;
139+
sorted[rank]
140+
}
141+
142+
fn ns_to_us(value: u128) -> u128 {
143+
value / 1_000
144+
}
145+
146+
fn signed_ns_to_us(value: i128) -> i128 {
147+
value / 1_000
148+
}
149+
150+
fn percentile_signed(sorted: &[i128], percentile: usize) -> i128 {
151+
if sorted.is_empty() {
152+
return 0;
153+
}
154+
155+
let rank = ((sorted.len() - 1) * percentile) / 100;
156+
sorted[rank]
157+
}
158+
159+
fn percentage_within(sorted: &[u128], limit_ns: u128) -> f64 {
160+
if sorted.is_empty() {
161+
return 0.0;
162+
}
163+
164+
let count = sorted.partition_point(|value| *value <= limit_ns);
165+
percentage(count, sorted.len())
166+
}
167+
168+
fn percentage(count: usize, total: usize) -> f64 {
169+
if total == 0 {
170+
return 0.0;
171+
}
172+
173+
(count as f64 * 100.0) / total as f64
174+
}

python/README.md

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,9 @@ maturin develop
3232
import fastuuidv7
3333

3434
raw = fastuuidv7.gen_id()
35+
raw_sub_ms_4 = fastuuidv7.gen_id_with_sub_ms_4()
36+
raw_sub_ms_8 = fastuuidv7.gen_id_with_sub_ms_8()
37+
raw_sub_ms_12 = fastuuidv7.gen_id_with_sub_ms_12()
3538
text = fastuuidv7.gen_id_str()
3639
data = fastuuidv7.gen_id_bytes()
3740

@@ -54,18 +57,24 @@ python3 bench/bench.py
5457

5558
The benchmark measures repeated single calls, so the numbers include Python/native boundary overhead.
5659

57-
Current results on linux amd64_x86 git runner:
60+
Current results on macOS arm64 with Python 3.14.4:
5861

5962
| Package | Callable | Time per call |
6063
| --- | --- | ---: |
61-
| `uuid6` | `uuid6.uuid7` | `2890.4 ns` |
62-
| `uuid7` | `uuid_extensions.uuid7` | `2767.7 ns` |
63-
| `fastuuid7` | `uuidv7.uuid7` | `313.5 ns` |
64-
| `fastuuidv7 ` | `fastuuidv7.uuid7` | `60.6 ns` |
64+
| `uuid6` | `uuid6.uuid7` | `1999.5 ns` |
65+
| `uuid-v7` | `uuid_v7.base.uuid7` | `2582.4 ns` |
66+
| `uuid7` | `uuid_extensions.uuid7` | `1969.2 ns` |
67+
| `uuid` | `uuid.uuid7` | `1579.9 ns` |
68+
| [`fastuuid7`](https://github.com/nekrasovp/uuidv7) | `uuidv7.uuid7` | `298.6 ns` |
69+
| [`uuid-utils`](https://github.com/aminalaee/uuid-utils) | `uuid_utils.uuid7` | `96.8 ns` |
70+
| `fastuuidv7` | `fastuuidv7.uuid7` | `51.0 ns` |
71+
72+
The linux x86 GH runner has similar times. It is running always in this repo.
6573

6674
Notes:
6775

6876
- These are single-call throughput measurements, so they include Python/native boundary overhead.
77+
- Install comparison packages such as `uuid-utils` before running; missing packages are skipped.
6978

7079
## Notes
7180

python/bench/bench.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@ def resolve_benchmarks():
6262
("uuid6", "uuid6", [("uuid6", ["uuid7"])]),
6363
("uuid-v7", "uuid-v7", [("uuid_v7", ["uuid7"]), ("uuid_v7.base", ["uuid7"])]),
6464
("uuid7", "uuid7", [("uuid7", ["uuid7"]), ("uuid_extensions", ["uuid7", "uuid7str"])]),
65+
("uuid-utils", "uuid-utils", [("uuid_utils", ["uuid7"])]),
6566
("uuidv7", "uuidv7", [("uuidv7", ["uuid7", "uuidv7", "generate"])]),
6667
(
6768
"fastuuid7",

0 commit comments

Comments
 (0)