Skip to content

Commit 38a9b9f

Browse files
committed
Admit SSSE3 encode backend
1 parent 0e6045e commit 38a9b9f

24 files changed

Lines changed: 510 additions & 286 deletions

CHANGELOG.md

Lines changed: 17 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,19 @@
22

33
## Unreleased
44

5+
- Admitted std `x86`/`x86_64` SSSE3/SSE4.1 encode dispatch for Standard and
6+
URL-safe alphabet families. The admitted path processes fixed 12-byte blocks
7+
with vector code after runtime CPU probing and falls back to scalar for
8+
unsupported CPUs, `no_std`, custom alphabets, tails, padding, in-place encode,
9+
and every decode path.
10+
- Updated runtime reporting, backend evidence, SIMD admission validation,
11+
unsafe inventory, and user documentation so SSSE3/SSE4.1 encode is reported
12+
as an admitted backend while AVX2, AVX-512 VBMI, NEON, wasm `simd128`,
13+
custom-alphabet, in-place, and decode acceleration remain prototype-only or
14+
scalar.
515
- Added a real non-dispatchable AVX-512 VBMI fixed-block encode prototype that
616
uses the provided alphabet table for all alphabets. The prototype remains
7-
test-only and active runtime backend selection remains scalar-only.
17+
test-only and is not reachable from runtime backend selection.
818
- Added AVX-512 SIMD equivalence coverage for patterned blocks, all 64 emitted
919
six-bit Base64 values, and a non-standard custom alphabet.
1020
- Added `scripts/generate_simd_asm_evidence.sh` to capture release
@@ -19,12 +29,12 @@
1929
Standard and URL-safe alphabets. Custom alphabets remain on the scalar
2030
scaffold path because portable wasm SIMD has no direct 64-byte lookup.
2131
- Added wasm `simd128` test-binary compile evidence to the SIMD feature-bundle
22-
check while keeping wasm cleanup/JIT caveats and scalar-only runtime dispatch.
32+
check while keeping wasm cleanup/JIT caveats and scalar runtime dispatch.
2333
- Hardened SIMD admission tooling and backend evidence manifests to distinguish
2434
real non-dispatchable prototypes from admitted active backends.
2535
- Added a draft SIMD encode admission package for future `1.2.0` activation,
2636
including runtime-report expectations, benchmark record shape, and release
27-
note wording rules while keeping `1.1.x` scalar-only.
37+
note wording rules for the remaining encode backends.
2838
- Added `scripts/validate-simd-encode-admission-draft.sh` and wired it into the
2939
standard checks so the future encode-dispatch admission contract remains
3040
packaged and machine-checked.
@@ -37,17 +47,16 @@
3747
- Inlined the test-only AArch64 NEON register cleanup macro into the prototype
3848
path so callee-saved `v8..v15` are not restored by a separate helper frame.
3949
- Added a real non-dispatchable AVX2 fixed-block encode prototype for Standard
40-
and URL-safe alphabets. The prototype remains test-only and active runtime
41-
backend selection remains scalar-only.
50+
and URL-safe alphabets. The prototype remains test-only and is not reachable
51+
from runtime backend selection.
4252
- Added AVX2 SIMD equivalence coverage for patterned blocks, all 64 emitted
4353
six-bit Base64 values, and custom alphabets that must fall back to scalar
4454
encoding.
4555
- Updated the SIMD unsafe inventory for the AVX-512, AVX2, and AArch64 NEON
4656
prototypes, including staged stack-copy wiping and vector-register cleanup.
4757
- Refreshed the SIMD roadmap, admission manifest, and backend evidence output
4858
so `1.1.x` checkpoint tags consistently describe the current state as real
49-
non-dispatchable prototype evidence while active runtime dispatch remains
50-
scalar-only until a future `1.2.0` admission package.
59+
non-dispatchable prototype evidence for backends that are not yet admitted.
5160
- Hardened the workspace publish helper so real crates.io publishing requires
5261
`HEAD` to match a verified signed `v<version>` tag, and documented the
5362
`git tag -v` release check.
@@ -63,8 +72,7 @@
6372
`1.1.12` checkpoints leading to that release.
6473
- Added a scalar-forced encode backend boundary and routed public slice,
6574
clear-tail, wrapped, alloc, and in-place encode paths through it, giving
66-
future SIMD encode admission one audited integration point while preserving
67-
scalar-only runtime behavior.
75+
future SIMD encode admission one audited integration point.
6876
- Added the matching scalar-forced decode backend boundary for future decode
6977
admission symmetry, changed the scalar in-place encode invariant to return an
7078
error instead of panicking in release builds, tightened panic-policy scanning

README.md

Lines changed: 20 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -31,9 +31,9 @@ The crate starts conservative: a small scalar implementation, strict RFC 4648 be
3131

3232
The current public release is `1.1.0`.
3333
Post-`1.1.0` GitHub checkpoint tags in the `1.1.x` line are evidence and
34-
integration milestones toward `1.2.0`; they keep active runtime dispatch
35-
scalar-only and are not intended as crates.io publishes unless a user-impacting
36-
fix requires it.
34+
integration milestones toward `1.2.0`; they may admit narrowly scoped encode
35+
backends after evidence review and are not intended as crates.io publishes
36+
unless a user-impacting fix requires it.
3737

3838
Implemented now:
3939

@@ -71,8 +71,10 @@ Implemented now:
7171
- Bounded Kani proof harnesses that run on Rust `1.90.0` with
7272
`cargo-kani 0.67.0`.
7373
- Constant-time assembly evidence generation for reviewer inspection.
74-
- Real non-dispatchable SSSE3/SSE4.1 fixed-block encode prototype for Standard
75-
and URL-safe alphabets behind the SIMD admission boundary.
74+
- Runtime-dispatched std `x86`/`x86_64` SSSE3/SSE4.1 fixed-block encode for
75+
Standard and URL-safe alphabets behind the SIMD admission boundary, with
76+
scalar fallback for unsupported CPUs, `no_std`, custom alphabets, tails,
77+
padding, in-place encode, and decode.
7678
- Optional `base64-ng-sanitization` companion crate for applications that
7779
already admit `sanitization` and want direct CT decode helpers into
7880
clear-on-drop secret containers.
@@ -85,9 +87,9 @@ Implemented now:
8587

8688
Planned behind admission evidence:
8789

88-
- Admitted AVX2, AVX-512, SSSE3/SSE4.1, ARM NEON, and wasm `simd128`
89-
fast paths after the SIMD admission evidence is complete. The `1.1.0`
90-
release remains scalar-only for active runtime dispatch.
90+
- Additional admitted AVX2, AVX-512, ARM NEON, wasm `simd128`, custom alphabet,
91+
in-place, and decode fast paths after the SIMD admission evidence is
92+
complete. Default builds and unsupported runtime CPUs remain scalar.
9193
- Full async streaming wrappers only after the `tokio` feature passes the
9294
cancellation-safety admission bar in [docs/ASYNC.md](docs/ASYNC.md). The
9395
`base64-ng-tokio` companion crate currently provides bounded async
@@ -104,8 +106,8 @@ Planned behind admission evidence:
104106
| License | `MIT OR Apache-2.0` |
105107
| MSRV | Rust `1.90.0` |
106108
| Runtime dependencies | Zero external crates |
107-
| Unsafe policy | Scalar encode/decode remains safe Rust; audited unsafe is limited to volatile wiping, CT comparison/barrier helpers, and test-only SIMD prototypes |
108-
| Active backend | Scalar only |
109+
| Unsafe policy | Scalar encode/decode remains safe Rust; audited unsafe is limited to volatile wiping, CT comparison/barrier helpers, and the reviewed SIMD boundary |
110+
| Active backend | Scalar by default; std x86/x86_64 SSSE3/SSE4.1 encode when `simd` is enabled and runtime CPU probing passes |
109111
| Strict decoding | Default, canonical, no whitespace |
110112
| Legacy compatibility | Explicit opt-in APIs |
111113
| Constant-time posture | Constant-time-oriented scalar validation/decode with isolated dudect-style timing evidence; no formal cryptographic guarantee |
@@ -947,7 +949,9 @@ Security commitments:
947949
dead-store elimination and are ordered before the cleanup boundary on
948950
supported native architectures. Constant-time comparison, byte accumulation,
949951
CT scan, and CT result-gate hardening remain audited in `src/ct/`.
950-
- Future unsafe SIMD remains isolated under `src/simd/`.
952+
- Unsafe SIMD remains isolated under `src/simd/`; the admitted SSSE3/SSE4.1
953+
encode path is std runtime-probed and all non-admitted backends remain
954+
prototype-only.
951955
- Local checks verify that `allow(unsafe_code)` is confined to the volatile
952956
wipe helpers and SIMD boundary, every unsafe function is inventoried, and
953957
every unsafe block has a nearby `SAFETY:` explanation. Architecture intrinsics,
@@ -959,9 +963,11 @@ Security commitments:
959963
async/Tokio API while the `tokio` feature remains inert.
960964
- [docs/DEPENDENCIES.md](docs/DEPENDENCIES.md) defines the dependency
961965
admission bar for any future external crate.
962-
- `runtime::backend_report()` exposes the active backend, detected candidate,
963-
candidate detection mode, SIMD feature status, scalar-only security posture,
964-
and a conservative unsafe-boundary posture flag for audit logging. The
966+
- `runtime::backend_report()` exposes the active admitted backend, detected
967+
candidate, candidate detection mode, SIMD feature status, security posture,
968+
and a conservative unsafe-boundary posture flag for audit logging. In the
969+
`1.1.x` line, non-scalar active values describe admitted encode dispatch;
970+
decode remains scalar. The
965971
unsafe-boundary flag is true only when the reserved `simd` feature is
966972
disabled; SIMD-enabled builds must rely on the release evidence scripts for
967973
boundary validation. On `no_std` and non-x86 targets, candidate detection is

docs/PLAN.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -674,14 +674,15 @@ Current `1.1.x` checkpoint state:
674674
Follow-up review also added the matching scalar-forced decode backend
675675
boundary so future decode acceleration has a symmetric admission point, and
676676
tightened panic-policy checks for production `assert_eq!`/`assert_ne!` sites.
677+
- `1.1.6`: admitted std-only x86/x86_64 SSSE3/SSE4.1 encode dispatch for
678+
Standard and URL-safe alphabet families. The active path uses runtime CPU
679+
probing, fixed 12-byte vector blocks, XMM cleanup, and scalar fallback for
680+
unsupported CPUs, `no_std`, custom alphabets, tails, padding, in-place encode,
681+
and decode. Runtime reports and admission evidence now identify
682+
SSSE3/SSE4.1 encode as the admitted backend.
677683

678684
Planned checkpoints to reach `1.2.0` fully working encode acceleration:
679685

680-
- `1.1.6`: admit std-only x86/x86_64 SSSE3/SSE4.1 encode dispatch for Standard
681-
and URL-safe alphabets if scalar parity, fallback, generated assembly,
682-
unsafe inventory, and runtime-report evidence are complete. Unsupported CPUs,
683-
custom alphabets, tails, wrapping, legacy profiles, and `no_std` builds must
684-
remain scalar.
685686
- `1.1.7`: admit std-only x86/x86_64 AVX2 encode dispatch with the same public
686687
API coverage and fallback requirements. Runtime priority must be explicit
687688
and tested, for example AVX2 above SSSE3/SSE4.1 when both are available.

docs/RELEASE_EVIDENCE.md

Lines changed: 34 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -113,11 +113,13 @@ The release gate runs:
113113
- reserved SIMD feature-bundle compile checks for AVX2, AVX-512 VBMI,
114114
SSSE3/SSE4.1, NEON, and wasm `simd128` under `no_std` when the corresponding
115115
Rust targets are installed
116-
- backend evidence capture for runtime backend reporting and inactive SIMD
117-
prototype scalar-equivalence output
118-
- scalar-only SIMD admission policy for the current release series, with no
119-
active accelerated dispatch and no SIMD performance claims unless a complete
120-
backend admission evidence package lands
116+
- backend evidence capture for runtime backend reporting, admitted SSSE3/SSE4.1
117+
encode dispatch when supported, and inactive SIMD prototype scalar-equivalence
118+
output for remaining candidates
119+
- SIMD admission policy for the current release series, with SSSE3/SSE4.1
120+
encode admitted only for std x86/x86_64 Standard and URL-safe alphabets, no
121+
decode acceleration, and no SIMD performance claims without complete local
122+
benchmark evidence
121123
- unsafe-boundary validation that confines `allow(unsafe_code)` to the audited
122124
cleanup helpers in `src/cleanup.rs`, CT barrier/comparison helpers in
123125
`src/ct/`, and the SIMD boundary in `src/simd/`
@@ -274,25 +276,25 @@ Capture local runtime backend and prototype evidence with:
274276
scripts/check_backend_evidence.sh
275277
```
276278

277-
The script runs the runtime backend-report test and the gated SIMD prototype
278-
scalar-equivalence scaffolding tests with `--nocapture`. The runtime report
279-
records `candidate_detection_mode`, which distinguishes x86/x86_64 `std`
280-
runtime CPU probing from compile-time target-feature reporting used by
281-
`no_std` and other compile-time-only targets. On CPUs with
282-
SSSE3/SSE4.1, AVX2, or the AVX-512 candidate bundle, those prototype tests
283-
execute the inactive prototype body and compare it against scalar output. The
284-
x86 prototypes exercise real fixed-block vector encode logic when the required
285-
CPU feature bundles are available. On AArch64 NEON-capable hosts, the NEON test
286-
exercises the inactive fixed-block vector prototype for Standard and URL-safe
287-
alphabets; 32-bit ARM remains scaffold evidence. Wasm `simd128` evidence is
288-
kept in `scripts/check_simd_feature_bundles.sh` as compile/test-binary evidence
289-
only because runtime JIT behavior is outside the crate's release gate. The
279+
The script runs the runtime backend-report test and the gated SIMD
280+
scalar-equivalence tests with `--nocapture`. The runtime report records
281+
`candidate_detection_mode`, which distinguishes x86/x86_64 `std` runtime CPU
282+
probing from compile-time target-feature reporting used by `no_std` and other
283+
compile-time-only targets. On CPUs with SSSE3/SSE4.1, the admitted encode path
284+
may be active for Standard and URL-safe alphabets. AVX2 and AVX-512 tests still
285+
execute inactive prototype bodies and compare them against scalar output. On
286+
AArch64 NEON-capable hosts, the NEON test exercises the inactive fixed-block
287+
vector prototype for Standard and URL-safe alphabets; 32-bit ARM remains
288+
scaffold evidence. Wasm `simd128` evidence is kept in
289+
`scripts/check_simd_feature_bundles.sh` as compile/test-binary evidence only
290+
because runtime JIT behavior is outside the crate's release gate. The
290291
script writes
291292
`target/release-evidence/backend/MANIFEST.txt`, `runtime-backend-report.txt`,
292293
and `simd-prototype-equivalence.txt` so local CPU evidence can be archived. The
293-
manifest labels prototype evidence as `real-non-dispatchable` and separately
294-
records `active_backend_admitted=false`, so audit logs do not confuse fixed
295-
block prototype execution with active dispatch admission.
294+
manifest labels prototype-only evidence as `real-non-dispatchable` and
295+
separately records `active_backend_admitted=ssse3-sse4.1-encode`, so audit logs
296+
do not confuse remaining fixed-block prototype execution with active dispatch
297+
admission.
296298

297299
The release gate also runs:
298300

@@ -301,10 +303,10 @@ scripts/validate-simd-admission.sh
301303
scripts/validate-simd-encode-admission-draft.sh
302304
```
303305

304-
That validator keeps active SIMD dispatch scalar-only until the release includes
305-
the required scalar differential tests, fuzz evidence, unsafe inventory updates,
306-
architecture evidence, benchmark evidence, release-note wording, and an updated
307-
`docs/SIMD_ADMISSION.md` manifest.
306+
That validator keeps active SIMD dispatch limited to admitted backends until a
307+
release includes the required scalar differential tests, fuzz evidence, unsafe
308+
inventory updates, architecture evidence, benchmark evidence, release-note
309+
wording, and an updated `docs/SIMD_ADMISSION.md` manifest.
308310

309311
For a future encode-dispatch release, use
310312
[`SIMD_ENCODE_ADMISSION_DRAFT.md`](SIMD_ENCODE_ADMISSION_DRAFT.md) as the
@@ -367,20 +369,19 @@ review focus, and artifact checksums. The LTO artifact exists so reviewers can
367369
check that cleanup primitives such as `wipe_bytes` and `wipe_barrier` remain
368370
visible call boundaries under aggressive optimization.
369371

370-
Capture generated assembly evidence for the inactive x86 SIMD encode
371-
prototypes with:
372+
Capture generated assembly evidence for x86 SIMD encode paths with:
372373

373374
```sh
374375
scripts/generate_simd_asm_evidence.sh
375376
```
376377

377378
On x86/x86_64 hosts, the script emits release test-harness assembly for the
378-
SSSE3/SSE4.1, AVX2, and AVX-512 VBMI feature bundles and checks for the
379-
expected byte-shuffle, byte-permute, vector-register, and cleanup instructions.
380-
On non-x86 hosts it records a skip manifest. The generated files are written to
381-
`target/release-evidence/simd-asm/` and are inactive prototype evidence only;
382-
runtime dispatch remains scalar-only until the SIMD admission manifest is
383-
updated in a future release.
379+
admitted SSSE3/SSE4.1 encode path and the AVX2 and AVX-512 VBMI prototype
380+
feature bundles, then checks for the expected byte-shuffle, byte-permute,
381+
vector-register, and cleanup instructions. On non-x86 hosts it records a skip
382+
manifest. The generated files are written to
383+
`target/release-evidence/simd-asm/`. AVX2 and AVX-512 remain inactive prototype
384+
evidence until the SIMD admission manifest names them as admitted backends.
384385

385386
## Performance Evidence
386387

0 commit comments

Comments
 (0)