Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(crypto): use ring for asm implementations of sha256/sha512 #27885

Merged
merged 5 commits into from
Jan 30, 2025

Conversation

nathanwhit
Copy link
Member

@nathanwhit nathanwhit commented Jan 30, 2025

Currently we are using the pure rust backend of sha2, which has subpar performance compared to asm implementations. We already depend on ring, so just use that instead of sha2 for sha256/sha512 digests.

This also speeds up things like S3 uploads, which calculate sha digests of the uploaded objects. On my local machine, this speeds up uploading a 100MB file (to a localhost s3 provider via@aws-sdk/client-s3) by about 2x

Benchmark:
import { createHmac } from "node:crypto";

for (
  const size of [1, 10, 100, 1_000, 10_000, 100_000, 1_000_000, 10_000_000]
) {
  const input = "a".repeat(size);
  Deno.bench({
    name: `sha256-${size}`,
    fn() {
      const _hash = createHmac("sha256", input).update(input).digest();
    },
  });
  Deno.bench({
    name: `sha512-${size}`,
    fn() {
      const _hash = createHmac("sha512", input).update(input).digest();
    },
  });
}
Results (arm64 macOS):
--- sha256-1 ---
../../deno/target/release/deno         2.527 µs    1.240 times faster
/Users/nathanwhit/.deno/bin/deno       3.132 µs    
--- sha512-1 ---
../../deno/target/release/deno         3.364 µs    1.071 times faster
/Users/nathanwhit/.deno/bin/deno       3.603 µs    
--- sha256-10 ---
../../deno/target/release/deno         3.060 µs    1.027 times faster
/Users/nathanwhit/.deno/bin/deno       3.144 µs    
--- sha512-10 ---
../../deno/target/release/deno         3.583 µs    1.047 times faster
/Users/nathanwhit/.deno/bin/deno       3.751 µs    
--- sha256-100 ---
../../deno/target/release/deno         3.695 µs    1.244 times faster
/Users/nathanwhit/.deno/bin/deno       4.598 µs    
--- sha512-100 ---
../../deno/target/release/deno         3.386 µs    1.188 times faster
/Users/nathanwhit/.deno/bin/deno       4.021 µs    
--- sha256-1000 ---
../../deno/target/release/deno         4.007 µs    3.230 times faster
/Users/nathanwhit/.deno/bin/deno      12.944 µs    
--- sha512-1000 ---
../../deno/target/release/deno         6.463 µs    1.466 times faster
/Users/nathanwhit/.deno/bin/deno       9.477 µs    
--- sha256-10000 ---
../../deno/target/release/deno        11.674 µs    6.981 times faster
/Users/nathanwhit/.deno/bin/deno      81.493 µs    
--- sha512-10000 ---
../../deno/target/release/deno        31.250 µs    1.740 times faster
/Users/nathanwhit/.deno/bin/deno      54.364 µs    
--- sha256-100000 ---
../../deno/target/release/deno        82.800 µs    9.393 times faster
/Users/nathanwhit/.deno/bin/deno     777.719 µs    
--- sha512-100000 ---
../../deno/target/release/deno       269.726 µs    1.851 times faster
/Users/nathanwhit/.deno/bin/deno     499.243 µs    
--- sha256-1000000 ---
../../deno/target/release/deno       808.662 µs    9.427 times faster
/Users/nathanwhit/.deno/bin/deno       7.623 ms    
--- sha512-1000000 ---
../../deno/target/release/deno         2.672 ms    1.795 times faster
/Users/nathanwhit/.deno/bin/deno       4.795 ms    
--- sha256-10000000 ---
../../deno/target/release/deno         7.823 ms    9.868 times faster
/Users/nathanwhit/.deno/bin/deno      77.201 ms    
--- sha512-10000000 ---
../../deno/target/release/deno        26.197 ms    1.846 times faster
/Users/nathanwhit/.deno/bin/deno      48.356 ms    
Results (x86_64 linux):
--- sha256-1 ---
/home/nathanwhit/.deno/bin/deno             10.726 µs    1.229 times faster
../../../deno/target/release-lite/deno      13.184 µs    
--- sha512-1 ---
/home/nathanwhit/.deno/bin/deno             13.177 µs    1.051 times faster
../../../deno/target/release-lite/deno      13.845 µs    
--- sha256-10 ---
/home/nathanwhit/.deno/bin/deno             13.156 µs    1.047 times faster
../../../deno/target/release-lite/deno      13.780 µs    
--- sha512-10 ---
/home/nathanwhit/.deno/bin/deno             14.386 µs    1.029 times faster
../../../deno/target/release-lite/deno      14.807 µs    
--- sha256-100 ---
/home/nathanwhit/.deno/bin/deno             14.580 µs    1.083 times faster
../../../deno/target/release-lite/deno      15.789 µs    
--- sha512-100 ---
/home/nathanwhit/.deno/bin/deno             13.477 µs    1.131 times faster
../../../deno/target/release-lite/deno      15.238 µs    
--- sha256-1000 ---
../../../deno/target/release-lite/deno      17.208 µs    1.116 times faster
/home/nathanwhit/.deno/bin/deno             19.198 µs    
--- sha512-1000 ---
../../../deno/target/release-lite/deno      21.168 µs    1.026 times faster
/home/nathanwhit/.deno/bin/deno             21.717 µs    
--- sha256-10000 ---
../../../deno/target/release-lite/deno      33.586 µs    1.990 times faster
/home/nathanwhit/.deno/bin/deno             66.837 µs    
--- sha512-10000 ---
../../../deno/target/release-lite/deno      53.338 µs    1.009 times faster
/home/nathanwhit/.deno/bin/deno             53.817 µs    
--- sha256-100000 ---
../../../deno/target/release-lite/deno     168.238 µs    3.063 times faster
/home/nathanwhit/.deno/bin/deno            515.354 µs    
--- sha512-100000 ---
../../../deno/target/release-lite/deno     383.311 µs    1.036 times faster
/home/nathanwhit/.deno/bin/deno            397.122 µs    
--- sha256-1000000 ---
../../../deno/target/release-lite/deno       1.474 ms    3.471 times faster
/home/nathanwhit/.deno/bin/deno              5.115 ms    
--- sha512-1000000 ---
../../../deno/target/release-lite/deno       3.658 ms    1.057 times faster
/home/nathanwhit/.deno/bin/deno              3.865 ms    
--- sha256-10000000 ---
../../../deno/target/release-lite/deno      16.438 ms    3.136 times faster
/home/nathanwhit/.deno/bin/deno             51.556 ms    
--- sha512-10000000 ---
../../../deno/target/release-lite/deno      37.128 ms    1.056 times faster
/home/nathanwhit/.deno/bin/deno             39.220 ms    

Copy link
Member

@bartlomieju bartlomieju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool!

@nathanwhit nathanwhit enabled auto-merge (squash) January 30, 2025 23:06
@nathanwhit nathanwhit merged commit bac8171 into denoland:main Jan 30, 2025
17 checks passed
bartlomieju pushed a commit that referenced this pull request Jan 30, 2025
Currently we are using the pure rust backend of `sha2`, which has subpar
performance compared to asm implementations. We already depend on
`ring`, so just use that instead of `sha2` for sha256/sha512 digests.

This also speeds up things like S3 uploads, which calculate sha digests
of the uploaded objects. On my local machine, this speeds up uploading a
100MB file (to a localhost s3 provider via`@aws-sdk/client-s3`) by about
2x

<details>

<summary>Benchmark:</summary>

```ts
import { createHmac } from "node:crypto";

for (
  const size of [1, 10, 100, 1_000, 10_000, 100_000, 1_000_000, 10_000_000]
) {
  const input = "a".repeat(size);
  Deno.bench({
    name: `sha256-${size}`,
    fn() {
      const _hash = createHmac("sha256", input).update(input).digest();
    },
  });
  Deno.bench({
    name: `sha512-${size}`,
    fn() {
      const _hash = createHmac("sha512", input).update(input).digest();
    },
  });
}
```

</details>

<details>

<summary>Results (arm64 macOS):</summary>

```
--- sha256-1 ---
../../deno/target/release/deno         2.527 µs    1.240 times faster
/Users/nathanwhit/.deno/bin/deno       3.132 µs    
--- sha512-1 ---
../../deno/target/release/deno         3.364 µs    1.071 times faster
/Users/nathanwhit/.deno/bin/deno       3.603 µs    
--- sha256-10 ---
../../deno/target/release/deno         3.060 µs    1.027 times faster
/Users/nathanwhit/.deno/bin/deno       3.144 µs    
--- sha512-10 ---
../../deno/target/release/deno         3.583 µs    1.047 times faster
/Users/nathanwhit/.deno/bin/deno       3.751 µs    
--- sha256-100 ---
../../deno/target/release/deno         3.695 µs    1.244 times faster
/Users/nathanwhit/.deno/bin/deno       4.598 µs    
--- sha512-100 ---
../../deno/target/release/deno         3.386 µs    1.188 times faster
/Users/nathanwhit/.deno/bin/deno       4.021 µs    
--- sha256-1000 ---
../../deno/target/release/deno         4.007 µs    3.230 times faster
/Users/nathanwhit/.deno/bin/deno      12.944 µs    
--- sha512-1000 ---
../../deno/target/release/deno         6.463 µs    1.466 times faster
/Users/nathanwhit/.deno/bin/deno       9.477 µs    
--- sha256-10000 ---
../../deno/target/release/deno        11.674 µs    6.981 times faster
/Users/nathanwhit/.deno/bin/deno      81.493 µs    
--- sha512-10000 ---
../../deno/target/release/deno        31.250 µs    1.740 times faster
/Users/nathanwhit/.deno/bin/deno      54.364 µs    
--- sha256-100000 ---
../../deno/target/release/deno        82.800 µs    9.393 times faster
/Users/nathanwhit/.deno/bin/deno     777.719 µs    
--- sha512-100000 ---
../../deno/target/release/deno       269.726 µs    1.851 times faster
/Users/nathanwhit/.deno/bin/deno     499.243 µs    
--- sha256-1000000 ---
../../deno/target/release/deno       808.662 µs    9.427 times faster
/Users/nathanwhit/.deno/bin/deno       7.623 ms    
--- sha512-1000000 ---
../../deno/target/release/deno         2.672 ms    1.795 times faster
/Users/nathanwhit/.deno/bin/deno       4.795 ms    
--- sha256-10000000 ---
../../deno/target/release/deno         7.823 ms    9.868 times faster
/Users/nathanwhit/.deno/bin/deno      77.201 ms    
--- sha512-10000000 ---
../../deno/target/release/deno        26.197 ms    1.846 times faster
/Users/nathanwhit/.deno/bin/deno      48.356 ms    
```

</details>

<details>

<summary>Results (x86_64 linux):</summary>

```
--- sha256-1 ---
/home/nathanwhit/.deno/bin/deno             10.726 µs    1.229 times faster
../../../deno/target/release-lite/deno      13.184 µs    
--- sha512-1 ---
/home/nathanwhit/.deno/bin/deno             13.177 µs    1.051 times faster
../../../deno/target/release-lite/deno      13.845 µs    
--- sha256-10 ---
/home/nathanwhit/.deno/bin/deno             13.156 µs    1.047 times faster
../../../deno/target/release-lite/deno      13.780 µs    
--- sha512-10 ---
/home/nathanwhit/.deno/bin/deno             14.386 µs    1.029 times faster
../../../deno/target/release-lite/deno      14.807 µs    
--- sha256-100 ---
/home/nathanwhit/.deno/bin/deno             14.580 µs    1.083 times faster
../../../deno/target/release-lite/deno      15.789 µs    
--- sha512-100 ---
/home/nathanwhit/.deno/bin/deno             13.477 µs    1.131 times faster
../../../deno/target/release-lite/deno      15.238 µs    
--- sha256-1000 ---
../../../deno/target/release-lite/deno      17.208 µs    1.116 times faster
/home/nathanwhit/.deno/bin/deno             19.198 µs    
--- sha512-1000 ---
../../../deno/target/release-lite/deno      21.168 µs    1.026 times faster
/home/nathanwhit/.deno/bin/deno             21.717 µs    
--- sha256-10000 ---
../../../deno/target/release-lite/deno      33.586 µs    1.990 times faster
/home/nathanwhit/.deno/bin/deno             66.837 µs    
--- sha512-10000 ---
../../../deno/target/release-lite/deno      53.338 µs    1.009 times faster
/home/nathanwhit/.deno/bin/deno             53.817 µs    
--- sha256-100000 ---
../../../deno/target/release-lite/deno     168.238 µs    3.063 times faster
/home/nathanwhit/.deno/bin/deno            515.354 µs    
--- sha512-100000 ---
../../../deno/target/release-lite/deno     383.311 µs    1.036 times faster
/home/nathanwhit/.deno/bin/deno            397.122 µs    
--- sha256-1000000 ---
../../../deno/target/release-lite/deno       1.474 ms    3.471 times faster
/home/nathanwhit/.deno/bin/deno              5.115 ms    
--- sha512-1000000 ---
../../../deno/target/release-lite/deno       3.658 ms    1.057 times faster
/home/nathanwhit/.deno/bin/deno              3.865 ms    
--- sha256-10000000 ---
../../../deno/target/release-lite/deno      16.438 ms    3.136 times faster
/home/nathanwhit/.deno/bin/deno             51.556 ms    
--- sha512-10000000 ---
../../../deno/target/release-lite/deno      37.128 ms    1.056 times faster
/home/nathanwhit/.deno/bin/deno             39.220 ms    
```

</details>
@nathanwhit nathanwhit deleted the sha256-ring branch January 30, 2025 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants