Support compression in sync push/pull

[yume-chan]

Android 11 added support for Brotli compression in sync push/pull, Android 12 further added lz4 and zstd support. Using compression can result in up to 2x transfer speed.

Algorithms:

• Brotli https://android-review.googlesource.com/c/platform/system/core/+/1250677
• lz4 https://android-review.googlesource.com/c/platform/system/core/+/1276917
• Zstd https://android-review.googlesource.com/c/platform/system/core/+/1319091

From https://android-review.googlesource.com/c/platform/system/core/+/1276917, pull speed in MB/s:

           none    brotli    lz4
USB 3.0     120       110    190
USB 2.0      38        75     63

No pure-JS implementations of all three algorithms were found on NPM. Need to create them ourselves.

Android 12 also added a dry-run flag in push/pull for testing. Shouldn't be useful in real situations: https://android-review.googlesource.com/c/platform/system/core/+/1276918

[gogogoghost]

Have to using pure-JS to implement it? How about WebAssembly? It can also be cross-platform and has good performance.

[yume-chan]

WebAssembly is also an option. Do you know any WebAssembly build for those compression algorithms?

[kuzalekon]

Hi @yume-chan

For example, we can use these packages: brotli-wasm, lz4-wasm and zstd-wasm.

[yume-chan]

I had already checked brotli-wasm and lz4-wasm.

brotli-wasm is the most promising one, but it does work (out-of-the-box) with Vite (httptoolkit/brotli-wasm#8), so it can't be used in a library that may be consumed by any bundler.

lz4-wasm doesn't expose streaming compression API so can't be used.

zstd-wasm has 0 documentation so I don't know what it does.

[yume-chan]

Both WebKit and Firefox now support Brotli in CompressionStream/DecompressionStream.

Proposal to add Brotli into spec: whatwg/compression#80

Chrome issue: https://issues.chromium.org/issues/463397980

[yume-chan]

Initial support has been added in #843, but since Chrome doesn't support it, this has not been tested.