Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 10 additions & 45 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,12 @@
- [Examples](#examples)
- [Usage](#usage)
- [Importing Conflux](#importing-conflux)
- [Package Manager](#package-manager)
- [CDN](#cdn)
- [Creating a ZIP](#creating-a-zip)
- [Example using `ReadableStream#pipeThrough`](#example-using-readablestreampipethrough)
- [Example using `writer.write`](#example-using-writerwrite)
- [Incorporating other streams](#incorporating-other-streams)
- [Reading ZIP files](#reading-zip-files)
- [Supporting Firefox](#supporting-firefox)
- [Supporting Legacy Browsers](#supporting-legacy-browsers)
- [License](#license)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->
Expand All @@ -43,7 +41,7 @@

## Blazing Fast

- ~100 kB import
- ~55 kB import
- Uses streams, minimizing memory overhead

## Compatibility
Expand All @@ -64,13 +62,7 @@

### Importing Conflux

#### Package Manager

```sh
# With Yarn
yarn add @transcend-io/conflux

# With NPM
npm install --save @transcend-io/conflux
```

Expand All @@ -79,17 +71,6 @@ npm install --save @transcend-io/conflux
import { Reader, Writer } from '@transcend-io/conflux';
```

#### CDN

```html
<script src="https://cdn.jsdelivr.net/npm/@transcend-io/conflux@3"></script>
```

```js
// Reader parses zip files, Writer builds zip files
const { Reader, Writer } = window.conflux;
```

### Creating a ZIP

#### Example using `ReadableStream#pipeThrough`
Expand All @@ -98,7 +79,7 @@ const { Reader, Writer } = window.conflux;
import { Writer } from '@transcend-io/conflux';
import streamSaver from 'streamsaver';

const s3 = 'https://s3-us-west-2.amazonaws.com/bencmbrook/';
const s3 = 'https://s3-us-west-2.amazonaws.com/your-bucket/';
const files = ['NYT.txt', 'water.png', 'Earth.jpg'].values();

const myReadable = new ReadableStream({
Expand Down Expand Up @@ -165,7 +146,7 @@ const fileStream = streamSaver.createWriteStream('conflux.zip');
});

const imgStream = await fetch(
'https://s3-us-west-2.amazonaws.com/bencmbrook/Earth.jpg',
'https://s3-us-west-2.amazonaws.com/your-bucket/Earth.jpg',
).then((r) => r.body);

writer.write({
Expand Down Expand Up @@ -195,31 +176,15 @@ fetch('https://cdn.jsdelivr.net/gh/Stuk/jszip/test/ref/deflate.zip').then(
);
```

## Supporting Firefox

Firefox [does not support ReadableStream#pipeThrough](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream#browser_compatibility), since it does not have `WritableStream` or `TransformStream` support yet. Conflux ponyfills `TransformStream` out of the box in Firefox, but if you're using the `myReadable.pipeThrough` and plan to support Firefox, you'll want to ponyfill `ReadableStream` like so:
## Supporting Legacy Browsers

```js
import { ReadableStream as ReadableStreamPonyfill } from 'web-streams-polyfill/ponyfill';

// Support Firefox with a ponyfill on ReadableStream to add .pipeThrough
const ModernReadableStream = window.WritableStream
? window.ReadableStream
: ReadableStreamPonyfill;
Conflux is compatible with all modern browsers since June 2022.

const myReadable = new ModernReadableStream({
async pull(controller) {
return controller.enqueue({
name: `/firefox.txt`,
stream: () => new Response.body('Firefox works!'),
});
},
});
If you need to support legacy browsers, you can add polyfills for:

myReadable
.pipeThrough(new Writer()) // see "Supporting Firefox" below
.pipeTo(streamSaver.createWriteStream('conflux.zip'));
```
- [`TransformStream`](https://developer.mozilla.org/en-US/docs/Web/API/TransformStream), and [`WritableStream`](https://developer.mozilla.org/en-US/docs/Web/API/WritableStream) (available in browsers since June 2022) by adding [web-streams-polyfill](https://www.npmjs.com/package/web-streams-polyfill).
- [`BigInt`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt) (available in browsers since January 2020) by setting `globalThis.JSBI` equal to [JSBI](https://github.com/GoogleChromeLabs/jsbi) before importing Conflux.
- [`globalThis`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/globalThis) (available in browsers since January 2020) by adding a polyfill like this [globalthis](https://www.npmjs.com/package/globalthis) or manually setting a shim.

## License

Expand Down
1 change: 0 additions & 1 deletion benchmark/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
</head>
<body>
<p id="loading">Benchmarking...</p>
<!-- <script src="https://cdn.jsdelivr.net/npm/web-streams-polyfill@2.0.2/dist/polyfill.min.js"></script> -->
<script src="https://cdn.jsdelivr.net/npm/lodash@4.17.11/lodash.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/benchmark@2.1.4/benchmark.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/jszip@3.2.2/dist/jszip.min.js"></script>
Expand Down
7 changes: 2 additions & 5 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@transcend-io/conflux",
"version": "4.1.2",
"version": "5.0.0",
"description": "Build zip files out of readable streams in the browser",
"main": "dist/conflux.umd.min.js",
"jsdelivr": "dist/conflux.umd.min.js",
Expand Down Expand Up @@ -98,10 +98,7 @@
"tape": "^5.7.2"
},
"dependencies": {
"@babel/runtime-corejs3": "^7.23.5",
"jsbi": "^4.3.0",
"pako": "^2.1.0",
"web-streams-polyfill": "^3.2.1"
"pako": "^2.1.0"
},
"packageManager": "yarn@4.5.3"
}
21 changes: 1 addition & 20 deletions rollup.config.js
Original file line number Diff line number Diff line change
@@ -1,25 +1,8 @@
import resolve from '@rollup/plugin-node-resolve';
import commonjs from '@rollup/plugin-commonjs';
import babel from '@rollup/plugin-babel';
import { terser } from 'rollup-plugin-terser';
import pkg from './package.json';

const babelDefaults = {
// see: https://github.com/rollup/plugins/tree/master/packages/babel#babelhelpers and the note about @babel/runtime for CJS/ES
babelHelpers: 'runtime',
configFile: './babel.config.js',
plugins: [
[
'@babel/plugin-transform-runtime',
{
regenerator: true,
corejs: { version: 3, proposals: true },
},
],
],
exclude: ['node_modules/**'],
};

export default [
// browser-friendly UMD build
{
Expand All @@ -34,7 +17,6 @@ export default [
plugins: [
resolve(), // so Rollup can find package dependencies
commonjs(), // so Rollup can convert package dependencies to an ES module
babel(babelDefaults),
terser(),
],
},
Expand All @@ -47,11 +29,10 @@ export default [
// `file` and `format` for each target)
{
input: 'src/index.js',
external: [/pako/, /jsbi/, /web-streams-polyfill/, /@babel\/runtime/],
external: [/pako/],
output: [
// { file: pkg.main, format: 'cjs' }, // don't need a Node import yet
{ file: pkg.module, format: 'es' },
],
plugins: [babel(babelDefaults)],
},
];
27 changes: 16 additions & 11 deletions src/bigint.js
Original file line number Diff line number Diff line change
@@ -1,18 +1,25 @@
/* global globalThis BigInt */

import JSBI from 'jsbi';

// eslint-disable-next-line import/no-mutable-exports
let jsbi;
if (!globalThis.BigInt && !globalThis.JSBI) {
throw new Error(
'BigInt is not supported in this browser.' +
' Conflux is unable to create zip files without BigInt support, or a JSBI polyfill.',
);
}

/**
* If BigInt is natively supported, change JSBI to use native expressions
* @see https://github.com/GoogleChromeLabs/jsbi/blob/master/jsbi.d.ts
* @see https://github.com/GoogleChromeLabs/babel-plugin-transform-jsbi-to-bigint/blob/master/src/index.js
* Use JSBI syntax for BigInt operations, instead of calling BigInt directly.
*
* This is NOT a polyfill. It uses native BigInt by default. Using this syntax simply makes it _possible_ to polyfill BigInt.
* If BigInt is not natively supported (ES2020+), library consumer MUST expose globalThis.JSBI before using Conflux.
*
* @see https://github.com/GoogleChromeLabs/jsbi - JSBI library for why BigInt
* @see https://github.com/GoogleChromeLabs/jsbi/blob/master/jsbi.d.ts - types
*/
if (globalThis.BigInt) {
jsbi = {};
const jsbi = globalThis.JSBI || {};

// If BigInt is natively supported, use it instead of JSBI.
if (globalThis.BigInt) {
// constructor
jsbi.BigInt = (a) => BigInt(a);

Expand Down Expand Up @@ -52,8 +59,6 @@ if (globalThis.BigInt) {
// static methods
jsbi.asIntN = (a, b) => BigInt.asIntN(a, b);
jsbi.asUintN = (a, b) => BigInt.asUintN(a, b);
} else {
jsbi = JSBI;
}

export default jsbi;
11 changes: 2 additions & 9 deletions src/write.js
Original file line number Diff line number Diff line change
@@ -1,13 +1,10 @@
/* global globalThis */
/**
* Conflux
* Build (and read) zip files with whatwg streams in the browser.
*
* @author Transcend Inc. <https://transcend.io>
* @license MIT
*/
// eslint-disable-next-line import/extensions
import { TransformStream as PonyfillTransformStream } from 'web-streams-polyfill/ponyfill';
import JSBI from './bigint.js';
import Crc32 from './crc.js';

Expand Down Expand Up @@ -93,6 +90,7 @@ class ZipTransformer {
zipObject.crc = new Crc32();
const reader = entry.stream().getReader();

// eslint-disable-next-line no-constant-condition
while (true) {
const it = await reader.read();
if (it.done) break;
Expand Down Expand Up @@ -167,12 +165,7 @@ class ZipTransformer {
}
}

const ModernTransformStream =
globalThis.TransformStream ||
globalThis.WebStreamsPolyfill?.TransformStream ||
PonyfillTransformStream;

class Writer extends ModernTransformStream {
class Writer extends TransformStream {
constructor() {
super(new ZipTransformer());
}
Expand Down
34 changes: 0 additions & 34 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -1329,16 +1329,6 @@ __metadata:
languageName: node
linkType: hard

"@babel/runtime-corejs3@npm:^7.23.5":
version: 7.23.5
resolution: "@babel/runtime-corejs3@npm:7.23.5"
dependencies:
core-js-pure: "npm:^3.30.2"
regenerator-runtime: "npm:^0.14.0"
checksum: 10c0/9bbad4ae7efea21e2c92ddee70b42ce9773a56e044cfc16267f9610b38ee531c87b465d84d39433fca93f7f567b47d5e40383e3d2cfe85dbeceea7fba8a52cc8
languageName: node
linkType: hard

"@babel/runtime@npm:^7.8.4":
version: 7.23.5
resolution: "@babel/runtime@npm:7.23.5"
Expand Down Expand Up @@ -1871,7 +1861,6 @@ __metadata:
"@babel/plugin-transform-classes": "npm:^7.23.5"
"@babel/plugin-transform-runtime": "npm:^7.23.4"
"@babel/preset-env": "npm:^7.23.5"
"@babel/runtime-corejs3": "npm:^7.23.5"
"@rollup/plugin-babel": "npm:^6.0.4"
"@rollup/plugin-commonjs": "npm:^25.0.7"
"@rollup/plugin-json": "npm:^6.0.1"
Expand All @@ -1885,7 +1874,6 @@ __metadata:
eslint-config-airbnb-base: "npm:^15.0.0"
eslint-plugin-import: "npm:2.29.0"
git-rev-sync: "npm:^3.0.2"
jsbi: "npm:^4.3.0"
karma: "npm:^6.4.4"
karma-browserstack-launcher: "npm:^1.6.0"
karma-chrome-launcher: "npm:^3.2.0"
Expand All @@ -1900,7 +1888,6 @@ __metadata:
rollup-plugin-node-polyfills: "npm:^0.2.1"
rollup-plugin-terser: "npm:^7.0.2"
tape: "npm:^5.7.2"
web-streams-polyfill: "npm:^3.2.1"
languageName: unknown
linkType: soft

Expand Down Expand Up @@ -3297,13 +3284,6 @@ __metadata:
languageName: node
linkType: hard

"core-js-pure@npm:^3.30.2":
version: 3.33.3
resolution: "core-js-pure@npm:3.33.3"
checksum: 10c0/97cf39cc013f6a4f77700762de36b495228b3c087fc04b61e86bfbfb475595529966cabbcf37e738e3a468c486e815c85118d120cc6fc4960da08a14caf69826
languageName: node
linkType: hard

"core-js@npm:^2.4.0, core-js@npm:^2.5.0":
version: 2.6.12
resolution: "core-js@npm:2.6.12"
Expand Down Expand Up @@ -5780,13 +5760,6 @@ __metadata:
languageName: node
linkType: hard

"jsbi@npm:^4.3.0":
version: 4.3.0
resolution: "jsbi@npm:4.3.0"
checksum: 10c0/1817ac1b50ea3f4438bcd84cadc9aee7a8657829f65b55ea6f151f401dbbd3babedbfdd3e4f481bd7b5472abb7823efa640fd7e5eee7c30cea6431f7a8b74696
languageName: node
linkType: hard

"jsesc@npm:^2.5.1":
version: 2.5.2
resolution: "jsesc@npm:2.5.2"
Expand Down Expand Up @@ -8714,13 +8687,6 @@ __metadata:
languageName: node
linkType: hard

"web-streams-polyfill@npm:^3.2.1":
version: 3.2.1
resolution: "web-streams-polyfill@npm:3.2.1"
checksum: 10c0/70ed6b5708e14afa2ab699221ea197d7c68ec0c8274bbe0181aecc5ba636ca27cbd383d2049f0eb9d529e738f5c088825502b317f3df24d18a278e4cc9a10e8b
languageName: node
linkType: hard

"webidl-conversions@npm:^3.0.0":
version: 3.0.1
resolution: "webidl-conversions@npm:3.0.1"
Expand Down
Loading