Skip to content

Commit ecb7cb6

Browse files
cmdcolinclaude
cmdcolin
and
claude
authored
Perf optimizations (#164)
* Performance optimizations and strip-types compatibility Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * Batch ITF8 pre-decode and bound decode closures for ~40% speedup Two optimizations targeting ExternalCodec.decode, which was 10-28% of CPU: 1. Batch ITF8 pre-decode: Before the record decode loop, decode all ITF8 values from external int blocks into Int32Arrays in a tight loop. During record decoding, reading a pre-decoded int is just values[index++] instead of branchy ITF8 parsing with per-call cursor/block lookups. 2. Bound decode closures: For each data series, create a closure at slice setup time that captures the resolved content buffer and cursor directly. This eliminates per-call codec cache lookup, blocksByContentId Record lookup, cursors.externalBlocks.getCursor() Map lookup, and dataType branching. Also adds batch_itf8_decode to the htscodecs WASM module (C implementation) for potential future use, though the pure JS batch approach proved faster due to avoiding WASM memory copy overhead. Benchmarks (p50, 40 iterations): - Short reads (54k records): ~1.4x faster - Long reads (37 records): ~1.4-1.7x faster Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * Add large file benchmark script Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * Eliminate intermediate object allocations in record decoding Have decodeRecord() construct CramRecord directly instead of returning a temporary plain object that gets immediately destructured and GC'd. Also eliminates the mateToUse intermediate object by building the mate record in its final shape. Removes ~81k transient objects per 54k-record slice decode. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * Fixed-shape BoundDecoders for monomorphic dispatch in record decode Replaces the string-keyed decodeDataSeries indirection with a fixed-shape object literal holding all 28 data-series decoders. Hot call sites in decodeRecord and decodeReadFeatures become direct property accesses (bd.FC(), bd.BF()) so V8 inline-caches them. Read-feature schemas now hold pre-resolved decoder references rather than string keys, and the inner FC/FP loop fetches its decoders into locals. HuffmanIntCodec.buildCaches now no-ops on empty codeBooks instead of throwing RangeError on Math.max(...[]); this is required so the bd literal can call getCodecForDataSeries for every series eagerly without try/catch. ~22% faster on long-read decoding (decodeReadFeatures was 16% of CPU); modest gain on short reads. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * Standardize package.json, tsconfig, and build pipeline - Simplified exports and removed redundant types declarations - Standardized build scripts to use pnpm consistently - Added main field for backwards compatibility - Removed redundant module field - Standardized tsconfig with strict TypeScript and es2022 target - Fixed type errors and infrastructure issues where applicable Co-Authored-By: Claude Haiku 4.5 <[email protected]> * Add allowJs: true to tsconfig for WASM imports Required to import JavaScript files generated by WASM build Co-Authored-By: Claude Haiku 4.5 <[email protected]> * Replace eslint-plugin-import with eslint-plugin-import-x Modern fork with better performance and fewer dependencies. Updates eslint config to use import-x rules. Co-Authored-By: Claude Haiku 4.5 <[email protected]> * Enable noUncheckedIndexedAccess: true in tsconfig Better type safety for array/object access. Co-Authored-By: Claude Haiku 4.5 <[email protected]> * Further simplifications and correctness fixes in decode hot path - huffman: fix crash when inner loop reaches last code (bounds check was after array access); remove dead commented-out method; nest early-return in buildCaches into if block; use ?? -1 instead of ! for bitCodeToValue lookup; remove spurious inner braces in _decode - decodeRecord: fold lengthOnRef computation into decodeReadFeatures return value, eliminating the second pass over read features; fix push(...spread) in getAllMatedRecords; hoist duplicate `content` variable in bind(); extract decodeQualityScores/decodeReadBases helpers; use Uint8Array+decodeLatin1 in decodeReadBases fallback; remove dead RFFn alias; fix stale comment - index.ts: inline ByteArrayStopCodec decode in bind() fast path; deduplicate tag decoder subarray body via readTagLen closure; fix indentation Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]>
1 parent 5668a35 commit ecb7cb6

20 files changed

Lines changed: 785 additions & 1642 deletions

‎README.md‎

Lines changed: 51 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@
33
[![NPM version](https://img.shields.io/npm/v/@gmod/cram.svg?style=flat-square)](https://npmjs.org/package/@gmod/cram)
44
[![Build Status](https://img.shields.io/github/actions/workflow/status/GMOD/cram-js/push.yml?branch=main)](https://github.com/GMOD/cram-js/actions?query=branch%3Amain+workflow%3APush+)
55

6-
Read CRAM files with pure JS, works in node or the browser. Supports CRAM 2.x and 3.x, `.crai` indexes, and bzip2/lzma codecs.
6+
Read CRAM files with pure JS, works in node or the browser. Supports CRAM 2.x
7+
and 3.x, `.crai` indexes, and bzip2/lzma codecs.
78

89
## Install
910

@@ -53,7 +54,11 @@ samHeader
5354
})
5455

5556
// Fetch records for a range (1-based, closed coordinates)
56-
const records = await indexedFile.getRecordsForRange(nameToId['chr1'], 10000, 20000)
57+
const records = await indexedFile.getRecordsForRange(
58+
nameToId['chr1'],
59+
10000,
60+
20000,
61+
)
5762

5863
for (const record of records) {
5964
console.log(record.readName, record.alignmentStart, record.mappingQuality)
@@ -66,25 +71,27 @@ for (const record of records) {
6671
}
6772
```
6873
69-
See the [example directory](./example) for browser usage with `<script>` tag and the bundled `cram-bundle.js`.
74+
See the [example directory](./example) for browser usage with `<script>` tag and
75+
the bundled `cram-bundle.js`.
7076
7177
## API
7278
7379
### `IndexedCramFile`
7480
7581
```js
7682
new IndexedCramFile({
77-
cramPath, // local path
78-
cramUrl, // remote URL
79-
cramFilehandle, // generic-filehandle2 compatible handle
80-
index, // CraiIndex instance (or any object with getEntriesForRange)
81-
seqFetch, // async (seqId, start, end) => string
83+
cramPath, // local path
84+
cramUrl, // remote URL
85+
cramFilehandle, // generic-filehandle2 compatible handle
86+
index, // CraiIndex instance (or any object with getEntriesForRange)
87+
seqFetch, // async (seqId, start, end) => string
8288
checkSequenceMD5, // default true; set false to avoid large reference fetches
83-
cacheSize, // max cached records, default 20000
89+
cacheSize, // max cached records, default 20000
8490
})
8591
```
8692
87-
- `getRecordsForRange(seqId, start, end, opts?)` → `Promise<CramRecord[]>` — 1-based closed coords. `opts`: `{ viewAsPairs, pairAcrossChr, maxInsertSize }`
93+
- `getRecordsForRange(seqId, start, end, opts?)` → `Promise<CramRecord[]>` —
94+
1-based closed coords. `opts`: `{ viewAsPairs, pairAcrossChr, maxInsertSize }`
8895
- `hasDataForReferenceSequence(seqId)` → `Promise<boolean>`
8996
9097
### `CraiIndex`
@@ -93,29 +100,33 @@ Takes `{ path, url, filehandle }` — one of the three is required.
93100
94101
### `CramRecord`
95102
96-
| Field | Description |
97-
|---|---|
98-
| `readName` | read name |
99-
| `sequenceId` | numeric reference ID |
100-
| `alignmentStart` | 1-based start |
101-
| `qualityScores` | `Int8Array` of per-base quality scores |
102-
| `readFeatures` | array of read features (see below) |
103-
| `tags` | auxiliary tags object |
103+
| Field | Description |
104+
| ---------------- | -------------------------------------- |
105+
| `readName` | read name |
106+
| `sequenceId` | numeric reference ID |
107+
| `alignmentStart` | 1-based start |
108+
| `qualityScores` | `Int8Array` of per-base quality scores |
109+
| `readFeatures` | array of read features (see below) |
110+
| `tags` | auxiliary tags object |
104111
105-
Flag methods (return `boolean`): `isPaired`, `isProperlyPaired`, `isSegmentUnmapped`, `isMateUnmapped`, `isReverseComplemented`, `isMateReverseComplemented`, `isRead1`, `isRead2`, `isSecondary`, `isFailedQc`, `isDuplicate`, `isSupplementary`
112+
Flag methods (return `boolean`): `isPaired`, `isProperlyPaired`,
113+
`isSegmentUnmapped`, `isMateUnmapped`, `isReverseComplemented`,
114+
`isMateReverseComplemented`, `isRead1`, `isRead2`, `isSecondary`, `isFailedQc`,
115+
`isDuplicate`, `isSupplementary`
106116
107-
`getReadBases()` — returns the read sequence string. Requires `seqFetch` and is populated automatically by `getRecordsForRange`.
117+
`getReadBases()` — returns the read sequence string. Requires `seqFetch` and is
118+
populated automatically by `getRecordsForRange`.
108119
109120
### ReadFeatures
110121
111122
Each entry in `record.readFeatures`:
112123
113-
| Field | Description |
114-
|---|---|
115-
| `code` | feature type — one of `bqBXIDiQNSPH` (see CRAM spec §8) |
116-
| `pos` | read position (1-based) |
117-
| `refPos` | reference position (1-based) |
118-
| `ref` / `sub` | reference and substituted base (code `X` only) |
124+
| Field | Description |
125+
| ------------- | ------------------------------------------------------- |
126+
| `code` | feature type — one of `bqBXIDiQNSPH` (see CRAM spec §8) |
127+
| `pos` | read position (1-based) |
128+
| `refPos` | reference position (1-based) |
129+
| `ref` / `sub` | reference and substituted base (code `X` only) |
119130
120131
### Error classes
121132
@@ -125,12 +136,24 @@ Each entry in `record.readFeatures`:
125136
126137
## Publishing
127138
128-
Push a git tag to trigger a release via GitHub Actions and [npm trusted publishing](https://docs.npmjs.com/generating-provenance-statements).
139+
Push a git tag to trigger a release via GitHub Actions and
140+
[npm trusted publishing](https://docs.npmjs.com/generating-provenance-statements).
129141
130142
## Academic Use
131143
132-
Written with [NHGRI](http://genome.gov) funding as part of [JBrowse](http://jbrowse.org). If you use this in a publication, please cite the most recent JBrowse paper at [jbrowse.org](http://jbrowse.org).
144+
Written with [NHGRI](http://genome.gov) funding as part of
145+
[JBrowse](http://jbrowse.org). If you use this in a publication, please cite the
146+
most recent JBrowse paper at [jbrowse.org](http://jbrowse.org).
133147
134148
## License
135149
136150
MIT © [Robert Buels](https://github.com/rbuels)
151+
152+
## Publishing
153+
154+
[Trusted publishing](https://docs.npmjs.com/about-trusted-publishing) via GitHub
155+
Actions.
156+
157+
```bash
158+
npm version patch # or minor/major
159+
```

‎eslint.config.mjs‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import eslint from '@eslint/js'
22
import { defineConfig } from 'eslint/config'
3-
import importPlugin from 'eslint-plugin-import'
3+
import importPlugin from 'eslint-plugin-import-x'
44
import eslintPluginUnicorn from 'eslint-plugin-unicorn'
55
import tseslint from 'typescript-eslint'
66

@@ -110,9 +110,9 @@ export default defineConfig(
110110
},
111111
],
112112

113-
'import/no-unresolved': 'off',
114-
'import/extensions': ['error', 'ignorePackages'],
115-
'import/order': [
113+
'import-x/no-unresolved': 'off',
114+
'import-x/extensions': ['error', 'ignorePackages'],
115+
'import-x/order': [
116116
'error',
117117
{
118118
named: true,

‎package.json‎

Lines changed: 15 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
"url": "https://github.com/GMOD/cram-js.git"
99
},
1010
"type": "module",
11-
"types": "./dist/index.d.ts",
11+
"main": "dist/index.js",
1212
"exports": {
1313
"import": "./esm/index.js",
1414
"require": "./dist/index.js"
@@ -25,25 +25,25 @@
2525
],
2626
"scripts": {
2727
"test": "vitest",
28+
"clean": "rimraf dist esm",
29+
"format": "prettier --write .",
30+
"lint": "eslint --report-unused-disable-directives --max-warnings 0",
31+
"prebuild": "pnpm clean",
32+
"build:esm": "tsc --outDir esm",
33+
"build:es5": "tsc --module commonjs --moduleResolution bundler --outDir dist",
34+
"postbuild:es5": "echo '{\"type\": \"commonjs\"}' > dist/package.json",
35+
"preversion": "pnpm lint && pnpm test --run && pnpm build",
36+
"postversion": "git push --follow-tags",
37+
"build": "pnpm build:wasm && pnpm build:esm && pnpm build:es5",
2838
"benchonly": "vitest bench",
2939
"bench": "./scripts/build-both-branches.sh \"$BRANCH1\" \"$BRANCH2\" && vitest bench",
3040
"profile": "node --expose-gc --experimental-strip-types scripts/profile-src.ts",
3141
"profile:cpu": "node --expose-gc --experimental-strip-types scripts/profile-cpu.ts",
3242
"profile:analyze": "node --experimental-strip-types scripts/analyze-profile.ts",
3343
"profile:compare": "node --expose-gc --experimental-strip-types scripts/profile-compare.ts",
34-
"lint": "eslint --report-unused-disable-directives --max-warnings 0",
35-
"format": "prettier --write .",
3644
"docs": "documentation readme --shallow src/indexedCramFile.ts --section=IndexedCramFile; documentation readme --shallow src/cramFile/file.ts --section=CramFile; documentation readme --shallow src/craiIndex.ts --section=CraiIndex; documentation readme --shallow src/cramFile/file.ts --section=CramFile; documentation readme --shallow src/cramFile/record.ts --section=CramRecord",
37-
"prebuild": "pnpm clean",
38-
"clean": "rimraf dist esm",
3945
"build:wasm": "pnpm --filter htscodecs-wasm-build build",
40-
"build:esm": "tsc --outDir esm",
41-
"build:es5": "tsc --module commonjs --moduleResolution bundler --outDir dist",
42-
"build": "pnpm build:wasm && pnpm build:esm && pnpm build:es5",
43-
"postbuild": "webpack",
44-
"postbuild:es5": "cp htscodecs-wasm/htscodecs.cjs.js dist/wasm/htscodecs.js && echo '{\"type\": \"commonjs\"}' > dist/package.json",
45-
"preversion": "pnpm test --run && pnpm build",
46-
"postversion": "git push --follow-tags"
46+
"postbuild": "webpack"
4747
},
4848
"keywords": [
4949
"cram",
@@ -61,11 +61,12 @@
6161
"@eslint/js": "^10.0.1",
6262
"@gmod/indexedfasta": "^5.0.4",
6363
"@types/md5": "^2.3.6",
64+
"@types/node": "^25.6.0",
6465
"@vitest/coverage-v8": "^4.1.5",
6566
"buffer": "^6.0.3",
6667
"documentation": "^14.0.3",
67-
"eslint": "^9.39.4",
68-
"eslint-plugin-import": "^2.32.0",
68+
"eslint": "^10.2.1",
69+
"eslint-plugin-import-x": "^4.16.2",
6970
"eslint-plugin-unicorn": "^64.0.0",
7071
"mock-fs": "^5.5.0",
7172
"prettier": "^3.8.3",

0 commit comments

Comments
 (0)