Skip to content

Inline external-sorting module to fix ESM usage#23

Merged
cmdcolin merged 3 commits intomasterfrom
fixes
Jan 18, 2026
Merged

Inline external-sorting module to fix ESM usage#23
cmdcolin merged 3 commits intomasterfrom
fixes

Conversation

@cmdcolin
Copy link
Copy Markdown
Collaborator

This crashed on windows, which used the non-GNU sort implementation of external-sorting but this package was badly configured for ESM usage

example error from windows jbrowse desktop

    [Browser SEVERE] file:///D:/a/jbrowse-components/jbrowse-components/products/jbrowse-desktop/dist/win-unpacked/resources/app.asar/build/static/js/main.b79dc5be.js 197830:16 TypeError: external_sorting__WEBPACK_IMPORTED_MODULE_0__ is not a function
    at sortLinesExternal (file:///D:/a/jbrowse-components/jbrowse-components/products/jbrowse-desktop/dist/win-unpacked/resources/app.asar/build/static/js/3939.cdf1e41e.chunk.js:74597:9)
    at makeIxWithJsSort (file:///D:/a/jbrowse-components/jbrowse-components/products/jbrowse-desktop/dist/win-unpacked/resources/app.asar/build/static/js/3939.cdf1e41e.chunk.js:19428:93)
    at makeIxStream (file:///D:/a/jbrowse-components/jbrowse-components/products/jbrowse-desktop/dist/win-unpacked/resources/app.asar/build/static/js/3939.cdf1e41e.chunk.js:19437:11)
    at ixIxxStream (file:///D:/a/jbrowse-components/jbrowse-components/products/jbrowse-desktop/dist/win-unpacked/resources/app.asar/build/static/js/3939.cdf1e41e.chunk.js:76090:74)
    at runIxIxx (file:///D:/a/jbrowse-components/jbrowse-components/products/jbrowse-desktop/dist/win-unpacked/resources/app.asar/build/static/js/5598.5f75b254.chunk.js:53709:42)
    at indexDriver (file:///D:/a/jbrowse-components/jbrowse-components/products/jbrowse-desktop/dist/win-unpacked/resources/app.asar/build/static/js/5598.5f75b254.chunk.js:53608:9)
    at perTrackIndex (file:///D:/a/jbrowse-components/jbrowse-components/products/jbrowse-desktop/dist/win-unpacked/resources/app.asar/build/static/js/5598.5f75b254.chunk.js:53542:11)
    at indexTracks (file:///D:/a/jbrowse-components/jbrowse-components/products/jbrowse-desktop/dist/win-unpacked/resources/app.asar/build/static/js/5598.5f75b254.chunk.js:53498:35)
    at TextIndexRpcMethod.execute (file:///D:/a/jbrowse-components/jbrowse-components/products/jbrowse-desktop/dist/win-unpacked/resources/app.asar/build/static/js/5598.5f75b254.chunk.js:53729:11)
    at file:///D:/a/jbrowse-components/jbrowse-components/products/jbrowse-desktop/dist/win-unpacked/resources/app.asar/build/static/js/5598.5f75b254.chunk.js:30675:12

This PR inlines external sorting and adds extra testing to ensure it works

@cmdcolin cmdcolin merged commit cc450be into master Jan 18, 2026
1 check passed
@cmdcolin
Copy link
Copy Markdown
Collaborator Author

AI summary:

Bug Fixes

  • Buffer concatenation (only appending actual bytes read)
  • Division by zero guard in optimizePrefixSize
  • Exit codes in bin.ts
  • Error propagation with Promise.race in makeIx.ts

Resource Management

  • FileParser now keeps file handle open (was reopening on every call)
  • Temp directory cleanup in sortLinesExternal
  • Try-finally for file handles

Stream Improvements

  • Replaced .pipe() chains with pipeline() for proper error handling
  • Simplified single-file merge case (15 lines → 3 lines)

Code Simplifications

  • Removed unused sortLinesInMemory
  • Simplified compare and swap functions
  • Moved PassThrough to static import
  • String concatenation → join() patterns

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant