Skip to content

Conversation

@luizirber
Copy link
Member

While the current implementation is simple, it is also

  • serial, reading one sig at a time
  • only write output at the end, when all sigs are processed

The changes here allow reading sigs in parallel, and writing manifest records as they are generated, which can be useful when building manifests for huge collections (like wort), and restarting calculation without recalculating everything.

@luizirber luizirber changed the title Write output for manifest generation in parallel feat: write output for manifest generation in parallel Aug 28, 2025
@luizirber luizirber force-pushed the lirber/parallel_manifest_write branch 3 times, most recently from 38f8b53 to 47e5a40 Compare August 29, 2025 14:23
@luizirber luizirber force-pushed the lirber/parallel_manifest_write branch from 47e5a40 to f200dca Compare August 29, 2025 15:38
@luizirber luizirber force-pushed the lirber/parallel_manifest_write branch from e0e9a23 to f403a80 Compare August 29, 2025 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants