Skip to content

Conversation

@rly
Copy link
Contributor

@rly rly commented Nov 2, 2025

Fix #165

This PR:

  • Updates the download benchmarks to run at most once (the lindi ones were running multiple times because it was so fast on my machine) with a long timeout (the big files were taking longer than 1 hour)
  • Updates the download benchmarks to download to a persistent directory: ~/.nwb_benchmarks/download
  • Updates the download benchmarks to always overwrite existing files instead of erroring if it exists
  • Moves the currently skipped benchmark that creates a LINDI JSON from a remote file into a separate file for clarity
  • Updates lindi benchmark to download the lindi file if it does not exist in the persistent directory during setup. If it does exist, use that. This saves a tiny bit of time.
  • Adds lindi "preloaded cache" benchmark where the read operation is done in setup
  • Adds dandi utility functions used in lindi benchmarks
  • Removes _download.py which is no longer used

@rly rly enabled auto-merge November 3, 2025 00:59
@rly rly merged commit f0f37b1 into main Nov 3, 2025
3 checks passed
@rly rly deleted the lindi_download branch November 3, 2025 03:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create LINDI download-then-read benchmark

3 participants