Features/alignment files indexing on ingestion json

[noctillion]

Index cram and bam files if the index is not available on ingestion in the experiments_json_with_files workflow.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.91%. Comparing base (1ddf753) to head (e46fa39).

Additional details and impacted files

@@           Coverage Diff            @@
##           develop     #574   +/-   ##
========================================
  Coverage    92.91%   92.91%           
========================================
  Files          124      124           
  Lines         4486     4486           
  Branches       391      391           
========================================
  Hits          4168     4168           
  Misses         228      228           
  Partials        90       90           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[davidlougheed]

this part of the code still isn't matching desired behaviour exactly - indices should be ingested if they're listed in the JSON; or, if no index is specified in the JSON, one should be generated (maybe in the python script using subprocess?) and put into the JSON. I think also prepare_for_drs should extract index files as well as file paths, and this task should stay naive to the type of file it is ingesting.

[davidlougheed]

I think for now the best way to write this workflow might unfortunately just be as a single big Python task which uses the requests library to ingest into DRS and, if needed, uses subprocess+samtools to generate indices for files missing the indices, rewriting the experiments JSON as it goes and then POSTing it to Katsu at the end.