Skip to content

Adding Dask support to mrun#927

Draft
tsmathis wants to merge 17 commits into
mainfrom
maggma-dask
Draft

Adding Dask support to mrun#927
tsmathis wants to merge 17 commits into
mainfrom
maggma-dask

Conversation

@tsmathis

Copy link
Copy Markdown
Collaborator

Summary

Major changes:

added a Dask work broker for use with mrun

Todos

  • test run a builder on a distributed cluster
  • add documentation for the differences needed to write Dask-compatible builders

Checklist

  • Google format doc strings added.
  • Code linted with ruff. (For guidance in fixing rule violates, see rule list)
  • Type annotations included. Check with mypy.
  • Tests added for new features/fixes.
  • I have run the tests locally and they passed.

Comment thread src/maggma/cli/__init__.py Fixed
else:
worker(url=url, port=port, num_processes=num_processes, no_bars=no_bars)
elif dask:
dask_executor(

Check failure

Code scanning / CodeQL

Potentially uninitialized local variable

Local variable 'dask_executor' may be used before it is initialized.
@codecov

codecov Bot commented Feb 27, 2024

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 25.37313% with 50 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.58%. Comparing base (53867d9) to head (245cf3e).
⚠️ Report is 550 commits behind head on main.

Files with missing lines Patch % Lines
src/maggma/cli/dask_executor.py 0.00% 47 Missing ⚠️
src/maggma/cli/__init__.py 85.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #927      +/-   ##
==========================================
- Coverage   81.56%   80.58%   -0.99%     
==========================================
  Files          46       47       +1     
  Lines        3938     4007      +69     
==========================================
+ Hits         3212     3229      +17     
- Misses        726      778      +52     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants