Skip to content

Prefer current and default clients over worker clients in distributed.worker.get_client() #8106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

minhnguyenxuan60
Copy link
Contributor

@minhnguyenxuan60 minhnguyenxuan60 commented Aug 15, 2023

Closes #7965

  • Tests added / passed
  • Passes pre-commit run --all-files

@GPUtester
Copy link
Collaborator

Can one of the admins verify this patch?

Admins can comment ok to test to allow this one PR to run or add to allowlist to allow all future PRs from the same author to run.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 15, 2023

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       21 files  ±    0         21 suites  ±0   9h 57m 38s ⏱️ - 1h 29m 15s
  3 784 tests +    7    3 669 ✔️ ±    0     107 💤  -   1    8 +  8 
35 540 runs   - 962  33 763 ✔️  - 927  1 745 💤  - 67  32 +32 

For more details on these failures, see this check.

Results for commit 1afd997. ± Comparison against base commit acb2809.

This pull request removes 1 and adds 8 tests. Note that renamed tests count towards both.
distributed.tests.test_client ‑ test_short_tracebacks
distributed.shuffle.tests.test_shuffle ‑ test_basic_integration_local_cluster[False]
distributed.shuffle.tests.test_shuffle ‑ test_basic_integration_local_cluster[True]
distributed.tests.test_client ‑ test_get_scheduler
distributed.tests.test_client ‑ test_short_tracebacks[gather]
distributed.tests.test_client ‑ test_short_tracebacks[result]
distributed.tests.test_client ‑ test_short_tracebacks_async[await]
distributed.tests.test_client ‑ test_short_tracebacks_async[gather]
distributed.tests.test_client ‑ test_short_tracebacks_async[result]

♻️ This comment has been updated with latest results.

@fjetter
Copy link
Member

fjetter commented Aug 15, 2023

I noticed some linting issues https://github.com/dask/distributed/actions/runs/5864563511/job/15899846304?pr=8106
I recommend install pre-commit hooks to avoid this and autoformat your code before committing. see also https://docs.dask.org/en/stable/develop.html#code-formatting

@fjetter
Copy link
Member

fjetter commented Aug 15, 2023

There are a couple of related tests failing. Let me know if you need help figuring out what's the correct behavior

@minhnguyenxuan60
Copy link
Contributor Author

Hi Florian, I've fixed the pre commit black issue. May I ask you to clarify what exactly do you think is right behavior. According to the way I understand, get_client method should try to get client.current() first and if that fails, _get_client() should return default first and if this fails, it will try to return worker client.

@minhnguyenxuan60
Copy link
Contributor Author

Hello, Florian. I'm still waiting for your response on the test I'm failing and what you believe is wrong with it. I'd also like to know what I might be able to do to fix it. Thank you!

@hendrikmakait hendrikmakait self-requested a review August 23, 2023 08:17
@hendrikmakait
Copy link
Member

Hi @minhnguyenxuan60, I'll take this over from @fjetter, who's unavailable this week. The best way to check if your behavior matches the desired behavior would be to add a test that mimics the reproducer in #7965.

According to the way I understand, get_client method should try to get client.current() first and if that fails, _get_client() should return default first and if this fails, it will try to return worker client.

From what I understand from #7965, get_client should return Client.current() if available. If not, return default_client() if this is available, otherwise, follow the previously existing fallback logic of returning a worker client or some newly client for the given address.

This breaks with the current behavior that focused on returning the worker client, so I assume it will break some tests. We can look into this once the reproducer from #7965 works as expected.

@hendrikmakait hendrikmakait changed the title Issue #7965: Changed order to pick worker client as suggested by Florian Jetter. Change order of precedence in distributed.worker.get_client() to favor the current and default clients over worker clients Aug 23, 2023
@hendrikmakait hendrikmakait changed the title Change order of precedence in distributed.worker.get_client() to favor the current and default clients over worker clients Prefer current and default clients over worker clients in distributed.worker.get_client() Aug 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Default and current client ignored in worker
4 participants