-
Notifications
You must be signed in to change notification settings - Fork 815
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Ready for Review] Support more convenient foreach variable/value search-ability. #1664
Conversation
410b3be
to
257b422
Compare
c5ae0b3
to
f337a5b
Compare
af9a112
to
66feea5
Compare
33c52a1
to
054b9fd
Compare
metaflow/runtime.py
Outdated
@@ -1027,7 +1027,13 @@ def results(self): | |||
@property | |||
def finished_id(self): | |||
# note: id is not available before the task has finished | |||
return (self.step, tuple(self.results["_foreach_stack"])) | |||
# index already identifies the task within the foreach, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is this needed? Isn't the presence of the value backward compatible with the old code?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for a node, we may want to quickly get the finish_id of the sibling node and check if they all finished (can check usage of finished_id
within _queue_task_join
). It is non-trivial to get value of the sibling, so we may as well not compare value at all (afterall, the index of the foreach already identifies the node).
c191b4f
to
56b081b
Compare
56b081b
to
e45e6c3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM with a small typo. Let's wait for someone from OB but we can merge afterwards.
e45e6c3
to
59babad
Compare
In this change, we
Currently, we try to get "value" of the for each in the following order:
["a", "b", "c"]
, we will use the literal value directlyThe end result will be of the form
[value]
. We will thentask_finish
function inside a custom decorator.An example for the updated search UI is like the following.
