Having node run conditionally? #1181
Replies: 6 comments 8 replies
-
So this isn't something we have out of the box - the idea to include some sort of conditional node is one I've been thinking about for a while. We've also been exploring more comprehensive DVC integration, but it's a way off. One way you could do it is to do an
It's not the prettiest and for now you would have to come up with the logic for defining if the data is different. |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
That would be interesting. We were trying to integrate DVC with kedro.
…On Thu, Jan 27, 2022 at 2:06 PM Joel Schwarzmann ***@***.***> wrote:
@wmoreiraa <https://github.com/wmoreiraa> nice! Would you be interested
in sharing a sample hook or project with the community?
—
Reply to this email directly, view it on GitHub
<#1181 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADHYYT7JWBFYEA2NK5VLWQTUYD72XANCNFSM5M3LIPPQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Sure, guys!
@datajoely Could we schedule a call early next week? Then we could discuss what we’re
aiming for and what we already have, then we could share with the community right after.
I will send you a DM with my e-mail
|
Beta Was this translation helpful? Give feedback.
-
There is a parallel conversation happening here too: #837 |
Beta Was this translation helpful? Give feedback.
-
Another ancient issue connected to this topic #104 |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Sorry if I missed it in the docs but is there a way to conditionally run a node in a pipeline? My use case is basically trying to do data version control where the first node of the pipeline reads in data from several tables in a database and hashes each of them (along the lines of DVC). Subsequent nodes would only run if that data had changed from what had been seen previously (i.e. if any of the tables had a hash that hadn't been previously seen).
Beta Was this translation helpful? Give feedback.
All reactions