Add a job query type for running a file from git - #1974
Open
amritghimire wants to merge 1 commit into
Open
Conversation
Studio can already clone a repository into a job's working directory, but the code it runs still has to be pasted into the job, so it drifts from the repo. Studio is adding a third query type where a job stores a path inside the repo instead of the source, and the worker runs that file straight out of the clone. The enum lives here, and both the Studio backend and its workers call JobQueryType(job.query_type) on the way to the worker, so the value has to exist here before any of that works. While we are here, the CLI mapped anything that was not "PYTHON" onto SHELL when mirroring a job into the local metastore. That was fine with two members and quietly wrong with three, so we look the member up by name instead.
Deploying datachain with
|
| Latest commit: |
edad8e4
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://7c871af4.datachain-2g6.pages.dev |
| Branch Preview URL: | https://amrit-run-from-git-ui.datachain-2g6.pages.dev |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
dreadatour
approved these changes
Sep 2, 2026
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The focused enum and mapping changes are correct and preserve existing behavior.
Pull request overview
Adds FROM_GIT job-query compatibility for Studio and SaaS integrations.
Changes:
- Adds
JobQueryType.FROM_GIT = 3. - Replaces binary query-type mapping with enum name lookup.
File summaries
| File | Description |
|---|---|
src/datachain/studio.py |
Maps query types directly to enum members. |
src/datachain/data_storage/job.py |
Defines the new FROM_GIT query type. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Studio clones a repo into the job working directory already, but the code that actually runs still has to be pasted into the job, so it drifts from the repo.
The Studio side adds a
FROM_GITquery type wherequeryholds a repo-relative path instead of source, and the worker runs that file straight out of the clone. This enum has to land first:JobFullResponse.from_modelin the backend andSaaSJobFull.from_dictindatachain_saasboth doJobQueryType(job.query_type), so a value of 3 raisesValueErroruntil the member exists here.Also fixes something the new member makes reachable.
create_jobmapped anything that was not"PYTHON"ontoSHELLwhen mirroring a job into the local metastore:Fine with two members, quietly wrong with three, so it looks the member up by name now. The value is derived from the filename right there in
create_job, so this was latent rather than biting anyone yet.Nothing else in this repo reads the new member.