DI-4173: scope job debugging to current project in troubleshooting skill - #128
DI-4173: scope job debugging to current project in troubleshooting skill#128theyostalservice wants to merge 2 commits into
Conversation
list_jobs_runs is account-wide. Without filtering by project_id first, the agent picks the most-recent run across the entire account. The new Step 0 instructs: get project_id → filter list_jobs → list_jobs_runs by job_id. Mermaid diagram and Step 1 examples updated to match. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Updates the troubleshooting-dbt-job-errors skill to prevent debugging the wrong dbt job run by explicitly scoping Admin API run queries to the current project_id before selecting “most recent failed” runs.
Changes:
- Adds a new “Step 0” that requires determining the current
project_id, listing jobs, and filtering jobs to that project before querying runs. - Updates the mermaid workflow diagram to reflect the new scoping flow (
list_jobs→ filter →list_jobs_runs(job_id=...)). - Updates the Admin API tool list and example usage to emphasize always passing
job_idtolist_jobs_runs.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| ## Step 0: Scope to the Current Project | ||
|
|
||
| **The Admin API run tools are account-wide, not project-scoped.** `list_jobs_runs` without a `job_id` returns runs from *every* project in the account, so "the most recent run" may belong to a different project than the one you are investigating. |
There was a problem hiding this comment.
This isn't always true - if the user configures x-dbt-project-id then this would be scoped to a single project - actually that's probably why the skill doesn't include this step because this was a recent addition into the admin API to make the MCP tools multi-project.
There was a problem hiding this comment.
Good point — I dug into the pinned dbt-mcp (1.20.4) in the ACA venv to get the full picture:
- You're right that the tools aren't inherently account-wide.
AdminApiConfig.prod_environment_idscopeslist_jobsto a specific environment when set. And trueproject_idscoping is in-flight —dbt_admin/tools.py:list_jobshas a commented-out TODO for it. - The account-wide behavior is specific to how ACA/Studio wires the config.
map_admin_tool_contextin ACA buildsAdminApiConfigwithaccount_idonly (noprod_environment_id). I can't cleanly thread the current project's environment either —request_context.environment_idin Studio is the user's dev environment, not where scheduled jobs run, so scopinglist_jobsto it would usually return nothing. Andlist_jobs_runshas no env/project filter at all in 1.20.4.
So the blanket 'account-wide, not project-scoped' framing in the skill was indeed wrong as a general statement. I've reworked the PR: the Studio-specific scoping instruction (filter to the current project_id from client context) now lives only in the ai-codegen-api system prompt (PR dbt-labs/ai-codegen-api#1143), and the skill just carries a client-agnostic note that results may span projects/environments depending on how the Admin API is configured — with a generic 'filter to the project/env you're investigating' pattern.
Longer term, the clean fix is the dbt-mcp project_id scoping you mentioned.
(AI-generated, on behalf of @theyostalservice)
Per PR review (wiggzz): remove Studio-specific 'Step 0: determine current project_id' and the blanket 'account-wide, not project-scoped' claim, which isn't universally true. Replace with a client-agnostic note that results may span projects/environments depending on Admin API config. Also fix mermaid, Step 1 example, and Quick Reference row (Copilot comment) to consistently show the list_jobs filter → list_jobs_runs(job_id=…) pattern rather than a bare account-wide list_jobs_runs call. Studio-specific scoping (always filter to client_context.project_id) lives in the ai-codegen-api system prompt (PR #1143), not this skill. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
| # List jobs and filter to the target project/environment (project_id = 1234 in this example) | ||
| jobs = list_jobs() | ||
| target_jobs = [j for j in jobs if j["project_id"] == 1234] | ||
|
|
||
| # Get recent failed runs for each job | ||
| for job in target_jobs: | ||
| list_jobs_runs(job_id=job["id"], status="error", limit=5) |
Why
When a Studio user asks the agent to debug job failures, it calls
list_jobs_runswithout ajob_idor project filter and debugs the most-recent run account-wide — which can be from a completely different project.What
Revises the
troubleshooting-dbt-job-errorsskill with a client-agnostic correctness fix:list_jobs/list_jobs_runsresults may span multiple projects/environments depending on how the Admin API is configured. Instructs the agent to always passjob_idand filter to the project/environment being investigated.list_jobs(filter) →list_jobs_runs(job_id=…)flow rather than a bare account-wide call.Studio-specific scoping (filter to
client_context.project_id) lives in the ai-codegen-api system prompt — PR dbt-labs/ai-codegen-api#1143.Notes
Dropped the earlier "Step 0: determine the current
project_id" framing per review feedback from @wiggzz: the tools aren't inherently account-wide —AdminApiConfigcan scope them server-side, and trueproject_idscoping is in-flight in dbt-mcp. The account-wide behaviour is specific to how ACA wires the config today.Refs
dbt-skillsrev there)Drafted by claude-sonnet-4-6 under the direction of @theyostalservice