Summary
With allowedDatasets set, bigquery-execute-sql documents that queries accessing tables outside the list are rejected. In practice the allowlist check runs only after a successful BigQuery dry-run, so a query against a non-existent foreign dataset returns a BQ 404 Not Found (via ProcessGcpError → AgentError) instead of the clear AgentError:
query accesses dataset 'PROJECT.DATASET', which is not in the allowed list
Security is usually still fail-safe (no data returned), but the policy signal is wrong and agents/evals cannot distinguish “table missing” from “dataset not allowlisted”.
Code path (main)
internal/tools/bigquery/bigqueryexecutesql/bigqueryexecutesql.go Invoke:
DryRunQuery(...) — on error, return immediately via ProcessGcpError
- Only then: writeMode checks +
allowedDatasets loop (dry-run ReferencedTables + TableParser)
So the SQL TableParser path never runs when dry-run fails for dataset/table not found.
Repro
Source config with e.g. allowedDatasets: [omnifox_admin, …] (any non-empty list).
SELECT 1 FROM `my-project.definitely_not_allowlisted.no_such_table` LIMIT 1
Actual: readable googleapi Error 404 Not found: Table …
Expected: AgentError mentioning definitely_not_allowlisted is not in the allowed list (independent of whether the table exists).
Contrast: when dry-run succeeds against a real table outside the list, the allowlist message appears as designed.
Ask
Run TableParser (or equivalent static dataset extraction) before dry-run when allowedDatasets is non-empty, and reject with the existing “not in the allowed list” AgentError. Keep dry-run for statement-type / referenced-table enrichment afterward.
Context
Follow-up to #3716 (403 → opaque 500). Same deploy; eval task scored PARTIAL because rejection was safe but not allowlist-shaped.
Checked against Toolbox 1.6.0 behavior and current main source (2026-07-27).
Summary
With
allowedDatasetsset,bigquery-execute-sqldocuments that queries accessing tables outside the list are rejected. In practice the allowlist check runs only after a successful BigQuery dry-run, so a query against a non-existent foreign dataset returns a BQ 404 Not Found (viaProcessGcpError→ AgentError) instead of the clear AgentError:query accesses dataset 'PROJECT.DATASET', which is not in the allowed listSecurity is usually still fail-safe (no data returned), but the policy signal is wrong and agents/evals cannot distinguish “table missing” from “dataset not allowlisted”.
Code path (
main)internal/tools/bigquery/bigqueryexecutesql/bigqueryexecutesql.goInvoke:DryRunQuery(...)— on error, return immediately viaProcessGcpErrorallowedDatasetsloop (dry-runReferencedTables+TableParser)So the SQL
TableParserpath never runs when dry-run fails fordataset/table not found.Repro
Source config with e.g.
allowedDatasets: [omnifox_admin, …](any non-empty list).Actual: readable googleapi Error 404 Not found: Table …
Expected: AgentError mentioning
definitely_not_allowlistedis not in the allowed list (independent of whether the table exists).Contrast: when dry-run succeeds against a real table outside the list, the allowlist message appears as designed.
Ask
Run
TableParser(or equivalent static dataset extraction) before dry-run whenallowedDatasetsis non-empty, and reject with the existing “not in the allowed list” AgentError. Keep dry-run for statement-type / referenced-table enrichment afterward.Context
Follow-up to #3716 (403 → opaque 500). Same deploy; eval task scored PARTIAL because rejection was safe but not allowlist-shaped.
Checked against Toolbox 1.6.0 behavior and current
mainsource (2026-07-27).