Add detection for outer DECLARE statements and update documentation#40
Add detection for outer DECLARE statements and update documentation#40max-ostapenko merged 23 commits intomainfrom
Conversation
- Implemented `hasOuterDeclare` function to check for outer level DECLARE statements in SQL. - Updated `autoAssignActions` to skip operations with outer DECLARE. - Enhanced README and Copilot instructions to clarify limitations of automated assignment. - Added tests for various scenarios involving outer DECLARE statements.
There was a problem hiding this comment.
Pull request overview
This PR adds detection for outer-level DECLARE statements in SQL operations to prevent syntax errors when prepending reservation SET statements. BigQuery requires DECLARE to be the first statement in a script, so prepending SET @@reservation before an outer DECLARE would cause the query to fail. The PR implements automatic detection and skipping of such operations.
Changes:
- Added
hasOuterDeclare()function to detect DECLARE statements at the outer level of SQL queries, stripping comments to find the first real statement - Updated
applyReservationToAction()to skip reservation assignment for operations with outer DECLARE - Added comprehensive test coverage for various DECLARE scenarios (mixed case, inside BEGIN...END blocks, inside EXECUTE IMMEDIATE, after comments, in arrays)
- Updated README and Copilot instructions to document this limitation of automated assignment
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| index.js | Implemented hasOuterDeclare() function and integrated DECLARE detection into reservation assignment logic |
| test/index.test.js | Added 5 test cases covering DECLARE detection in various scenarios |
| README.md | Added "Limitations of Automated Assignment" section documenting DECLARE restriction |
| .github/copilot-instructions.md | Added section 5 documenting outer DECLARE detection logic |
Comments suppressed due to low confidence (1)
index.js:204
- The DECLARE detection doesn't handle cases where
queriesis passed as a function that returns SQL with an outer DECLARE statement. The check on line 191 only evaluates the function object itself (which will stringify to something like "function (ctx) { ... }"), not the SQL it returns.
When the function is executed later (lines 196-204), the result could contain an outer DECLARE, which would cause a syntax error since the reservation SET statement would be prepended before it. Consider either checking the function's return value at execution time, or documenting this as a known limitation requiring manual assignment.
// Check for outer DECLARE before wrapping
if (hasOuterDeclare(queries)) {
return originalQueriesFn.apply(this, [queries])
}
if (typeof queries === 'function') {
queriesArray = (ctx) => {
const result = queries(ctx)
if (typeof result === 'string') {
return [statement, result]
} else if (Array.isArray(result)) {
return [statement, ...result]
}
return result
}
Bumps [eslint](https://github.com/eslint/eslint) from 10.0.0 to 10.0.1. - [Release notes](https://github.com/eslint/eslint/releases) - [Commits](eslint/eslint@v10.0.0...v10.0.1) --- updated-dependencies: - dependency-name: eslint dependency-version: 10.0.1 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [@dataform/core](https://github.com/dataform-co/dataform) from 3.0.46 to 3.0.47. - [Release notes](https://github.com/dataform-co/dataform/releases) - [Commits](dataform-co/dataform@3.0.46...3.0.47) --- updated-dependencies: - dependency-name: "@dataform/core" dependency-version: 3.0.47 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [@dataform/cli](https://github.com/dataform-co/dataform) from 3.0.46 to 3.0.47. - [Release notes](https://github.com/dataform-co/dataform/releases) - [Commits](dataform-co/dataform@3.0.46...3.0.47) --- updated-dependencies: - dependency-name: "@dataform/cli" dependency-version: 3.0.47 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Max Ostapenko <1611259+max-ostapenko@users.noreply.github.com>
Bumps [eslint](https://github.com/eslint/eslint) from 10.0.1 to 10.0.2. - [Release notes](https://github.com/eslint/eslint/releases) - [Commits](eslint/eslint@v10.0.1...v10.0.2) --- updated-dependencies: - dependency-name: eslint dependency-version: 10.0.2 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [globals](https://github.com/sindresorhus/globals) from 17.3.0 to 17.4.0. - [Release notes](https://github.com/sindresorhus/globals/releases) - [Commits](sindresorhus/globals@v17.3.0...v17.4.0) --- updated-dependencies: - dependency-name: globals dependency-version: 17.4.0 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the npm_and_yarn group with 1 update in the / directory: [minimatch](https://github.com/isaacs/minimatch). Bumps the npm_and_yarn group with 1 update in the /test-project directory: [minimatch](https://github.com/isaacs/minimatch). Updates `minimatch` from 3.1.2 to 3.1.5 - [Changelog](https://github.com/isaacs/minimatch/blob/main/changelog.md) - [Commits](isaacs/minimatch@v3.1.2...v3.1.5) Updates `minimatch` from 9.0.5 to 9.0.9 - [Changelog](https://github.com/isaacs/minimatch/blob/main/changelog.md) - [Commits](isaacs/minimatch@v3.1.2...v3.1.5) --- updated-dependencies: - dependency-name: minimatch dependency-version: 3.1.5 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: minimatch dependency-version: 9.0.9 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [eslint](https://github.com/eslint/eslint) from 10.0.0 to 10.0.1. - [Release notes](https://github.com/eslint/eslint/releases) - [Commits](eslint/eslint@v10.0.0...v10.0.1) --- updated-dependencies: - dependency-name: eslint dependency-version: 10.0.1 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* refactor: deduplicate logic in applyReservationToAction - Extracted duplicated logic for prepending statements into `prependStatement` and `isArrayOrString` helpers. - Refactored `applyReservationToAction` to use these helpers for `contextablePreOps`, `contextableQueries`, `proto.preOps`, and `proto.queries`. - Simplified the `action.queries` monkeypatch using `prependStatement`. - Ensured non-mutating behavior for array prepending. - Restored `test-project/workflow_settings.yaml` that was accidentally renamed during matrix tests. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * refactor: update exports and enhance test coverage with new utility functions --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* ⚡ Optimize reservation lookup using Map - Refactored `preprocessConfig` to build an optimized `Map` for reservation lookups. - Updated `findReservation` to perform O(1) lookups using the `Map`. - Preserved "first match wins" logic and existing validation. - Maintained backward compatibility by preserving original data structures in preprocessed config. - Improved lookup performance by ~24x in benchmarks. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Refactor findReservation tests to use Map for action-reservation mapping --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
…ps) (#46) * 🧪 test: add coverage for compiled objects and proto.preOps - Added comprehensive tests for compiled objects in `test/compiled_objects.test.js`. - Covered `proto.preOps` and `proto.queries` handling for both array and string types. - Verified fallback mechanisms and monkeypatching logic. - Increased line coverage of `index.js` to 100%. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * fix: standardize string quotes in compiled objects tests * refactor: simplify monkeypatched queries tests with parameterized cases --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
…d update isNativeSupported logic in verify_compilation.js
hasOuterDeclarefunction to check for outer level DECLARE statements in SQL.autoAssignActionsto skip operations with outer DECLARE.