Skip to content
This repository was archived by the owner on Oct 16, 2025. It is now read-only.

Conversation

@dup05
Copy link
Contributor

@dup05 dup05 commented Sep 22, 2023

Summary (Short summary of what is being done) :

Simplify REID workflow by making the query parameter optional

Description (Describe in detail the fix made) :

The REID workflow required users to compulsarily provide SQL query to read from the BigQuery. This involved manual steps to create a GCS object containing sql query and passing its path in the pipeline parameter. With this change, the queryPath parameter has been made optional and users can run REID pipeline without needing to provide a SQL query.

Bug ID (if any) :

b/293427326

Public Documentation (if any) :

TESTED (Test Cases with scenario and description - must have 1 positive and 1 negative scenario) :

  1. Tested REID without passing queryPath and verified the results
  2. Tested REID by passing the query

@codecov
Copy link

codecov bot commented Oct 11, 2023

Codecov Report

❌ Patch coverage is 0% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 12.43%. Comparing base (0691475) to head (01f8279).
⚠️ Report is 117 commits behind head on master.

Files with missing lines Patch % Lines
...arm/tokenization/common/BigQueryReadTransform.java 0.00% 15 Missing ⚠️
...ava/com/google/swarm/tokenization/common/Util.java 0.00% 3 Missing ⚠️

❌ Your patch check has failed because the patch coverage (0.00%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@             Coverage Diff              @@
##             master     #162      +/-   ##
============================================
- Coverage     12.45%   12.43%   -0.03%     
  Complexity       63       63              
============================================
  Files            53       53              
  Lines          2480     2484       +4     
  Branches        207      208       +1     
============================================
  Hits            309      309              
- Misses         2152     2156       +4     
  Partials         19       19              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

.usingStandardSql()
.withMethod(Method.DEFAULT))
.apply("AddTableNameAsKey", WithKeys.of(tableRef()));
throw new IllegalArgumentException("Export method not supported");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May not be the intended behaviour but since the EXPORT case used to work with DEFAULT read earlier, are we declaring that somewhere before removing the support altogether?

.apply("AddTableNameAsKey", WithKeys.of(tableRef()));
}

case EXPORT:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just club EXPORT and DIRECT_READ together and throw the exception "Only DEFAULT Read Method supported"?

case EXPORT: case DIRECT_READ: throw new IllegalArgumentException("Only DEFAULT READ method supported");

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants