improvement: expose wait timeout

[mutianf]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new configuration parameter, bigtableReadPartialRowTimeoutMs, to the Cloud Bigtable to GCS Dataflow template. This addition allows users to specify a timeout for partial row reads, improving control over scan operations. Additionally, the project dependency for Bigtable Beam imports has been upgraded to version 2.20.1 to support the latest features and fixes.

Highlights

• Configuration Update: Added the bigtableReadPartialRowTimeoutMs parameter to the Bigtable-to-GCS Dataflow template configuration.
• Dependency Upgrade: Updated the bigtable-beam-import version from 2.18.3 to 2.20.1 in the project pom.xml.
• Documentation: Updated the README documentation to include the new timeout parameter in both gcloud and Maven execution examples.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new optional parameter 'bigtableReadPartialRowTimeoutMs' to configure the read partial row timeout for Bigtable, and updates the 'bigtable-beam-import.version' dependency to '2.20.1'. The review feedback points out that the README file was modified manually; since this file is auto-generated, these manual changes should be reverted and regenerated using the template utility.

[gemini-code-assist]

The README_Cloud_Bigtable_to_GCS_SequenceFile.md file is auto-generated from the template metadata annotations (as noted on line 16: 'Do not change this file directly.'). Manual changes to this file will be overwritten during the next build/generation cycle. Please revert the manual changes to this file and instead run the template utility (e.g., using the templatesSpec Maven profile) to regenerate the README automatically.

References

1. Do not manually edit generated documentation files (such as READMEs generated from Metadata Annotations). Any corrections, such as incorrect directory paths, should be made in the generator code or source annotations.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 55.93%. Comparing base (9d1e211) to head (75231ed).

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #4221      +/-   ##
============================================
- Coverage     55.93%   55.93%   -0.01%     
+ Complexity     7482     7031     -451     
============================================
  Files          1134     1134              
  Lines         70175    70175              
  Branches       8023     8023              
============================================
- Hits          39255    39253       -2     
- Misses        28371    28373       +2     
  Partials       2549     2549              

Components	Coverage Δ
spanner-templates	84.69% <ø> (-0.02%)	⬇️
spanner-import-export	69.03% <ø> (+0.03%)	⬆️
spanner-live-forward-migration	88.66% <ø> (-0.04%)	⬇️
spanner-live-reverse-replication	81.37% <ø> (-0.03%)	⬇️
spanner-bulk-migration	89.06% <ø> (-0.02%)	⬇️
gcs-spanner-dv	87.88% <ø> (-0.04%)	⬇️

Files with missing lines	Coverage Δ
...cloud/teleport/templates/ExportJobPlaceholder.java	0.00% <ø> (ø)

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Abacn]

Just note that additional-pipeline-option is now supported for classic templates. User can use

curl --request POST --url https://googleapis.com{projectId}/locations/{location}/templates:launch?gcsPath=gs://dataflow-templates/latest/Cloud_Bigtable_to_GCS_SequenceFile  --header "Authorization: Bearer $(gcloud auth application-default print-access-token)" -H "x-goog-user-project: {projectId}" --header "Content-Type: application/json" --data '{
    "jobName": "{JOB_NAME}",
    "parameters": {
      ...
    },
    "environment": {
      "additionalPipelineOptions": ["bigtableReadPartialRowTimeoutMs=600000"],
    }
  }'

to pass a runtime pipeline option for a classic template invocation.

[mutianf]

@Abacn This is awesome, but we'll still need to update the dependency version.

I noticed some dependency issue during testing. So will update the bigtable client first.