Skip to content

ReadFromBigQuery fails with ValueProvider for table #20518

Open
@damccorm

Description

@damccorm

2 errors are present:

  1. ReadFromBigQuery passing a ValueProvider for the table param gets this error:

'_CustomBigQuerySource' object has no attribute 'dataset'

File "./lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 656, in estimate_size self.table_reference.get(), self.dataset, self.project)

There are a 2 places where self.dataset are referenced, but it is never set to any value in the CustomBigQuerySource class.  This should be set init() method.

  1. _export_files references table_ref as a str before checking if its a ValueProvider.

Imported from Jira BEAM-10792. Original Jira may contain additional context.
Reported by: mballou.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions