Skip to content

BigQuery: Cannot specify a billing/connection project independently of the target table project #180

@mbyrne00

Description

@mbyrne00

Bug Description
In BigQuery the project id used for the connection is used to execute jobs, attribute billing, etc. In our org the target table often resides in a different project. We cannot use that target project for executing jobs - it's for storing data.

In your bigquery code:

  • BigQueryConnection uses the project arg. This is your own class, but wraps Google's bigquery.Client(project=project, credentials=credentials)
  • upload_raster uses the same project to create the table

While this is a sensible default, args should allow the caller to differentiate.

Expected behavior
Leave current behaviour as the default behaviour, but allow the caller to specify a connection and target project independently.

Screenshots
In our org setup I get ...

 User does not have bigquery.jobs.create permission in project <project>

System information
[Run carto info in a terminal and add the output here, overwriting the
text below.]

Raster Loader version: 0.11.0
Python version: 3.11.9
Platform: macOS-15.6.1-arm64-arm-64bit
System version: Darwin 24.6.0
Machine: arm64
Processor: arm
Architecture: 64bit

Additional context
[Add any other context about the problem here.]

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions