[data] Zero-sized blocks crashes write_bigquery

[FredrikNoren]

What happened + What you expected to happen

Here: https://github.com/ray-project/ray/blob/master/python/ray/data/_internal/datasource/bigquery_datasink.py#L79

We need to add:

            if block.num_rows == 0: return

Otherwise we're getting:

google.api_core.exceptions.BadRequest: 400 Table testy_embeddings_d6c21277_8ef5_4ed7_9c85_304fb04ed6fe_source does not have a schema.; reason: invalid, message: Table testy_embeddings_d6c21277_8ef5_4ed7_9c85_304fb04ed6fe_source does not have a schema.

Versions / Dependencies

Ray 2.44.1

Reproduction script

Filter out everything in a dataset and try to write it to bigquery

Issue Severity

High: It blocks me from completing my task.

[richardliaw]

Hi @FredrikNoren - thanks for reporting; do you want to submit a simple PR to fix?

[FredrikNoren]

@richardliaw Sure, here you go: #51919

[patina27]

I am encountering the same issue here. what's the recommended workaround?

[richardliaw]

#51919 was a good approach; unfortunately the previous PR was closed (and we couldn't merge it due to lint issues). If you'd like to try opening a similar PR with lint fixed, happy to help shepherd!