Skip to content

Commit 3a55196

Browse files
authored
feat: BigQuery Storage v1beta1 API migration guide (#14345)
* feat: BigQuery Storage v1beta1 API migration guide * Add bigquery-readapi-team to subfolder code owner * Move to .github/CODEOWNERS
1 parent 08e44b9 commit 3a55196

2 files changed

Lines changed: 125 additions & 0 deletions

File tree

.github/CODEOWNERS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,7 @@
8484
/bigquery-datatransfer/**/* @GoogleCloudPlatform/api-bigquery @GoogleCloudPlatform/python-samples-reviewers @GoogleCloudPlatform/cloud-samples-reviewers
8585
/bigquery-migration/**/* @GoogleCloudPlatform/api-bigquery @GoogleCloudPlatform/python-samples-reviewers @GoogleCloudPlatform/cloud-samples-reviewers
8686
/bigquery-reservation/**/* @GoogleCloudPlatform/api-bigquery @GoogleCloudPlatform/python-samples-reviewers @GoogleCloudPlatform/cloud-samples-reviewers
87+
/bigquery_storage/**/* @GoogleCloudPlatform/bigquery-readapi-team @GoogleCloudPlatform/python-samples-reviewers @GoogleCloudPlatform/cloud-samples-reviewers
8788
/connectgateway/**/* @GoogleCloudPlatform/connectgateway @GoogleCloudPlatform/python-samples-reviewers @GoogleCloudPlatform/cloud-samples-reviewers
8889
/dlp/**/* @GoogleCloudPlatform/googleapis-dlp @GoogleCloudPlatform/python-samples-reviewers @GoogleCloudPlatform/cloud-samples-reviewers
8990
/functions/spanner/* @GoogleCloudPlatform/api-spanner-python @GoogleCloudPlatform/functions-framework-google @GoogleCloudPlatform/python-samples-reviewers @GoogleCloudPlatform/cloud-samples-reviewers
Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# Migrating BigQuery Storage API from v1beta1 to v1: Python
2+
3+
This guide shows how to migrate Python code using the BigQuery Storage API from
4+
version `v1beta1` to `v1`.
5+
6+
## Key Changes
7+
8+
* **Package Imports**: `google.cloud.bigquery_storage_v1beta1` ->
9+
`google.cloud.bigquery_storage_v1`
10+
* **Service Client**: `BigQueryStorageClient` is replaced by
11+
`BigQueryReadClient`.
12+
* **Table Reference**: `TableReference` type is replaced by a simple string
13+
representation of the table path in `ReadSession.table`.
14+
* **Session Configuration**: Configuration fields (table, format, read
15+
options) have moved into `ReadSession` type, which is passed in
16+
`CreateReadSessionRequest`.
17+
* **Parallelism**: `requested_streams` parameter in `create_read_session` is
18+
replaced by `max_stream_count`.
19+
* **Sharding Strategy**: `sharding_strategy` is removed. The server now
20+
automatically balances the streams.
21+
* **Read Rows Request**: `StreamPosition` is flattened. You now pass the
22+
stream name directly to `read_rows`.
23+
24+
## Code Comparison
25+
26+
### 1. Client Initialization
27+
28+
**v1beta1:**
29+
30+
```python
31+
from google.cloud import bigquery_storage_v1beta1
32+
33+
client = bigquery_storage_v1beta1.BigQueryStorageClient()
34+
```
35+
36+
**v1:**
37+
38+
```python
39+
from google.cloud import bigquery_storage_v1
40+
41+
client = bigquery_storage_v1.BigQueryReadClient()
42+
```
43+
44+
### 2. Creating a Read Session
45+
46+
**v1beta1:**
47+
48+
```python
49+
from google.cloud import bigquery_storage_v1beta1
50+
51+
table_ref = bigquery_storage_v1beta1.types.TableReference()
52+
table_ref.project_id = "bigquery-public-data"
53+
table_ref.dataset_id = "usa_names"
54+
table_ref.table_id = "usa_1910_current"
55+
56+
read_options = bigquery_storage_v1beta1.types.TableReadOptions()
57+
read_options.selected_fields.append("name")
58+
read_options.row_restriction = 'state = "WA"'
59+
60+
session = client.create_read_session(
61+
table_reference=table_ref,
62+
parent='projects/read-session-project',
63+
read_options=read_options,
64+
requested_streams=1,
65+
format=bigquery_storage_v1beta1.types.DataFormat.AVRO,
66+
sharding_strategy=bigquery_storage_v1beta1.types.ShardingStrategy.LIQUID
67+
)
68+
```
69+
70+
**v1:**
71+
72+
```python
73+
from google.cloud import bigquery_storage_v1
74+
75+
# Table path is now a string: projects/{project}/datasets/{dataset}/tables/{table}
76+
table_path = "projects/bigquery-public-data/datasets/usa_names/tables/usa_1910_current"
77+
78+
read_options = bigquery_storage_v1.types.ReadSession.TableReadOptions()
79+
read_options.selected_fields.append("name")
80+
read_options.row_restriction = 'state = "WA"'
81+
82+
# ReadSession holds the session configuration
83+
read_session = bigquery_storage_v1.types.ReadSession(
84+
table=table_path,
85+
data_format=bigquery_storage_v1.types.DataFormat.AVRO, # format renamed to data_format
86+
read_options=read_options
87+
)
88+
89+
session = client.create_read_session(
90+
parent="projects/read-session-project",
91+
read_session=read_session,
92+
max_stream_count=1 # requested_streams renamed to max_stream_count
93+
)
94+
```
95+
96+
### 3. Reading Rows
97+
98+
**v1beta1:**
99+
100+
```python
101+
from google.cloud import bigquery_storage_v1beta1
102+
103+
position = bigquery_storage_v1beta1.types.StreamPosition(
104+
stream=session.streams[0]
105+
)
106+
107+
reader = client.read_rows(read_position=position)
108+
109+
for row in reader.rows(session):
110+
print(row["name"])
111+
```
112+
113+
**v1:**
114+
115+
```python
116+
# read_rows accepts stream name and offset directly.
117+
# Note that the parameter name in the library helper is 'name', which maps to 'read_stream' in proto.
118+
reader = client.read_rows(session.streams[0].name)
119+
120+
# In v1, you don't need to pass session to rows() if schema info is already present,
121+
# but the client library handles it.
122+
for row in reader.rows():
123+
print(row["name"])
124+
```

0 commit comments

Comments
 (0)