Skip to content

Commit fdb285d

Browse files
authored
Add native Beam schematransform autogen snippet (apache#34515)
* add native java io example * fix typo
1 parent 1cf9755 commit fdb285d

File tree

1 file changed

+17
-0
lines changed

1 file changed

+17
-0
lines changed

website/www/site/content/en/documentation/sdks/python-custom-multi-language-pipelines-guide.md

+17
Original file line numberDiff line numberDiff line change
@@ -270,6 +270,23 @@ inspect.signature(MyTransform)
270270

271271
This metadata is generated directly from the provider's implementation. The class documentation is generated from the [optional **description** method](#additional-metadata). The signature information is generated from the `@SchemaFieldDescription` annotations in the [configuration object](#implement-a-configuration).
272272

273+
### Using Beam native Java SchemaTransforms
274+
If there's an existing Beam native Java SchemaTransform you'd like to use, and you know which expansion service module it's in, you can connect to it using `BeamJarExpansionService`:
275+
276+
```python
277+
from apache_beam.transforms.external_transform_provider import ExternalTransformProvider
278+
from apache_beam.transforms.external import BeamJarExpansionService
279+
280+
identifier = "beam:schematransform:org.apache.beam:bigquery_fileloads:v1"
281+
expansion_service = "sdks:java:io:google-cloud-platform:expansion-service:shadowJar"
282+
283+
provider = ExternalTransformProvider(BeamJarExpansionService(expansion_service))
284+
BqFileLoads = provider.get_urn(identifier)
285+
286+
with beam.Pipeline(argv=args) as p:
287+
p | beam.Create(...) | BqFileLoads(table="project.dataset.table")
288+
```
289+
273290
## Appendix
274291

275292
### Portable transform

0 commit comments

Comments
 (0)