Skip to content

[Bug]: withOutputTags (aka side output) requires ParDo to run on each output separately #34333

Open
@shai-rubin

Description

@shai-rubin

What happened?

Apache Beam Java sdk version: 2.63.0 Spark version: 3.5.4

Running ParDo on a CoGrouped PCollection and expecting to retrieve multiple outputs using withOutputTags.

On Spark UI I see that the ParDo runs on each "output" separately instead of 1 time for all.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions