Skip to content

[Bug]: Portable runner queryMetrics returns duplicate result #32001

@Abacn

Description

@Abacn

What happened?

For portable runner pipeline result, it is found running PipelineResult.metrics().queryMetrics(...).getCounter/getDistribution/etc returns each metric twice, found in #31999 (comment)

The reson is in

monitoringInfoList.addAll(jobMetrics.getAttemptedList());
monitoringInfoList.addAll(jobMetrics.getCommittedList());

it adds both attempted list and committed list, which essentially has the same info, then in convert...MonitoringInfoTo... always creates attempted metrics, e.g. here

return MetricResult.create(key, false, decodeInt64Counter(counterMonInfo.getPayload()));

(second parameter isCommitSupported=False)

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions