Skip to content

[Bug]: DatastoreIO.Write fails multiple mutations of same entity on finishBundle #34608

Open
@bitnahian

Description

@bitnahian

What happened?

Similar to #18521

Fails on the edge case when finishBundle is executed at the end of a bundle but flushBatch does not take into account duplicates before trying to write to Datastore.

Possible solutions include removing duplicate mutations from mutations ArrayList before flushBatch or just using a HashMap of mutations instead of maintaining an ArrayList which can ensure uniqueness.

org.apache.beam.sdk.Pipeline$PipelineExecutionException: com.google.datastore.v1.client.DatastoreException: A non-transactional commit may not contain multiple mutations affecting the same entity., code=INVALID_ARGUMENT
java.lang.RuntimeException: org.apache.beam.sdk.Pipeline$PipelineExecutionException: com.google.datastore.v1.client.DatastoreException: A non-transactional commit may not contain multiple mutations affecting the same entity., code=INVALID_ARGUMENT

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions