Skip to content

Write with custom indexName and docId not present in document itself #20640

Open
@damccorm

Description

@damccorm

ElasticIO connector write method is taking currently as input a "PCollection<String>" element ; string value will be passed as ElasticSearch document body. With methods withIdFn, and withIndexFn, we can overload Elastic document id and index name at runtime.

There is some kind of situation where indexName and/or docId are not present in document itself (for example from Google Pubsub metadata or other).

Is there a way to implement this usecase with current ElasticIO ? Otherwise, if it makes sense for you, what kind of improvment can be performed on connector ?

Imported from Jira BEAM-11276. Original Jira may contain additional context.
Reported by: lionelsetan.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions