Skip to content

[BUG] Pipeline fails to process null change event (document DB source) #6106

@JonahCalvo

Description

@JonahCalvo

Describe the bug
User noticed a very sharp spike in the ChangeEventsProcessingErrors metric (1.11M)

Looking into logs at the time, the following was repeatedly being output:

2025-09-29T14:07:47.565 [documentdb-source-3] ERROR org.opensearch.dataprepper.plugins.mongo.stream.StreamWorker - Failed to add records to buffer with error
java.lang.NullPointerException: Cannot invoke "org.bson.Document.toJson(org.bson.json.JsonWriterSettings)" because the return value of "com.mongodb.client.model.changestream.ChangeStreamDocument.getFullDocument()" is null

Later on after these logs had stop, the following warning was repeatedly displayed (unsure if related):


2025-09-29T14:07:50.573 [documentdb-pipeline-s3-sink-worker-4-thread-2] WARN  org.opensearch.dataprepper.plugins.sink.opensearch.BulkRetryStrategy - index = parts operation = Index, error = [66ead8422f9ba6016c4c365a]: version conflict, current version [1759154109000203] is higher or equal to the one provided [1759153858000095] |  

A circuit breaker flip was also observed.

To Reproduce
I'm not familiar enough with the ChangeStreamDocument to know if this is expected to be null or if this is a bug from the other service

Expected behavior
Null document should be ignored?

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    Status

    Unplanned

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions