Skip to content

[Bug]: OpenSearch DocStatus bulk item failures can be silently ignored #3355

Description

@VectorPeak

Describe the bug

OpenSearchDocStatusStorage.upsert() calls _run_chunked_async_bulk(..., raise_on_error=False) but previously discarded the returned failed collection. OpenSearch bulk requests can complete at the request level while individual items fail, so a DocStatus write can be treated as successful even when a single item reports a mapping error, retryable status, or other per-item failure.

DocStatus records are LightRAG pipeline bookkeeping for document processing states. If a DocStatus row is not actually written but the pipeline continues as if it was, a document can lose the reliable record of whether it is pending, failed, or processed.

Steps to reproduce

  1. Use the OpenSearch DocStatus backend.
  2. Make the OpenSearch bulk helper return a response-level success with item-level failures, for example:
d1 -> status=400 mapper_parsing_exception
d2 -> status=503 transient failure
  1. Call OpenSearchDocStatusStorage.upsert() for those document ids.

Before PR #3354, upsert() ignored the returned failed list and completed without surfacing either item failure.

Expected Behavior

OpenSearchDocStatusStorage.upsert() should inspect per-item bulk failures and fail fast when any DocStatus write was not accepted by OpenSearch.

LightRAG Config Used

OpenSearch DocStatus storage backend. The failure mode is independent of a specific deployment config because it follows from OpenSearch bulk response semantics.

Logs and screenshots

No live OpenSearch service is required to demonstrate the bug. PR #3354 adds mocked regression coverage for the bulk helper returning item-level failures.

Additional Information

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions