Skip to content

Persist RLI index bootstrap records only if estimation is required and add unpersist #14068

@lokeshj1703

Description

@lokeshj1703

Bug Description

What happened:
During bootstrap, RLI index records are persisted for calculating the file group count. Users can configure the record index file group count by configuring the min and max file group count and setting them to the same value. In such a case, estimation is not required and therefore the record index records need not be persisted.

What you expected:
If the min and max file group count for record index is configured and set to the same value, record index records should not be persisted.

Steps to reproduce:
1.
2.
3.

Environment

Hudi version: master
Query engine: Spark
**Relevant Configs: **
hoodie.metadata.partitioned.record.index.min.filegroup.count
hoodie.metadata.partitioned.record.index.max.filegroup.count
hoodie.metadata.record.index.min.filegroup.count
hoodie.metadata.record.index.max.filegroup.count

Logs and Stack Trace

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    type:bugFor Issues and PRs that fix bugs

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions