Skip to content

Chunk overlap prefix is on even word boundary >= overlap character count. #2886

Open
@scanny

Description

@scanny

Problem
Chunk text begins mid-word when overlap is specified.

image

Desired solution
Compute the overlap prefix as the next even-word boundary greater than or equal to overlap characters from the end of the prior chunk.

Metadata

Metadata

Assignees

No one assigned

    Labels

    chunkingRelated to element chunking.enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions