Skip to content

Row alignment validation across configs #5

@davanstrien

Description

@davanstrien

`load_config_dataset()` merges configs by positional index — no alignment key. This is safe when all model runs use the same `--seed`/`--max-samples` and the source dataset doesn't change, but fragile.

Add a content hash column (e.g. hash of the source image) during `ocr-bench run` and validate alignment during `judge`.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions