DAGE-73: MtebWriter corpus bug#237

Merged

nicolo-rinaldi merged 21 commits intodataset-generatorfrom

DAGE-73_MtebWriter-bug-corpus

Oct 9, 2025

Commits on Oct 8, 2025

[DAGE-73] bug fixed by adding all doc to datastore after saving it
nicolo-rinaldi
committed
[DAGE-73] added tests
nicolo-rinaldi
committed
[DAGE-73] Updated version with new DataStore autosave feature
nicolo-rinaldi
committed
[DAGE-73] added `uv sync --group dev` to README.md
nicolo-rinaldi
committed
[DAGE-73] typo in main.py datastore variable
nicolo-rinaldi
committed
[DAGE-73] added robustness to fetch_all
nicolo-rinaldi
committed
[DAGE-73] rolled back to `_write_embeddings_jsonl` old version in EmbeddingWriter
nicolo-rinaldi
committed
[DAGE-73] 2 big fixes
nicolo-rinaldi
committed
[DAGE-73] reverted back max_tokens to be ignored by ruff
nicolo-rinaldi
committed
[DAGE-73] workaround modified to avoid changing the datastore persistency
nicolo-rinaldi
committed
[DAGE-73] fixed description bug
nicolo-rinaldi
committed
[DAGE-73] extended fetch_all() method to be used with vespa_search_engine.py
nicolo-rinaldi
committed
[DAGE-73] uniformed config param in M1 and M2
nicolo-rinaldi
committed
[DAGE-73] centralized function setup_logging
nicolo-rinaldi
committed
[DAGE-73] fixed hardcoded description and coded the streamed version of fetch_all(), that uses a prefixed batch of 100
nicolo-rinaldi
committed
[DAGE-73] added _get_total_hits to do the streamed version of fetch_all for all search engines but vespa. Also fixed tests
nicolo-rinaldi
committed
[DAGE-73] vespa should be fixed, I need to update the tests since fetch_all related ones are missing
nicolo-rinaldi
committed
[DAGE-73] fixed vespa tests
nicolo-rinaldi
committed
[DAGE-73] addressing Naz comments
nicolo-rinaldi
committed
[DAGE-73] added break in while loop
nicolo-rinaldi
committed

Commits on Oct 9, 2025

[DAGE-73] addressing Daniele's comments
nicolo-rinaldi
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DAGE-73: MtebWriter corpus bug#237

DAGE-73: MtebWriter corpus bug#237
nicolo-rinaldi merged 21 commits intodataset-generatorfrom
DAGE-73_MtebWriter-bug-corpus

Commits on Oct 8, 2025

[DAGE-73] bug fixed by adding all doc to datastore after saving it

[DAGE-73] added tests

[DAGE-73] Updated version with new DataStore autosave feature

[DAGE-73] added `uv sync --group dev` to README.md

[DAGE-73] typo in main.py datastore variable

[DAGE-73] added robustness to fetch_all

[DAGE-73] rolled back to `_write_embeddings_jsonl` old version in EmbeddingWriter

[DAGE-73] 2 big fixes

[DAGE-73] reverted back max_tokens to be ignored by ruff

[DAGE-73] workaround modified to avoid changing the datastore persistency

[DAGE-73] fixed description bug

[DAGE-73] extended fetch_all() method to be used with vespa_search_engine.py

[DAGE-73] uniformed config param in M1 and M2

[DAGE-73] centralized function setup_logging

[DAGE-73] fixed hardcoded description and coded the streamed version of fetch_all(), that uses a prefixed batch of 100

[DAGE-73] added _get_total_hits to do the streamed version of fetch_all for all search engines but vespa. Also fixed tests

[DAGE-73] vespa should be fixed, I need to update the tests since fetch_all related ones are missing

[DAGE-73] fixed vespa tests

[DAGE-73] addressing Naz comments

[DAGE-73] added break in while loop

Commits on Oct 9, 2025

[DAGE-73] addressing Daniele's comments