2.7.9

2.7.9 (2026-01-24)

Fix

• fix: add kwargs to pub chem load data (#3990)

add kwargs to pub chem load data (aeb22cd)

2.7.8

2.7.8 (2026-01-20)

Fix

• fix: Filled active_parameter_overiride for GritLM/GritLM-8x7B nomic-ai/nomic-embed-text-v2-moe (#3967)

• Filled active_parameter_overiride for ritLM/GritLM-8x7B and nomic-ai/nomic-embed-text-v2-moe

• add correct parameters for nomic-ai/nomic-embed-text-v2-moe (dbd4287)

2.7.7

2.7.7 (2026-01-20)

Fix

• fix: leaderboard Nan handling (#3965)

• fix leaderboard

• fix loading aggregated tasks

• Update mteb/results/task_result.py

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>

---

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com> (a369c26)

• fix: Add fill_missing parameter in get_model_meta (#3801)

• Add compute missing parameter in get_model_meta

• fix logs

• fix

• fix from comments

• apply suggestion

• fix method

• add test and fix logic

• address comments

• rename compute_missing to fill_missing

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (90536d4)

• fix: Ensure that retrieval tasks only evaluate on specified subsets instead of all (#3946)

• fix dataset loading

• update logging

• add test (8186392)

Unknown

• Merge branch 'main' of https://github.com/embeddings-benchmark/mteb (523b5bc)

• model: Adding voyage-4-large (2048d) model configs (#3970)

• Adding voyage-4-large (2048d) model configs

• Adding voyage-4-large 2048d model configs

• Adding voyage-4-large 2048d model configs (961a43b)

2.7.6

2.7.6 (2026-01-20)

Fix

• fix: saving aggregated tasks (#3915)

fix saving (ced5f71)

2.7.5

2.7.5 (2026-01-19)

Fix

• fix: use num_proc for dataset processing (#3832)

• add typehint for encode kwargs

• remove num_proc

• start adding num_proc

• remove all num proc

• fix import

• add num proc to transform

• add to push to hub

• use num proc in vidore v2

• move num proc to evaluate

• pass num proc everywhere

• fix tests

• fix pylate

• fix image text pair

• fix num workers

• add kwargs to load_data (daf2b6f)

2.7.4

2.7.4 (2026-01-19)

Fix

• fix: Update metadata to include active number of parameter to ModelMeta (#3837)

• Add active parameter column on LB

• update ModelMeta with parameters

• update ModelMeta of models

• Delete parameter_update_results.csv

• fix test

• fix tests

• delete script

• rename for consistency

• convert active_parameter to property

• rename and fix property

• update embedding parameters for model2vec models

• remove duplicate loading of models

• fix

• lintter

• fix

• remove separate method for embedding parameter calculation

• fix embedding calculation to pass typecheck

• lintter

• fix checking

• rename active parameters

• upd docstring

• fix tests

• remove n_active_parameters_override from ModelMeta of all models

• lintter

• rename file instead of merging main

• fix tests

• correct tests

• Delete model total and active parameters - model_parameters.csv

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (a45359e)

Refactor

• refactor: split BRIGHT benchmark into individual subset tasks (#3285)

• refactor: split BRIGHT benchmark into individual subset tasks

• readd bright

• readd bright subset tasks

• feat: add descriptive stats for BRIGHT subsets retrieval tasks

• feat: add top_ranked for excluded_ids handling

• change main score to recall@1 for long version

• improve BRIGHT task descriptions

• add prompts to BRIGHT retrieval tasks

• refactor: BRIGHT(v1.1)

• calculate descriptive stats for BRIGHTLongRetrieval

• update prompts

• normalize names in prompts

• don't filter tasks

• remove filter_queries_without_positives and update revision

• don't create top ranked if not necessary

• get back naucs

• fix instructions

• add warning

• fix import

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (2c9b9e9)

2.7.3

2.7.3 (2026-01-19)

Fix

• fix: temporarily remove private column from RTEB (#3932)

• fix: temporarily remove private column from RTEB

Link is still missing the note as I am waiting for @isaac-chung and @Samoed to confirm the write-up.

fixes #3902

• added issue link

• fix remove mean (Task)

• lint

• merge in fixes to remove_private (#3940)

fix: exclude private tasks from Borda rank calculation in RTEB

Co-authored-by: bflhc <kunka.xgw@gmail.com>

---

Co-authored-by: bflhc <kunka.xgw@gmail.com> (b968433)

• fix: correct inverted unload_data condition in evaluate (#3929)

Add tests verifying preloaded data is preserved.

Co-authored-by: Daniel Svonava <daniel@superlinked.com> (1c5d9c6)

Refactor

• refactor: Activate TC (#3800)

• activate tc

• activate TC

• small import fix

• fix imports

• fix imports

• fix pil import

• fix benchmark result validation

• full benchmark fix

• update

• fix unpack imports

• upd vllm type (16e0211)

Unknown

• dedup colnomic_7b and fix loader (#3957)

• dedup colnomic_7b and fix loader

• remove flash_attention_2 (8c96b79)

• fix colSmol-256M revision (#3956) (b3c2718)

• fix typo (#3954) (d7acd57)

2.7.2

2.7.2 (2026-01-15)

Documentation

• docs: fix vllm broken link (#3936)

fix vllm link (d045d53)

Fix

• fix: expose ResultCache directly as mteb.ResultCache (#3912)

• fix: expose ResultCache directly as mteb.ResultCache

fixes #3910

• docs: Update docs usage of ResultCache (3103f97)

• fix: computation of results with missing scores (#3874)

• fix computation of results with missing scores

• fix test

• change 0 to nan

• change 0 to nan

• remove fill_missing_scores (d60e916)

Unknown

• model: add pixie_models (#3938)

• model: add pixie_models

• Apply lint formatting (8b54f0e)

• model: mixedbread-ai/mxbai-edge-colbert-v0-32m and mixedbread-ai/mxbai-edge-colbert-v0-17m (#3931)

• Add model: mixedbread-ai/mxbai-edge-colbert-v0-32m and mixedbread-ai/mxbai-edge-colbert-v0-17m

• Lintter

• Add quotes

• Update dataset name

• Apply suggestions from code review

• Update mixedbread_ai_models.py

---

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (5de2194)

2.7.1

2.7.1 (2026-01-14)

Fix

• fix: Minor logging fixes by activate LOG rule (#3820)

activate logger rule (65313c9)

Unknown

• model: Adding voyage-4 model (#3927)

• Adding voyage-4 model

• Adding voyage-4 model configs (b80da30)

• Update references and citations for ViDoRe V3 benchmark (#3930)

• fix: Update references and citations for ViDoRe V3 benchmark

• foramat citation

• format again

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (e7d077e)

• model: add nemotron rerank (#3750)

• add nemotron rerank

• move to nvidia models

• removed extra params

• Apply suggestions from code review

Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

• remove or

• add docstring

• Update mteb/models/model_implementations/nvidia_models.py

Co-authored-by: Yauhen Babakhin <ybabakhin@nvidia.com>

• update

---

Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>
Co-authored-by: Yauhen Babakhin <ybabakhin@nvidia.com> (330a601)

• dataset: Add EuroPIRQRetrieval dataset (#3924)

• dataset: Add EuroPIRQRetrieval dataset

• Removed unnecessary load dataset functions (7966e06)

• dataset: add ChemRxivRetrieval task to ChemTEB benchmark (#3923)

• dataset: add ChemRxivRetrieval task to ChemTEB benchmark

• fix: add descriptive statistics

• feat: add ChemTEB v1.1 with ChemRxivRetrieval task

• fix: chemteb v1.1 alias (86359fd)

2.7.0

2.7.0 (2026-01-13)

Documentation

• docs: Resolve problems with missing documentation links (#3834)

• resolve problems with missing documentation links

• split into files (0d277cd)

Feature

• feat: Add vLLM support (#3794)

• init

• init

Signed-off-by: wang.yuqi <noooop@126.com>

• ruff

Signed-off-by: wang.yuqi <noooop@126.com>

• • vllm_loader

Signed-off-by: wang.yuqi <noooop@126.com>

• • TYPE_CHECKING

Signed-off-by: wang.yuqi <noooop@126.com>

• Make vLLM exit properly.

Signed-off-by: wang.yuqi <noooop@126.com>

• rename

Signed-off-by: wang.yuqi <noooop@126.com>

• support rerank

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• refine

Signed-off-by: wang.yuqi <noooop@126.com>

• refine

Signed-off-by: wang.yuqi <noooop@126.com>

• Update mteb/models/vllm_wrapper.py

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• refine

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• • docs

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• • benchmark

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• • more benchmark

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• Update docs/advanced_usage/vllm_wrapper.md

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• Update docs/advanced_usage/vllm_wrapper.md

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• refine docs

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• refine docs

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• fix typing

• move type ignore

• doc upd

• add test

• Update Makefile

• add support for prompts

• add support for prompts

• • demo

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• make mypy happy

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• fix typehints

• update pyproject

• update pyproject

• update pyproject

• The pooling + dp fails to run.

• fix uv lock

• fix docs

• simplify conflicts

• upd lock

• upd lock

• Update docs/advanced_usage/vllm_wrapper.md

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• Update docs/advanced_usage/vllm_wrapper.md

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• Update docs/advanced_usage/vllm_wrapper.md

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• Update docs/advanced_usage/vllm_wrapper.md

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• Apply suggestions from code review

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• Update docs/advanced_usage/vllm_wrapper.md

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• Apply suggestion from @Samoed

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• update

Signed-off-by: wang.yuqi <noooop@126.com>

• update

Signed-off-by: wang.yuqi <noooop@126.com>

---

Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>
Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>
Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (4d568a8)

Unknown

• model: Update the nemo retriever reversions to avoid error when loading the model (#3925)

• Update the nemo retriever versions to fix the crash issue with visual_config

• Update mteb/models/model_implementations/nvidia_llama_nemoretriever_colemb.py

• Update mteb/models/model_implementations/nvidia_llama_nemoretriever_colemb.py

---

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me> (dcd31fa)

• model: Adding voyage-4-large, voyage-4 and voyage-4-lite (#3885)

• Adding voyage-4-large and voyage-4-lite

• Adding voyage-4-large and voyage-4-lite

• Adding voyage-4

• Reverting voyage-4 (as the tokenizer is not yet available publicly)

• added superseeded_by

---

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com> (64ce6ba)