关于rerank的训练和评估的疑问

[zengwb-lx]

你好，我看了你的训练代码中加载数据是：
class RerankTrainDataset(Dataset):
...
dataset = datasets.load_dataset(data_name_or_path)
dataset = dataset[dataset_split]

eval_retrieval2.py中加载数据是：
dataset = load_dataset("C-MTEB/T2Reranking", split="dev")
ds = dataset.train_test_split(test_size=0.1, seed=42)

这是不是相当于从训练集中加载了10%去做测试？

[LongxingTan]

您好，训练集是划分后单独保存的，剩下的10%用来评测的

from datasets import load_dataset

dataset = load_dataset("C-MTEB/T2Reranking", split="dev")
ds = dataset.train_test_split(test_size=0.1, seed=42)

ds_train = ds["train"].filter(
    lambda x: len(x["positive"]) > 0 and len(x["negative"]) > 0
)

ds_train.to_json("t2_ranking.jsonl", force_ascii=False)