尝试复现但有些问题
你好这是我对table1和table5的尝试复现和论文结果有些出,我的这些计算方式是否有问题,或者是论文的具体参数设置
这是我单卡H200 140G跑出的Table 5:
依照论文进行的参数设置:
BON
"prompts": "AMC",
"max_prompts": null,
"backend": "huggingface",
"n": 8,
"max_generation_len": 1024,
"temperature": 0.5,
"top_p": 0.95,
"top_k": null,
"seed": 42,
"dtype": "bfloat16",
"attn_implementation": "flash_attention_2",
"reward_batch_size": 8
SC
"prompts": "AMC",
"max_prompts": null,
"backend": "huggingface",
"n": 8,
"max_generation_len": 1024,
"temperature": 0.5,
"top_p": 0.95,
"top_k": null,
"seed": 42,
"dtype": "bfloat16",
"attn_implementation": "flash_attention_2",
尝试复现但有些问题
你好这是我对table1和table5的尝试复现和论文结果有些出,我的这些计算方式是否有问题,或者是论文的具体参数设置
这是我单卡H200 140G跑出的Table 5:
依照论文进行的参数设置:
BON
"prompts": "AMC",
"max_prompts": null,
"backend": "huggingface",
"n": 8,
"max_generation_len": 1024,
"temperature": 0.5,
"top_p": 0.95,
"top_k": null,
"seed": 42,
"dtype": "bfloat16",
"attn_implementation": "flash_attention_2",
"reward_batch_size": 8
SC
"prompts": "AMC",
"max_prompts": null,
"backend": "huggingface",
"n": 8,
"max_generation_len": 1024,
"temperature": 0.5,
"top_p": 0.95,
"top_k": null,
"seed": 42,
"dtype": "bfloat16",
"attn_implementation": "flash_attention_2",