【BUG】occur error：AttributerError：'DeepSpeedHybridEngine' object has no attribute 'mp_group' whiling run llama7b for step3/rlhf/ppo

[Pattaro]

[Chevolier]

Any solutions? I encountered the same issue with bloomz model, so far I just removed --enable_hybrid_engine to bypass the issue, and the program runs. However, guess the efficiency would reduce.

[DwarfWarriors]

It seems enable_hybrid_engine only support for OPT models.

[kkk935208447]

try adjusting the --inference_tp_size to a lower number, it may be you don't have enough GPUs across your nodes.

[bug]AttributeError: 'DeepSpeedHybridEngine' object has no attribute 'mp_group' #525