[QUESTION] When will multimodal evaluation support pipeline parallelism ?

[wangxiaoyang-dev]

In currently, multimodal evaluation does not support pipeline parallelism, refer to

Megatron-LM/examples/multimodal/run_text_generation.py

Line 944 in db7d37b

logging.getLogger(__name__).warning("Models using pipeline parallelism are not supported yet.")

In some multimodals such as InternVL2-Llama3-76B, its vision model has 25 attention heads which does not support tensor parallelism, so it needs pipeline parallelism to evaluation. so i want to ask when will multimodal evaluation support pipeline parallelism ?

thank.