Why does VLM Pipeline Removes all the Enrichments From ConvertPipeline?

[Sinha-Ujjawal]

Question

I can see that VlmPipeline is setting enrichment_pipe to empty list. here.

As I can understand VlmPipeline is a subclass of ConvertPipeline, so why does VlmPipeline resets the enrichment_pipe to empty when it was set with document classification and picture description in ConvertPipeline

[dosubot]

VlmPipeline resets enrichment_pipe to an empty list because it processes documents using Visual Language Models (VLMs), which generate all necessary structure and enrichment directly in their output. This makes traditional enrichment steps like document classification and picture description redundant or incompatible for VLM-based workflows. The code intentionally overrides the inherited enrichment modules to avoid unnecessary post-processing, focusing the pipeline on VLM-specific tasks instead see source.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[Sinha-Ujjawal]

@dosu Hmm.. but picture description has stopped working now.

[dosubot]

'Picture description' stopped working because VlmPipeline explicitly disables all enrichment steps—including picture description—by setting enrichment_pipe to an empty list in its constructor. This is a design choice to avoid redundant post-processing when using VLMs, but it means enrichment modules from ConvertPipeline (like picture description) are not run by default see code.

If you want to restore picture description in VlmPipeline, you can manually add the picture description model to enrichment_pipe after constructing the pipeline, using the same factory and options as ConvertPipeline does see implementation. This lets you selectively enable enrichment steps as needed, even in VLM workflows.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[dolfim-ibm]

The current VLM pipeline is designed as a single-shot prediction without any extra step. Including no enrichment step. The current idea is that the VLM model should do all at once.

[Sinha-Ujjawal]

@dolfim-ibm I guess I get that, but it will be really cool to have something inbuilt to use Vlm alongside StandardPdfPipeline. For now I have used the enrichment trick to "enhance" the OCR.

https://github.com/Sinha-Ujjawal/embed-ncert/blob/main/app_config/docling_config/text_enhancer_pipeline.py