Suggestion: add WFGY 16 Problem Map as a RAG debugging checklist

[onestardao]

Hi, thanks a lot for open sourcing baml and for the work on bringing stronger "engineering" into prompt engineering.

I wanted to share one external resource that might pair well with baml users who are building real RAG and multi step workflows.

What it is

• Name: WFGY 16 Problem Map
• Link: https://github.com/onestardao/WFGY/tree/main/ProblemMap/README.md
• Type: vendor neutral troubleshooting map for LLM pipelines, focused on RAG, vector stores and long context prompts
• License: MIT

The map defines 16 concrete failure modes such as:

• ingestion and parsing drift
• chunking and context contract mismatch
• vector store fragmentation and metric problems
• retrieval order issues and prompt injection
• evaluator mismatch for long horizon chains

For each failure type there is a short description, symptoms that users can observe, and a minimal fix strategy. The idea is to treat it as a checklist you can run through when a RAG or agentic workflow behaves in strange ways.

Why I think it might fit baml

baml already helps enforce stronger structure and contracts at the code and prompt level. A lot of real world failures still come from data and retrieval issues around the model. The 16 Problem Map is designed to complement that by giving teams a shared language for the typical RAG and vector store breakpoints.

This could be useful for:

• teams using baml to design production agents and wanting a more systematic debug playbook
• docs readers who search for "why is my RAG broken" and need a high level map of where to look

Possible integration

Totally up to you, but some low friction options might be:

• a short bullet under a "Further reading" or "RAG debugging" section in the docs
• one sentence mention in any place where you already talk about failure modes, reliability or context management

If this feels out of scope for the project that is completely fine. Either way, thank you for the work on baml and for pushing structured prompt engineering forward.

[linear]

BAML-586

[onestardao]

Thanks for wiring this into your Linear tracker!

I tried opening the BAML-586 link but I get an “Authentication error”, which I assume just means it is internal to your workspace. That’s totally fine – if you need any extra details or a concrete example from my side, I’m very happy to provide them.

[onestardao]

Hi team — just a small follow-up update here.

Since opening this issue, the WFGY 16-problem RAG failure map has been integrated or cited in several production and research RAG projects, including:

• LlamaIndex (RAG troubleshooting docs)
• RAGFlow (official RAG failure checklist guide)
• ToolUniverse (Harvard MIMS Lab tool wrapper for incident triage)
• Rankify (Univ. of Innsbruck re-ranking diagnostics)
• Multimodal RAG Survey (QCRI LLM Lab)

I thought this might be relevant context, since baml focuses on bringing engineering structure to prompt and LLM workflows. The map is being used as a shared failure vocabulary layer on top of existing infra.

No rush at all — just wanted to share the ecosystem update in case it helps with internal evaluation of BAML-586.

Happy to provide a minimal PR draft anytime if useful.