[SIP-157] Proposal for Agentic Query In Dashboard

[Abhishek-kumar-samsung]

[SIP-157] Proposal for Agentic Query In Dashboard

Motivation

This SIP aims to provide users with the flexibility to ask any question they desire from the dashboard. Our motivation is to place a search button at the top of the dashboard, enabling users to input any natural language query that can be answered using the dashboard's charts.

Proposed Change

We propose to add an LLM agentic query search button at the top of the dashboard if the feature flag is set to True.

Use cases:

1. Users can perform natural language query-based searches on the dashboard.
2. Users can ask specific questions about any chart using natural language.
3. Users can request a summary of a chart or the entire dashboard by asking the search button to do so.
4. Feature Flag: Users have an option to make feature flag as False if they don't want to use the feature.

New or Changed Public Interfaces

Views changed:

1. Dashboard view changed based on feature flag in superset_config SHOW_DASHBOARD_AGENT_QUERY.
2. Added a new action in dashboard react components to call query dashboard rest api.
3. Added a new REST api in dashboard apis to query dashboard.

New dependencies

1. Added two PyPI packages langchain and langchain-openai, both have MIT license.
2. To run this feature user has to update LLM configurations in superset_config, mainly api_key if they want to use openai or base_url if they want to use locally running llm.

Mockups and Screenshots

When feature flag is True

When feature flag is False i.e default view

CLI logs when query runs looks like :

PR Raised

PR link : #32649

[Abhishek-kumar-samsung]

PR link : #32649

[Abhishek-kumar-samsung]

Related to : #32408

[mistercrunch]

Pointing out that this feature/SIP would best be implemented as an extension as SIPped here #31932 .

Some the reasons why I think this is a good fit for an extension:

• design patterns for RAG / AI-integrations are still emerging, there's probably a multitude of ways this could be implement from a UX perspective. Picking one specific UX may prevent us from getting a wider variety of shapes this could take as it evolves
• there's probably a need to vectorize things / create and manage embedding to retrieve the right context through semantic search (datasets, sample data, result sets, related-charts/objects, ...) . I'm guessing the whole context is too large for the context window, and that there's quite a bit of RAG needed to support this and other AI-related features
• feature would need to be highly configurable, so that it can talk with a variety of AI APIs (GPT, Claude, Grok, Gemini...), support different vector dbs (pincone, pgvector, ...), ... bringing potentially a lot of extra libs to the main project (langchain/langraph, openai, ...)
• an extension would allow for different AI extension experiments to compete and mature in isolation, without putting a lot of eggs/complexity in the main repo/app
• this would help test-prove and push [SIP-151] The vision for a new Superset Plugins Architecture #31932 forward