Simpler approach

[pablonyx]

In the Search pipeline, we need to first prune, and then merge. This desired behavior was not adhered to anymore and has been re-established.

prompt_config and contextual_pruning _config both need to be set for this to work.

This fix is also related to https://linear.app/danswer/issue/DAN-1626/dalberg-projects-not-being-foundcited-as-expected

Description

Key changes:

• passed contextual_pruning_config to Search pipeline
• used prune_and_merge to create final chunks
• if prompt_config not done or contextual_pruning_config not available, default to earlier behavior of calling _merge_sections

How Has This Been Tested?

Locally. Tested with default Search, and then also an Assistant that does not do AI filtering

Backporting (check the box to trigger backport action)

Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.

• This PR should be backported (make sure to check that the backport attempt succeeds)
• [Optional] Override Linear Check

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
internal-search	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 1, 2025 11:54pm