Skip to content

feat(webui): Make max search results configurable via Helm#2246

Open
goynam wants to merge 1 commit intoy-scope:mainfrom
goynam:feat/configurable-webui-max-results
Open

feat(webui): Make max search results configurable via Helm#2246
goynam wants to merge 1 commit intoy-scope:mainfrom
goynam:feat/configurable-webui-max-results

Conversation

@goynam
Copy link
Copy Markdown
Contributor

@goynam goynam commented May 5, 2026

The maximum number of search results returned by the WebUI was hardcoded to 1000. This makes it configurable via:

  • Environment variable: SEARCH_MAX_NUM_RESULTS
  • Helm values: clpConfig.webui.max_num_results

The default remains 1000 for backward compatibility.

Description

Checklist

  • The PR satisfies the contribution guidelines.
  • This is a breaking change and that has been indicated in the PR title, OR this isn't a
    breaking change.
  • Necessary docs have been updated, OR no docs need to be updated.

Validation performed

Summary by CodeRabbit

  • New Features

    • Search results limit is now configurable (default 1,000). The limit is read at startup and applied to search queries.
  • Chores

    • Deployment templates and defaults updated to expose and set the new max-results setting; chart version bumped.

@goynam goynam requested a review from a team as a code owner May 5, 2026 06:12
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 5, 2026

Walkthrough

Adds a configurable search max-results limit: exposes SEARCH_MAX_NUM_RESULTS in Fastify config/schema and Helm values, switches the route constant to a mutable exported variable with a setter, and initializes that variable from fastify.config at plugin startup.

Changes

Search Max Results Configuration

Layer / File(s) Summary
Configuration Schema
components/webui/server/src/plugins/external/env.ts
Extends FastifyInstance.config with SEARCH_MAX_NUM_RESULTS: number and adds SEARCH_MAX_NUM_RESULTS to env validation schema.properties (type: number, default: 1_000, minimum: 1). Not added to required.
Mutable State & Setter
components/webui/server/src/routes/api/search/typings.ts
Converts SEARCH_MAX_NUM_RESULTS from const to let (initialized to 1000) and adds/export setSearchMaxNumResults(maxResults: number) to update it at runtime.
Runtime Initialization
components/webui/server/src/routes/api/search/index.ts
Imports setSearchMaxNumResults and calls it on plugin startup with fastify.config.SEARCH_MAX_NUM_RESULTS so the runtime value reflects configuration before handlers run.
Deployment Configuration
tools/deployment/package-helm/templates/webui-deployment.yaml, tools/deployment/package-helm/values.yaml, tools/deployment/package-helm/Chart.yaml
Adds clpConfig.webui.max_num_results: 1000 to values.yaml and injects SEARCH_MAX_NUM_RESULTS into the webui container from that value (default 1000). Bumps Helm chart version from 0.3.2-dev.0 to 0.3.2-dev.1.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat(webui): Make max search results configurable via Helm' directly and clearly summarizes the main change: making search result limits configurable through Helm configuration.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tools/deployment/package-helm/templates/webui-deployment.yaml`:
- Around line 58-59: The YAML linter flags the Helm template expression on the
SEARCH_MAX_NUM_RESULTS line for the "braces" rule; suppress it by adding an
inline yamllint disable comment for the braces rule on that line in
tools/deployment/package-helm/templates/webui-deployment.yaml (e.g., add a "#
yamllint disable-line braces" style comment right after or on the same line as
the SEARCH_MAX_NUM_RESULTS mapping) so the diff-scoped linter won't fail while
preserving the existing quoted Helm template expression.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 06efef7b-5726-44af-a977-25b87ffe04b3

📥 Commits

Reviewing files that changed from the base of the PR and between 351dd0e and b42745f.

📒 Files selected for processing (5)
  • components/webui/server/src/plugins/external/env.ts
  • components/webui/server/src/routes/api/search/index.ts
  • components/webui/server/src/routes/api/search/typings.ts
  • tools/deployment/package-helm/templates/webui-deployment.yaml
  • tools/deployment/package-helm/values.yaml

Comment thread tools/deployment/package-helm/templates/webui-deployment.yaml
@goynam goynam force-pushed the feat/configurable-webui-max-results branch from b42745f to c52cca1 Compare May 5, 2026 07:15
The maximum number of search results returned by the WebUI was hardcoded
to 1000. This makes it configurable via:
- Environment variable: SEARCH_MAX_NUM_RESULTS
- Helm values: clpConfig.webui.max_num_results

The default remains 1000 for backward compatibility.
@goynam goynam force-pushed the feat/configurable-webui-max-results branch from c52cca1 to e7f85dc Compare May 5, 2026 07:20
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@components/webui/server/src/routes/api/search/index.ts`:
- Line 19: The import statement bringing in SEARCH_MAX_NUM_RESULTS and
setSearchMaxNumResults from "./typings.js" should be split across multiple lines
to satisfy import-newlines/enforce and `@stylistic/object-curly-newline`; update
the import for the module "./typings.js" so the named specifiers
(SEARCH_MAX_NUM_RESULTS and setSearchMaxNumResults) each appear on their own
line (or otherwise broken across lines) inside the braces while preserving the
same identifiers and relative order.

In `@components/webui/server/src/routes/api/search/typings.ts`:
- Around line 16-24: Replace the magic numeric literal by introducing a named
constant and use it to initialize the mutable variable: add a const like
DEFAULT_SEARCH_MAX_NUM_RESULTS (value 1000) and change the mutable
SEARCH_MAX_NUM_RESULTS initialization to use that constant; keep the setter
setSearchMaxNumResults as-is so the runtime mutability remains but the literal
is no longer inline.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: d3762c92-35f9-49a6-987f-c940371f7f4f

📥 Commits

Reviewing files that changed from the base of the PR and between c52cca1 and e7f85dc.

📒 Files selected for processing (6)
  • components/webui/server/src/plugins/external/env.ts
  • components/webui/server/src/routes/api/search/index.ts
  • components/webui/server/src/routes/api/search/typings.ts
  • tools/deployment/package-helm/Chart.yaml
  • tools/deployment/package-helm/templates/webui-deployment.yaml
  • tools/deployment/package-helm/values.yaml


import settings from "../../../../settings.json" with {type: "json"};
import {SEARCH_MAX_NUM_RESULTS} from "./typings.js";
import {SEARCH_MAX_NUM_RESULTS, setSearchMaxNumResults} from "./typings.js";
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Split the named import across lines to satisfy the active lint rules.

Line 19 fails import-newlines/enforce and @stylistic/object-curly-newline, which blocks the lint job.

Proposed fix
-import {SEARCH_MAX_NUM_RESULTS, setSearchMaxNumResults} from "./typings.js";
+import {
+    SEARCH_MAX_NUM_RESULTS,
+    setSearchMaxNumResults,
+} from "./typings.js";
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
import {SEARCH_MAX_NUM_RESULTS, setSearchMaxNumResults} from "./typings.js";
import {
SEARCH_MAX_NUM_RESULTS,
setSearchMaxNumResults,
} from "./typings.js";
🧰 Tools
🪛 GitHub Actions: clp-lint

[error] 19-19: ESLint import-newlines/enforce: Imports must be broken into multiple lines if there are more than 1 elements.


[error] 19-19: ESLint @stylistic/object-curly-newline: Expected a line break after this opening brace.


[error] 19-19: ESLint @stylistic/object-curly-newline: Expected a line break before this closing brace.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@components/webui/server/src/routes/api/search/index.ts` at line 19, The
import statement bringing in SEARCH_MAX_NUM_RESULTS and setSearchMaxNumResults
from "./typings.js" should be split across multiple lines to satisfy
import-newlines/enforce and `@stylistic/object-curly-newline`; update the import
for the module "./typings.js" so the named specifiers (SEARCH_MAX_NUM_RESULTS
and setSearchMaxNumResults) each appear on their own line (or otherwise broken
across lines) inside the braces while preserving the same identifiers and
relative order.

Comment on lines +16 to +24
let SEARCH_MAX_NUM_RESULTS = 1000;

/**
* Sets the max number of search results from the fastify config.
* @param maxResults
*/
const setSearchMaxNumResults = (maxResults: number) => {
SEARCH_MAX_NUM_RESULTS = maxResults;
};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Fix the CI-blocking no-magic-numbers lint error in the mutable search limit.

Line 16 currently initializes a let with a numeric literal, which violates the configured ESLint rule and fails clp-lint.

Proposed fix
+const DEFAULT_SEARCH_MAX_NUM_RESULTS = 1_000;
+
 /**
  * The maximum number of results to retrieve for a search.
  * Configured via the SEARCH_MAX_NUM_RESULTS environment variable (default: 1000).
  */
-let SEARCH_MAX_NUM_RESULTS = 1000;
+let SEARCH_MAX_NUM_RESULTS = DEFAULT_SEARCH_MAX_NUM_RESULTS;
🧰 Tools
🪛 GitHub Actions: clp-lint

[error] 16-16: ESLint no-magic-numbers: Number constants declarations must use 'const'.


[warning] 19-19: ESLint jsdoc/tag-lines: Expected 1 lines after block description.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@components/webui/server/src/routes/api/search/typings.ts` around lines 16 -
24, Replace the magic numeric literal by introducing a named constant and use it
to initialize the mutable variable: add a const like
DEFAULT_SEARCH_MAX_NUM_RESULTS (value 1000) and change the mutable
SEARCH_MAX_NUM_RESULTS initialization to use that constant; keep the setter
setSearchMaxNumResults as-is so the runtime mutability remains but the literal
is no longer inline.

@junhaoliao
Copy link
Copy Markdown
Member

Thanks for working on this! To give some context: the max result count isn't really a proper mechanism for scalability - we'll likely drop it entirely in the future - but in the meantime, making it configurable is the right step, and it's great to see someone take it on.

I've been thinking through the same problem and wanted to share some design considerations for the approach. The core difference is around how the configuration flows through the system and how complete the coverage is.

1. Configuration mechanism: settings.json vs environment variable

In the webui, environment variables are used for credentials (e.g., database passwords, AWS keys), while settings.json handles general configuration (e.g., ClpQueryEngine, MaxDatasetsPerQuery, MongoDbSearchResultsMetadataCollectionName). Since max search results is a general config value rather than a credential, it fits naturally in settings.json.

This also matters for the next point — settings.json is the mechanism that delivers config to both the client and the server from a single source.

2. Client-side coverage

This PR only configures the server-side limit, but there's also a client-side cursor limit that independently caps displayed results at 1000:

  • client/src/pages/SearchPage/SearchResults/SearchResultsTable/typings.tsSEARCH_MAX_NUM_RESULTS = 1000
  • Used in useSearchResults.ts (MongoDB cursor limit) and usePrestoSearchResults.ts (same)

Even if the server is configured to store more results, the client will only fetch and display 1000 because the cursor limit is still hardcoded. For the feature to work end-to-end, the client needs to receive the same configured value — which is where settings.json naturally delivers it to both sides.

3. Presto search coverage

There are two additional hardcoded limits in the Presto search path that should also be configurable:

  • server/src/routes/api/presto-search/typings.tsMAX_PRESTO_SEARCH_RESULTS = 1000 (caps MongoDB storage)
  • client/src/pages/SearchPage/SearchControls/Presto/Guided/presto-guided-search-requests.tsDEFAULT_SEARCH_LIMIT = 1000 (SQL LIMIT clause in guided mode)

If the goal is "max search results is configurable," all three search paths (CLP server, Presto server, Presto guided SQL) should respect the same setting.

4. Avoiding mutable global state

The PR introduces let SEARCH_MAX_NUM_RESULTS with a setSearchMaxNumResults() setter. Since settings.json is already imported at module load time in search/index.ts, the value can be read directly as settings.MaxSearchResults — no mutable state or setter function needed. This keeps the code simpler and consistent with how the server reads other settings.

5. Standalone/Docker deployment coverage

The PR adds the Helm deployment path, but the standalone/Docker deployment (via clp-package) uses controller.py's _set_up_env_for_webui() to generate settings.json from clp-config.yaml. That path needs updating too:

  • Add max_search_results to the WebUi class in clp_config.py
  • Add "MaxSearchResults" to client_settings_json_updates and server_settings_json_updates in controller.py
  • Add the key to both settings.json template files (required because _read_and_update_settings_json validates that keys exist before updating)
  • Update components/package-template/src/etc/clp-config.template.json.yaml

Suggested approach (summary)

Following the MaxDatasetsPerQuery precedent:

Step File Change
Config definition clp_config.py Add max_search_results: PositiveInt = 1000 to WebUi
Templates client/public/settings.json, server/settings.json Add "MaxSearchResults": 1000
Propagation controller.py Add to both client & server settings_json_updates
Client types client/src/settings.ts Add MaxSearchResults: number to Settings
Client config client/src/config/index.ts Export SETTINGS_MAX_SEARCH_RESULTS
Client usage (3 sites) typings.ts, useSearchResults.ts, usePrestoSearchResults.ts Replace hardcoded constant with SETTINGS_MAX_SEARCH_RESULTS
Presto guided presto-guided-search-requests.ts Replace DEFAULT_SEARCH_LIMIT with SETTINGS_MAX_SEARCH_RESULTS
Server usage (3 sites) search/typings.ts, search/index.ts, search/utils.ts, presto-search/typings.ts, presto-search/index.ts Use settings.MaxSearchResults instead of hardcoded/mutable constants
Package template clp-config.template.json.yaml Add commented max_search_results: 1000
Helm values.yaml, configmap.yaml Add max_search_results to webui section and both settings JSON blocks

This covers all search paths end-to-end, uses the existing config flow, and avoids introducing mutable global state.

Happy to help if you'd like to iterate on any of this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants