This repository was archived by the owner on Nov 7, 2025. It is now read-only.
Fix /*/_mapping and /*/_field_caps handling
#1475
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nablaone
changed the title
/*/_mapping and /*/_field_caps handling
…doesn't check if we can query across multiple tables.
Contributor
There was a problem hiding this comment.
Pull Request Overview
This pull request addresses issues with handling mappings and field capabilities for all indices by introducing a Meta pipeline and updating the table resolver logic. Key changes include:
- Adding a MetaPipeline constant and associated resolver support.
- Refactoring decision merging to use a basicDecisionMerger interface.
- Enhancing endpoint handlers and e2e tests to properly filter out a common table index.
Reviewed Changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| platform/v2/core/decision.go | Added MetaPipeline constant. |
| platform/table_resolver/table_resolver.go | Changed decisionMerger to an interface; updated index update logic. |
| platform/table_resolver/rules.go | Modified basicDecisionMerger implementation and usage. |
| platform/processors/es_to_ch_query/elasticsearch_to_clickhouse_query_processor.go | Updated HandleGetIndexMapping call to include context and log manager. |
| platform/functionality/field_capabilities/field_caps.go | Added filtering of common table from index list. |
| platform/frontend_connectors/router_v2.go | Updated router registrations to use the new MetaPipeline. |
| platform/frontend_connectors/route_handlers.go | Enhanced HandleGetIndexMapping to iterate over filtered indexes. |
| platform/frontend_connectors/es_responses.go | Updated response generation functions for mappings. |
| http_requests/_mapping.http | Added new HTTP request tests for mapping and field_caps endpoints. |
| ci/it/testcases/test_common_table_and_regular_table.go | Added e2e tests for common and regular table handling. |
| ci/it/integration_test.go | Included integration test for the common table scenario. |
| ci/it/configs/quesma-common-table-and-regular-table.yml.template | New configuration template supporting common table along with regular tables. |
pdelewski
reviewed
Jun 25, 2025
pdelewski
reviewed
Jun 25, 2025
| mappings := elasticsearch.GenerateMappings(hierarchicalSchema) | ||
| allMappings := make(map[string]map[string]any) | ||
|
|
||
| var filteredIndexes []string |
Contributor
There was a problem hiding this comment.
The same as below, it would be better to filter out common_table in ResolveIndexPattern
pdelewski
reviewed
Jun 26, 2025
| const ( | ||
| QueryPipeline = "Query" | ||
| IngestPipeline = "Ingest" | ||
| MetaPipeline = "Meta" |
Contributor
There was a problem hiding this comment.
Have some concerns about these pipelines, but that's a matter of broader discussion
Member
Author
There was a problem hiding this comment.
Yes. This is quite controversial. We should reassign the responsibility of the table resolver.
Contributor
|
/run-it |
Contributor
|
It seems that the integration tests currently take more than 10 minutes and are failing due to this time limit. |
Contributor
|
/run-it |
pdelewski
force-pushed
the
que-278-quesma-request-failed-common-table-is-not-allowed-to-be
branch
from
e405e4a to
7527a51
Compare
Contributor
|
/run-it |
Contributor
pdelewski
approved these changes
Jun 26, 2025
nablaone
deleted the
que-278-quesma-request-failed-common-table-is-not-allowed-to-be
branch
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes getting mappings and fields caps for all indices.
Details: