You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This page provides an overview of how to use {{esql}} for search use cases.
17
17
18
18
::::{tip}
19
-
Prefer to get started with a hands-on tutorial? Check out [Search and filter with {{esql}}](esql-search-tutorial.md).
19
+
For a hands-on tutorial check out [Search and filter with {{esql}}](esql-search-tutorial.md).
20
20
::::
21
21
22
+
## {{esql}} search quick reference
22
23
23
-
## Overview
24
+
The following table summarizes the key search features available in [{{esql}}](/explore-analyze/query-filter/languages/esql.md) and when they were introduced, organized chronologically by release.
24
25
25
-
The following table summarizes the key search features available in [{{esql}}](/explore-analyze/query-filter/languages/esql.md) and when they were introduced.
26
+
| Feature | Description | Available since |
27
+
|---------|-------------|----------------|
28
+
|[Match function/operator](#match-function-and-operator)| Perform basic text searches with `MATCH` function or match operator (`:`) | 8.17 |
29
+
|[Query string function](#query-string-qstr-function)| Execute complex queries with `QSTR` using Query String syntax | 8.17 |
30
+
|[Relevance scoring](#esql-for-search-scoring)| Calculate and sort by relevance with `METADATA _score`| 8.18/9.0 |
31
+
|[Semantic search](#semantic-search)| Perform semantic searches on `semantic_text` field types | 8.18/9.0 |
32
+
|[Hybrid search](#hybrid-search)| Combine lexical and semantic search approaches with custom weights | 8.18/9.0 |
33
+
|[Kibana Query Language](#kql-function)| Use Kibana Query Language with the `KQL` function | 8.18/9.0 |
34
+
|[Match phrase function](#match_phrase-function)| Perform phrase matching with `MATCH_PHRASE` function | 8.19/9.1 |
26
35
27
-
| Feature | Available since | Description |
28
-
|---------|----------------|-------------|
29
-
|[Full text search functions](elasticsearch://reference/query-languages/esql/functions-operators/search-functions.md)| 8.17 | Perform basic text searches with `MATCH` function or match operator (`:`) |
30
-
|[Query string function](#esql-for-search-query-string)| 8.17 | Execute complex queries with `QSTR` using Query String syntax |
31
-
|[Relevance scoring](#esql-for-search-scoring)| 8.18/9.0 | Calculate and sort by relevance with `METADATA _score`|
32
-
| Enhanced match options | 8.18/9.0 | Configure text searches with additional parameters for the `MATCH` function |
33
-
|[Kibana Query Language](#esql-for-search-kql)| 8.18/9.0 | Use Kibana Query Language with the `KQL` function |
34
-
|[Semantic search](#esql-for-search-semantic)| 8.18/9.0 | Perform semantic searches on `semantic_text` field types |
35
-
|[Hybrid search](#esql-for-search-hybrid)| 8.18/9.0 | Combine lexical and semantic search approaches with custom weights |
36
+
## How search works in {{esql}}
36
37
37
-
## Filtering vs. searching [esql-filtering-vs-searching]
38
+
{{esql}} provides two distinct approaches for finding documents: filtering and searching. Understanding the difference is crucial for building effective queries and choosing the right approach for your use case.
38
39
39
-
{{esql}} can be used for both simple filtering and relevance-based searching:
40
+
**Filtering** removes documents that don't meet your criteria. It's a binary yes/no decision - documents either match your conditions or they don't. Filtering is faster because it doesn't calculate relevance scores and leverages efficient index structures for exact matches, ranges, and boolean logic.
40
41
41
-
***Filtering** removes non-matching documents without calculating relevance scores
42
-
***Searching** both filters documents and ranks them by how well they match the query
42
+
**Searching** both filters documents and ranks them by relevance. It calculates a score for each matching document based on how well the content matches your query, allowing you to sort results from most relevant to least relevant. Search functions use advanced text analysis including stemming, synonyms, and fuzzy matching.
43
43
44
-
Note that filtering is faster than searching, because it doesn't require score calculations.
- Any scenario where you want all matching results without ranking
45
49
46
-
### Relevance scoring [esql-for-search-scoring]
47
-
48
-
To get the most relevant results first, you need to use `METADATA _score` and sort by score. For example:
49
-
50
-
```esql
51
-
FROM books METADATA _score
52
-
| WHERE match(title, "Shakespeare") OR match(plot, "Shakespeare")
53
-
| SORT _score DESC
54
-
```
55
-
56
-
### How `_score` works [esql-for-search-how-scoring-works]
50
+
**When to choose searching:**
51
+
- Text queries where some results are more relevant than others
52
+
- Finding documents similar to a search phrase
53
+
- Any scenario where you want the "best" matches first
54
+
- You want to use [analyzers](elasticsearch://reference/elasticsearch/mapping-reference/analyzer.md) or [synonyms](/solutions/search/full-text/search-with-synonyms.md)
57
55
58
-
When working with relevance scoring in {{esql}}:
56
+
{{esql}}'s search functions address several key limitations that existed for text filtering: they work directly on multivalued fields, leverage analyzers for proper text analysis, and use optimized Lucene index structures for better performance.
59
57
60
-
* If you don't include `METADATA _score` in your query, this only performs filtering operations with no relevance calculation.
61
-
* When you include `METADATA _score`, any search function included in `WHERE` conditions contribute to the relevance score. This means that every occurrence of `MATCH`, `QSTR` and `KQL` will affect the score.
62
-
* Filtering operations that are not search functions, like range conditions and exact matches, don't affect the score.
63
-
* Including `METADATA _score` doesn't automatically sort your results by relevance. You must explicitly use `SORT _score DESC` or `SORT _score ASC` to order your results by relevance.
58
+
### Relevance scoring [esql-for-search-scoring]
64
59
65
-
## Full text search [esql-for-search-full-text]
60
+
To get relevance-ranked results, you must explicitly request scoring with `METADATA _score` and sort by the score. Without this, even search functions like `MATCH` will only filter documents without ranking them.
66
61
67
-
### `MATCH` function and operator [esql-for-search-match-function-operator]
62
+
**Without `METADATA _score`**: All operations are filtering-only, even `MATCH`, `QSTR`, and `KQL` functions. Documents either match or don't match - no ranking occurs.
68
63
69
-
{{esql}} offers two syntax options for `match`, which replicate the functionality of [match](elasticsearch://reference/query-languages/query-dsl/query-dsl-match-query.md)queries in Query DSL.
64
+
**With `METADATA _score`**: [Search functions](elasticsearch://reference/query-languages/esql/functions-operators/search-functions.md)contribute to relevance scores, while filtering operations (range conditions, exact matches) don't affect scoring. You must explicitly use `SORT _score DESC` to see the most relevant results first.
70
65
71
-
Use the compact operator syntax (`:`) for simple text matching with default parameters.
66
+
This gives you full control over when to use fast filtering versus slower but more powerful relevance-based searching.
72
67
73
-
```esql
74
-
FROM logs | WHERE message: "connection error"
75
-
```
68
+
## Search functions
76
69
77
-
Use the `match()` function syntax when you need to pass additional parameters:
70
+
The following functions provide text-based search capabilities in {{esql}} with different levels of precision and control.
78
71
79
-
```esql
80
-
FROM products | WHERE match(name, "laptop", { "boost": 2.0 })
81
-
```
72
+
### `MATCH` function and operator
82
73
83
-
These full-text functions address several key limitations that existed for text filtering in {{esql}}:
74
+
{{esql}} offers two syntax options for match, which replicate the functionality of [match](elasticsearch://reference/query-languages/query-dsl/query-dsl-match-query.md) queries in Query DSL.
84
75
85
-
* They work directly on multivalued fields, returning results when any value in a multivalued field matches the query
86
-
* They leverage analyzers, ensuring the query is analyzed with the same process as the indexed data (enabling case-insensitive matching, ASCII folding, stopword removal, and synonym support)
87
-
* They are highly performant, using Lucene index structures rather than pattern matching or regular expressions to locate terms in your data
76
+
- Use the compact [operator syntax (:)](elasticsearch://reference/query-languages/esql/functions-operators/operators.md#esql-match-operator) for simple text matching with default parameters.
77
+
- Use the [MATCH function syntax](elasticsearch://reference/query-languages/esql/functions-operators/search-functions.md#esql-match) for more control over the query, such as specifying analyzers, fuzziness, and other parameters.
88
78
89
-
Refer to this blog for more context: [Introducing full text filtering in {{esql}}](https://www.elastic.co/search-labs/blog/filtering-in-esql-full-text-search-match-qstr).
79
+
Refer to the [tutorial](esql-search-tutorial.md#step-3-basic-search-operations) for examples of both syntaxes.
90
80
91
-
::::{tip}
92
-
See [Match field parameters](elasticsearch://reference/query-languages/esql/functions-operators/search-functions.md#esql-match) for more advanced options using match.
93
-
::::
81
+
### `MATCH_PHRASE` function
94
82
95
-
::::{important}
96
-
These queries match documents but don't automatically sort by relevance. To get the most relevant results first, you need to use `METADATA _score` and sort by score. See [Relevance scoring](#esql-for-search-scoring) for more information.
97
-
::::
83
+
Use the [`MATCH_PHRASE` function](elasticsearch://reference/query-languages/esql/functions-operators/search-functions.md#esql-match_phrase) to perform a `match_phrase` query on the specified field. This is equivalent to using the [match_phrase query](elasticsearch://reference/query-languages/query-dsl/query-dsl-match-query-phrase.md) in Query DSL.
98
84
99
-
### Query string (`QSTR`) function [esql-for-search-query-string]
85
+
For exact phrase matching rather than individual word matching, use `MATCH_PHRASE`.
100
86
101
-
The [`qstr` function](elasticsearch://reference/query-languages/esql/functions-operators/search-functions.md#esql-qstr) provides the same functionality as the Query DSL's `query_string` query. This is for advanced use cases, such as wildcard searches, searches across multiple fields, and more.
87
+
### Query string (`QSTR`) function
102
88
103
-
```esql
104
-
FROM articles METADATA _score
105
-
| WHERE QSTR("(new york city) OR (big apple)")
106
-
| SORT _score DESC
107
-
| LIMIT 10
108
-
```
89
+
The [`qstr` function](elasticsearch://reference/query-languages/esql/functions-operators/search-functions.md#esql-qstr) provides the same functionality as the Query DSL's `query_string` query. This enables advanced search patterns with wildcards, boolean logic, and multi-field searches.
109
90
110
91
For complete details, refer to the [Query DSL `query_string` docs](elasticsearch://reference/query-languages/query-dsl/query-dsl-query-string-query.md).
111
92
112
-
### `KQL` function [esql-for-search-kql]
113
-
114
-
Use the [KQL function](elasticsearch://reference/query-languages/esql/functions-operators/search-functions.md#esql-kql) to use the [Kibana Query Language](/explore-analyze/query-filter/languages/kql.md) in your {{esql}} queries:
115
-
116
-
```esql
117
-
FROM logs*
118
-
| WHERE KQL("http.request.method:GET AND agent.type:filebeat")
119
-
```
120
-
121
-
The `kql` function is useful when transitioning queries from Kibana's Discover, Dashboard, or other interfaces that use KQL. This will allow you to gradually migrate queries to {{esql}} without needing to rewrite them all at once.
93
+
### `KQL` function
122
94
123
-
## Semantic search [esql-for-search-semantic]
95
+
Use the [KQL function](elasticsearch://reference/query-languages/esql/functions-operators/search-functions.md#esql-kql) to use the [Kibana Query Language](/explore-analyze/query-filter/languages/kql.md) in your {{esql}} queries.
124
96
125
-
You can perform semantic searches over [`semantic_text`](elasticsearch://reference/elasticsearch/mapping-reference/semantic-text.md) field types using the same match syntax as full-text search.
97
+
For migrating queries from other Kibana interfaces, the `KQL` function preserves existing query syntax and allows gradual migration to {{esql}} without rewriting existing Kibana queries.
126
98
127
-
This example uses the match operator `:`:
99
+
## Advanced search capabilities
128
100
129
-
```esql
130
-
FROM articles METADATA _score
131
-
| WHERE semantic_content: "What are the impacts of climate change on agriculture?"
132
-
| SORT _score DESC
133
-
```
101
+
### Semantic search
134
102
135
-
This example uses the match function:
103
+
[Semantic search](/solutions/search/semantic-search.md) leverages machine learning models to understand the meaning of text, enabling more accurate and context-aware search results.
136
104
137
-
```esql
138
-
FROM articles METADATA _score
139
-
| WHERE match(semantic_content, "What are the impacts of climate change on agriculture?")
140
-
| SORT _score DESC
141
-
```
105
+
In {{esql}}, you can perform semantic searches on [`semantic_text`](elasticsearch://reference/elasticsearch/mapping-reference/semantic-text.md) field types using the same match syntax as full-text search.
142
106
143
-
## Hybrid search [esql-for-search-hybrid]
107
+
Refer to [semantic search with semantic_text](/solutions/search/semantic-search/semantic-search-semantic-text.md) for an example or follow the [tutorial](esql-search-tutorial.md#step-5-semantic-search-and-hybrid-search).
144
108
145
-
[Hybrid search](/solutions/search/hybrid-search.md) combines lexical and semantic search with custom weights:
109
+
### Hybrid search
146
110
147
-
```esql
148
-
FROM books METADATA _score
149
-
| WHERE match(semantic_title, "fantasy adventure", { "boost": 0.75 })
150
-
OR match(title, "fantasy adventure", { "boost": 0.25 })
151
-
| SORT _score DESC
152
-
```
111
+
[Hybrid search](/solutions/search/hybrid-search.md) combines lexical and semantic search with custom weights to leverage both exact keyword matching and semantic understanding.
153
112
154
-
## Limitations [esql-for-search-limitations]
155
-
156
-
Refer to [{{esql}} limitations](elasticsearch://reference/query-languages/esql/limitations.md#esql-limitations-full-text-search) for a list of known limitations.
113
+
Refer to [hybrid search with semantic_text](hybrid-semantic-text.md) for an example or follow the [tutorial](esql-search-tutorial.md#step-5-semantic-search-and-hybrid-search).
157
114
158
115
## Next steps [esql-for-search-next-steps]
159
116
160
117
### Tutorials and how-to guides [esql-for-search-tutorials]
161
118
162
-
-[Search and filter with {{esql}}](esql-search-tutorial.md): Hands-on tutorial for getting started with search tools in {{esql}}
163
-
-[Semantic search with semantic_text](semantic-search/semantic-search-semantic-text.md): Learn how to use the `semantic_text` field type
119
+
-[Search and filter with {{esql}}](esql-search-tutorial.md): Hands-on tutorial for getting started with search tools in {{esql}}, with concrete examples of the functionalities described in this page
-[Search functions](elasticsearch://reference/query-languages/esql/functions-operators/search-functions.md): Complete reference for all search functions
168
-
-[Limitations](elasticsearch://reference/query-languages/esql/limitations.md#esql-limitations-full-text-search): Current limitations for search in {{esql}}
-[Analysis](/manage-data/data-store/text-analysis.md): Learn how text is processed for full-text search
173
-
-[Semantic search](semantic-search.md): Get an overview of semantic search in Elasticsearch
174
-
-[Query vs filter context](elasticsearch://reference/query-languages/query-dsl/query-filter-context.md): Understand the difference between query and filter contexts in Elasticsearch
124
+
-[Limitations](elasticsearch://reference/query-languages/esql/limitations.md#esql-limitations-full-text-search): Current limitations for search functions in {{esql}}
0 commit comments