Skip to content

Query: Enable promql-experimental-functions #8191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

Query: Enable promql-experimental-functions #8191

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
8aff32e
enable promql-experimental-functions
heysujal Apr 5, 2025
e5b0d3c
fix lint error
heysujal Apr 5, 2025
95f9ce6
update changelog
heysujal Apr 5, 2025
f53f873
Sidecar: increase default prometheus timeout (#8192)
MichaHoffmann Apr 7, 2025
f85b8e7
enable experimental function using parser
heysujal Apr 19, 2025
706b3c2
fix build
heysujal Apr 19, 2025
194f433
Merge branch 'main' into main
heysujal Apr 19, 2025
38bc09c
Merge branch 'main' into main
heysujal Apr 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ We use *breaking :warning:* to mark changes that are not backward compatible (re

### Added

- [#8191](https://github.com/thanos-io/thanos/pull/8191) Query: Add `--enable-feature=promql-experimental-functions` flag to enable experimental functions in query.

### Changed

- [#8192](https://github.com/thanos-io/thanos/pull/8192) Sidecar: fix default get config timeout
Expand Down
12 changes: 8 additions & 4 deletions cmd/thanos/query.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,10 @@ import (
)

const (
promqlNegativeOffset = "promql-negative-offset"
promqlAtModifier = "promql-at-modifier"
queryPushdown = "query-pushdown"
promqlNegativeOffset = "promql-negative-offset"
promqlAtModifier = "promql-at-modifier"
queryPushdown = "query-pushdown"
promqlExperimentalFunctions = "promql-experimental-functions"
)

// registerQuery registers a query command.
Expand Down Expand Up @@ -135,7 +136,7 @@ func registerQuery(app *extkingpin.App) {

activeQueryDir := cmd.Flag("query.active-query-path", "Directory to log currently active queries in the queries.active file.").Default("").String()

featureList := cmd.Flag("enable-feature", "Comma separated experimental feature names to enable.The current list of features is empty.").Hidden().Default("").Strings()
featureList := cmd.Flag("enable-feature", "Comma separated experimental feature names to enable. The current list of features is: promql-experimental-functions (enables experimental PromQL functions).").Hidden().Default("").Strings()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this flag still need to be hidden, I wonder


enableExemplarPartialResponse := cmd.Flag("exemplar.partial-response", "Enable partial response for exemplar endpoint. --no-exemplar.partial-response for disabling.").
Hidden().Default("true").Bool()
Expand Down Expand Up @@ -317,6 +318,7 @@ func registerQuery(app *extkingpin.App) {
*enableTargetPartialResponse,
*enableMetricMetadataPartialResponse,
*enableExemplarPartialResponse,
false,
Copy link
Contributor

@Saumya40-codes Saumya40-codes Apr 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if --enable-feature=promql-experimental-functions is set, this will still take it as false.

https://github.com/thanos-io/thanos/pull/8191/files#diff-f9808a0ac67a474a18f6a7ea767593f85c216aeafe322c1ff5d17c81ababff95R211-R221

maybe we can intercept by checking from here if the set value is promql-experimental-functions and pass true/false based on that.

*activeQueryDir,
time.Duration(*instantDefaultMaxSourceResolution),
*defaultMetadataTimeRange,
Expand Down Expand Up @@ -379,6 +381,7 @@ func runQuery(
enableTargetPartialResponse bool,
enableMetricMetadataPartialResponse bool,
enableExemplarPartialResponse bool,
enableQueryExperimentalFunctions bool,
activeQueryDir string,
instantDefaultMaxSourceResolution time.Duration,
defaultMetadataTimeRange time.Duration,
Expand Down Expand Up @@ -464,6 +467,7 @@ func runQuery(
lookbackDelta,
defaultEvaluationInterval,
extendedFunctionsEnabled,
enableQueryExperimentalFunctions,
activeQueryTracker,
queryMode,
)
Expand Down
5 changes: 5 additions & 0 deletions pkg/api/query/engine.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import (
"github.com/go-kit/log"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/prometheus/promql"
"github.com/prometheus/prometheus/promql/parser"
"github.com/prometheus/prometheus/storage"

"github.com/thanos-io/promql-engine/api"
Expand Down Expand Up @@ -88,10 +89,14 @@ func NewQueryFactory(
lookbackDelta time.Duration,
evaluationInterval time.Duration,
enableXFunctions bool,
enablePromQLExperimentalFunctions bool,
activeQueryTracker *promql.ActiveQueryTracker,
mode PromqlQueryMode,
) *QueryFactory {
makeOpts := func(registry prometheus.Registerer) engine.Opts {
// Set global experimental functions flag
parser.EnableExperimentalFunctions = enablePromQLExperimentalFunctions

Comment on lines +97 to +99
Copy link
Contributor

@Saumya40-codes Saumya40-codes Apr 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a Question: As I was working with this in engine (re: thanos-io/promql-engine#547) wonder if its ok to have this flag set logic in thanos side or we should keep parser related logic limited to engine.

Copy link
Contributor

@Saumya40-codes Saumya40-codes Apr 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps, adding a new field in engine struct like EnableExperimentalFunctions might be a good way if want to continue with later 👀

https://github.com/thanos-io/promql-engine/blob/main/engine/engine.go#L57C1-L84C2 and then when we do engine.New we can use this logic there (parser.EnableExperimentalFunctions=EnableExperimentalFunctions)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it sets the global in parser package, wouldn't that reflect everywhere (including in the thanos engine)? I think this is an ok pattern to follow. Would having extra option on the engine struct help other cases? 🙂

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right!

Current looks to be right then
We can resolve this!

opts := engine.Opts{
EngineOpts: promql.EngineOpts{
Logger: logutil.GoKitLogToSlog(logger),
Expand Down
1 change: 1 addition & 0 deletions pkg/api/query/v1_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ var (
5*time.Minute,
30*time.Second,
true,
false,
nil,
PromqlQueryModeLocal,
)
Expand Down
23 changes: 12 additions & 11 deletions pkg/queryfrontend/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,17 +202,18 @@ type Config struct {
LabelsConfig
DownstreamTripperConfig

CortexHandlerConfig *transport.HandlerConfig
CompressResponses bool
CacheCompression string
RequestLoggingDecision string
DownstreamURL string
ForwardHeaders []string
NumShards int
TenantHeader string
DefaultTenant string
TenantCertField string
EnableXFunctions bool
CortexHandlerConfig *transport.HandlerConfig
CompressResponses bool
CacheCompression string
RequestLoggingDecision string
DownstreamURL string
ForwardHeaders []string
NumShards int
TenantHeader string
DefaultTenant string
TenantCertField string
EnableXFunctions bool
EnableQueryExperimentalFunctions bool
Copy link
Contributor

@Saumya40-codes Saumya40-codes Apr 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can use this field at cmd/thanos/query_frontend.go now

e.g. https://github.com/thanos-io/thanos/blob/main/cmd/thanos/query_frontend.go#L161

}

// QueryRangeConfig holds the config for query range tripperware.
Expand Down
6 changes: 6 additions & 0 deletions test/e2e/e2ethanos/services.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -282,6 +282,7 @@ type QuerierBuilder struct {
queryDistributedWithOverlappingInterval bool
enableXFunctions bool
deduplicationFunc string
enableQueryExperimentalFunctions bool

replicaLabels []string
tracingConfig string
Expand Down Expand Up @@ -391,6 +392,11 @@ func (q *QuerierBuilder) WithEnableXFunctions() *QuerierBuilder {
return q
}

func (q *QuerierBuilder) WithEnableExperimentalFunctions() *QuerierBuilder {
q.enableQueryExperimentalFunctions = true
return q
}

func (q *QuerierBuilder) WithDeduplicationFunc(deduplicationFunc string) *QuerierBuilder {
q.deduplicationFunc = deduplicationFunc
return q
Expand Down
Loading