Generalizing plan execution

[pdelewski]

This PR extends the current Quesma workflow to allow interrupting query execution when the execution plan contains multiple queries and we’ve already retrieved enough results.

The idea is to split the original query into n queries with different time ranges. If the first one (covering a 15-minute range) returns enough results, we will stop the execution.

The problem is reproducible on kibana_sample_data_flights with the following query:

GET /kibana_sample_data_flights/_search 
{
  "_source": false,
  "fields": [
    {
      "field": "*",
      "include_unmapped": "true"
    },
    {
      "field": "@timestamp",
      "format": "strict_date_optional_time"
    },
    {
      "field": "timestamp",
      "format": "strict_date_optional_time"
    }
  ],
  "highlight": {
    "fields": {
      "*": {}
    },
    "fragment_size": 2147483647,
    "post_tags": [
      "@/kibana-highlighted-field@"
    ],
    "pre_tags": [
      "@kibana-highlighted-field@"
    ]
  },
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "timestamp": {
              "format": "strict_date_optional_time",
              "gte": "2025-05-13T08:46:01.926Z",
              "lte": "2025-05-13T09:01:01.926Z"
            }
          }
        }
      ],
      "must": [],
      "must_not": [],
      "should": []
    }
  },
  "runtime_mappings": {
    "hour_of_day": {
      "script": {
        "source": "emit(doc['timestamp'].value.getHour());"
      },
      "type": "long"
    }
  },
  "script_fields": {},
  "size": 500,
  "sort": [
    {
      "timestamp": {
        "format": "strict_date_optional_time",
        "order": "desc",
        "unmapped_type": "boolean"
      }
    },
    {
      "_doc": {
        "order": "desc",
        "unmapped_type": "boolean"
      }
    }
  ],
  "stored_fields": [
    "*"
  ],
  "track_total_hits": false,
  "version": true
}

[trzysiek]

Loving the direction. Can we merge it already?

[pdelewski]

Loving the direction. Can we merge it already?

Not ready yet.

[pdelewski]

/run-it

[pdelewski]

/run-it

[pdelewski]

/run-it

[pdelewski]

/run-it

[pdelewski]

/run-it

[pdelewski]

/run-it

[pdelewski]

/run-it

[mieciu]

some minor things, other than that - nice one 💪

[cloudflare-workers-and-pages]

Deploying quesma with    Cloudflare Pages

Latest commit:	4329e3f
Status:	✅  Deploy successful!
Preview URL:	https://abe2613e.quesma.pages.dev
Branch Preview URL:	https://generalizing-execution-plan.quesma.pages.dev

View logs

[pdelewski]

/run-it