Parameterize queued_jobs_aggregate ClickHouse query (#8007)

**Impact:** AWS autoscaler Lambda and any future consumers of the
`queued_jobs_aggregate` HUD API endpoint
**Risk:** low

## What
Replaces hardcoded filter values in the `queued_jobs_aggregate`
ClickHouse query with named parameters that have sensible defaults, and
adds support for a `defaults` mechanism in `queryClickhouseSaved` so
callers can omit parameters and get the previous behavior.

## Why
The `queued_jobs_aggregate` query powers the AWS autoscaler
(`scale-up-chron` Lambda) that decides how many CI runners to spin up
based on queued job counts. The query previously hardcoded the
queued-time threshold (30 min), max age window (3 days), and org filter
(`pytorch`, `pytorch-labs`, `meta-pytorch`) — making it impossible for
different callers (e.g., a per-repo autoscaler or a new fleet) to
customize behavior without duplicating the query.

Part of the work tracked in
[pytorch/ci-infra#499](pytorch/ci-infra#499)
and companion changes in
[pytorch/ci-infra#500](pytorch/ci-infra#500) and
[jeanschmidt/actions-runner-controller#1](jeanschmidt/actions-runner-controller#1).

## How
- Introduced a `"defaults"` key in `params.json` — a new convention for
ClickHouse saved queries. When a caller omits a parameter, the default
value is used instead, preserving backward compatibility for existing
callers that pass `{}`.
- Used ClickHouse's `toIntervalMinute()` / `toIntervalDay()` functions
instead of `INTERVAL` literals to accept parameterized values.
- Added an optional `repo` filter (`{repo: String}`) that, when empty
string (the default), matches all repos — allowing callers to scope
results to a single repository without changing the query structure.

## Changes
- **`torchci/clickhouse_queries/queued_jobs_aggregate/query.sql`**
- `INTERVAL 30 MINUTE` → `toIntervalMinute({queuedThresholdMinutes:
Int64})`
  - `INTERVAL 3 DAY` → `toIntervalDay({maxAgeDays: Int64})`
  - Hardcoded org list → `{orgs: Array(String)}` parameter
  - Added optional `{repo: String}` filter (no-op when empty string)
- **`torchci/clickhouse_queries/queued_jobs_aggregate/params.json`**
- Declared four typed parameters: `queuedThresholdMinutes`,
`maxAgeDays`, `orgs`, `repo`
- Added `defaults` block preserving the original hardcoded values (30
min, 3 days, pytorch orgs, all repos)
  - Added a test case with zeroed-out / empty values
- **`torchci/lib/clickhouse.ts`**
- `queryClickhouseSaved` now reads `defaults` from `params.json` and
falls back to them when `inputParams` does not provide a value (via
`inputParams[key] !== undefined ? inputParams[key] : defaults[key]`)
- Backward compatible: existing queries without a `defaults` key default
to `{}` via `?? {}`

## Notes
- This is the first query to use the `defaults` convention. Other
queries can adopt the same pattern if needed.
- The autoscaler Lambda currently calls this endpoint with no parameters
— it will automatically get the default values and see no behavior
change.
- The companion ci-infra PR
([#500](pytorch/ci-infra#500)) updates the
autoscaler to pass custom parameters when needed.

## Testing
- Verify existing behavior is preserved: call
`/api/clickhouse/queued_jobs_aggregate?parameters={}` and confirm the
response matches the previous hardcoded query results.
- Test parameter overrides: call with explicit `queuedThresholdMinutes`,
`maxAgeDays`, `orgs`, and `repo` values and verify the results reflect
the custom filters.
- Test partial overrides: call with only some parameters (e.g., just
`repo`) and confirm defaults are applied for the rest.
- Run `yarn build` in `torchci/` to verify TypeScript compilation.

Signed-off-by: Jean Schmidt <contato@jschmidt.me>

Author: jeanschmidt

torchci/clickhouse_queries/queued_jobs_aggregate/params.json

@@ -1,4 +1,22 @@
 {
-  "params": {},
-  "tests": []
-}
+  "params": {
+    "queuedThresholdMinutes": "Int64",
+    "maxAgeDays": "Int64",
+    "orgs": "Array(String)",
+    "repo": "String"
+  },
+  "defaults": {
+    "queuedThresholdMinutes": 30,
+    "maxAgeDays": 3,
+    "orgs": ["pytorch", "pytorch-labs", "meta-pytorch"],
+    "repo": ""
+  },
+  "tests": [
+    {
+      "queuedThresholdMinutes": 0,
+      "maxAgeDays": 0,
+      "orgs": [],
+      "repo": ""
+    }
+  ]
+}

torchci/clickhouse_queries/queued_jobs_aggregate/query.sql

@@ -14,12 +14,12 @@ WITH possible_queued_jobs AS (
         status = 'queued'
         AND created_at < (
         -- Only consider jobs that have been queued for a significant period of time
-            CURRENT_TIMESTAMP() - INTERVAL 30 MINUTE
+            CURRENT_TIMESTAMP() - toIntervalMinute({queuedThresholdMinutes: Int64})
         )
         AND created_at > (
         -- Queued jobs are automatically cancelled after this long. Any allegedly pending
         -- jobs older than this are actually bad data
-            CURRENT_TIMESTAMP() - INTERVAL 3 DAY
+            CURRENT_TIMESTAMP() - toIntervalDay({maxAgeDays: Int64})
         )
 ),
 
@@ -49,9 +49,8 @@ queued_jobs AS (
     WHERE
         job.id IN (SELECT id FROM possible_queued_jobs)
         AND workflow.id IN (SELECT run_id FROM possible_queued_jobs)
-        AND workflow.repository.owner.login IN (
-            'pytorch', 'pytorch-labs', 'meta-pytorch'
-        )
+        AND workflow.repository.owner.login IN {orgs: Array(String)}
+        AND ({repo: String} = '' OR workflow.repository.name = {repo: String})
         AND job.status = 'queued'
         /* These two conditions are workarounds for GitHub's broken API. Sometimes */
         /* jobs get stuck in a permanently "queued" state but definitely ran. We can */

torchci/lib/clickhouse.ts

@@ -84,14 +84,15 @@ export async function queryClickhouseSaved(
     `${process.cwd()}/clickhouse_queries/${queryName}/query.sql`,
     "utf8"
   );
-  let paramsText =
-    require(`clickhouse_queries/${queryName}/params.json`).params;
-  if (paramsText === undefined) {
-    paramsText = {};
-  }
+  const paramsJson = require(`clickhouse_queries/${queryName}/params.json`);
+  const paramsText = paramsJson.params ?? {};
+  const defaults: Record<string, unknown> = paramsJson.defaults ?? {};
 
   const queryParams = new Map(
-    Object.entries(paramsText).map(([key, _]) => [key, inputParams[key]])
+    Object.entries(paramsText).map(([key, _]) => [
+      key,
+      inputParams[key] !== undefined ? inputParams[key] : defaults[key],
+    ])
   );
   return await thisModule.queryClickhouse(
     query,