Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions packages/cli/src/commands/checks/list.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,9 @@ export default class ChecksList extends AuthCommand {
const { page, limit } = flags

try {
// All filtering is server-side — single paginated API call
const [paginated, statuses] = await Promise.all([
// Paginated + filtered for the table; full account fetch drives the
// account-wide summary bar, which never reacts to filters.
const [paginated, allChecks, statuses] = await Promise.all([
api.checks.getAllPaginated({
limit,
page,
Expand All @@ -71,6 +72,7 @@ export default class ChecksList extends AuthCommand {
search: flags.search,
status: flags.status,
}),
api.checks.fetchAll(),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we fetch all, why also fetching paginated? That looks like quite a waste. I get we're skipping the pagination logic that'd need to be replicated locally, but also 😩
Idk, that feels a bit much.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I need to dig a bit deeper tomorrow

Basically; the status bar requires data that is not efficiently returned via our public api, i'll check tomorrow if there's an easy workaround, extending the public api or simplifying the status bar further

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Skip full-account fetch for non-terminal list output

checks list now always calls api.checks.fetchAll() before branching on --output, but JSON/Markdown paths never render the summary bar that needs account-wide data. On large accounts this turns a single paginated request into a full-account scan (many extra API calls), which can noticeably slow scripted --output json usage and increase failure risk from rate limits/timeouts even though the returned payload is unchanged.

Useful? React with 👍 / 👎.

api.checkStatuses.fetchAll().catch(() => []),
])

Expand Down Expand Up @@ -112,7 +114,7 @@ export default class ChecksList extends AuthCommand {
// Table output
const output: string[] = []

output.push(formatSummaryBar(statuses, totalChecks))
output.push(formatSummaryBar(allChecks, statuses))
output.push('')

if (checks.length === 0) {
Expand Down
25 changes: 15 additions & 10 deletions packages/cli/src/commands/checks/stats.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ import { AuthCommand } from '../authCommand'
import { outputFlag } from '../../helpers/flags'
import * as api from '../../rest/api'
import { batchQuickRangeValues, type BatchQuickRange } from '../../rest/batch-analytics'
import type { Check } from '../../rest/checks'
import type { CheckStatus } from '../../rest/check-statuses'
import type { CheckWithStatus, PaginationInfo } from '../../formatters/checks'
import { formatSummaryBar, formatPaginationInfo } from '../../formatters/checks'
import type { OutputFormat } from '../../formatters/render'
Expand Down Expand Up @@ -72,32 +74,39 @@ export default class ChecksStats extends AuthCommand {

let checksWithStatus: CheckWithStatus[]
let totalChecks: number
let allChecks: Check[]
let allStatuses: CheckStatus[]

if (explicitIds.length > 0) {
// Fetch all checks (paginate through all pages), filter to requested IDs
const [allChecks, statuses] = await Promise.all([
;[allChecks, allStatuses] = await Promise.all([
api.checks.fetchAll(),
api.checkStatuses.fetchAll().catch(() => []),
])
const statusMap = new Map(statuses.map(s => [s.checkId, s]))
const statusMap = new Map(allStatuses.map(s => [s.checkId, s]))
const idSet = new Set(explicitIds)
checksWithStatus = allChecks
.filter(c => idSet.has(c.id))
.map(c => ({ ...c, status: statusMap.get(c.id) }))
totalChecks = checksWithStatus.length
} else {
// Paginated fetch with filters
const [paginated, statuses] = await Promise.all([
// Paginated fetch with filters for the table; account-wide fetch drives
// the summary bar, which doesn't react to filters.
const paginatedResult = await Promise.all([
api.checks.getAllPaginated({
limit,
page,
tag: flags.tag,
checkType: flags.type,
search: flags.search,
}),
api.checks.fetchAll(),
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid unconditional full check fetch in stats pagination path

In the non-explicit-ID path, checks stats now always does api.checks.fetchAll() even when --output json or --output md is used, but those formats never print the terminal summary bar. This introduces unnecessary full-account pagination for common machine-readable workflows, increasing runtime and API load and making otherwise valid filtered stats calls more likely to fail under large-account or rate-limited conditions.

Useful? React with 👍 / 👎.

api.checkStatuses.fetchAll().catch(() => []),
])
const statusMap = new Map(statuses.map(s => [s.checkId, s]))
const paginated = paginatedResult[0]
allChecks = paginatedResult[1]
allStatuses = paginatedResult[2]
const statusMap = new Map(allStatuses.map(s => [s.checkId, s]))
checksWithStatus = paginated.checks.map(c => ({ ...c, status: statusMap.get(c.id) }))
totalChecks = paginated.total
}
Expand Down Expand Up @@ -161,11 +170,7 @@ export default class ChecksStats extends AuthCommand {

// Terminal output
const output: string[] = []
const statuses = checksWithStatus
.map(c => c.status)
.filter((s): s is NonNullable<typeof s> => s != null)
const activeCheckIds = new Set(checksWithStatus.map(c => c.id))
output.push(formatSummaryBar(statuses, totalChecks, activeCheckIds))
output.push(formatSummaryBar(allChecks, allStatuses))
output.push('')
output.push(formatBatchStats(rows, range, fmt))
output.push('')
Expand Down
65 changes: 45 additions & 20 deletions packages/cli/src/formatters/__tests__/checks.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@
formatCheckDetail,
formatResults,
formatErrorGroups,
getActivatedStatuses,
} from '../checks'
import { scenario } from '../../rest/__tests__/__fixtures__/api'
import {
passingCheck,
failingCheck,
Expand All @@ -23,10 +25,10 @@
checkWithLowSsl,
checkWithMediumSsl,
checkWithNoSsl,
passingStatus,

Check warning on line 28 in packages/cli/src/formatters/__tests__/checks.spec.ts

View workflow job for this annotation

GitHub Actions / lint

'passingStatus' is defined but never used
failingStatus,

Check warning on line 29 in packages/cli/src/formatters/__tests__/checks.spec.ts

View workflow job for this annotation

GitHub Actions / lint

'failingStatus' is defined but never used
degradedStatus,

Check warning on line 30 in packages/cli/src/formatters/__tests__/checks.spec.ts

View workflow job for this annotation

GitHub Actions / lint

'degradedStatus' is defined but never used
errorStatus,

Check warning on line 31 in packages/cli/src/formatters/__tests__/checks.spec.ts

View workflow job for this annotation

GitHub Actions / lint

'errorStatus' is defined but never used
apiCheckResult,
browserCheckResult,
activeErrorGroup,
Expand All @@ -45,30 +47,53 @@
vi.useRealTimers()
})

describe('formatSummaryBar', () => {
it('shows counts for passing, degraded, and failing', () => {
const statuses = [passingStatus, failingStatus, degradedStatus]
const result = stripAnsi(formatSummaryBar(statuses, 10))
expect(result).toContain('1 passing')
expect(result).toContain('1 degraded')
expect(result).toContain('1 failing')
expect(result).toContain('10 total checks')
describe('getActivatedStatuses', () => {
it('excludes statuses whose check is deactivated', () => {
const { checks, statuses } = scenario('mixed')
const deactivatedIds = new Set(checks.filter(c => !c.activated).map(c => c.id))
const result = getActivatedStatuses(checks, statuses)
// mixed: 16 statuses (13 active + 3 stale on deactivated). Expected: 13.
expect(result).toHaveLength(13)
expect(result.every(s => !deactivatedIds.has(s.checkId))).toBe(true)
})

it('counts hasErrors status as failing', () => {
const statuses = [passingStatus, errorStatus]
const result = stripAnsi(formatSummaryBar(statuses, 2))
expect(result).toContain('1 passing')
expect(result).toContain('1 failing')
it('returns empty for an all-deactivated account', () => {
const { checks, statuses } = scenario('all-deactivated')
expect(getActivatedStatuses(checks, statuses)).toEqual([])
})
})

it('filters by activeCheckIds when provided', () => {
const statuses = [passingStatus, failingStatus, degradedStatus]
const activeIds = new Set(['check-1'])
const result = stripAnsi(formatSummaryBar(statuses, 3, activeIds))
expect(result).toContain('1 passing')
expect(result).not.toContain('failing')
expect(result).not.toContain('degraded')
describe('formatSummaryBar', () => {
it('renders account-wide 4-bucket bar with "Account wide:" prefix and no total label', () => {
const { checks, statuses } = scenario('mixed')
const result = stripAnsi(formatSummaryBar(checks, statuses))
// mixed: 8 passing, 2 degraded, 3 failing, 1 no-status (not counted),
// 5 deactivated -> all in inactive bucket.
expect(result).toContain('Account wide:')
expect(result).toContain('8 passing')
expect(result).toContain('2 degraded')
expect(result).toContain('3 failing')
expect(result).toContain('5 inactive')
expect(result).not.toContain('total checks')
})

it('does not count stale passing statuses on deactivated checks as passing', () => {
// Regression: 3 of the 5 deactivated checks carry stale hasFailures=false statuses.
// Before the fix they leaked into the passing count. With all-deactivated only the
// inactive bucket should remain.
const { checks, statuses } = scenario('all-deactivated')
expect(stripAnsi(formatSummaryBar(checks, statuses))).toBe('Account wide: ⊘ 5 inactive')
})

it('hides zero-count buckets', () => {
const { checks, statuses } = scenario('all-passing')
const result = stripAnsi(formatSummaryBar(checks, statuses))
expect(result).toContain('6 passing')
expect(result).not.toMatch(/degraded|failing|inactive/)
})

it('returns an empty string when the account has no checks', () => {
expect(formatSummaryBar([], [])).toBe('')
})
})

Expand Down
25 changes: 14 additions & 11 deletions packages/cli/src/formatters/checks.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,23 +55,26 @@ function boolSymbol (value: boolean, format: OutputFormat): string {

// --- Summary bar (terminal only) ---

export function formatSummaryBar (statuses: CheckStatus[], totalChecks: number, activeCheckIds?: Set<string>): string {
const counted = activeCheckIds
? statuses.filter(s => activeCheckIds.has(s.checkId))
: statuses
const passing = counted.filter(s => !s.hasFailures && !s.hasErrors && !s.isDegraded).length
const degraded = counted.filter(s => s.isDegraded && !s.hasFailures && !s.hasErrors).length
const failing = counted.filter(s => s.hasFailures || s.hasErrors).length
export function getActivatedStatuses (checks: Check[], statuses: CheckStatus[]): CheckStatus[] {
const activated = new Set(checks.filter(c => c.activated).map(c => c.id))
return statuses.filter(s => activated.has(s.checkId))
}

export function formatSummaryBar (checks: Check[], statuses: CheckStatus[]): string {
const activated = getActivatedStatuses(checks, statuses)
const passing = activated.filter(s => !s.hasFailures && !s.hasErrors && !s.isDegraded).length
const degraded = activated.filter(s => s.isDegraded && !s.hasFailures && !s.hasErrors).length
const failing = activated.filter(s => s.hasFailures || s.hasErrors).length
const inactive = checks.filter(c => !c.activated).length

const parts: string[] = []
if (passing > 0) parts.push(chalk.green(`${logSymbols.success} ${passing} passing`))
if (degraded > 0) parts.push(chalk.yellow(`${logSymbols.warning} ${degraded} degraded`))
if (failing > 0) parts.push(chalk.red(`${logSymbols.error} ${failing} failing`))
if (inactive > 0) parts.push(chalk.dim(`⊘ ${inactive} inactive`))

const displayTotal = activeCheckIds ? activeCheckIds.size : totalChecks
const total = chalk.dim(`(${displayTotal} total checks)`)
if (parts.length === 0) return total
return parts.join(' ') + ' ' + total
if (parts.length === 0) return ''
return chalk.dim('Account wide:') + ' ' + parts.join(' ')
}

// --- Type breakdown (terminal only) ---
Expand Down
87 changes: 87 additions & 0 deletions packages/cli/src/rest/__tests__/__fixtures__/api/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# API response fixtures

Sanitized samples of `/v1/checks` and `/v1/check-statuses` responses, drawn
from a real Checkly production account and scrubbed of account-specific data.
Use via the typed helpers in `./index.ts`:

```ts
import { scenario } from './'

const { checks, statuses } = scenario('all-deactivated')
```

## Contents

- `checks.json` — 19 checks covering a mix of active/deactivated states across
API, BROWSER, HEARTBEAT, MULTI_STEP, PLAYWRIGHT, AGENTIC check types.
- `check-statuses.json` — 16 statuses (3 of the 19 checks legitimately lack a
status entry — they're either freshly created or deactivated without a run).

## Scenarios (via `scenario(name)`)

| Name | Description |
| --------------------- | ------------------------------------------------------ |
| `mixed` | Full 19-check fixture. The canonical dataset. |
| `all-passing` | 6 active checks, all passing. Healthy-account baseline.|
| `all-deactivated` | 5 deactivated checks (3 with stale statuses). |
| `active-no-statuses` | 1 active check with no status entry. New-check case. |
| `empty` | `{ checks: [], statuses: [] }`. |

For edge cases not represented in real data (pure `hasErrors`,
deactivated+failing), mutate the returned fixture in the test that needs it.

## Sanitization

The following were remapped or stripped:

- `id` → `check-NNN` (sequential)
- `name` → `"Check NNN"`
- `description` → `null`
- `request.url` → `https://example.com/check-NNN`
- `tags`, `privateLocations` → sequential synthetic names
- `groupId` → sequential small integers
- `created_at` / `updated_at` → synthetic dates starting `2025-01-01`
- `longestRun` / `shortestRun` → rounded to nearest 100ms
- `script`, `environmentVariables`, `alertChannelSubscriptions`,
`alertSettings`, `localSetupScript`, `localTearDownScript`,
`setupSnippetId`, `tearDownSnippetId`, `sslCheckDomain` → dropped entirely

Retained as-is (not sensitive):
- `checkType`, `activated`, `muted`
- `frequency`, `frequencyOffset`
- AWS region codes in `locations`
- Public `runtimeId` version strings
- `heartbeat` config on HEARTBEAT checks
- Status booleans (`hasFailures`, `hasErrors`, `isDegraded`) and
`sslDaysRemaining`

## Empirical API quirks verified at capture time

Documented here because they materially affect how the CLI should treat the
data — they're not obvious from the type declarations.

1. **`/v1/check-statuses` ignores `limit` and `page`** query parameters. It
always returns all statuses that exist for the account, in one response.
The `content-range` header can be misleading if those params are passed.

2. **`/v1/check-statuses` includes statuses for deactivated checks.** The
webapp filters these out client-side when rendering its summary bar; the
CLI must do the same. This fixture reproduces that: `check-015`, `-016`,
and `-017` are deactivated but appear in `check-statuses.json`.

3. **Not every check has a status entry.** Brand-new checks that have never
run are absent from the status response (e.g. `check-013` here).
Deactivated checks that never ran are also absent.

4. **`/v1/checks?status=X` server filters already exclude deactivated checks**
— confirmed empirically. No client-side `activated` check is needed when
applying `status=` to the checks endpoint; only the summary-bar needs
client-side activated filtering, because the status endpoint isn't
filter-aware.

5. **Per-type shape varies.** HEARTBEAT checks have no `frequency`,
`locations`, `privateLocations`, or `groupId`; instead they carry a
`heartbeat` object with `period`/`periodUnit`/`grace`/`graceUnit`. Other
types carry `frequency` and `locations`. The declared `Check` interface in
`packages/cli/src/rest/checks.ts` does not currently reflect this
divergence — something to be aware of if tests run into missing fields.
Loading
Loading