Skip to content

RHINENG-23942 - Add support for expiration_date to remediations endpoints in the API backend#1005

Open
brantleyr wants to merge 5 commits intomasterfrom
RHINENG-23942-retention-policy-expiration-date
Open

RHINENG-23942 - Add support for expiration_date to remediations endpoints in the API backend#1005
brantleyr wants to merge 5 commits intomasterfrom
RHINENG-23942-retention-policy-expiration-date

Conversation

@brantleyr
Copy link
Collaborator

@brantleyr brantleyr commented Mar 4, 2026

Summary by Sourcery

Add expiration_date support to remediation plans, including database schema, API contracts, and defaulting behavior, and update configuration, seed data, and tests accordingly.

New Features:

  • Expose expiration_date as a required field on remediation detail and list APIs, with optional creation and update via request body and defaulting based on a configurable retention policy.
  • Support filtering and sorting remediation listings by expiration_date using before/after and expiring_within_days filters.

Enhancements:

  • Add expiration_date column with indexed backfill migration for existing remediations, and configure retention and migration batch size via environment-driven config.
  • Include expiration_date formatting and propagation through remediation models, formatters, and integration/unit tests.
  • Adjust seeders to generate non-expired expiration dates for test and demo remediations using shared helpers.

Build:

  • Extend deployment configuration and parameters to surface PLAN_RETENTION_DAYS and MIGRATION_BACKFILL_BATCH_SIZE environment variables.

Tests:

  • Add and update unit and integration tests plus snapshots to cover expiration_date sorting, filtering, serialization, validation, and empty PATCH handling.

@sourcery-ai
Copy link
Contributor

sourcery-ai bot commented Mar 4, 2026

Reviewer's Guide

Adds expiration_date as a first-class field on remediations, including DB schema/migration, model, config, seed data, API contract, query filtering/sorting, write semantics, and comprehensive tests around the new behavior.

Sequence diagram for create and patch remediations with expiration_date

sequenceDiagram
    actor User
    participant APIBackend
    participant ControllerWrite
    participant Remediation
    participant Config

    %% Create remediation with explicit expiration_date
    User->>APIBackend: POST /v1/remediations { name, auto_reboot, expiration_date }
    APIBackend->>ControllerWrite: create(req, res)
    ControllerWrite->>ControllerWrite: parseExpirationDate(bodyExpirationDate)
    ControllerWrite->>Remediation: create({ name, auto_reboot, expiration_date, ... })
    Remediation-->>ControllerWrite: persisted remediation
    ControllerWrite-->>APIBackend: 201 { id }
    APIBackend-->>User: response

    %% Create remediation without expiration_date (default from retention policy)
    User->>APIBackend: POST /v1/remediations { name, auto_reboot }
    APIBackend->>ControllerWrite: create(req, res)
    ControllerWrite->>Config: read planRetentionDays
    Config-->>ControllerWrite: days
    ControllerWrite->>ControllerWrite: defaultExpirationDate()
    ControllerWrite->>Remediation: create({ name, auto_reboot, expiration_date=now+days, ... })
    Remediation-->>ControllerWrite: persisted remediation
    ControllerWrite-->>APIBackend: 201 { id }
    APIBackend-->>User: response

    %% Patch remediation updating expiration_date
    User->>APIBackend: PATCH /v1/remediations/:id { expiration_date }
    APIBackend->>ControllerWrite: patch(req, res)
    ControllerWrite->>Remediation: findByPk(id)
    Remediation-->>ControllerWrite: remediation instance
    ControllerWrite->>ControllerWrite: parseExpirationDate(bodyExpirationDate)
    ControllerWrite->>Remediation: set expiration_date
    ControllerWrite->>Remediation: save()
    Remediation-->>ControllerWrite: updated remediation
    ControllerWrite-->>APIBackend: 200 { id }
    APIBackend-->>User: response
Loading

ER diagram for remediations table with expiration_date

erDiagram
    REMEDIATIONS {
        uuid id PK
        string name
        boolean auto_reboot
        boolean needs_reboot
        boolean archived
        string account_number
        string tenant_org_id
        string created_by
        string updated_by
        string workspace_id
        date expiration_date
        timestamp created_at
        timestamp updated_at
    }

    REMEDIATION_ISSUES {
        uuid id PK
        uuid remediation_id FK
        string issue_id
        string resolution
        integer precedence
    }

    REMEDIATIONS ||--o{ REMEDIATION_ISSUES : has
Loading

Class diagram for remediations expiration_date support

classDiagram
    class Config {
        +number planRetentionDays
        +number migrationBackfillBatchSize
    }

    class DbHelpers {
        +string getDefaultExpirationDate()
    }

    class RemediationModel {
        +UUID id
        +string name
        +boolean auto_reboot
        +boolean needs_reboot
        +boolean archived
        +string account_number
        +string tenant_org_id
        +string created_by
        +string updated_by
        +string workspace_id
        +Date expiration_date
        +Date created_at
        +Date updated_at
        +Date defaultValueExpirationDate()
    }

    class RemediationsControllerWrite {
        +Date defaultExpirationDate()
        +Date parseExpirationDate(value)
        +create(req, res)
        +patch(req, res)
    }

    class RemediationsQueries {
        +list(tenant_org_id, created_by, system, primaryOrder, asc, filter, detailed, limit, offset)
    }

    class MigrationAddExpirationDateToRemediations {
        +up(q, DataTypes)
        +down(q)
    }

    Config --> RemediationModel : uses planRetentionDays
    Config --> RemediationsControllerWrite : uses planRetentionDays
    Config --> MigrationAddExpirationDateToRemediations : uses planRetentionDays
    Config --> MigrationAddExpirationDateToRemediations : uses migrationBackfillBatchSize

    DbHelpers --> Config : uses
    DbHelpers --> RemediationsSeeders : used by

    RemediationsControllerWrite --> RemediationModel : create/patch instances
    RemediationsControllerWrite --> Config : uses planRetentionDays

    RemediationsQueries --> RemediationModel : reads list with expiration_date
    MigrationAddExpirationDateToRemediations --> RemediationModel : schema change

    class RemediationsSeeders {
        +up(q)
    }

    RemediationsSeeders --> DbHelpers : calls getDefaultExpirationDate
Loading

File-Level Changes

Change Details Files
Introduce expiration_date column on remediations with migration-time backfill and indexing, wired to configurable retention days and batch size.
  • Add Sequelize migration to create nullable expiration_date, backfill in batches using created_at + planRetentionDays, then enforce NOT NULL and add an index
  • Extend remediation model with non-null DATE expiration_date and defaultValue based on config.planRetentionDays
  • Add migrationBackfillBatchSize and planRetentionDays configuration + env wiring in config and clowdapp templates
db/migrations/20251201120000-add-expiration-date-to-remediations.js
src/remediations/models/remediation.js
src/config/index.js
deployment/clowdapp.yml
Expose expiration_date through the public API for list/detail responses and allow sorting and filtering by expiration-related criteria.
  • Extend OpenAPI schemas and enums to include expiration_date on remediation list and detail responses, and allow sort=±expiration_date
  • Document new filters expiration_before, expiration_after, and expiring_within_days with appropriate formats/patterns
  • Update list/detail response formatting to serialize expiration_date as YYYY-MM-DD and mark it as required in response schemas
src/api/openapi.yaml
src/remediations/remediations.format.js
Allow clients to set or update expiration_date when creating or patching remediations, with validation and defaults.
  • Add helpers in controller.write to compute default expiration_date from planRetentionDays and to parse/validate user-provided expiration_date
  • On create, always persist an expiration_date: either parsed from request body or derived from defaultExpirationDate
  • On patch, treat expiration_date as an updatable field, validate it when present, and update EMPTY_REQUEST error messaging/tests to include expiration_date
src/remediations/controller.write.js
src/remediations/write.integration.js
Support querying remediations by expiration_date and sorting by it at the DB layer, with unit and integration coverage.
  • Extend REMEDIATION_ATTRIBUTES so expiration_date is selected in list queries
  • Implement expiration_date filters in list(): expiration_before, expiration_after, and expiring_within_days using Sequelize Op and date math
  • Add unit tests for query construction around expiration filters/sorting and integration tests for API behavior (sorting, filters, and response payloads)
src/remediations/remediations.queries.js
src/remediations/remediations.queries.unit.js
src/remediations/read.integration.js
Update all relevant seeders and test expectations so test data has non-expired expiration_date values computed at seed time.
  • Introduce db/helpers.getDefaultExpirationDate to compute a default future expiration date based on config.planRetentionDays
  • Update all remediation-related seeders to set expiration_date via getDefaultExpirationDate and, where needed, propagate to additional test remediations
  • Adjust existing snapshot and response-shape tests to account for new expiration_date field while remaining resilient to the dynamic date value
db/helpers.js
db/seeders/20180823074644-tests-read.js
db/seeders/20181121125041-demo.js
db/seeders/20181123074644-tests-write.js
db/seeders/20190107133125-tests-read-single.js
db/seeders/20190212130752-status.js
db/seeders/20191223133315-fifi.js
db/seeders/20240326162729-scalability.js
src/remediations/read.integration.js.snap
src/remediations/write.integration.js.snap
src/config/__snapshots__/config.unit.js.snap

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 3 issues, and left some high level feedback:

  • The logic for computing default expiration dates is now duplicated in the migration, model default, controller.write (defaultExpirationDate), and db/helpers.getDefaultExpirationDate; consider centralizing this into a single helper to avoid subtle drift if the behavior needs to change later.
  • In the expiration_date backfill migration, the batched UPDATE uses a subselect with LIMIT but no ORDER BY; adding an ORDER BY (e.g., created_at or id) would make the batching deterministic and can help the planner perform more predictable, index-friendly scans.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The logic for computing default expiration dates is now duplicated in the migration, model default, controller.write (defaultExpirationDate), and db/helpers.getDefaultExpirationDate; consider centralizing this into a single helper to avoid subtle drift if the behavior needs to change later.
- In the expiration_date backfill migration, the batched UPDATE uses a subselect with LIMIT but no ORDER BY; adding an ORDER BY (e.g., created_at or id) would make the batching deterministic and can help the planner perform more predictable, index-friendly scans.

## Individual Comments

### Comment 1
<location path="src/remediations/remediations.queries.js" line_range="260-261" />
<code_context>

+        // expiration_date filters (before, after, or expiring within N days)
+        const expConditions = [];
+        if (filter.expiration_before) {
+            expConditions.push({ [Op.lte]: new Date(filter.expiration_before) });
+        }
+        if (filter.expiration_after) {
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Date-only filters using `new Date('YYYY-MM-DD')` may behave differently across timezones.

`new Date('2026-12-31')` depends on how the engine interprets date-only strings, which varies by timezone (UTC vs local). That can shift the effective filter boundary versus the DB’s `DATE`/`TIMESTAMP` column. If these are calendar date filters, consider explicitly building a UTC range (start/end of day) instead of relying on `new Date(string)` parsing.

Suggested implementation:

```javascript
const ISSUE_ATTRIBUTES = ['issue_id', 'resolution', 'precedence'];

function parseDateOnlyToUtcStart(dateStr) {
    if (typeof dateStr !== 'string') {
        return null;
    }

    const match = /^(\d{4})-(\d{2})-(\d{2})$/.exec(dateStr);
    if (!match) {
        return null;
    }

    const year = Number(match[1]);
    const month = Number(match[2]) - 1; // JS months are 0-based
    const day = Number(match[3]);

    return new Date(Date.UTC(year, month, day, 0, 0, 0, 0));
}

function parseDateOnlyToUtcEnd(dateStr) {
    if (typeof dateStr !== 'string') {
        return null;
    }

    const match = /^(\d{4})-(\d{2})-(\d{2})$/.exec(dateStr);
    if (!match) {
        return null;
    }

    const year = Number(match[1]);
    const month = Number(match[2]) - 1; // JS months are 0-based
    const day = Number(match[3]);

    return new Date(Date.UTC(year, month, day, 23, 59, 59, 999));
}

const PLAYBOOK_RUN_ATTRIBUTES = [

```

```javascript
        // expiration_date filters (before, after, or expiring within N days)
        const expConditions = [];
        if (filter.expiration_before) {
            const endOfDayUtc = parseDateOnlyToUtcEnd(filter.expiration_before);
            if (endOfDayUtc) {
                expConditions.push({ [Op.lte]: endOfDayUtc });
            }
        }
        if (filter.expiration_after) {
            const startOfDayUtc = parseDateOnlyToUtcStart(filter.expiration_after);
            if (startOfDayUtc) {
                expConditions.push({ [Op.gte]: startOfDayUtc });
            }
        }

```

```javascript
        if (filter.expiring_within_days !== undefined && filter.expiring_within_days !== '') {
            const n = parseInt(filter.expiring_within_days, 10);
            if (!Number.isNaN(n) && n >= 0) {
                const now = new Date();
                now.setUTCHours(0, 0, 0, 0);
                const end = new Date(now);
                end.setUTCDate(end.getUTCDate() + n);
                expConditions.push({ [Op.gte]: now });

```
</issue_to_address>

### Comment 2
<location path="db/migrations/20251201120000-add-expiration-date-to-remediations.js" line_range="9-10" />
<code_context>
+
+module.exports = {
+  up: async (q, {DATE}) => {
+    const retentionDays = config.planRetentionDays;
+    const batchSize = config.migrationBackfillBatchSize;
+
+    await q.addColumn('remediations', 'expiration_date', {
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Relying on application config in a migration can make behavior environment-dependent and harder to reproduce.

Because `planRetentionDays` and `migrationBackfillBatchSize` come from runtime config, rerunning this migration or running it in another environment with different env vars will change the backfilled values and batch sizes. For schema/data migrations, it’s safer to hardcode these values (or at least assert/log the expected ones) so the behavior is fixed and reproducible even if `PLAN_RETENTION_DAYS` changes later.

Suggested implementation:

```javascript
'use strict';

const config = require('../../src/config');

// Fixed values for this migration to keep behavior reproducible even if config/env changes later.
const RETENTION_DAYS = 30; // snapshot of planRetentionDays at time of writing this migration
const BACKFILL_BATCH_SIZE = 1000; // snapshot of migrationBackfillBatchSize at time of writing this migration

// Surface any drift between migration expectations and current runtime config.
if (
  config.planRetentionDays !== RETENTION_DAYS ||
  config.migrationBackfillBatchSize !== BACKFILL_BATCH_SIZE
) {
  // eslint-disable-next-line no-console
  console.warn(
    `Migration 20251201120000-add-expiration-date-to-remediations is using fixed values ` +
      `RETENTION_DAYS=${RETENTION_DAYS}, BACKFILL_BATCH_SIZE=${BACKFILL_BATCH_SIZE}, ` +
      `but runtime config is planRetentionDays=${config.planRetentionDays}, ` +
      `migrationBackfillBatchSize=${config.migrationBackfillBatchSize}.`
  );
}

const EXPIRATION_DATE_INDEX = 'remediations_expiration_date_idx';

```

```javascript
  up: async (q, {DATE}) => {
    const retentionDays = RETENTION_DAYS;
    const batchSize = BACKFILL_BATCH_SIZE;

```

1. Replace the placeholder values `RETENTION_DAYS = 30` and `BACKFILL_BATCH_SIZE = 1000` with the actual values currently used in your `config.planRetentionDays` and `config.migrationBackfillBatchSize` so this migration accurately captures the intended behavior at the time it was written.
2. If your linting rules differ (e.g., no top-level console usage), you may need to adjust or remove the `console.warn` and use your project's preferred logging mechanism instead.
</issue_to_address>

### Comment 3
<location path="db/helpers.js" line_range="11-13" />
<code_context>
+ */
+function getDefaultExpirationDate() {
+    const config = require('../src/config');
+    const d = new Date();
+    d.setDate(d.getDate() + (config.planRetentionDays ?? 270));
+    return d.toISOString().split('T')[0];
+}
+
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Returning a date string for `expiration_date` while the column is `DATE`/`TIMESTAMP` can introduce subtle type/format inconsistencies.

This helper makes seeders write `YYYY-MM-DD` strings into a `DATE` column, while other flows (model default, controller create/patch) send JS `Date` instances. That inconsistency can lead to dialect-specific casting differences and forces downstream code to handle both strings and Dates. Consider returning a `Date` and letting Sequelize serialize it so the type matches the rest of the code path.

Suggested implementation:

```javascript

```

```javascript
/**
 * Returns a default expiration_date for seed data: today + planRetentionDays.
 * Computed at seed time so remediation plans stay "not yet expired" whenever
 * tests run. Future culling jobs can test with explicitly past expiration_date
 * in dedicated seed data or fixtures.
 *
 * Returns a Date instance so Sequelize can serialize it consistently with
 * other code paths (model defaults, controllers, etc).
 */
function getDefaultExpirationDate() {
    const config = require('../src/config');
    const d = new Date();
    d.setDate(d.getDate() + (config.planRetentionDays ?? 270));
    return d;

```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +260 to +261
if (filter.expiration_before) {
expConditions.push({ [Op.lte]: new Date(filter.expiration_before) });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): Date-only filters using new Date('YYYY-MM-DD') may behave differently across timezones.

new Date('2026-12-31') depends on how the engine interprets date-only strings, which varies by timezone (UTC vs local). That can shift the effective filter boundary versus the DB’s DATE/TIMESTAMP column. If these are calendar date filters, consider explicitly building a UTC range (start/end of day) instead of relying on new Date(string) parsing.

Suggested implementation:

const ISSUE_ATTRIBUTES = ['issue_id', 'resolution', 'precedence'];

function parseDateOnlyToUtcStart(dateStr) {
    if (typeof dateStr !== 'string') {
        return null;
    }

    const match = /^(\d{4})-(\d{2})-(\d{2})$/.exec(dateStr);
    if (!match) {
        return null;
    }

    const year = Number(match[1]);
    const month = Number(match[2]) - 1; // JS months are 0-based
    const day = Number(match[3]);

    return new Date(Date.UTC(year, month, day, 0, 0, 0, 0));
}

function parseDateOnlyToUtcEnd(dateStr) {
    if (typeof dateStr !== 'string') {
        return null;
    }

    const match = /^(\d{4})-(\d{2})-(\d{2})$/.exec(dateStr);
    if (!match) {
        return null;
    }

    const year = Number(match[1]);
    const month = Number(match[2]) - 1; // JS months are 0-based
    const day = Number(match[3]);

    return new Date(Date.UTC(year, month, day, 23, 59, 59, 999));
}

const PLAYBOOK_RUN_ATTRIBUTES = [
        // expiration_date filters (before, after, or expiring within N days)
        const expConditions = [];
        if (filter.expiration_before) {
            const endOfDayUtc = parseDateOnlyToUtcEnd(filter.expiration_before);
            if (endOfDayUtc) {
                expConditions.push({ [Op.lte]: endOfDayUtc });
            }
        }
        if (filter.expiration_after) {
            const startOfDayUtc = parseDateOnlyToUtcStart(filter.expiration_after);
            if (startOfDayUtc) {
                expConditions.push({ [Op.gte]: startOfDayUtc });
            }
        }
        if (filter.expiring_within_days !== undefined && filter.expiring_within_days !== '') {
            const n = parseInt(filter.expiring_within_days, 10);
            if (!Number.isNaN(n) && n >= 0) {
                const now = new Date();
                now.setUTCHours(0, 0, 0, 0);
                const end = new Date(now);
                end.setUTCDate(end.getUTCDate() + n);
                expConditions.push({ [Op.gte]: now });

Comment on lines +11 to +13
const d = new Date();
d.setDate(d.getDate() + (config.planRetentionDays ?? 270));
return d.toISOString().split('T')[0];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): Returning a date string for expiration_date while the column is DATE/TIMESTAMP can introduce subtle type/format inconsistencies.

This helper makes seeders write YYYY-MM-DD strings into a DATE column, while other flows (model default, controller create/patch) send JS Date instances. That inconsistency can lead to dialect-specific casting differences and forces downstream code to handle both strings and Dates. Consider returning a Date and letting Sequelize serialize it so the type matches the rest of the code path.

Suggested implementation:

/**
 * Returns a default expiration_date for seed data: today + planRetentionDays.
 * Computed at seed time so remediation plans stay "not yet expired" whenever
 * tests run. Future culling jobs can test with explicitly past expiration_date
 * in dedicated seed data or fixtures.
 *
 * Returns a Date instance so Sequelize can serialize it consistently with
 * other code paths (model defaults, controllers, etc).
 */
function getDefaultExpirationDate() {
    const config = require('../src/config');
    const d = new Date();
    d.setDate(d.getDate() + (config.planRetentionDays ?? 270));
    return d;

@brantleyr brantleyr requested a review from rexwhite March 4, 2026 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants