Scheduled notifications improvements, fix scheduled_for

[Scott-S]

User Story - Business Need

• Ticket is understood, and QA has been contacted (if the ticket has a QA label).

User Story(ies)

As a API client using the scheduled notifications feature
I want to schedule notifications using UTC timestamps that are stored and returned exactly as provided
So that notifications are sent at the time I specify without unexpected timezone conversions causing 4-8 hour delays

Additional Info and Resources

GitHub Spike Issue: department-of-veterans-affairs/notification-api#1439 - Review TIME for scheduled_for and determine how to improve it

QA Observed Behavior (from tests here) / Current Issue:

• User sent: "2023-09-06 17:00" (without timezone this is ambiguous, which led to the error)
• What actually happened:
  • POST response returned: "scheduled_for": "2023-09-06 17:00" (echoed input back)
  • Actual notification delivery: 21:15 UTC (3:15 PM MDT) - 4 hours late
  • GET response showed: "scheduled_for": "2023-09-07T01:00:00.000000Z" (next day at 1:00 AM UTC = 7:00 PM MDT) - 8 hours late
• Why this happened:
  • Bug 1: Code treated "17:00" as 17:00 EDT and converted to 21:00 UTC (+4 hours)
  • Bug 2: Code treated DB value 21:00 UTC as 21:00 EDT and converted to 01:00 UTC next day (+4 more hours)

Test Created:
tests/app/dao/notification_dao/test_notification_dao.py::test_persist_scheduled_notification_stores_utc_without_conversion

This test currently fails and exposes both bugs with clear output showing the 4-hour and 8-hour discrepancies.

WIP Branch:
A branch with the failing test has been created to demonstrate the bug. This branch should be used as the starting point for implementing the fix.

• Branch name: https://github.com/department-of-veterans-affairs/notification-api/tree/2680-scheduled-notifications-improvements
• Branch includes: Failing test that exposes both Bug 1 and Bug 2
• Test output shows: Exact 4-hour and 8-hour discrepancies matching QA observations
• Start work from this branch to ensure the fix makes the test pass

Proposed Solution Approach:

Implement the UTC Everywhere pattern, and potentially require ISO 8601 with explicit timezone:

1. Store UTC in database (DB already does this correctly)
2. Work in UTC internally (remove incorrect timezone conversions)
3. Accept (possibly convert) ISO 8601 at API boundary (client can send any timezone, as long as it's a valid ISO 8601 with an explicit timezone, and we'll convert to UTC).
   Note, if we don't require the explicit timezone, then we'll need to infer a timezone... which trade-off do we want?

API Contract:

• Input: Client MUST send ISO 8601 format with timezone indicator
  • Valid examples:
    • 2023-09-06T17:00:00Z - UTC using Z notation
    • 2023-09-06T17:00:00+00:00 - UTC using explicit offset
    • 2023-09-06T13:00:00-04:00 - East Coast in summer (EDT, automatically UTC-4)
  • Invalid? (likely a breaking change): 2023-09-06T17:00:00 (missing timezone is ambiguous.. other option is we guess.)
  • Invalid: 2023-09-06 17:00 (not ISO 8601, no timezone)
• Processing: Parse timezone from input, convert to UTC, store as UTC
• Storage: Store exactly as UTC (no timezone info in DB)
• Output: Return UTC timestamps in ISO 8601 format with 'Z' suffix (e.g., 2023-09-06T17:00:00.000000Z)
• Format consistency: Input and output both use ISO 8601 with explicit UTC

Example:

• Client sends: "2023-09-06T13:00:00-04:00" (1 PM EDT)
• We convert to UTC and store: 2023-09-06 17:00:00 (UTC, no timezone info)
  • We store like this because scheduled_for is a DateTime object in the db.
• We return: "2023-09-06T17:00:00.000000Z" (ISO 8601 UTC)
• Notification sends at: 17:00 UTC exactly

Acceptance Criteria

• scheduled_for accepts ISO 8601 format with timezone indicator (e.g., 2023-09-06T17:00:00Z, 2023-09-06T13:00:00-04:00)
• Reject timestamps without timezone info (e.g., reject 2023-09-06T17:00:00 or 2023-09-06 17:00)
• Input timezone is parsed and converted to UTC for storage
• scheduled_for value is stored in DB as UTC (no timezone info)
• GET responses return scheduled_for in ISO 8601 UTC format with Z suffix (e.g., 2023-09-06T17:00:00.000000Z)
• Notifications are sent at the exact UTC time (no 4-hour delay)
• The failing test test_persist_scheduled_notification_stores_utc_without_conversion passes
• Existing tests for scheduled notifications are updated for new ISO 8601 format
• Update API documentation (if we have API documentation?) and code comment: "scheduled_for must be ISO 8601 format with timezone (e.g., 2023-09-06T17:00:00Z for UTC or 2023-09-06T13:00:00-04:00 for EDT)"

QA Considerations

Potential Dependencies

---

[npmartin-oddball]

Take into account when working in ENP. Ultimately, not strictly following implementation.

[npmartin-oddball]

Documentation in Portal will need to be updated. DO NOT DO before feature is launched.