Skip to content

fix: use timezone-aware UTC for the default cloudwatch end_time - #3438

Open
hsusul wants to merge 1 commit into
aws:mainfrom
hsusul:fix/cloudwatch-naive-utc-end-time
Open

fix: use timezone-aware UTC for the default cloudwatch end_time#3438
hsusul wants to merge 1 commit into
aws:mainfrom
hsusul:fix/cloudwatch-naive-utc-end-time

Conversation

@hsusul

@hsusul hsusul commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Feature or Bugfix

  • Bugfix

Detail

wr.cloudwatch.start_query (and therefore run_query / read_logs) computes its default
end_time with datetime.datetime.utcnow(), which returns a naive datetime. The very
next line calls .timestamp() on it, and datetime.timestamp() interprets a naive datetime
as local time. The resulting epoch value is therefore shifted by the machine's UTC offset.

The default start_time right above it already uses an aware value
(datetime.datetime(1970, 1, 1, tzinfo=datetime.timezone.utc)), so the two boundaries of the
same query window are converted under two different assumptions.

awswrangler/cloudwatch.py (before):

start_time = (
    start_time if start_time else datetime.datetime(year=1970, month=1, day=1, tzinfo=datetime.timezone.utc)
)
end_time = end_time if end_time else datetime.datetime.utcnow()   # naive

start_timestamp: int = int(1000 * start_time.timestamp())
end_timestamp: int = int(1000 * end_time.timestamp())

Reproduction (no AWS credentials needed)

import datetime, os, time
from unittest.mock import MagicMock, patch

os.environ["TZ"] = "Asia/Tokyo"
time.tzset()

import awswrangler as wr

logs = MagicMock()
logs.start_query.return_value = {"queryId": "q1"}

with patch("awswrangler._utils.client", return_value=logs):
    wr.cloudwatch.start_query(query="fields @timestamp", log_group_names=["lg"])

sent = logs.start_query.call_args.kwargs["endTime"]
now = int(datetime.datetime.now(datetime.timezone.utc).timestamp() * 1000)
print("skew (hours):", (sent - now) / 3_600_000)   # -9.0

Current vs corrected behavior

current corrected
endTime sent to CloudWatch Logs on a UTC+9 host 9 hours in the past current UTC instant
endTime sent on a UTC host correct correct
start_time=now-5min with default end_time on a UTC+9 host raises InvalidArgumentCombination: start_time must be inferior to end_time. query starts normally

So on any host whose local timezone is ahead of UTC, wr.cloudwatch.read_logs(...) without an
explicit end_time silently drops the most recent offset-worth of log events, and combining a
recent start_time with the default end_time fails outright. On hosts behind UTC the window is
extended into the future instead.

Implementation

One line in awswrangler/cloudwatch.py:

-    end_time = end_time if end_time else datetime.datetime.utcnow()
+    end_time = end_time if end_time else datetime.datetime.now(tz=datetime.timezone.utc)

This also removes the last use of datetime.datetime.utcnow() in awswrangler/, which is
deprecated since Python 3.12.

No public API, signature, default, or return type changes. Explicitly supplied start_time /
end_time values are converted exactly as before.

Regression tests

Added to tests/unit/test_moto.py (the module the Minimal Tests workflow runs), plus a
local_timezone fixture that sets TZ / time.tzset() and restores the previous value
afterwards. It skips where time.tzset() is unavailable (Windows) or where the requested zone
is missing from the timezone database.

  • test_cloudwatch_start_query_default_end_time_is_utc_now — the endTime sent to
    logs:StartQuery is the current UTC instant under Asia/Tokyo and America/New_York, and
    the default startTime is still 0.
  • test_cloudwatch_start_query_recent_start_time_with_default_end_time — a start_time five
    minutes in the past no longer raises against the default end_time.
  • test_cloudwatch_start_query_explicit_times_are_preserved — explicit aware boundaries are
    still converted straight to epoch milliseconds (no behavior change).

Against the unpatched module these fail as expected: both parametrizations of the first test, and
the Asia/Tokyo case of the second (the America/New_York case correctly passes, since a
timezone behind UTC moves the default end_time into the future rather than the past).

Validation

Run locally on macOS 15.6 / Python 3.13.5, AWS_DEFAULT_REGION=us-east-1:

check command result
new tests, unpatched source pytest tests/unit/test_moto.py -k cloudwatch 3 failed, 2 passed (expected)
new tests, patched source pytest tests/unit/test_moto.py -k cloudwatch 5 passed
moto suite (as in CI) pytest -n 4 tests/unit/test_moto.py 51 passed
CI unit modules pytest tests/unit/test_moto.py tests/unit/test_utils.py tests/unit/test_session.py tests/unit/test_metadata.py 71 passed
format ruff format --check . 280 files already formatted
lint ruff check . All checks passed
types mypy awswrangler 19 errors, byte-identical to the same command on unpatched upstream/main (pre-existing, all in unrelated modules)
docs doc8 --max-line-length 120 docs/source clean
lockfile uv lock --check resolved, unchanged
build uv build --wheel awswrangler-3.17.1-py3-none-any.whl
whitespace git diff --check clean

Not run: the AWS integration tests (tests/unit/test_cloudwatch.py and the rest of the
credential-dependent suites), which need a live account and the CDK test infrastructure. The
behavior changed here is the local timestamp conversion, which the added tests cover without
any AWS call.

Relates

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant