Skip to content

Cloudwatch log query sorting is the wrong way around #9818

@StuartBertram

Description

@StuartBertram

I looks like the Cloudwatch log query sorting is the wrong way round, and has been since it was implemented.

The query obviously doesn't implement everything, and only does timestamp sorting regardless of which field is passed. But when running a query with sort ... asc then the results come back newest first and with sort ... desc then they come back oldest first.

from uuid import uuid7

import boto3
import pytest
from moto import mock_aws
from moto.core import DEFAULT_ACCOUNT_ID
from moto.logs import logs_backends


@mock_aws
@pytest.mark.parametrize(
    "order, expected_order_func", [("asc", list), ("desc", reversed)]
)
def test_moto_cloudwatch_sort_order(order, expected_order_func):
    cloudwatch = boto3.client("logs")
    log_group = f"test-log-group-{uuid7()}"
    cloudwatch.create_log_group(logGroupName=log_group)
    log_stream = f"test-log-stream-{uuid7()}"
    cloudwatch.create_log_stream(logGroupName=log_group, logStreamName=log_stream)
    # Avoid the "must be recent" filtering and poke it straight into the Moto backend
    logs_backends[DEFAULT_ACCOUNT_ID]["eu-west-1"].groups[log_group].put_log_events(
        log_stream,
        [{"timestamp": i * 1000, "message": f"Message {i}"} for i in range(5)],
    )
    response = cloudwatch.start_query(
        logGroupName=log_group,
        startTime=0,
        endTime=1000,
        queryString=f"fields @timestamp, @message | sort @timestamp {order}",
    )
    results = cloudwatch.get_query_results(queryId=response["queryId"])
    assert results["results"] == [
        [
            {"field": "@ptr", "value": str(i)},
            {"field": "@timestamp", "value": str(i * 1000)},
            {"field": "@message", "value": f"Message {i}"},
        ]
        for i in expected_order_func(range(5))
    ]

This appears to be because sort_reversed() is true iff the sort order is asc and that value is passed to the reversed parameter in sorted(), which does a descending sort when reversed.

The current unit tests seem to confirm this, because the events are put into a list in ascending order (event_1 from 1 second ago is oldest and comes first first) but the default query (with descending order) asserts that they're in their insertion order and the simplified query (with ascending order) expects them in reversed order.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions