Skip to content

logs search/aggregate silently return partial results when time range exceeds storage-tier retention #717

Description

@dgilman-perplexity

Summary

pup logs search and pup logs aggregate silently return partial results when the requested time range extends past the retention of the storage tier being queried. There is no warning, no metadata field indicating truncation, and no hint in --help that --storage is the knob to reach for.

Repro

On an org where the default (indexes) tier retains N days and Flex retains substantially more:

pup logs aggregate --query="service:some-service" --compute=count --from=30d
Returns counts consistent with only the last N days of data. Re-running the same query with --storage=flex returns the full 30-day count.

The output structure is identical in both cases — no truncated, partial, or retention_limit field on the response, and nothing on stderr.

Impact

For investigations that span more than the default tier's retention (incident forensics, audit lookbacks, long-horizon trend queries), the caller gets a plausible-looking answer that is silently incomplete. This is particularly risky for AI-agent callers, which do not know to cross-check against a second storage tier and will confidently report the truncated result as ground truth.

Suggested fixes

  1. stderr warning when --from reaches or exceeds detected retention for the selected tier, pointing at --storage.
  2. Response metadata — a retention_limited: true (or equivalent) field so programmatic callers can detect truncation.
  3. --storage=auto that picks the narrowest tier covering the requested range, and prints which tier it chose.
  4. --help cross-reference on logs search / logs aggregate noting that long lookbacks may require --storage=flex or --storage=online-archives.

Options 1 and 4 are the lowest-risk and would substantially close the gap on their own. Option 3 is the most user-friendly but is a larger design change.

Environment

pup 0.29.3

Behavior reproduced against the Datadog Logs v2 API path used by logs search and logs aggregate.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions