Skip to content

[DBMON-6882] Make Postgres check teardown idempotent - #24852

Merged
eric-weaver merged 4 commits into
masterfrom
eric.weaver/DBMON-6882-postgres
Aug 13, 2026
Merged

[DBMON-6882] Make Postgres check teardown idempotent#24852
eric-weaver merged 4 commits into
masterfrom
eric.weaver/DBMON-6882-postgres

Conversation

@eric-weaver

@eric-weaver eric-weaver commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Makes PostgreSql._finalize() run at most once, guarded by a _finalized flag under the
existing _cancel_lock, and moves self.log.check = None after the final log statement.

Motivation

Two independent problems on the teardown path, both surfaced while preparing #24844 (lifting
this protocol into DatabaseCheck).

Teardown is not idempotent, and the check is already finalized more than once in practice:
run_one_check() calls cancel() and the integration_check fixture calls it again on
teardown, so every integration test closes the main connection, calls db_pool.close_all()
and re-runs async job shutdown twice. It only survives today because each individual step
happens to tolerate a second call.

Nulling log.check before the trailing debug line is a latent crash.
CheckLoggingAdapter.process re-reads self.check.check_id on every call for checks whose
check_id was never resolved, so _finalize() raises AttributeError: 'NoneType' object has no attribute 'check_id' as soon as debug logging is enabled. Verified locally against an
unmodified check.

This also unblocks #24844 from carrying transitional logic. When the base class gains the same
protocol, PostgreSql.run() reaches it through super().run(), so the inner and outer wrappers
would both finalize on an in-flight cancel. With teardown idempotent here, the base wrapper
needs no special case for integrations that have not yet migrated, and this guard moves into the
base once Postgres adopts it.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

_finalize() now runs at most once. The check is already cancelled more than once in
practice -- run_one_check() cancels and the integration_check fixture cancels again on
teardown -- so teardown re-closed connections and re-tore down the async jobs each time.

Nulling log.check also moves after the final log line. CheckLoggingAdapter.process reads
back through that attribute for checks whose check_id was never resolved, so the trailing
debug call raised AttributeError once debug logging was enabled.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 13, 2026

Copy link
Copy Markdown

evalya-impact-summary

evalya impact analysis
Impact analysis: RUN-ALL — every test task will run
Trigger:         empty diff (default branch, scheduled run, or shallow-clone fallback)
Test tasks:      0 (all selected)
Publish tasks:   1 (always emitted)
Diff:            empty (no diff information)

Learn more about CI impact filtering

@datadog-official

datadog-official Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Tests  Code Coverage

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🚧 1 test that failed was ignored due to quarantine View in Datadog

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 94.19%

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 937883c | Docs | Datadog PR Page | Give us feedback!

Drop the logging-hazard test and the rationale comment on the idempotency test.

Co-authored-by: Cursor <cursoragent@cursor.com>
eric-weaver added a commit that referenced this pull request Aug 13, 2026
An integration carrying its own copy of the protocol reaches this wrapper through
super().run(), so both frames see the cancellation and call _finalize(). Rather than have
the base wrapper special-case that nesting, Postgres's teardown was made idempotent
(#24852), which is the guard the wrappers need while they coexist.

Co-authored-by: Cursor <cursoragent@cursor.com>
@eric-weaver
eric-weaver marked this pull request as ready for review August 13, 2026 13:36
@eric-weaver
eric-weaver requested review from a team as code owners August 13, 2026 13:36

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 85b133b654

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread postgres/datadog_checks/postgres/postgres.py Outdated
azhou-datadog
azhou-datadog previously approved these changes Aug 13, 2026
The first teardown nulls log.check, so a second call cannot log through the adapter --
which is the very hazard this change is fixing elsewhere.

Co-authored-by: Cursor <cursoragent@cursor.com>
@temporal-github-worker-1
temporal-github-worker-1 Bot dismissed azhou-datadog’s stale review August 13, 2026 13:46

Review from azhou-datadog is dismissed. Related teams and files:

  • database-monitoring-agent
    • postgres/datadog_checks/postgres/postgres.py
@dd-octo-sts

dd-octo-sts Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Validation Report

All 21 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and code coverage settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
qa-label Validate the pull request declares whether it needs QA for the next Agent release
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

@eric-weaver
eric-weaver added this pull request to the merge queue Aug 13, 2026
Merged via the queue into master with commit 640de87 Aug 13, 2026
72 checks passed
@eric-weaver
eric-weaver deleted the eric.weaver/DBMON-6882-postgres branch August 13, 2026 13:58
@dd-octo-sts dd-octo-sts Bot added this to the 7.83.0 milestone Aug 13, 2026
eric-weaver added a commit that referenced this pull request Aug 13, 2026
The scenario it covered belongs to the integration that still wraps run(), and is exercised
by the idempotency test in #24852.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants