Skip to content

The Elasticsearch reindexing step of the image data refresh triggers DB alerts #5379

Open
@krysal

Description

@krysal

Problem

We keep getting this unactionable alarm during the DR process, and to avoid creeping the alerts channel, it'd be better to silence it while the process is complete, as we do with ES alarms.

Image


Image

Description

Include the name of the alarm, API Production DB CPU above 50% utilization, in the list of alarms to silence.

cw_wrapper = CloudWatchWrapper(cloudwatch.get_conn())
for percentage in [50, 90]:
cw_wrapper.enable_alarm_actions(
f"ES Production CPU utilization above {percentage}%", enable
)

Or change the mechanism to get the list from an Airflow variable; this way, future changes won't require code modifications, as it can be tweaked through the Airflow UI.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ✨ goal: improvementImprovement to an existing user-facing feature💻 aspect: codeConcerns the software code in the repository🟨 priority: mediumNot blocking but should be addressed soon🧱 stack: catalogRelated to the catalog and Airflow DAGs

    Type

    No type

    Projects

    Status

    📋 Backlog

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @krysal

        Issue actions

          The Elasticsearch reindexing step of the image data refresh triggers DB alerts · Issue #5379 · WordPress/openverse