Skip to content

Keep metrics endpoint up during graceful long shutdown #11817

@danlsgiga

Description

@danlsgiga

Feature description

Hey folks, we run multiple types of flows in Kestra and we have configured Kestra to wait for up to 48 hrs for all threads / flows / processes to exit cleanly before shutting down by setting this in the config:

kestra:
  server:
    terminationGracePeriod: 48h
    workerTaskRestartStrategy: IMMEDIATELY

This is working as expected, however we noticed that we are getting alerts regarding the up metric being down during these shutdowns, which means our scraper keeps trying to scrape metrics from the metrics endpoint but fails.

Can we have the metrics listener (8081 by default) to shutdown only after the complete Kestra shutdown?

Metadata

Metadata

Assignees

Labels

area/backendNeeds backend code changesenhancementNew feature or request

Projects

Status

In review

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions