Skip to content

Persist live SQL telemetry data after job ends for post-mortem debugging #2977

@dulinriley

Description

@dulinriley

When doing debugging and investigations of jobs, the job might terminate either successfully or with a failure. The job might also fail because the client disconnects or kills the job.

In these cases, we don't want to lose the live telemetry data stored in data fusion, as this can be very valuable for post-mortem debugging.
Right now the data is stored purely in memory and is owned (eventually) by the client. If the client disconnects or sends "stop" commands, right now the actor will just stop and drop the data it holds.

Extend the system to persist the data somehow such that it could be retrieved after the job completes

Metadata

Metadata

Assignees

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions