Add API to respond with for running backfills #343
Conversation
supports the ability to more easily see what's currently running when investigating possible issues or looking to turn things off temporarily. UI changes to come in separate PR
|
Mind reposting the screenshots, they don't seem to be showing in the preview? |
|
Make sure to rebase on top of the variant PR. |
| private val queryFactory: Query.Factory, | ||
| ) : WebAction { | ||
|
|
||
| @Get("/backfills/running") |
There was a problem hiding this comment.
Probably needs to be paginated.
There was a problem hiding this comment.
yeah - was kinda assuming the number of backfills concurrently running wouldn't generally be enough to require pagination, but that's I guess that's not necessarily the case.
There was a problem hiding this comment.
Actually, looks like we don't have pagination on any of the endpoints apart from the /services/{service}/variants/{variant}/backfill-runs one. (which makes sense because it's completely unbounded)
Feels like we might be able to avoid it for this endpoint given it's only returning currently running ones. WDYT?
There was a problem hiding this comment.
You are listing runs so of course it needs to be paginated.
The other ones might need a rethink soon given how popular Backfila has become.
Why did you pick runs to show rather than services?
👍 done. (not sure what happened there) |
I've rebased. It's not immediately obvious to me how the variant changes work, I've just hardcoded some values to make the tests I added pass. |
When an incident occurs it's often really valuable to be able to see what Backfills are currently running in order to:
This PR exposes a new endpoint to surface the currently running backfills.
As part of this change I've also included instructions on starting mysql locally for development and fixed some issues that occur locally when building / running tests on Apple silicon.
Subsequent PR will integrate this endpoint into the UI to create a screen something like:
