Skip to content

Conversation

@jtwaleson
Copy link
Contributor

Here we add a priority queue to the postgres bg worker. This way, we can prioritize jobs.

Now we can do Worker::perform_later_with_priority(ctx, params, Some(100)) to schedule a high prio job or with Some(0) for a low prio job.

I brought this up in #1462

What I'm using this for: we sync and analyze 100s of git repos. Each one has 3 stages: fetch, quick analysis, thorough analysis. As we want to show the results of the quick analysis as soon as possible, we give the following prios: fetch: 5, quick analysis: 10, thorough analysis: 1. This way, we always fetch first, but after fetching, we do a quick analysis right away. When everything is fetched and "quick analyzed", we handle the thorough analysis.

Please let me know what you think, and if we would want this for all queue backends or if we are fine with just postgres. If we don't want this at all, also fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant