Library Job Processing layout

[4tuw2tuv0]

I'm looking at the how Oban's being used, now, and I'm either just not as familiar with Elixir, yet, (first time seeing @spec, @callback, and @impl) or just taking a bit to start to ingest things so I grasp them well.

Could I just get a sort of explanation of how the code is laid out, organizationally?

If I've understood things, PublisherWorker and ReceiverWorker handle performing jobs which have been queued with WorkerHelper just providing sort of helpful/useful utility support for the former two…? That's, honestly, just a guess, though; I noticed it has , queue: wherever it's invoked so maybe it's a feature of Oban I just haven't discovered/read, yet.

And it looks like Federator implements just the functions relevant to the jobs (such as perform and queuing the jobs).

And some of those functions are fully implemented in ActivityPubWeb.Publisher, which implements the behavior of ActivityPubWeb.Federator.Publisher? Was that a requirement of Oban or just a design decision? Queuetopia was insired by Oban (and has the most recent of recent commits) so I'm trying to see how to integrate it in so I'm trying understand how the Oban-related files are working to understand how the Queuetopia bits might fit in with the current setup.

[mayel]

Yeah... I was thinking about this yesterday and it would probably be a good opportunity to cleanup and refactor these modules a bit (besides abstracting the batching and queuing so other queuing systems can be used, or having the option of queuing being skipped entirely and jobs being processed straight away, though preferably async with some simple supervision). I can't really answer your questions without studying the code more, as I didn't write it (and some of it was originally inherited from Pleroma), and the fact that it's confusing at first look is reason enough to at the very least document it better.

[4tuw2tuv0]

Agreed; I've been staring at this for the past few days and thought I was making some sense of the design decision but it seems very embedded in how Oban works; I was going to incrementally alter files to genericize things but I'm not sure how to be able to (this thing weaves back and forth between the same files so often that I'm not entirely certain it isn't cyclical). I may just start with a very different job library, instead, and see what commonalities become apparent.

Would you be able to describe a high-level responsibility of each job and what events trigger each job and its responsibilities? I'm trying to understand what the jobs and doing and why to figure out how to write out the new libraries that'll be pluggable.