Enqueue using just PgTransaction

[jakob-lilliemarck]

Hey 👋 good morning 🌻

I am looking for a way to enqueue a task using a PgTransaction without having to pass a pool around. Job, Worker and Queue all require async construction, but all I would like is to be able to dispatch new tasks using the transaction I have at hand - is there currently any way to do that?

[maxcountryman]

This is what enqueue_using is for.

[jakob-lilliemarck]

Sure, I did see that. However I was curious to know if there was just a pure function somewhere, or an impl on some other struct that does not hold a reference to the pool at all, since I am not using that anyway?

I assume passing Queue down is the way to go then, otherwise please point me in the right direction 🙏
thanks for coming back to me

[maxcountryman]

The queue still uses a pool to ensure it exists in the database at build time + the pool is used to dequeue tasks (this could be a transaction passed as an argument). If your goal is to not use a pool at all, that's not currently supported by the design.

[jakob-lilliemarck]

Thanks for replying. I do have one more question.

When experimenting with this it seemed that each queue each polls for its jobs, rather than having a central polling loop that polls for jobs of any kind - is that correct? Is it configurable?

I'm typically writing monoliths, and database connections is a bit of a scarce resource, so I would like job-server to only consume one connection for polling, if possible.

[maxcountryman]

Because it’s a strongly typed model, you can’t have multiple types in that way.

Instead what you can do is use an enum for input and follow something like the “multitask” example.

[jakob-lilliemarck]

Hmm, I think you could do that without compromising with type saftey 🤔 I did something similar for an event handler registry recently - It uses downcasting with Any, but only after the types already have been checked in the function signature:
https://github.com/jakob-lilliemarck/fx-event-bus/blob/main/src/handler/handler_registry.rs#L49C1-L71C6

Sure I could use an enum and a single queue but that would require me to define the input for all jobs in a single place, whereas jobs logically may belong to specific service or subdomain in the app. It will break modularity and create coupling.

I think the "one polling loop per queue" is a bit of a dealbreaker for me using this library with my current project.

[maxcountryman]

I’m not interested in using Any.

[jakob-lilliemarck]

Ok, fair enough, I understand it doesn't align with your design goals for this crate. However I am curious though to know why? I can see that using Any in a way that compromises type-saftey is of course problematic, but what's the argument against using it in a type-safe manner as proposed above?

Thinking about it some more, I don't think you would even need Any for jobs as they typically only have a single "handler" per argument type. A type-erasing trait and generic blanket impl for desertializing the message and passing it to the strongly typed handler should do 🤔

Asking purely out of curiosity, not trying to impose a design on your library or anything like that.