-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
We can either use multiprocessing or subprocess to start an offline ingestor in the different process.
| - | multiprocessing |
subprocess |
|---|---|---|
| pros | - easy to attach callback - easy to post-process the result of the offline-ingestor - doesn't need to use arbitrary command |
- offline-ingestor will need a command entry point anyways, so don't need to implement another accessible interface |
| cons | - needs a function type interface to run an offline-ingestor |
- limited access to the result - not possible to attach a callback function - needs to wrap the pipe-object with an async call to continue to the next message while offline-ingestor is running |
Since we don't use non-python program at all,
I would like to go for the multiprocess.Process...?
In case we keep using subprocess, we need to make sure if it should be asynchronously run or not.
Metadata
Metadata
Assignees
Labels
No labels