Open
Description
Is your feature request related to a problem or challenge?
- Related to Document DataFusion Threading / tokio runtimes (how to separate IO and CPU bound work) #12393
Describe the solution you'd like
As explained on the following PR:
I implemented a version of DedicatedExecutor
and an example showing how to use multiple thread pools with DataFusion. However, the example is currently very large as it uses an abstraction called DedicatedExecutor
to manage sending work back and forth between the runtimes
If users want to do the same thing, they will need to copy DedicatedExecutor
into their code (or make something similar).
Describe alternatives you've considered
If DedicatedExecutor
is a useful API for multiple people I think we should consider moving it into one of the core crates
Additional context
No response