Conversation
| // all stages to workers | ||
| let (task_datas, final_addrs) = | ||
| assign_to_workers(query_id, &stages, workers.values().collect(), codec)?; | ||
| assign_to_workers(query_id, stages, workers.values().collect(), codec)?; |
There was a problem hiding this comment.
@LiaCastaneda made this &[DDStage] in this https://github.com/datafusion-contrib/datafusion-distributed/pull/48/files and would not need another & herer
| pub fn create_flight_info_response( | ||
| &self, | ||
| query_plan: QueryPlan, | ||
| ) -> Result<FlightInfo, Box<Status>> { |
There was a problem hiding this comment.
Cargo clippy complains this status is over 176 bytes and we want to put this in heap instead
| } | ||
|
|
||
| pub mod tests { | ||
| #[cfg(test)] |
There was a problem hiding this comment.
Add this to avoid compile warnings that we import wrong places
| done: Arc<Mutex<bool>>, | ||
|
|
||
| /// Optional customizer for our context and proto serde | ||
| #[allow(dead_code)] |
There was a problem hiding this comment.
I know we use customizer else where but get complain here because it is not used. Put the deadcode here to avoid it.
Also, I wonder if the unit or integration test will help avoid this warning @LiaCastaneda
There was a problem hiding this comment.
maybe I missed something, but do we need to keep the customizer field in DDWorkerHandler? I don't see its being used in DDWorkerHandler, except for creating the PhysicalExtensionCodec , so in theory any customization will be automatically applied/used when we call try_decode on DDCodec 🤔
There was a problem hiding this comment.
It is used in execute_plan_and_build_stream
datafusion-distributed/src/proxy_service.rs
Line 130 in 07e26de
I think we need this to talk with different datasources
| handler: self.handler.clone(), | ||
| }; | ||
|
|
||
| #[allow(clippy::result_large_err)] |
There was a problem hiding this comment.
Since this function is passed to external FlightServiceServer::with_interceptor that use Status, we cannot change it to Box. So add clippy here to avoid complains
|
|
||
| Ok(Box::pin(out_stream)) | ||
| } | ||
| #[allow(clippy::result_large_err)] |
There was a problem hiding this comment.
Ditto. This function implements trait FlightHandler that return Status
3 participants
To avoid
compileandclippywarnings, I need to changes these;&fromstages#[cfg(test)]to import test-only stuffs#[allow(dead_code)]forcustomizerStatusstackBox<Status.heap.#[allow(clippy::result_large_err)]where we do not want t(or cannot) o makeStatusBox<Status>