-
Notifications
You must be signed in to change notification settings - Fork 71
Description
In order to use the Optimizer or generate a TableLayout, the existence and registration of a Velox Connector object is required
ConnectorMetadata
For the TableLayout, a non-owning pointer to the connector is an explicit parameter:
https://github.com/facebookincubator/axiom/blob/main/axiom/connectors/ConnectorMetadata.h#L331-L338
This is not exposed except to access the Connector ID and through a direct getter:
/// The Connector to use for generating ColumnHandles and TableHandles for
/// operations against this layout.
velox::connector::Connector* connector() const {
return connector_;
}
This is explicitly null checked by the TableLayout constructor:
https://github.com/facebookincubator/axiom/blob/main/axiom/connectors/ConnectorMetadata.cpp#L154
Optimizer
In the Optimizer, it is only used to access the Connector ID to seed the ConnectorSession:
https://github.com/facebookincubator/axiom/blob/main/axiom/optimizer/ToVelox.cpp#L169-L171
And in one place to lookup the Metadata:
https://github.com/facebookincubator/axiom/blob/main/axiom/optimizer/ToVelox.cpp#L1467-L1469
Rationale for Removal
Given that Velox is the execution library, and Axiom is the query frontend, it may not make sense to require the setup of the query execution layer in order to plan a query; you can imagine a scenario where Connector initialization is undesirable in the query frontend layer (for example, if the Connector eagerly allocates some significant resources)
Some metadata implementations (TpchConnectorMetadata, HiveConnectorMetadata, TestConnectorMetadata) accept the Connector as a parameter, but these contracts tightly couple Axiom and Velox initialization. Open to other thoughts though
cc @mbasmanova