Skip to content

Prevent import cycle issues between relation and expressions #160

@vbarua

Description

@vbarua

Most relations in Substrait include expression in them. Some expressions in Substrait can include relations, specifically, subqueries. This is pretty standard mutual recursion.

However, in order to model data like this in Go we must be wary of introducing import cycles in which package A depends on B, and package B depends on A. Unfortunately, this is somewhat the situation that we've found ourselves in with the existences of a plan package containing relations, and an expr package for expressions.

When support for subqueries was added in #134 a number of kludges had to be made to avoid the dreaded import cycle not allowed:

  1. Subqueries, which are expressions in Substrait, had to be added to the plan package because they contain both relations and expressions.
  2. A subqueryConverter interface had to be introduced into the expr package to allow for the injection of subquery handling code to avoid directly reference the plan package in the expr package.

While we can work around the cyclic dependency by adding indirection like this, it does not lend itself to a clean API, especially as we add more code that wishes to handle both together.

A different approach to this issue would be to merge the expr package into the plan package. While this would be a larger upfront change for users, it would effectively help us, and them, avoid a whole class of problem that comes up naturally while processing Substrait.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions