Skip to content

Unparse of Joins is ignoring projections #15688

Open
@nuno-faria

Description

@nuno-faria

Describe the bug

The unparse of Join operators is ignoring the projected columns, ending up projecting everything.

Two conditions cause this to happen:

  • the final projected columns match the columns in the TableScans, meaning the Projection is optimized away with the optimize_projections rule;
  • the try_transform_to_simple_table_scan_with_filters function in the unparser module -- used when processing joins -- causes the TableScan projection to be discarded.

I could try to fix this by modifying the join unparser, but I would like to be sure that this is the best approach here.

To Reproduce

use datafusion::error::Result;
use datafusion::prelude::SessionContext;
use datafusion::sql::unparser::Unparser;
use datafusion::sql::unparser::dialect::PostgreSqlDialect;

#[tokio::main]
async fn main() -> Result<()> {
    let ctx = SessionContext::new();

    ctx.sql("create table test (k int, v int)")
        .await?
        .collect()
        .await?;

    let df = ctx.sql("select t1.v, t2.v from test t1, test t2").await?;

    println!("{}\n", df.logical_plan());

    let plan = df.into_optimized_plan()?;
    println!("{}\n", plan);

    println!(
        "{}",
        Unparser::new(&PostgreSqlDialect {})
            .plan_to_sql(&plan)
            .unwrap()
    );

    Ok(())
}
Projection: t1.v, t2.v
  Cross Join: 
    SubqueryAlias: t1
      TableScan: test
    SubqueryAlias: t2
      TableScan: test

Cross Join:
  SubqueryAlias: t1
    TableScan: test projection=[v]
  SubqueryAlias: t2
    TableScan: test projection=[v]

SELECT * FROM "test" AS "t1" CROSS JOIN "test" AS "t2"

Expected behavior

Output the query with the projected fields:

SELECT "t1".v, "t2".v FROM "test" AS "t1" CROSS JOIN "test" AS "t2"

Additional context

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions