Skip to content

Predicate Pushdown for Attached ADBC Tables #2

Description

@SamArch27

Consider the following SQL query that selects a subset of rows from an attached ADBC table.

D ATTACH 'profile://mydb' AS mydb (TYPE adbc);
D USE mydb.public;
D SELECT * FROM games WHERE name = 'Monopoly';
┌───────┬──────────┬─────────────────┬───────┬─────────┬─────────────┬─────────────┐
│  id   │   name   │    inventor     │ year  │ min_age │ min_players │ max_players │
│ int32 │ varcharvarchar     │ int16 │  int16  │    int16    │    int16    │
├───────┼──────────┼─────────────────┼───────┼─────────┼─────────────┼─────────────┤
│   1   │ Monopoly │ Elizabeth Magie │ 1903826      │
└───────┴──────────┴─────────────────┴───────┴─────────┴─────────────┴─────────────┘
D EXPLAIN SELECT * FROM games WHERE name = 'Monopoly';
┌─────────────────────────────┐
│┌───────────────────────────┐│
││       Physical Plan       ││
│└───────────────────────────┘│
└─────────────────────────────┘
┌───────────────────────────┐
│         PROJECTION        │
│    ────────────────────   │
│             id            │
│            name           │
│          inventor         │
│            year           │
│          min_age          │
│        min_players        │
│        max_players        │
│                           │
│           ~1 row          │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│           FILTER          │
│    ────────────────────   │
│    (name = 'Monopoly')    │
│                           │
│           ~1 row          │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         READ_ADBC         │
│    ────────────────────   │
│    Function: READ_ADBC    │
│                           │
│        Projections:       │
│            name           │
│             id            │
│          inventor         │
│            year           │
│          min_age          │
│        min_players        │
│        max_players        │
│                           │
│           ~5 rows         │
└───────────────────────────┘

The ADBC extension scans games by executing a SELECT * FROM games query on the ADBC database with read_adbc and then applies the predicate name = 'Monopoly' with a local filter inside DuckDB. Pushing down predicates into the query on the ADBC database (i.e., SELECT * FROM games WHERE name = 'Monopoly') would improve performance by reading only the required rows.

Although predicate pushdown is possible with the ADBC extension by calling read_adbc (see below), it would be convenient for the extension to apply it automatically for attached ADBC tables.

D CREATE MACRO read_mydb(query) AS TABLE SELECT * FROM read_adbc('profile://mydb', query);
D SELECT * FROM read_mydb('SELECT * FROM games WHERE name = ''Monopoly''')
┌───────┬──────────┬─────────────────┬───────┬─────────┬─────────────┬─────────────┐
│  id   │   name   │    inventor     │ year  │ min_age │ min_players │ max_players │
│ int32 │ varcharvarchar     │ int16 │  int16  │    int16    │    int16    │
├───────┼──────────┼─────────────────┼───────┼─────────┼─────────────┼─────────────┤
│   1   │ Monopoly │ Elizabeth Magie │ 1903826      │
└───────┴──────────┴─────────────────┴───────┴─────────┴─────────────┴─────────────┘

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions