Fill Operations Feature

[hawkfish]

A common pair of data prep operations are fill down and fill up. Coming in DuckDB 2.0, you can write window functions in extensions, which would let you add these operations in an extension. A simple version of fill down is already included in the demo extension code (it could use a default argument.)

[SouravRoy-ETL]

Working on it!

[SouravRoy-ETL]

Done, shipped both directions today.

xf.fill_forward already existed, so I just added the sibling xf.fill_backward next to it - together they cover the fill down / fill up pair you mentioned.

I went with a plain window function over the system DuckDB CLI rather than the 2.0 extension route you suggested:

SELECT * REPLACE (
  first_value(col IGNORE NULLS) OVER (
    PARTITION BY ... ORDER BY ts
    ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
  ) AS col
) FROM upstream

Reasoning: works on every DuckDB version Duckle currently ships against, single pass, no extra LOAD step on the user's machine, no extension binary to package. The 2.0 window-function extension is on my radar though - if it ends up materially faster in practice I can swap the implementation behind the same component schema without anything breaking.

Both live in the Rows group of the Transforms palette. Required props: column, orderBy. Optional: partitionBy (fills each group independently). Round-trip test against real DuckDB at crates/duckdb-engine/tests/execution.rs::fill_backward_propagates_next_non_null.

Goes out with the next release. Appreciate the nudge.

[SouravRoy-ETL]

@all-contributors please add @hawkfish for infrastructure, tests and code

[allcontributors]

@SouravRoy-ETL

I've put up a pull request to add @hawkfish! 🎉