Python Polars 1.42.1

⚠️ Deprecations

• Deprecate strict parameter of pl.concat, replace with new how='horizontal_extend' (#27965)

🚀 Performance improvements

• Add a sampled resolve mode for multi-file parquet metadata (#28111)
• Speed up small dtype series sums with a upcast sum kernel (#27958)
• Don't block on path expansion (#28073)

✨ Enhancements

• Close ResourceWarning leaks in database tests (#28066)

🐞 Bug fixes

• Incorrect sorted fast-path returning nan for grouped max (#28129)
• Fix panic on scan_parquet filter of fixed-size binary column (#28122)
• Fix decimal dynamic float cast (#28126)
• Fix pyiceberg key dot HDFS prefix filter (#28109)
• Fix projection pushdown panic on select(len()) after groupby (#28108)
• Fully resolve remaining ResourceWarning leaks in database/iceberg tests (#28107)
• Expiry time loading for AWS DeferredRefreshableCredentials (#28099)
• Close ResourceWarning leaks in database tests (#28066)
• Avoid panic on temporal extraction for datetime columns with nulls (#28054)
• Prevent panic in replace when old/new contain Expr or object dtype values (#27433)

📖 Documentation

• Fix dataset regression in the examples (#28131)
• Emphasize to generally not ever write Rust tests (#28082)
• Mention Polars skill in User Guide's LLM section (#28113)

🛠️ Other improvements

• Disallow usage of tokio::spawn in clippy (#28123)
• Improve Makefile with respect to venv robustness (#28110)
• Remove legacy prelude glob imports from leaf files (#28114)
• Skip pandas 3.0.4 due to pd.TimeDelta segfault (#28125)
• Update mypy to the new 2.x release (#28116)
• Ensure pyrefly checks run as part of make pre-commit (#28095)
• Fix test_select_explode_height_filter_order_by failure when POLARS_MAX_THREADS=2 (#28090)
• Add Morsel::height (#28060)

Thank you to all our contributors for making this release possible!
@0guban0v, @Kevin-Patyk, @TNieuwdorp, @alexander-beedie, @azimafroozeh, @dsprenkels, @kdn36, @nameexhaustion, @orlp, @raphaelroshan, @ritchie46 and @sar-cheng

Python Polars 1.42.0

⚠️ Deprecations

• Deprecate casts from string to temporal dtypes (#28056)

🚀 Performance improvements

• Add bytes-based concurrency control for cloud IO (#27924)
• Stop flushing multiplexer on phase change (#28013)
• Prevent empty chunks from triggering copy in rechunk (#27953)
• Improve performance of struct unnest inside list eval on streaming (#27905)
• Eliminate filters with contradictory predicates (#27775)

✨ Enhancements

• Deprecate casts from string to temporal dtypes (#28056)
• Add naive out-of-core spilling to Polars (#27998)
• Add experimental strict mode (#28023)
• Optimize non-negative len and null_count comparisons (#28002)
• Optimize len and null_count comparisons (#27727)
• Support SQL's implicit JOIN syntax (#27890)
• Add Expr.is_sorted (#26708)
• Add suggestions in wrong api usage or mismatched column names (#27954)
• Add external object_store support for non-native schemes (#27871)
• Skip Delta files and parquet row groups using per-field struct statistics (#27887)
• Improve IR sortedness detection for with_row_index and explode (#27875)
• Add DataFrame.is_sorted() (#27870)
• Expose fixed-size rolling window expressions in Python visitor (#27108)
• Fix SchemaError using lazy HConcat->Sink (#27770)
• Add tiered multi-file parquet metadata resolver (#27720)
• Cache and shuffle DNS for cloud object_store (#27659)

🐞 Bug fixes

• Add a deprecation warning for .explode() without empty_as_null argument (#28040)
• Fix select(len()) incorrectly returned 0 when using scan_iceberg with pyiceberg as reader override (#28044)
• Ensure slice expr always observes input order (#28041)
• Fix projection pushdown with row index out of order (#28043)
• Fix cross-join predicates not applied when using sink_* functions (#28042)
• Grammar corrections (recommend to → recommend, allows to → allows using) (#28038)
• Ensure optimisations aren't omitted by cached IR nodes (#28004)
• Computed GROUP keys in SQL projections (#27989)
• Handle empty DataFrames in dt.truncate and dt.round (#26120)
• Address join alias state leakage in SQL CTEs (#27985)
• Support scanning file:// URIs with percent-encoded paths (#27876)
• Lower SQL [NOT] IN (subquery) to semi/anti join (#27888)
• Check for continuity across morsels in AsOf join node (#27461)
• Add test for projection pushdown with hconcat regression (#27937)
• Traverse caches only once in explain (#27939)
• Preserve null placement when collapsing sort.reverse() into single sort (#27918)
• Incorrect schema order for hstack in projection pushdown optimizer (#27920)
• Fix duplicated map_elements warning on streaming engine (#27919)
• Use block_in_place_on for calls which can come from executor thread (#27855)
• Mismatch in max_threads -> pipeline configuration (#27854)
• Keep maintain_order on sliced unique (#27852)
• Fix SchemaError using lazy HConcat->Sink (#27770)
• Fix incorrect projection height when selecting only literals (#27825)
• Select with expr slice and len gave incorrect len (#27824)
• Prevent import panic when environment variable set to unexpected value (#27831)

📖 Documentation

• New On-Prem release (#27979)
• Add note on OOM behavior on Kubernetes (#27972)
• Properly render backslashes in CSV eol_char docstrings (#27962)
• Add entry for DataFrame.is_sorted() (#27913)
• Clarify compat_level export docs (#27715)
• Add robots.txt disallowing stale docs and added sitemap for better SEO (#27901)
• Update Polars <> Spark comparison (#27873)
• Bump to patched version (#27851)
• Replace Typeform sign-up URL with new enterprise link (#27838)
• Correct wrong head call (#27848)
• Add Polars On-Prem 0.5.0 release (#27849)
• Correct onprem license helm values (#27847)
• Update connecting Polars Cloud to AWS documentation (#27823)
• Correct cost/connectivity sections (#27813)
• Add sections about egress (#27805)
• Tragic typo in On-Prem docs (#27804)

🛠️ Other improvements

• Add AI agent disclsoure (#28064)
• Bump memmap2 version (#28050)
• Update sqlparser dependency (#28021)
• Update pyo3 and rust-numpy to 0.29.0 (#27970)
• Add 2.0 branch as primary branch (#27978)
• Patch jemalloc-sys for Makefile bug (#27945)
• Restore Cargo.lock (#27964)
• Add test for projection pushdown with hconcat regression (#27937)
• Update deny.toml configuration (#27949)
• Doctest skip refactor and add hugging face rate limited tests (#27885)
• Run Pyrefly type-checker on source code (#27722)
• Bump Rust dependencies (#27874)
• Changes needed for Rust 0.54.x (#27853)
• Use Vec instead of PlHashMap for ProjectionInfo.map (#27856)
• Reduce codegen-units (#27835)
• Refactor to avoid unbound-name Pyright/Pyrefly errors (#27827)

Thank you to all our contributors for making this release possible!
@0guban0v, @April-Sonnet, @BitWeaverDev, @EndPositive, @JakubValtar, @Kevin-Patyk, @Liyixin95, @MarcoGorelli, @Matt711, @TNieuwdorp, @alexander-beedie, @ankane, @azimafroozeh, @cBournhonesque, @carnarez, @dsprenkels, @gautamvarmadatla, @kdn36, @nameexhaustion, @orlp, @ritchie46, @tolleybot, @toreerdmann, @uurl and @xixixao

Rust Polars 0.54.4

🏆 Highlights

• Add LazyFrame.gather (#27501)
• Nested common subplan elimination (#27340)
• Stabilize streaming engine (#27497)
• Speed up parquet metadata decode with hand-written Thrift (#27427)
• Add streaming support for grouped AsOf join (#27293)

🚀 Performance improvements

• Eliminate filters with contradictory predicates (#27775)
• Update to new jemalloc (#27797)
• Do not materialize ScalarColumn in Column split_at (#27782)
• Avoid materializing broadcast in array.shift (#27740)
• Avoid materializing broadcast list in list.sample(n) and list.sample(frac) (#27679)
• Adaptive size dispatch to hashset or radix sort + capacity-aware reset in agg_n_unique (#27719)
• Dispatch {list,arr}.{unique,n_unique,reverse} to group_by engine (#27278)
• Improve in-memory grouped non-null count (#27702)
• Factor shared conjuncts out of OR-of-ANDs predicates (#27627)
• Skip downloading IPC batches exceeding slice bounds (#27683)
• Faster Series::is_sorted for logical / non-primitive types (#27567)
• Avoid materializing broadcast list in list.shift (#27628)
• Optimise json_decode Datetime string parsing (#27559)
• Speed up to_numpy C-order via cache-blocked transpose (#27522)
• Optimize select(len()) for non-strict horizontal concat (#27516)
• Pushdown slices to inputs on left/right/full join (#27508)
• Don't infer CSV schema if schema is set (#27507)
• Nested common subplan elimination (#27340)
• Make is_in row-group pruning precise on null-containing haystacks (#27495)
• Don't do fused-multiply-add on scalars (#27479)
• List full fast path (#27477)
• Make is_in row-group pruning precise on multi-value lists (#27475)
• Add streaming GatherNode (#27465)
• Lower non-elementwise FunctionExprIR to ColumnarFunctionNode (#27462)
• Speed up parquet metadata decode with hand-written Thrift (#27427)
• Skip validity mask processing in __array_ufunc__ when no inputs have nulls (#27358)
• Create IR slice from expr slice pushdown (#27200)
• Add streaming support for grouped AsOf join (#27293)
• Avoid unnecessary rechunk when sorting already sorted DataFrame (#27264)
• Lower basic over() to streaming primitives (#27303)
• Lower drop_{nulls,nans} in streaming group_by aggregations (#27296)
• Lower entropy to streaming reductions (#27174)
• Add native streaming interpolate (#27185)
• Streaming strptime with format=None (#27056)
• Lower skew / kurtosis to streaming aggregations (#27176)
• Post apply pyarrow filter in Polars' engine instead of pyarrow (#27192)
• Optimize drop_nulls().{first,last}() to {first,last}(ignore_nulls=True) (#27187)
• Always process pyarrow scan in batches (#27183)
• Make cut output Enum and mark as elementwise (#27173)
• Remove unused expression sorts (#27075)
• Use delta stats for mixed hive and non-hive predicate pushdown (#27102)
• Take into account size per row in join sampling (#27098)
• Streaming is_first_distinct and unique(maintain_order=True) (#27052)
• Streaming cov and corr (#27008)
• Add sorted unique node to streaming engine (#26990)
• Ensure Expr.append is lowered in streaming engine (#27022)
• Collapse consecutive Sort nodes (#26965)
• Drop maintain_order=True requirement in sink_delta (#27007)
• Lower index_of to streaming engine (#26923)
• Streaming native backward_fill (#26967)
• Native streaming forward_fill (#26922)
• Drop unused filter column above cache (#26955)
• Optimize .replace() from a single value (#26948)
• Add a streaming range-join (#26790)
• Lower arg_{min,max} to streaming engine (#26845)
• Additional IR slice pushdown after filter pushdown (#26815)
• Streaming first/last on Enum through physical (#26783)
• Fast filter for scalar predicates (#26745)
• Allow SimpleProjection in streaming engine to rename (#26709)
• Streaming cloud download for scan_csv (#26637)
• Drop columns only needed for predicates after the predicate is applied (#26703)
• Run projection pushdown after predicate pushdown (#26688)
• Comparison literal downcasting (#26663)
• Add dynamic predicates for TopK (#26495)
• Increase minimum default parquet row group prefetch to 8 (#26632)
• Partial predicate conversion to PyArrow (#26567)
• Streaming cloud download for scan_ndjson / scan_lines (#26563)
• Grab GIL fewer times during Object join materialization (#26587)
• Improve CSV and NDJSON cloud sink performance (#26545)
• Tune cloud writer performance (#26518)
• Allow parallel InMemorySinks in streaming engine (#26501)
• Add streaming AsOf join node (#26398)

✨ Enhancements

• Expose fixed-size rolling window expressions in Python visitor (#27108)
• Expose IR::Scan hive parts in the python node visitor (#27829)
• Expose IRFunctionExpr::DynamicPred in the python visitor (#27616)
• Fix SchemaError using lazy HConcat->Sink (#27770)
• Add pinning and queuing logic to polars-ooc (#27791)
• Add tiered multi-file parquet metadata resolver (#27720)
• Cache and shuffle DNS for cloud object_store (#27659)
• Update to new jemalloc (#27797)
• Allow deeper expressions (#27768)
• Add is_inherently_nondeterministic helper for AExpr (#27687)
• Use true division for the / operator in Polars SQL (#27391)
• Add Rust backend for Expr.has_nulls (#27590)
• Add block_in_place to Polars' async executor (#27612)
• Stabilize float16 (#27607)
• Add Expr.is_empty (#27583)
• Add support for the SQL FILTER clause for aggregate functions, and STRING_AGG (#27564)
• Make parquet FileMetadata prunable for IR-plan dispatch (#27535)
• Broadcast scalar input for list.slice (#27487)
• Add LazyFrame.gather (#27501)
• Add null_on_oob in {Expr/Series}.gather (#27327)
• Stabilize streaming engine (#27497)
• Process batched arr.eval on overflow boundaries (#27496)
• Process batched list.eval on overflow boundaries (#27483)
• Print SLICED UNION in LazyFrame explain (#27467)
• Cargo deny (#27363)
• Add maintain_order parameter to merge_sorted (#27263)
• Add ignore_nulls to {list,arr}.{any,all} (#27186)
• Lock-free memory manager with spill-to-disk and fully OOC multiplexer (#26774)
• Add is_unique to list/array dtypes (#27290)
• Add pl.merge_sorted operating on multiple frames (#27014)
• Add fast_alloc feature flag, remove default_alloc (#27206)
• Add a GPU slot to OptFlags so we can control CSE (similar to streaming) (#27026)
• Allow group_by() without key exprs (#27141)
• Collapse consecutive Sort nodes (#26965)
• Use UUIDv7 for sink_iceberg directory name generation (#26958)
• Truncate large binary/utf8 Parquet statistics values (#26764)
• Error if PartitionBy path provider returns absolute path that does not begin with base path, or contains '..' (#26894)
• Support Delta deletion vectors in scan_delta (#26867)
• Support Decimal32/64 in scan_parquet (#26941)
• Support casting Duration to String in ISO 8601 format (#26860)
• Add a streaming range-join (#26790)
• Support Expr for holidays in business day calculations (#26193)
• Parameter for pivot to always include value column name (#26730)
• Raise error in .collect_schema() when arr.get() is out-of-bounds (#26866)
• Extend Expr.reinterpret to all numeric types of the same size (#26401)
• Add missing_columns parameter to scan_csv (#26787)
• Clear no-op scan projections (#26858)
• Support nested datatypes for {min,max}_by (#26849)
• Support SQL ARRAY init from typed literals (#26622)
• Accept table identifier string in scan_iceberg() (#26826)
• Add a convenience make fresh command to the Makefile (#26809)
• Add unstable LazyFrame.sink_iceberg (#26799)
• Add maintain order argument on implode (#26782)
• Implement predicate pushdown for aliased groupby keys (#26597)
• Speed up casting primitive to bool by at least 2x (#26823)
• Enable rowgroup skipping for float columns (#26805)
• Add expression context to errors (#26716)
• Add Decimal support for product reduction (#26725)
• Support all Iceberg V2 arrow types in sink_parquet arrow_schema parameter (#26669)
• Re-work behavior of arrow_schema parameter on sink_parquet (#26621)
• Add contains_dtype() method for Schema (#26661)
• Implement truncate as a "to_zero" rounding mode (#26677)
• Expose AExpr::Rolling in the python visitor (#26715)
• More generic streaming GroupBy lowering (#26696)
• Add basic MemoryManager to track buffered dataframes for out-of-core support later (#26443)
• Add truncate Expression for numeric values (#26666)
• Better error messages for hex literal conversion issues in the SQL interface (#26657)
• Add SQL support for LPAD and RPAD string functions (#26631)
• Support SQL "FROM-first" SELECT query syntax (#26598)
• Speed up any() and all() for nulls (#26615)
• Bump Chrono to 0.4.24, enabling stricter parsing of %.3f/%.6f/%.9f specifiers (#26075)
• Expose unstable assert_schema_equal in py-polars (#24869)
• Allow parsing of compact ISO 8601 strings (#24629)
• Streaming cloud download for scan_ndjson / scan_lines (#26563)
• Configuration to cast integers to floats in cast_options for scan_parquet (#26492)
• Add escaping to quotes and newlines when reading JSON object into string (#26578)
• Standardise on RFC-5545 when doing datetime arithmetic on timezone-aware datetimes (#26425)
• Support sas_token in Azure credential provider (#26565)
• Expose HConcat options in the python node visitor (#26551)
• Relax SQL requirement for derived tables and subqueries to have aliases (#26543)
• Add polars-config and pl.Config.reload_env_vars() (#26524)
• Record path for object store error raised from sinks (#26541)
• Use CRC64NVME for checksum in aws sinks (#26522)
• Add get() for binary Series (#26514)
• Add streaming AsOf join node (#26398)

🐞 Bug fixes

• Fix skip_batches not handling negation of bool dtype with None values (#27452)
• Use block_in_place_on for calls...

Python Polars 1.41.2

🚀 Performance improvements

• Update to new jemalloc (#27797)
• Do not materialize ScalarColumn in Column split_at (#27782)
• Avoid materializing broadcast in array.shift (#27740)
• Avoid materializing broadcast list in list.sample(n) and list.sample(frac) (#27679)

✨ Enhancements

• Update to new jemalloc (#27797)

🐞 Bug fixes

• Broken link to AI Policy corrected (#27793)
• Update to new jemalloc (#27797)

📖 Documentation

• Update Polars On-Prem version stamp (#27799)
• Broken link to AI Policy corrected (#27793)
• Add release dates to the On-Prem releases page (#27787)
• Improve on-prem docs (#27788)
• Add query profiler video to On-Prem user guide (#27786)
• Add EKS/AKS/GKE guides (#27774)
• Add Polars On-Prem 0.4.2 (#27780)

🛠️ Other improvements

• Run Pyrefly on _utils and functions (#27789)
• Harden against async blocking deadlocks (take 2) (#27767)

Thank you to all our contributors for making this release possible!
@ButteryPaws, @EndPositive, @Kevin-Patyk, @MarcoGorelli, @TNieuwdorp, @azimafroozeh, @carnarez, @kdn36, @lun3x, @orlp and @ritchie46

Python Polars 1.41.1

🚀 Performance improvements

• Adaptive size dispatch to hashset or radix sort + capacity-aware reset in agg_n_unique (#27719)

✨ Enhancements

• Allow deeper expressions (#27768)

🐞 Bug fixes

• Raise length mismatch in multiple sort_by in group_by (#27772)
• Respect min_samples for rolling_by ops with nulls (#27706)
• Fix memory usage regression affecting TPCH Q22 (#27758)
• Add POLARS_ALLOW_NESTED_CSPE env var and make nested CSPE opt-in (#27765)
• Post-apply residual pyarrow predicates (#27764)
• Fix loss of precision for smaller floating types(#27662) (#27732)
• Filter at scan dropped in CSPE filter pushdown (#27763)
• Fix portstate assertion error on is_in (#27757)
• Fix incorrect when/then after forward fill / reverse in groupby (#27745)
• Accept empty Thrift list encoded as bare 0x00 byte in parquet metadata (#27754)
• Stabilize object store credentialprovider cache key (#27712)
• Add to merge_sorted docs that the input must be nulls first (#27743)

📖 Documentation

• Docs fixes (#27766)
• Sync from Polars Cloud (#27751)
• Add to merge_sorted docs that the input must be nulls first (#27743)

🛠️ Other improvements

• Attribute annotations for CatalogCredentialProvider (#27739)
• Solve type: ignore in _AioDataFrameResult (#27311)
• Remove dead code in _write_utils.py (#27721)
• Remove unnecessary not isinstance(v, DataType) check (#27723)

Thank you to all our contributors for making this release possible!
@EndPositive, @JakubValtar, @MarcoGorelli, @NicoOhR, @azimafroozeh, @carnarez, @dsprenkels, @jorenham, @kdn36, @nameexhaustion, @orlp and @ritchie46

Python Polars 1.41.0

🏆 Highlights

• Add LazyFrame.gather (#27501)
• Nested common subplan elimination (#27340)
• Stabilize streaming engine (#27497)
• Speed up parquet metadata decode with hand-written Thrift (#27427)

⚠️ Deprecations

• Deprecate the StringCache (#27580)

🚀 Performance improvements

• Dispatch {list,arr}.{unique,n_unique,reverse} to group_by engine (#27278)
• Improve in-memory grouped non-null count (#27702)
• Factor shared conjuncts out of OR-of-ANDs predicates (#27627)
• Skip downloading IPC batches exceeding slice bounds (#27683)
• Avoid materializing broadcast list in list.shift (#27628)
• Optimise json_decode Datetime string parsing (#27559)
• Speed up to_numpy C-order via cache-blocked transpose (#27522)
• Optimize select(len()) for non-strict horizontal concat (#27516)
• Pushdown slices to inputs on left/right/full join (#27508)
• Don't infer CSV schema if schema is set (#27507)
• Nested common subplan elimination (#27340)
• Make is_in row-group pruning precise on null-containing haystacks (#27495)
• Don't do fused-multiply-add on scalars (#27479)
• List full fast path (#27477)
• Make is_in row-group pruning precise on multi-value lists (#27475)
• Add streaming GatherNode (#27465)
• Lower non-elementwise FunctionExprIR to ColumnarFunctionNode (#27462)
• Speed up parquet metadata decode with hand-written Thrift (#27427)

✨ Enhancements

• Use true division for the / operator in Polars SQL (#27391)
• Add Rust backend for Expr.has_nulls (#27590)
• Stabilize float16 (#27607)
• Add Expr.is_empty (#27583)
• Add support for the SQL FILTER clause for aggregate functions, and STRING_AGG (#27564)
• Make parquet FileMetadata prunable for IR-plan dispatch (#27535)
• Broadcast scalar input for list.slice (#27487)
• Add LazyFrame.gather (#27501)
• Add null_on_oob in {Expr/Series}.gather (#27327)
• Stabilize streaming engine (#27497)
• Process batched arr.eval on overflow boundaries (#27496)
• Process batched list.eval on overflow boundaries (#27483)
• Print SLICED UNION in LazyFrame explain (#27467)

🐞 Bug fixes

• Panic in scan of empty IPC with slice (#27708)
• Persist object_store rebuild state in cache (#27707)
• Sort flag on GroupsType only applies to first element (#27684)
• Invalid unwrap_unchecked when length isn't exact (#27685)
• Don't unwrap channel send in streaming join_asof (#27688)
• Fix merge_sorted panic when List in frame (#27568)
• Put AsOf join buffered Morsels back the front of the deque if we cannot process them rn (#27658)
• Fix skip_batches logic for NaN (#27673)
• Raise TypeError when calling next() directly on GroupBy objects (#27562)
• Data type comparison for extension types (#27632)
• Share last-morsel split budget across files in streaming multi-scan (#27630)
• Bytes scalars were not being broadcast in dataframe constructor (#27621)
• Reset the sort-options in Series::is_sorted() after row-encoding columns (#27614)
• Rayon deadlock with re-entrant io sources (#27600)
• Don't push negative-offset slices through HConcat (#27570)
• Logic error in streaming is_empty (#27602)
• Fix incorrect CSE with large is_in literal (#27575)
• AnonymousFunction can qualify as SQL aggregator (#26986)
• Fix CSPE panic in cloud (#27594)
• Set merge-join streaming node to Finished if its sending port is Done (#27572)
• Widen decimal precision on sum aggregation at runtime (#27579)
• Fix str.to_time was raising unnecessarily when input was all nulls (#27574)
• Prevent panic when switching from one extension dtype to another (#27566)
• Fix DataFrame.write_database(..., if_table_exists="append", engine="adbc") not handling missing tables correctly (#26913)
• Ensure json_decode doesn't fail for Date and Time string deserialization (#27554)
• Incorrect RUSTFLAGS passing in Makefile (#27555)
• Fix panic on reading IPC with 0-row compressed bitmap (#27551)
• Set HEAD_RESPONSE_SIZE_ESTIMATE to 0 (#27548)
• Fix lazy concat horizontal didn't raise on mismatching heights after projection pushdown (#27506)
• Prevent join panic when suffix="" and coalesce=True (#27376)
• Do not make a FastCount for csv if pre_slice is set (#27536)
• Support duplicate names in over (#27544)
• Reassign sequence numbers when distributing input morsels in streaming AsOf join node (#27538)
• Do not reverse dataframes when sorting with all-null key columns (#27517)
• Incorrect length check on streaming zip (#27505)
• Remove invalid type annotation Sequence[int] from DataFrame.\_\_setitem\_\_ key (#27355)
• Respect nulls_last for descending over(order_by) in group_by().agg() (#27486)
• Fix perf regression in scan_csv select(len()) when collected on streaming engine (#27504)
• Harden extend strictness (#27476)
• Prevent deadlock when using to_arrow() in a multithreaded context (#27472)
• Do not flatten sliced union (#27466)
• Prevent deadlock when using to_pandas() in multithreaded context (#27451)
• Struct rechunk bug and add Series::with_validity (#27446)
• Handle column indexing in read_parquet/read_csv with pyarrow reader (#27397)
• Export enum as ordered dictionary to arrow (#27432)
• Ensure sample() respects shuffle=False (#27248)
• Return empty DataFrame from concat_list with lit and empty column (#27305)
• Read parquet MAP columns without LogicalType annotation (#27404)
• Raise DuplicateError on parquet files with duplicate column names (#27399)

📖 Documentation

• Document Expr.list.__getitem__ (#27689)
• Add cloudpickle requirement (#27703)
• Clarify from_arrow schema ordering (#27493)
• Fix a typo in join_asof docstring (#27682)
• Clarify schema column order (#27681)
• Document horizontal string concatenation (#27542)
• Document all valid engine options on LazyFrame collect/sink/explain methods (#27374)
• Orchestration docs check (#27605)
• Drop redundant Pattern 2 from Dagster integration page (#27581)
• Update to remove Dockerhub PAT references (#27582)
• Modernize Dagster integration example for Polars Cloud (#27560)
• Use Polars random seed in sample example (#27537)
• Clarify full join description (#27530)
• Make expressions operations RNG deterministic (#27494)
• Document struct field order (#27492)
• Improve over:order_by description (#27520)
• Clarify join output columns (#27449)
• Document null propagation in pl.format (#27447)
• Document gzip support in read_csv (#27434)
• Add See Also sections for datetime docstrings (#27316)
• Polars On-Prem release (#27439)
• Rename to Polars On-Prem (#27435)
• Clarify null handling in unique operations (#27431)
• Document write_ipc buffer behavior with file=None (#27430)

📦 Build system

• Also split debug info in debug-release (#27609)
• Use split-debuginfo on linux (#27608)
• Bump deltalake to 1.5.1 in CI (#27387)

🛠️ Other improvements

• Remove redundant DSL::AGG::Unique (#27718)
• Harden against async blocking deadlocks (#27653)
• Print Python traceback when POLARS_TIMEOUT_MS is exceeded (#27657)
• Format missed in previous PR (#27700)
• Bump pytest and remove codspeed (#27686)
• Remove client-side allow_local_scans option for prepare_cloud_plan (#27663)
• Remove superfluous test (#27676)
• Cleanup streaming flags (#27671)
• Expose unordered concatenation in python visitor (#27666)
• Bump deltalake and fix CI (#27660)
• Add impl IntoAExprBuilder for ExprIR (#27656)
• Split _expand_selector_dicts into multiple functions so return type is simple and accurate (#27618)
• Update object_store patch repo (#27650)
• Match NumPy signature in DataFrame.__array__ and Series.__array__ (#27634)
• Add ImageVersion to rust-cache key (#27626)
• Run Pyrefly on tests (#27459)
• Fix is_empty test (#27597)
• Fix tz type difference pandas assert, take 2 (#27596)
• Fix CSPE panic in cloud (#27594)
• Fix tz type difference pandas assert (#27593)
• Add contributing note about conventional comments (#27543)
• Nested common subplan elimination (#27340)
• Deduplicate interns (#27470)
• Fix merge conflict in ColumnarFunction (#27464)
• Keep the schema ordered in scan projection pushdown (#27429)
• Remove unused type: ignore statements (#27360)
• Remove redundant PhysNodeKind::AsOfJoin::{left_right}_by fields (#27400)
• Resolve type-ignores in udfs.py (#27341)
• Bump rustls-webpki (#27382)

Thank you to all our contributors for making this release possible!
@0guban0v, @EndPositive, @JakubValtar, @Jesse-Bakker, @Kevin-Patyk, @Liyixin95, @MarcoGorelli, @NedJWestern, @Shoeboxam, @SuryaSunil1326, @TNieuwdorp, @alexander-beedie, @aryansri05, @ashler-herrick, @azimafroozeh, @carnarez, @coastalwhite, @dependabot[bot], @dsprenkels, @gab23r, @gautamvarmadatla, @ilya-pevzner, @jonathansergio, @junnythemarksman, @kdn36, @lun3x, @nameexhaustion, @orlp, @pablogsal, @ritchie46, @uurl, @waamm, @wence-, @wmoss, @xronocode and dependabot[bot]

Python Polars 1.40.1

🚀 Performance improvements

• Skip validity mask processing in __array_ufunc__ when no inputs have nulls (#27358)

✨ Enhancements

• Cargo deny (#27363)
• Add maintain_order parameter to merge_sorted (#27263)

🐞 Bug fixes

• Honor having predicate in GroupBy iter (#27370)
• Use the physical dtype for NumUnorderedImplodeReducer arrow ListArray (#27375)
• Address bug in reduce_balanced for certain input length lists affecting pl.concat (#27352)
• Ensure list.sample() allows fraction > 1 when with_replacement=True (#27350)
• Ensure append() errors when upcast=False (#27346)
• Always rechunk sorts, prune sorts even in eager execution (#27356)
• Fix typing for DataFrame.__init__ and Series.__init__ so they don't require all optional dependencies to be installed (#27348)

📖 Documentation

• Split out openlineage docs into guide and configuration (#27371)
• Add explanation on the observatory sqlite db file (#27354)

🛠️ Other improvements

• Disable mypy type checking for pyarrow calls (#27377)
• Disable debug symbols in macos coverage tests (#27361)
• Cargo deny (#27363)

Thank you to all our contributors for making this release possible!
@EndPositive, @Kevin-Patyk, @MarcoGorelli, @carnarez, @dsprenkels, @gab23r, @jonathanchang31, @kdn36, @mzjp2 and @ritchie46

Python Polars 1.40.0

🏆 Highlights

• Add streaming support for grouped AsOf join (#27293)

⚠️ Deprecations

• Deprecate support for dataframe interchange protocol (#27214)

🚀 Performance improvements

• Create IR slice from expr slice pushdown (#27200)
• Add streaming support for grouped AsOf join (#27293)
• Avoid unnecessary rechunk when sorting already sorted DataFrame (#27264)
• Lower basic over() to streaming primitives (#27303)
• Lower drop_{nulls,nans} in streaming group_by aggregations (#27296)
• Lower entropy to streaming reductions (#27174)
• Add native streaming interpolate (#27185)
• Streaming strptime with format=None (#27056)
• Lower skew / kurtosis to streaming aggregations (#27176)
• Post apply pyarrow filter in Polars' engine instead of pyarrow (#27192)
• Optimize drop_nulls().{first,last}() to {first,last}(ignore_nulls=True) (#27187)
• Always process pyarrow scan in batches (#27183)
• Make cut output Enum and mark as elementwise (#27173)
• Remove unused expression sorts (#27075)
• Use delta stats for mixed hive and non-hive predicate pushdown (#27102)
• Take into account size per row in join sampling (#27098)
• Streaming is_first_distinct and unique(maintain_order=True) (#27052)
• Streaming cov and corr (#27008)
• Add sorted unique node to streaming engine (#26990)
• Ensure Expr.append is lowered in streaming engine (#27022)
• Collapse consecutive Sort nodes (#26965)
• Drop maintain_order=True requirement in sink_delta (#27007)

✨ Enhancements

• Add ignore_nulls to {list,arr}.{any,all} (#27186)
• Lock-free memory manager with spill-to-disk and fully OOC multiplexer (#26774)
• Add is_unique to list/array dtypes (#27290)
• Streaming pyarrow datasets sources (#27230)
• Add pl.merge_sorted operating on multiple frames (#27014)
• Allow group_by() without key exprs (#27141)
• Change default scan/read_lines column name from "lines" to "line" (#27122)
• Make unnest() effective on all columns by default (#27029)
• Collapse consecutive Sort nodes (#26965)

🐞 Bug fixes

• Update groups to correct length for Implode (#27282)
• Fix scan_csv missing_columns='insert' overwrote existing data with NULLs (#27297)
• Raise on non-numeric inputs in pl.int_ranges (#27294)
• Fix always-true filter conversion to Iceberg filter (#27119)
• Do not skip nulls when enumerating over rows in grouped AsOf join (#27275)
• Fix pivot dropping data for null on values (#27273)
• Resolve multiple files deadlock in CSV async reader (#27073)
• Widen decimal precision on sum aggregation (#27270)
• Correct lf.remote type (#27261)
• Default LazyFrame.map_batches to no optimizations (#27262)
• Extend StructEval schema context in StackOptimizer (#27243)
• Preserve nulls when casting from all-null Series to Struct (#27241)
• Fix scan_delta filter on empty dataframe (#27244)
• Prevent DataFrame creation panic on list[struct] with heterogenous types (#27217)
• Named aggregation __structify was being ignored (#27148)
• Skip null group entries when collecting AsOf-by groups (#27215)
• Fix panic with empty order_by in over expression (#27088)
• Write field ID from sink_parquet (#27196)
• Fix statistics for Null columns in Parquet (#27021)
• Do not prune sort nodes containing slice with dyn predicate (#27140)
• Correct grouped Binary arg_min/arg_max and String single-element arg indices (#27172)
• Resolve multiple files deadlock in NDJSON async reader (#27204)
• Overflow panic in interpolate nearest (#27205)
• Using checked arithmetic in int96_to_i64_ns to prevent overflow panic (#27129)
• Don't trigger csv fast count if predicate is pushed down (#27190)
• Support all integer dtypes for Series index assignment (#27188)
• Streaming sort by-expressions were lowered incorrectly (#27158)
• Replace multiprocessing.dummy.Pool with ThreadPoolExecutor (#27175)
• Reset IO metrics instead of consuming (#27156)
• Output SVG if output_path ends with '.svg' in show_graph (#27144)
• Skip extension types for min/max in describe (#27120)
• Address a potential overflow in from_epoch scaling (#27118)
• Fix incorrect IO metrics on multi-phase streaming execution (#27123)
• Use delta stats for mixed hive and non-hive predicate pushdown (#27102)
• Make the files used in docs available locally (#27121)
• Apply scalar bound in clip when the Series bound contains nulls (#27087)
• Ignore ddof parameter in rolling_corr and deprecate (#27104)
• Preserve casts for horizontal ops with untyped literals (#27011)
• Reject invalid input to sql_expr (#27084)
• Ensure SQL COUNT(<lit>) expressions return the correct value (#27085)
• Regression in replace_strict for enums (#27066)
• Make test_group_by_arg_max_boolean_26978 non-flaky for max_by ties (#27048)
• Null count for aggregated list inside count aggregation (#27032)
• Panic in streaming MergeSortedNode (#27024)
• Prevent panic in transpose() with mixed List and non-List columns (#27038)
• Set sorted flag for Boolean and Time (#27035)
• Missing src/ subdirectory to CI Python docs step (#27025)
• Resolve stack overflow on merge_sorted and union (#27018)
• Make pl.DataFrame.fill_null work on columns with Null dtype (#27020)
• Fix repeated word typos in comments (#26917)
• Covariance with constant is zero, not NaN (#27015)
• Don't remove set_sorted in projection pushdown (#27006)
• Infer nulls when df create from empty-struct (#26991)
• Correct suggestion in multi-expr filter error (#27003)
• Implement agg_arg_min/agg_arg_max for boolean data type (#26997)
• Ensure sample() respects the global set seed (#26992)

📖 Documentation

• Add documentation for openlineage on-premises (#27334)
• Release page (#27335)
• Update uv pip install polars-on-premises cmd (#27330)
• Fix outdated LazyGroupBy.map_groups docstring (#27292)
• Add deny_anonymous_users to scheduler config (#27287)
• Slurm documentation (#27259)
• Add link to concepts in index.md (#27077)
• Add docs entry for merge_sorted (#27224)
• Fix typo (#27212)
• Make the files used in docs available locally (#27121)
• Put first-time contribution requirements in its own linkable section (#27113)
• Add missing docstrings for Expr.struct.__getitem__ and Series.__setitem__ (#27092)
• Normalise Series docstring whitespace indents (#27082)
• Change Polars Cloud API to 0.6.0 (#27005)
• Improve write_parquet docstring for use_pyarrow (#26988)

📦 Build system

• Really do not install pyiceberg-core 0.9.0 (#27017)

🛠️ Other improvements

• Add regression test for instantiating polars DataFrame from pandas Timestamp (#27332)
• Bump Python Polars version (#27315)
• Resolve bad instantiations in test_iceberg (#27314)
• Sink DSL and callback for Iceberg (#27258)
• Wait for morsel consumption in merge_sorted streaming node (#27288)
• Use more precise internal typing (pt. iii) (#27232)
• Mark scan_ipc cache arguments as deprecated (#27216)
• Consolidate reordered compare functions (#27229)
• Fix test_dtype_concat_3735 not actually iterating through numeric dtypes (#27178)
• Remove dead code in test_scan_lines (#27213)
• Move/genericize _balanced_reduce to Python utils (#27100)
• Remove unused attributes (#27191)
• Avoid unnecessary recompilation due to changing env vars (#27166)
• Update nightly Rust compiler version (#27145)
• Simplify pyarrow scan and process in batches (#26982)
• Make internal typing more precise (part ii) (#27117)
• Add None & Dataframe to FrameInitTypes (#27126)
• Remove unused expression sorts (#27075)
• Improve internal typing ahead of using ty / pyrefly (#27050)
• Add explicit ResourceWarning coverage (#27083)
• Add sinked paths callback (#26995)
• Pin maturin due to compile time regression (#27062)
• Missing src/ subdirectory to CI Python docs step (#27025)
• Really do not install pyiceberg-core 0.9.0 (#27017)
• Naming for named scopes (#26999)
• Enable hypothesis tests when POLARS_AUTO_NEW_STREAMING=1 (#26818)
• Fix CI by excluding missing wheel version of pyiceberg (#27001)
• Remove indirection in calling python scans (#26981)
• Polars versions (#26980)

Thank you to all our contributors for making this release possible!
@0xRozier, @EndPositive, @HCYT, @Kevin-Patyk, @MarcoGorelli, @NeejWeej, @RedZapdos123, @TNieuwdorp, @abhidotsh, @alexander-beedie, @andyjessen, @azimafroozeh, @borchero, @carnarez, @coastalwhite, @debnathshoham, @dpinol, @dsprenkels, @dydev012, @farouk-01, @gab23r, @gautamvarmadatla, @joaquinhuigomez, @kdn36, @nameexhaustion, @orlp, @ritchie46, @wence-, @xenzh, @yangsong97 and @yonatan-genai

Python Polars 1.39.3

• No changes

Thank you to all our contributors for making this release possible!
@ritchie46

Python Polars 1.39.2

• No changes

Thank you to all our contributors for making this release possible!
@nameexhaustion and @ritchie46