Skip to content
This repository was archived by the owner on Apr 17, 2025. It is now read-only.

Release v0.3.0

Choose a tag to compare

View all tags
@danielgafni danielgafni released this 29 Jan 21:18
· 4 commits to master since this release
v0.3.0
fed5350
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

✨ What's Changed

  • feat: add lazyframe support for parquet and delta by @ion-elgreco in #52. sink_df_to_path method can be used to implement LazyFrame sinking
  • cloud_storage_options config field added. Supports passing values with dagster.EnvVar, for example:
PolarsDeltaIOManager(cloud_storage_options={"aws_secret_access_key": EnvVar("AWS_SECRET_ACCESS_KEY")})
  • removed legacy pyarrow parquet reader

💥 Breaking Changes

  • minimal compatible polars version has been increased to 0.20.0
  • BasePolarsUPathIOManager - renamed dump_df_to_path to write_df_to_path

🤝 New Contributors

  • @ion-elgreco made their first contribution in #52. Thank you for your work!

Full Changelog: v0.2.2...v0.3.0

Contributors

ion-elgreco
Assets 2
Loading