Releases: pgcentralfoundation/pgrx
v0.7.0-beta.0
This is pgx v0.7.0-beta.0. Other than pgx' initial release, this is our biggest release yet! It contains a number of safety improvements, segfault fixes, API changes, minor performance improvements, and an overhauled Spi interface now with prepared statements and cursors!
Quite a number of people have contributed to this release, so a big thank you is necessary right here at the top. Some notable names are @yrashk and @EdMcBane for their work on Spi. And a shout-out to the new contributors: @jaskij, @kianmeng, and @ChuckHend.
To use this beta release, install cargo-pgx by version: $ cargo install cargo-pgx --version 0.7.0-beta.0 --locked and make sure to update your extension crate dependencies to use 0.7.0-beta.0 too.
API Changes
v0.7.0 contains some backwards incompatible API changes and your extension code may need to be updated as a result. Leaning on the compiler should help you through the changes.
pgbox::WhoAllocated<T>doesn't need to be generic over anyTby @eeeebbbbrrrr in #901- Lift
UnwindSafe + RefUnwindSafebounds in PgMemoryContexts by @eeeebbbbrrrr in #986 impl Clone for PgTupleDescby @mhov in #962AnyElement::into()needs to be unsafe by @eeeebbbbrrrr in #928- Lets make
pg_guard_ffi_boundarypublic for everyone to enjoy! by @eeeebbbbrrrr in #942 - Mark many functions
unsafethat should be by @eeeebbbbrrrr in #995 - allow
#![forbid(unsafe_op_in_unsafe_fn)]to work with pgx-generated… by @eeeebbbbrrrr in #996 pg_sys::Oidis now a newtype wrapper around au32by @workingjubilee in #1000 & #1004
Spi
Spi has received a major overhaul in v0.7.0. It now supports cursors, prepared statements, and runtime datum conversion error detection.
Much of the Spi API has been overhauled in a backwards-incompatible way, and is too much to document in release notes. However, the biggest change is that the various getter functions on Spi and SpiTupleTable and SpiHeapTupleData now return pgx::spi::Result<Option<T>>. The error type is pgx::spi::Error and can not only report Postgres-specific SPI_ERROR_XXX errors from Postgres but also represent runtime situations where the requested Rust type T isn't compatible with the backing Datum type.
Cursor Support Changes
- Add support for Spi cursors by @EdMcBane in #579
- ensure multiple open SpiTupTables can coexist by @EdMcBane in #938
- Cursor API should not require
&self mutby @yrashk in #934 - Minor code improvement for open_cursor by @yrashk in #935
Prepared Statement Support
Spi Statement "readonly" Management
- Make non-updating queries use
readonlyuntil transaction end unless there was a prior mutation by @yrashk/@eeeebbbbrrrr in #963 & #992 fix issue #983 by @eeeebbbbrrrr in #984(was replaced by #992)
Spi Error Handling
Because most Spi-related functions now return spi::Result<T>, pgx now knows how to convert a Result<T: IntoDatum, E: Any + Display> into a Datum. This means #[pg_extern]-style functions (including #[pg_test] functions) can now return a Result. This makes working with Spi much more "rust-like" and fluent.
If result is Ok then its payload is converted into a Datum. If the result is Err, then pgx will automatically raise a Postgres ERROR. Furthermore, if the Err variant's payload is a pgx ErrorReport, then that'll be raised, which can include a specific SQL error code, detail, hint, and context message.
The general work on Spi error handling happened in these PRs, with much help from @yrashk and @EdMcBane:
- Build upon #902 by @yrashk: Spi Error Handling by @eeeebbbbrrrr in #969
- Proper error handling for SPI by @eeeebbbbrrrr in #977
- Add type/oid details to Datum conversion errors by @EdMcBane in #985
And the generalized support for handling Results-as-Datums came in through:
impl IntoDatum for Result<T: IntoDatum, E: Display>by @eeeebbbbrrrr in #972- Change bounds on
IntoDatum for Result<T, E>such thatE: Any + Displayby @eeeebbbbrrrr in #973 - Support returning
Result<TableIterator/SetOfIterator, E>by @eeeebbbbrrrr in #975 - Fix a bug trying to return
Result<pg_sys::Oid>by @eeeebbbbrrrr in #976
General Cleanups
- Improve the safety documentation for
pg_guard_ffi_boundaryby @thomcc in #957 - Fix typos by @kianmeng in #959
- cargo-pgx: deny clippy::perf by @workingjubilee in #933
- Silence clippy in
pgx-version-updaterby @workingjubilee in #930 - sql-entity-graph: fix clippy::if_same_then_else by @workingjubilee in #931
- Fix pgx-sql-entity-graph impls of derivable traits. by @thomcc in #929
pgx-utilscrate renamed topgx-sql-entity-graphby @eeeebbbbrrrr in #911- pgx-pg-sys: cshim: Makefile: user AR variable by @jaskij in #937
- Fully qualify all pgx symbol references during code generation by @eeeebbbbrrrr in #925
- fully qualify the path to #[pg_guard] during code generation by @eeeebbbbrrrr in #943
- Document and simplify
cargo-pgxversion matching requirement by @yrashk in #964 - Remove specific versions from getting started section by @rustprooflabs in #991
- Revert "Revert "Replace rayon"" by @workingjubilee in #945
- localize some use statements by @eeeebbbbrrrr in #944
- add doc to pg_test module in template by @ChuckHend in #970
- fix compilation on arm by @eeeebbbbrrrr in #993
- Preliminary support for cross-compilation on Linux<-->Linux for x86_64 to aarch64 by @thomcc in #1003
This one is particularly interesting as it now allows cargo test --all --features "pgXX ... ..." to work from a top-level workspace crate:
- Teach
pgx-tests/src/framework.rsto detectcargo testfeature arguments by @eeeebbbbrrrr in #967
Stability, Correctness, and Performance
- Problem: can't use
pg_guardon parameterized functions by @yrashk in #918 - Handle panics in PgMemoryContexts::switch_to by @yrashk in #920
- Ensure setting owned memory context as current twice won't lead to problems (bugfix) by @yrashk in #956
- Improve error handling during code generation by converting some panics into
syn::Errors by @thomcc in #919 - Set returning
#[pg_extern]functions can cause segfault by @eeeebbbbrrrr in #982- Move generated SRF code into managed code by @eeeebbbbrrrr in #987
- PgLwLock: skip releasing the lock when unwinding from an
elogcall in postgres code by @EdMcBane in #989 - :enh: bail out early in do_ereport if errstart returns false by @EdMcBane in #980
- Rewrite
StringInfoby @eeeebbbbrrrr in #903 - Optimize our String/&str/Vec/&[u8] IntoDatum conversions by @eeeebbbbrrrr in #952
Feature Flags
- A
pgx/no-schema-generationfeature flag by @eeeebbbbrrrr in #915
When enabled, pgx will not generate any code related to the "sql entity graph", thereby causing cargo-pgx to generate an empty schema.
- Add a default
pgx/cshimfeature flag by @eeeebbbbrrrr in #958 - The
cshimfeature should not be a default inpgx-pg-sys. by @eeeebbbbrrrr in #990
When enabled (which is the default), pgx (specifically, pgx-pg-sys) will compile its "cshim" code and have it statically linked into the extension. Disabling this feature flag might make pgx compilation easier for platforms where the necessary Postgres C extension build requirements aren't available, however you'll lose access to a few pgx modules: hooks, list, namespace and spinlock.
Our ultimate goal is to not have a "cshim" at all, and to that end much of the old "cshim" has been ported to Rust:
- Port c-shim.c
heap_getattr,HeapTupleHeaderGetXmin, andHeapTupleHeaderGetRawCommandIdto Rust by @eeeebbbbrrrr in #953 - Port c-shim.c
pgx_GETSTRUCTandpgx_HeapTupleHeaderGetOidto Rust by @eeeebbbbrrrr in #950 - Port various c-shim.c
SET_VARSIZE*functions to Rust by @eeeebbbbrrrr in #949 PgMemoryContexts::get_context_for_pointer()is wildly unsafe. by @eeeebbbbrrrr in #947- Port
c-shim.c'spgx_ereport()function to rust by @eeeebbbbrrrr in #946 - Port various c-shim.c
ARR_*...
v0.6.1
Welcome to pgx v0.6.1. This is a minor release that fixes a few safety/crashing bugs along with fixing up the --profile argument to cargo pgx.
What's Changed
cleanup cargo-pgx handling of the --profile argument by @eeeebbbbrrrr in #908
v0.6.0 introduced the --profile argument. While it works for any of the targets such as run, test, install, it was primary intended for cargo pgx package so that a specific, "more optimized" release profile could be used. It of course, didn't work and caused cargo to raise an error about conflicting arguments. It now works as expected.
Ensuring owned memory context is not current when dropping it (extension) by @yrashk in #922
@yrashk found a bug that would crash Postgres when Rust drops a PgMemoryContexts::Owned that was also set as the pg_sys::CurrentMemoryContext.
Note that this introduces a minor API breakage as the function
PgMemoryContexts::set_as_current()now takes&mut selfinstead of&self.
Handle panics in PgMemoryContexts::switch_to by @yrashk in #920
@yrashk found another bug that would crash Postgres if a panic occurred while inside a PgMemoryContexts::switch_to(|| ... ) closure.
#[pg_guard] can now be applied to generic functions by @yrashk in #918
Additionally, pgx will emit a compiler error if you try to attach both #[pg_guard] and #[no_mangle] to the same extern "C" function which is generic over anything but a lifetime.
Once again, thanks @yrashk!
Full Changelog: v0.6.0...v0.6.1
v0.6.0
This is the full release of pgx 0.6.0!
Remember to rustup update and then cargo install cargo-pgx, but that also brings us to the first two updates!
It's important to remember to update your rustc installation first because the rustc version must be the same as what is used to build cargo pgx and for compiling crates. We now we enforce that thanks to thomcc.
You may now build cargo pgx with rustls thanks to @felipe-vaultree, using:
cargo install cargo-pgx \
--locked \
--version 0.6.0 \
--features rustlsrustls can help if the "native TLS" cargo pgx would default to is OpenSSL.
This is not a comment on OpenSSL's security, only on ease of installing and using it.
New Logo!
Check it out! We added a new logo by Cenza Della Donna!
Breaking Changes
This version brings a number of breaking changes.
Postgres Major Support
This release introduces support for Postgres 15 and drops support for Postgres 10, thanks to @Smittyvb, @BradyBonnette, @yrashk, and @steve-chavez!
Numeric
Thanks to @eeeebbbbrrrr, proper support for SQL's NUMERIC has landed in PGX, using two types:
AnyNumericNumeric<const P: u32, const S: u32>
Obviously, this breaks code using Numeric as it currently is, which is more like AnyNumeric.
Usage Note: As far as we are able to discern, AnyNumeric is the type used by Postgres on SQL function entry and exit, even if you specify a precision and scale in SQL, so for the case of #[pg_extern] you should probably prefer AnyNumeric, and deliberately convert to Numeric<P, S> inside functions if needed. Numeric<P,S> helps most with serialization, writing to tables, and other cases Postgres also uses strict precision and scale.
Error Handling has been revamped
Error handling has been revamped in a way that may be breaking if you were directly handling PGX's error-handling functions, which are intended to be used when you extend this with your own error-handling abstractions. This includes a lot more support for the kinds of patterns allowed by PG_TRY and PG_CATCH. Take a look at PgTryBuilder.
SpiClient now discourages misuse, plus other Spi* improvements
@yrashk and @thomcc have significantly revised the way that SpiClient works, although a lot of users won't notice a difference. It now uses a scoped lifetime, existing as more of a consumable token, rather than something you can just arbitrarily instantiate. Also see:
- Ensure SpiClient is only used within boundaries of a connection by @yrashk in #896
SpiClient::updateno longer requires&mut selfby @yrashk in #897- Non-leaking, consumable SpiClient by @yrashk in #898
- Ensure the lifetime of
SpiClientis scoped to the connection by @thomcc in #900
SpiHeapTupleDataEntrygains an::oidgetter thanks to @workingjubileeSpiTupleTableexposes some getters for information oncolumns, etc. thanks to @yrashkSpi::connectcan now return values without theFromDatum + IntoDatumbound.- We now have
{run,explain}_with_argsthanks to @montanalow
BackgroundWorker::transaction fixes
@yrashk expanded the ability of various contexts to allow data to escape them: background workers can return values from transaction, and no longer require Copy to do so.
PgNode is now sealed
This probably won't affect most users, but PgNode is now a "sealed trait", preventing you from implementing it on things. Sorry! PGX relies on the soundness of its implementation, and probably will rely on that more in the future.
PgRelation will promise less
Rust functions that are safe should not cause UB with arbitrary inputs, so @workingjubilee made PgRelation::with_lock unsafe in #886. Thanks to @JohnHVancouver for catching that one!
Deprecated code
Most deprecations that began in 0.5.0 have now been made good on:
Array::overis no more 🎉. This will allow us to speed upArray<'a, T>in the future and make it more sound.- Most of the
::newfunctions on datetime types.
You can still opt in or out of the time crate as a dependency using
pgx = { version = "0.6.0", features = "time-crate" }
This will let you use various TryFrom or From impls.
typo lol
The indicies function now is indices thanks to @jteplitz. This is technically a breaking change!
New in 0.6.0
pgx::datum::Range<T>
We now can map "range types" between Rust and Postgres thanks to @mhov!
pgx now defends against FFI while multithreading
It's now almost reasonable to offload work onto multiple threads while using PGX, as @thomcc gave us a true check for calling FFI from multiple threads that is more resilient to being fooled by renaming threads or other oddness you might get up to. @eeeebbbbrrrr made pg_sys functions also #[track_caller] to help detect and track this. Remember, however, that Postgres is fundamentally single-threaded, so all interaction with Postgres must be on a single thread... and whether PGX functions call into Postgres may change between versions without warning!
- Include caller location in error message when pgx detects a Postgres FFI was called on a not-the-main-thread by @eeeebbbbrrrr in #862
- "ereport" messages (
pgx::log!(), pgx::error!(), etc) no longer add a CONTEXT message to the ereport by @eeeebbbbrrrr in #875
pgx-pg-sys build improvements
Pertinent to our Postgres version support updates, @BradyBonnette taught pgx-pg-sys how to warn if you use an unsupported version of Postgres.
We also gained some bindings:
- Include extension & namespace catalogs by @yrashk in #836
- Add bindings for typedefs within replication/logical.h to pg_sys by @agamble in #827
- add the
catalog/indexing.hheader by @eeeebbbbrrrr in #874
Compatibility with various environments
cargo pgx will now configure databases to use C.UTF-8 locale when it creates them (e.g. for testing) thanks to @Smittyvb, or use the "C" locale instead as a fallback.
/And we now also test multiple distros thanks to @BradyBonnette!
Thanks to @pcnc we caught some AArch64 compatibility issues related to c_char.
Miscellaneous Improvements
Most other changes are more subtle quality of life improvements:
- There are now safe wrappers for
SpinLockthanks to @thomcc - We can now handle your esoteric patched Postgres version string thanks to @jteplitz
- Thanks to @thomcc
#[pg_guard]should now preserve attributes - @jteplitz added support for a new PgHook
post_parse_analyze - PostgresType now supports Rust enums as well thanks to @yrashk
- @eeeebbbbrrrr made
direct_function_callhave less overhead - Improve pgx-test's shutdown handling by @thomcc in #833
Documentation and example updates
A fair amount of PGX documentation got some polish.
- Update
PgTryBuilderdocs by @yrashk in #865 - Provide an example of
PostgresTypederivation on enums by @yrashk in #861 - Document
pg_extern(name)by @yrashk in #868 - Thanks to @MaxKingPor we found the macro docs still mentioned
impl Iterator! It now recommendsTableIteratorappropriately. Please let us know if you find any issues with the PGX documentation or have a question!
New Contributors
- @jteplitz made their first contribution in #788
- @montanalow made their first contribution in #689
- @felipe-vaultree made their first contribution in #842
- @agamble made their first contribution in #827
- @pcnc made their first contribution in #870
Full Changelog: v0.5.6...v0.6.0
v0.6.0-alpha.2
This is pgx v0.6.0-alpha.2. It's mainly a followup to alpha.1 with issues reported from the wild.
What's Changed
- Include caller location in error message when pgx detects a Postgres FFI was called on a not-the-main-thread by @eeeebbbbrrrr in #862
- Remove the
BackgroundWorker::transactionR: Copybound by @yrashk in #863 - pgx compiles under rustc v1.61 by @thomcc in #867
- Update
PgTryBuilderdocs by @yrashk in #865 - Remove stale references to "ZomboDB" by @workingjubilee in #859
- Provide an example of
PostgresTypederivation on enums by @yrashk in #861 - Use
rustlsas the cargo-pgx feature name by @workingjubilee in #860 - Aarch64 compatibility fixes related to
c_charby @pcnc in #870 - "ereport" messages (
pgx::log!(), pgx::error!(), etc) no longer add a CONTEXT message to the ereport by @eeeebbbbrrrr in #875 - add the
catalog/indexing.hheader by @eeeebbbbrrrr in #874 - Add a Rust version check to cargo-pgx by @thomcc in #873
- new logo by Cenza Della Donna in #876
- Document
pg_extern(name)by @yrashk in #868 Range<T>Type mapping by @mhov in #663- Improve pgx-test's shutdown handling by @thomcc in #833
New Contributors
v0.6.0-alpha.0
Welcome to the first prerelease of pgx 0.6.0!
Remember to use cargo install cargo-pgx --version 0.6.0-alpha.0 --locked!
Postgres Major Support
This release introduces support for Postgres 15 and drops support for Postgres 10, thanks to @Smittyvb, @BradyBonnette, @yrashk, and @steve-chavez.
Numeric
Thanks to @eeeebbbbrrrr, proper support for SQL's NUMERIC has landed in PGX, using two types:
AnyNumericNumeric<const P: u32, const S: u32>
Obviously, this breaks code using Numeric as it currently is, which is more like AnyNumeric.
Usage Note: As far as we are able to discern, AnyNumeric is the type used by Postgres on SQL function entry and exit, even if you specify a precision and scale in SQL, so for the case of #[pg_extern] you should probably prefer AnyNumeric, and deliberately convert to Numeric<P, S> inside functions if needed. Numeric<P,S> helps most with serialization, writing to tables, and other cases Postgres also uses strict precision and scale.
Error Handling has been revamped
Error handling has been revamped in a way that may be breaking if you were directly handling PGX's error-handling functions, which are intended to be used when you extend this with your own error-handling abstractions. This includes a lot more support for the kinds of patterns allowed by PG_TRY and PG_CATCH. Take a look at PgTryBuilder.
cargo pgx can use rustls!
You can now configure cargo-pgx to use rustls instead of native SSL, but you must intentionally build it that way for now.
Data can now actually return!
@yrashk expanded the ability of various contexts to allow data to escape them: background workers can return values from transaction, and Spi::connect can now return values without the FromDatum + IntoDatum bound.
Performance with soundness?
- It's now almost reasonable to use PGX with multithreading, as @thomcc gave us a true check for calling FFI from multiple threads that is more resilient to being fooled by renaming threads or other oddness you might get up to.
Array::overis no more 🎉 andArrayis now slightly faster thanks to @workingjubilee. More changes will bring further improvements.- @eeeebbbbrrrr made
direct_function_callhave less overhead
pgx-pg-sys build improvements
Pertinent to our Postgres version support updates, @BradyBonnette taught pgx-pg-sys how to warn if you use an unsupported version of Postgres. @thomcc also removed our need for rayon to build. We also gained some bindings:
- Include extension & namespace catalogs by @yrashk in #836
- Add bindings for typedefs within replication/logical.h to pg_sys by @agamble in #827
Miscellaneous Improvements
Most other changes are more subtle quality of life improvements:
- There are now safe wrappers for
SpinLockthanks to @thomcc SpiHeapTupleDataEntrygains an::oidgetter thanks to @workingjubileecore::ffi::CStris now available in Rust, and now PGX uses it consistently throughout API thanks to @thomccSpiTupleTableexposes some getters for information oncolumns, etc. thanks to @yrashk- We can now handle your esoteric patched Postgres version string thanks to @jteplitz
- PGX will now configure databases to use C.UTF-8 locale when it creates them thanks to @Smittyvb
- Thanks to @thomcc
#[pg_guard]should now preserve attributes - We now have
{run,explain}_with_argsthanks to @montanalow - We now test multiple distros thanks to @BradyBonnette!
- The
indiciesfunction now isindicesthanks to @jteplitz - @jteplitz added support for a new PgHook
post_parse_analyze - PostgresType now supports Rust enums as well thanks to @yrashk
New Contributors
- @jteplitz made their first contribution in #788
- @montanalow made their first contribution in #689
- @felipe-vaultree made their first contribution in #842
- @agamble made their first contribution in #827
Full Changelog: v0.5.6...v0.6.0-alpha.0
v0.5.6
This is pgx v0.5.6. It is a small bugfix release that includes the fix from PR #807.
Please cargo install --locked cargo-pgx and update your Cargo.toml:
pgx = "0.5.6"Fixed Bugs
- Functions using the
name!()macro in their return type (returning aTableIteratoror(rust, tuple)) can now use Rust reserved keywords as the first "column name" argument. This was a regression introduced in 0.5.3.
Full Changelog: v0.5.4...v0.5.6
v0.5.4
pgx v0.5.4 is out, please cargo install --locked cargo-pgx and update your Cargo.toml:
pgx = "0.5.4"This is a fairly small release, containing a bugfix and some new API surface:
- An incorrect handling of some Datum arguments fixed thanks to @eeeebbbbrrrr in #795
- Some missed SpinLock macro bindings were added thanks to @thomcc in #793
- A new base API for dynamic background worker registration was introduced thanks to @yrashk in #710!
We're excited to see further improvements to the Dynamic Background Worker API and are looking forward to any feedback you have!
Full Changelog: v0.5.3...v0.5.4
v0.5.3
What was presumed to be a no-op change after testing on multiple operating systems nonetheless broke the build on Red Hat distros. We're looking into adding another distro (presumably one that is fond of fedoras?) in our CI just to prevent this from happening again and allow us to better anticipate repackages of Postgres with a nonzero number of quirks. This release primarily is to fix that by removing allowlist_* from our bindgen invocation for now, thanks to @BradyBonnette in #760.
It also fixes a use-after-free that PGX was inducing in Postgres: Datum arguments were being unboxed (into... new boxes, usually) during set-returning functions, but not using the multi-call memory context. This was fixed thanks to @eeeebbbbrrrr in #784
However, it does have a number of other changes.
Less unnecessary dynamic linkage
Some of the dynamic linkage PGX was asking for was largely unnecessary. In new PGX projects, you should now be able to use exotic linkers which don't accept various command line arguments other linkers do... on Linux, as the dynamic lookup argument remains necessary on MacOS (for now), thanks to "Only specify -undefined dynamic_lookup on macOS" by @thomcc in #754
To get this benefit in an existing project, replace the existing [build] key in your ./.cargo/config{,.toml} with
[target.'cfg(target_os="macos")']
# Postgres symbols won't be available until runtime
rustflags = ["-Clink-arg=-Wl,-undefined,dynamic_lookup"]cargo-pgx improvements
cargo pgx init now accepts the flags --base-port and --base-testing-port, thanks to "Problem: working with independent extensions simultaneously" by @yrashk in #765. These values can be later changed in $PGX_HOME/config.toml. This is intended to make it easier to work with multiple extensions at once.
We now support --profile <profile name> in cargo-pgx thanks to @thomcc in #758, to allow you to better tune the testing and packaging profiles you may want to use.
New pg_sys bindings
- Enable low-level access to the database catalog by @yrashk in #709
- Added includes for "storage/spin.h" by @osawyerr in #775
Other minor fixups
- We've stopped distorting the signatures of variadic functions thanks to @thomcc in #757
- The
--bgworkertemplate was found to have fallen out of sync and was patched by @workingjubilee in #768 - Remove lifetimes from FunctionMetadata impl by @thomcc in #772
- derive Debug for PgLogLevel and PgSqlErrorCode by @yrashk in #786
- Make syntect an optional dependency, only used by cargo-pgx by @thomcc in #755
- Avoid some dependencies in pgx-pg-sys by @thomcc in #756
New Contributors
Full Changelog: v0.5.2...v0.5.3
v0.5.2
This is pgx v0.5.2. It is a very minor release that fixes issue #762.
Full Changelog: v0.5.0...v0.5.2
v0.5.0
This is the release of pgx 0.5.0! Special thanks to @BradyBonnette, @JamesGuthrie, @Smittyvb, and @willmurnane for contributing fixes for bugs that we noticed in the last round of testing for 0.5.0-beta.1. Most of these release notes will be a reprise of those if you have seen them already. For everyone else, welcome aboard!
With a rustup or cargo toolchain installed, you can download pgx from crates.io, just cargo install --locked cargo-pgx --version 0.5.0 and add this to your Cargo.toml:
pgx = "0.5.0"API documentation is available on docs.rs.
For newer users
Code written using pgx can now use pgx::prelude::*; instead of use pgx::*; and in many cases this will make the extension compile with only a few additional explicit imports. Existing extensions may require extra effort and may want to approach this more incrementally.
Upgrading
Most of the transition should be smooth, except for certain extensions. You will need to reinstall cargo pgx first: everything regarding SQL generation changed internally, and cargo pgx is the binary that handles most of the actual SQL compilation. We now recommend using cargo install --locked cargo-pgx --version 0.5.0 for this.
There are four major pain points you may immediately encounter: code that directly handles datums, using FromDatum, date/time types, and set-returning or table-returning functions. But after that compiles, you will also want to address any soundness or behavior questions that might arise regarding Arrays in your code.
Handling Datums
Since 0.5.0-beta.0, pg_sys::Datum is no longer type Datum = usize;, but a "newtype": a struct wrapping a pointer that indicates it is an opaque type. This is for correctness and to better represent the unique nature of the type with respect to Postgres. A fair amount of your code may have relied on T as pg_sys::Datum or datum as T working. Certainly some in pgx did, resulting in patterns like:
// generating fresh arrays for data
let mut datums: Vec<pg_sys::Datum> = vec![0; N];
let mut nulls = vec![false; N];
// treating a datum as a pass-by-value type
let index = datum as i32;
// treating a datum as a pointer type
let varlena = datum as *mut pg_sys::varlena;Now it doesn't! To make the transition easy, From<T> for pg_sys::Datum is implemented for everything it is reasonable to convert directly into it in a "by-value" fashion. Use IntoDatum for anything more complex. To get the usize hiding inside Datum out, use Datum::value(self) -> usize, to cast it as a pointer, Datum::cast_mut_ptr::<T>(self) -> *mut T is available.
// (getting|putting) data (from|into) Postgres as a datum array
let mut datums = vec![pg_sys::Datum::from(0); N];
let mut nulls = vec![false; N];
// treating a datum as a pass-by-value type
let index = datum.value() as i32;
// treating a datum as a pointer type
let varlena = datum.cast_mut_ptr::<pg_sys::varlena>();Because a pg_sys::Datum is a union of types, only certain traits that are implemented on usize are also implemented on Datum, as it was deemed safest to limit implementations to those that are also valid if Datum is a pointer or float. This can induce code to reveal any assumptions it was making.
If you were an early upgrader to 0.5.0-beta.0: First, thanks for trying things out! Second, you may have used Datum::ptr_cast: it has the new name of cast_mut_ptr. Datum::to_void was simply dropped as it is unlikely to be what you want, and if it is, cast_mut_ptr::<c_void> works.
FromDatum::from_polymorphic_datum
FromDatum::from_polymorphic_datum now holds the behavior of FromDatum::from_datum and a new FromDatum::from_datum which only accepts 2 arguments (omitting the pg_sys::Oid argument) has been added. By default, the new from_datum calls from_polymorphic_datum with pg_sys::InvalidOid as the third argument. Most people use the provided implementations, but you should override this if you have a custom impl of FromDatum for a polymorphic-like type like AnyElement, or ignore the third parameter in your implementation of FromDatum::from_polymorphic_datum if you don't.
Otherwise, you should be fine with a find-replace for the most part.
Date/Time types
Previously, pgx leaned on the excellent support for dates and times provided by the time crate. Unfortunately, that also meant that we ran into problems like "not representing all time values that Postgres understands". It also meant using heavy conversions when touching these types, even to just copy them out of Postgres, resulting in large amounts of switching between Rust and Postgres during control flow for e.g. constructing a Vec<Timestamp> from a pg_sys::ArrayType that was a handle to a varlena of pg_sys::Timestamp. To allow fixing these performance and correctness problems, pgx now implements these types in terms of the Postgres representations in Rust code.
This also means that all the {Date,Time,Timestamp}{,WithTimeZone}::new functions are now deprecated.
For easy transition, {,Try}{Into,From} for types in the time crate are available with pgx { version = "0.5.0.beta-1", features = ["time-crate"] }.
With thanks to @mhov, @workingjubilee, and @jsaied99's contributions:
- Opaque Date type with +/- infinity support by @mhov in #622
- Opaque Timestamp/Tz types by @mhov in #643
- Match the Postgres repr in Time and TimeWithTimeZone by @workingjubilee in #677
- Adding Clone to Timestamp and Date types by @jsaied99 in #700
Set/Table-returning functions
Functions that previously would be RETURNS SET OF or RETURNS TABLE used to return impl Iterator. Unfortunately, not being able to name the type of the iterator greatly complicated pgx 0.5.0's redesign for SQL generation lead by @Hoverbear and @workingjubilee. pgx::iter::SetOfIterator and pgx::iter::TableIterator now need to be used instead to return these values. In all existing cases you should be able to simply wrap the previous expression in {SetOf,Table}Iterator::new, but for cases where you are returning a single row, TableIterator::once makes for a sweeter alternative.
For example, this:
#[pg_extern]
fn fluffy() -> impl Iterator<Item = i32> {
vec![1,2,3].into_iter()
}Becomes this:
#[pg_extern]
fn fluffy() -> SetOfIterator<'static, i32> {
SetOfIterator::new(vec![1,2,3].into_iter())
}Array soundness issues
There were several soundness issues regarding interactions between the several ways of constructing pgx::datum::Array<'a, T>, Array::as_slice, and impl<'a, T> Drop for Array<'a, T>. While the critical soundness issue for Drop has been fixed and the resulting leak plugged, these have resulted in the deprecation ofArray::over and Array::as_slice.
Array::as_slice allowed wildly incorrect slices, or viewing data Postgres had marked as "null" in the SQL sense. Instead, this function now panics if either of these conditions would be met, or returns a newly-correct slice for simple data types. Because the usually-trivial fn as_slice panicking is rarely expected, this function is deprecated.
Array::over allows constructing improper Arrays from pointers to random data that is probably controlled by Rust and not Postgres, and thus is data in a form Rust prefers, when Array is supposed to be a zero-copy type that handles ArrayType data in the form Postgres prefers. This makes it almost impossible for the underlying type to be useful for both cases without severely impacting correctness and performance.
Both expose implications about the way that Postgres represents data that are not always true. If you still want to interact with the underlying representation of a Postgres Array and you know the Array was correctly constructed, consider the new experimental pgx::array::RawArray type, created with support from bitvec. It is possible pgx::datum::Array<'a, T> may be replaced entirely in the future by a new type that exposes less assumptions about internal repr.
New features and fixes
In general, PGX added significant improvements for working with uncommon use-cases or existing configurations, with thanks to @steve-chavez, @anth0nyleung, and @TimeToogo.
We also moved some code into a new crate: pgx-pg-config.
Fixes in cargo pgx and pgx-pg-sys
- Hang onto libraries by @Hoverbear in #573
Acargo pgxbug that could cause segfaults on old versions of glibc was fixed by no longer usingdlcloseon the libraries we dynamically load until the process ends.
We also found and fixed some unsound behavior in the way PGX handled symbols on MacOS thanks to @Smittyvb. T...
