v0.5.0-beta.1

This is a pilot release for 0.5.0 and no further functional changes are intended from now until the 0.5.0 release. Bikeshedding, renaming, and purely architectural/internal rearrangements may still occur, but the only things anyone should have to change would be within the scope of simple find-replace rules. Bugfixes and other non-functional changes will be merged. We do not intend to merge any additional new features.

For newer users

Code written using pgx can now use pgx::prelude::*; instead of use pgx::*; and in many cases this will make the extension compile with only a few additional explicit imports. Existing extensions may require extra effort and may want to approach this more incrementally.

Upgrading

Most of the transition should be smooth, except for certain extensions. You will need to reinstall cargo pgx first: everything regarding SQL generation changed internally, and cargo pgx is the binary that handles most of the actual SQL compilation. We now recommend using cargo install --locked cargo-pgx --version 0.5.0-beta.1 for this.

There are three major pain points you may immediately encounter: code that directly handles datums, date/time types, and set-returning or table-returning functions. But after that compiles, you will also want to address any soundness or behavior questions that might arise regarding Arrays in your code.

Handling Datums

Since 0.5.0-beta.0, pg_sys::Datum is no longer type Datum = usize;, but a "newtype": a struct wrapping a pointer that indicates it is an opaque type. This is for correctness and to better represent the unique nature of the type with respect to Postgres. A fair amount of your code may have relied on T as pg_sys::Datum or datum as T working. Certainly some in pgx did, resulting in patterns like:

// generating fresh arrays for data
let mut datums: Vec<pg_sys::Datum> = vec![0; N];
let mut nulls = vec![false; N];

// treating a datum as a pass-by-value type
let index = datum as i32;

// treating a datum as a pointer type
let varlena = datum as *mut pg_sys::varlena;

Now it doesn't! To make the transition easy, From<T> for pg_sys::Datum is implemented for everything it is reasonable to convert directly into it in a "by-value" fashion. Use IntoDatum for anything more complex. To get the usize hiding inside Datum out, use Datum::value(self) -> usize, to cast it as a pointer, Datum::cast_mut_ptr::<T>(self) -> *mut T is available.

// (getting|putting) data (from|into) Postgres as a datum array
let mut datums = vec![pg_sys::Datum::from(0); N];
let mut nulls = vec![false; N];

// treating a datum as a pass-by-value type
let index = datum.value() as i32;

// treating a datum as a pointer type
let varlena = datum.cast_mut_ptr::<pg_sys::varlena>();

Because a pg_sys::Datum is a union of types, only certain traits that are implemented on usize are also implemented on Datum, as it was deemed safest to limit implementations to those that are also valid if Datum is a pointer or float. This can induce code to reveal any assumptions it was making.

If you were an early upgrader to 0.5.0-beta.0: First, thanks for trying things out! Second, you may have used Datum::ptr_cast: it has the new name of cast_mut_ptr. Datum::to_void was simply dropped as it is unlikely to be what you want, and if it is, cast_mut_ptr::<c_void> works.

Date/Time types

Previously, pgx leaned on the excellent support for dates and times provided by the time crate. Unfortunately, that also meant that we ran into problems like "not representing all time values that Postgres understands". It also meant using heavy conversions when touching these types, even to just copy them out of Postgres, resulting in large amounts of switching between Rust and Postgres during control flow for e.g. constructing a Vec<Timestamp> from a pg_sys::ArrayType that was a handle to a varlena of pg_sys::Timestamp. To allow fixing these performance and correctness problems, pgx now implements these types in terms of the Postgres representations in Rust code.

This also means that all the {Date,Time,Timestamp}{,WithTimeZone}::new functions are now deprecated.

For easy transition, {,Try}{Into,From} for types in the time crate are available with pgx { version = "0.5.0.beta-1", features = ["time-crate"] }.

With thanks to @mhov, @workingjubilee, and @jsaied99's contributions:

• Opaque Date type with +/- infinity support by @mhov in #622
• Opaque Timestamp/Tz types by @mhov in #643
• Match the Postgres repr in Time and TimeWithTimeZone by @workingjubilee in #677
• Adding Clone to Timestamp and Date types by @jsaied99 in #700

Set/Table-returning functions

Functions that previously would be RETURNS SET OF or RETURNS TABLE used to return impl Iterator. Unfortunately, not being able to name the type of the iterator greatly complicated pgx 0.5.0's redesign for SQL generation lead by @Hoverbear and @workingjubilee. pgx::iter::SetOfIterator and pgx::iter::TableIterator now need to be used instead to return these values. In all existing cases you should be able to simply wrap the previous expression in {SetOf,Table}Iterator::new, but for cases where you are returning a single row, TableIterator::once makes for a sweeter alternative.

For example, this:

#[pg_extern]
fn fluffy() -> impl Iterator<Item = i32> {
   vec![1,2,3].into_iter()
}

Becomes this:

#[pg_extern]
fn fluffy() -> SetOfIterator<'static, i32> {
   SetOfIterator::new(vec![1,2,3].into_iter())
}

Array soundness issues

There were several soundness issues regarding interactions between the several ways of constructing pgx::datum::Array<'a, T>, Array::as_slice, and impl<'a, T> Drop for Array<'a, T>. While the critical soundness issue for Drop has been fixed and the resulting leak plugged, these have resulted in the deprecation ofArray::over and Array::as_slice.

Array::as_slice allowed wildly incorrect slices, or viewing data Postgres had marked as "null" in the SQL sense. Instead, this function now panics if either of these conditions would be met, or returns a newly-correct slice for simple data types. Because the usually-trivial fn as_slice panicking is rarely expected, this function is deprecated.

Array::over allows constructing improper Arrays from pointers to random data that is probably controlled by Rust and not Postgres, and thus is data in a form Rust prefers, when Array is supposed to be a zero-copy type that handles ArrayType data in the form Postgres prefers. This makes it almost impossible for the underlying type to be useful for both cases without severely impacting correctness and performance.

Both expose implications about the way that Postgres represents data that are not always true. If you still want to interact with the underlying representation of a Postgres Array and you know the Array was correctly constructed, consider the new experimental pgx::array::RawArray type, created with support from bitvec. It is possible pgx::datum::Array<'a, T> may be replaced entirely in the future by a new type that exposes less assumptions about internal repr.

New features and fixes

In general, PGX added significant improvements for working with uncommon use-cases or existing configurations, with thanks to @steve-chavez, @anth0nyleung, and @TimeToogo.

This includes a soundness fix for the way PGX handled symbols on MacOS thanks to @Smittyvb. This should fix a number of random crashes users were experiencing, but we can't tell for sure (by definition: UB!). Please give it a try and let us know if there are any issues remaining.

We also moved some code into a new crate: pgx-pg-config.

Better soundness

@willmurnane contributed several soundness and correctness improvements, allowing input functions to validate or reject input (now accepting Options).

BackgroundWorker functions may now panic if used from unregistered workers. Thanks to @EdMcBane for catching the previously unsound behavior.

Optional CREATE OR REPLACE FUNCTION

In 0.4.5, PGX switched to using CREATE FUNCTION by default, as this is nondestructive and more secure. However, that prevented migration beyond 0.4.4 for some users, [as they relied on CREATE OR ...

v0.5.0-beta.0

This is an experimental version for the upcoming 0.5.0 release.
Make sure to run cargo install --locked cargo-pgx --version 0.5.0-beta.0.

Handling Datums

pg_sys::Datum is no longer type Datum = usize;, but a "newtype": a struct wrapping a pointer that indicates it is an opaque type. This is for correctness and to better represent the unique nature of the type with respect to Postgres. A fair amount of your code may have relied on T as pg_sys::Datum or datum as T working. Certainly some in pgx did, resulting in patterns like:

// generating fresh arrays for data
let mut datums: Vec<pg_sys::Datum> = vec![0; N];
let mut nulls = vec![false; N];

// treating a datum as a pass-by-value type
let index = datum as i32;

// treating a datum as a pointer type
let varlena = datum as *mut pg_sys::varlena;

Now it doesn't! To make the transition easy, From<T> for pg_sys::Datum is implemented for everything it is reasonable to convert directly into it in a "by-value" fashion. Use IntoDatum for anything more complex. To get the usize hiding inside Datum out, use Datum::value(self) -> usize, to cast it as a pointer, Datum::ptr_cast::<T>(self) -> *mut T is available.

// (getting|putting) data (from|into) Postgres as a datum array
let mut datums = vec![pg_sys::Datum::from(0); N];
let mut nulls = vec![false; N];

// treating a datum as a pass-by-value type
let index = datum.value() as i32;

// treating a datum as a pointer type
let varlena = datum.ptr_cast::<pg_sys::varlena>();

Because a pg_sys::Datum is a union of types, only certain traits that are implemented on usize are also implemented on Datum, as it was deemed safest to limit implementations to those that are also valid if Datum is a pointer or float. This can induce code to reveal any assumptions it was making.

Enhancements

Considerable enhancements have been made to heap tuple and trigger support, introducing pg_trigger and a new PgHeapTuple type! Try them out!

• Improved "HeapTuple" support by @eeeebbbbrrrr in #532
• Trigger support improvements by @Hoverbear in #558

Fixes

• Hang onto libraries by @Hoverbear in #573
  A cargo pgx bug that could cause segfaults on old versions of glibc was fixed by no longer using dlclose on the libraries we dynamically load until the process ends.

Experimental features

You may have noticed the "postgrestd" feature. This is a highly experimental feature for using PGX to back a very special configuration of Rust as a trusted procedural language: PL/Rust. It's best to assume this feature can induce unsound changes if they aren't combined with the rest of the configuration for the trusted language handler's building and running of functions.

Full Changelog: v0.4.5...v0.5.0-beta.0

v0.4.5

This is pgx v0.4.5 It's generally a minor release without backwards incompatible changes.

As usual, make sure to run cargo install cargo-pgx.

What's Changed

• Lockdown semver on pgx deps by @Hoverbear in #559
• Annotate and refactor pgx-tests by @workingjubilee in #563
• Clone less while beginning to unwind by @workingjubilee in #564
• Do not use CREATE OR REPLACE FUNCTION by @Hoverbear in #554
• Remove dubious thread assertion by @workingjubilee in #562
• Pre-patch fixes by @workingjubilee in #568
• Bump to v0.4.5 by @workingjubilee in #569

Full Changelog: v0.4.4...v0.4.5

v0.4.4

This is pgx v0.4.4. A minor release which adds support for "versioned .so" files.

Notable Changes

• Optionally emit versioned .so by @JamesGuthrie in #546
• Add impl Debug for Numeric by @Hoverbear in #551
• Fix namespace.rs safety issues by @eeeebbbbrrrr in #553
• Add "parser/parse_oper.h" to includes by @JamesGuthrie in #549

Minor Changes

• Fix spelling mistakes by @kiwicopple in #548
• Fix url typo on Detailed cargo pgx usage section by @msAlcantara in #550
• Add aggregate article by @Hoverbear in #555

New Contributors

• @kiwicopple made their first contribution in #548
• @msAlcantara made their first contribution in #550

Full Changelog: v0.4.3...v0.4.4

v0.4.3

This is pgx v0.4.3. It brings a number of new code-level features, faster compilation, more PostgreSQL header bindings, and other fun stuff!

When upgrading, please ensure to run cargo install cargo-pgx and to also update your crate dependencies to reference 0.4.3.

What's Changed

General Compilation and Bindings

• #[pg_guard] cleanup by @eeeebbbbrrrr in #526
  • This significantly improves compilation time of the pgx-pg-sys crate
• Add pg_operator.h by @einarmo in #512
• Add support for pg_authid by @nikkhils in #527

cargo-pgx changes

• Remove dead --workspace flag on cargo pgx test by @Hoverbear in #520
• Support --pgcli by @Hoverbear in #538 (see cargo-pgx docs
• Pass RUSTFLAGS to rustc on schema generate by @Hoverbear in #513

Source-code Additions

• Add Copy,Clone for pgx::Timestamp* types by @jamessewell in #509
• Add From<Timestamp> impls by @jamessewell in #510
• Get TimestampWithTimeZone from OffsetDateTime by @einarmo in #539
• Allow compilation on OpenBSD by @ledeuns in #359

Misc

• Updates Github Action runners for PGX by @BradyBonnette in #541
• update copyright headers by @eeeebbbbrrrr in #511

New Contributors

• @BradyBonnette made their first contribution in #541
• @ledeuns made their first contribution in #359

Full Changelog: v0.4.2...v0.4.3

0.4.2

This is pgx v0.4.2. It fixes a few small issues with cargo pgx, adds new Postgres header bindings, and other minor things.

Upgrading

Please make sure to run cargo install cargo-pgx --version 0.4.2 and update all the pgx extension Cargo.toml's pgx* versions to 0.4.2.

What's Changed

• PR #520: The --workspace flag has been removed from cargo pgx test. This was an unintentional addition as a result of #475
• PR #513: We now support compiling on musl-based systems, such as Alpine Linux, by honoring any RUSTFLAGS that might already be set in the environment
• PR #512: Include Postgres' pg_operator.h header in our generated bindings
• PR #510: Various From impls for pgx::Timestamp to pg_sys::timestamp and pgx::TimestampWithTimeZone to pg_sys:: timestamptz
• PR #509: Derive Copy, Clone for pgx' various "Timestamp" types

Thanks!

Thanks to all the contributors and users!

Full Changelog: v0.4.1...v0.4.2

0.4.1

This release includes UX improvements with cargo pgx schema, and performance improvements for Internal and Aggregate.

Upgrading

Please make sure to run cargo install cargo-pgx --version 0.4.1 and update all the pgx extension Cargo.toml's pgx* versions to 0.4.1.

What's Changed

• Internal::get_or_insert_default, and similar functions were doing some unnecessary allocations. Removing them improves the speed of these functions significantly. (#488)
• We now #[inline(always)] on pgx::Aggregate's memory context swapping code. (#487)
• cargo pgx status will by default show the status of all pgx managed PostgreSQLs, instead of the default if one exists in the Cargo.toml. (#505)
• cargo pgx schema was producing color codes outside of the normal xterm-256color codes, this produced unpleasant output on some terminals (notably Mac's terminal.app). When the output of cargo pgx schema is a TTY it will now produce xterm-256color style codes, and use the user's terminal colors. This removes the need to specify if the user was using a light theme. (#507)
• If cargo pgx test experiences a cargo related error, we don't report a full eyre error, we just report the cargo errors. (#491)
• Documentation fixes. (#496)

v0.4.0

This release reworks the SQL generation process present in 0.2.x and 0.3.x, resulting in approximately 50% faster linking times for most commands.

The usability of #[pgx(sql = some_func)] has also improved, as we are now invoking these functions from the shared object itself, instead of some separate binary.

Upgrading

This release requires some manual intervention for all users, as we have updated the SQL generation process. We discovered a new (significantly faster and more safe) way to generate SQL that doesn't require the sql-generator binary or linker scripts.

Additionally, only users of #[pg_extern(sql = some_func)] (and thus using the pgx::datum::sql_entity_graph) should be aware that pgx::datum::sql_entity_graph was merged with pgx_utils::sql_entity_graph and is available at pgx::utils::sql_entity_graph now.

Steps to upgrade:

Please make sure to run cargo install cargo-pgx --version 0.4.0 and update all the pgx extension Cargo.toml's pgx* versions to 0.4.0.

Then:

• Remove .cargo/pgx-linker-script.sh, src/bin/sql-generator.rs.

  rm .cargo/pgx-linker-script.sh src/bin/sql-generator.rs

• Replace .cargo/config with:

  [build]
  # Postgres symbols won't be available until runtime
  rustflags = ["-C", "link-args=-Wl,-undefined,dynamic_lookup"]

• In your Cargo.toml:

    [lib]
  + crate-type = ["cdylib"]
  - crate-type = ["cdylib", "rlib"]

What's Changed

• cargo pgx schema now: (#441, #446, #465, #478, #471, #480, #441)
  1. Uses --message-format=json on some cargo invocations and captures that output.
  2. Generates a 'stub' of the relevant postmaster binary (found via pg_config).
  3. dlopens that 'stub'.
  4. dlopens the extension.
  5. Calls the pgx_utils::sql_entity_graph::PgxSql builders as the sql-generator used to.
• Most cargo pgx commands now support the --package arg matching cargo's.
  Please note unlike cargo, cargo pgx does not support multiple --package args at this time. (#475 )
• pgx::datum::sql_entity_graph and pgx_utils::sql_entity_graph are merged and present in pgx_utils::sql_entity_graph together. (#441)
• cargo pgx schema now outputs to /dev/stdout by default, use --out to specify a destination, or pipe the stdout. (#465 )
• pgx_utils is now exported from at pgx::utils, this is primarily to expose pgx_utils::sql_entity_graph without requiring users have pgx_utils in their Cargo.toml. (#441)
• The sql-generator binary, .cargo/pgx-linker-script.sh and related .cargo/config settings are no longer needed. The old .cargo/config from 0.1.2x is back. (#441)
• pgx_graph_magic!() now expands with __pgx_source_only_sql_mappings and __pgx_typeid_sql_mappings symbols, allowing cargo-pgx to get the appropriate SQL mappings. (#441)
• cargo pgx test and cargo pgx install had several changes to make them more appropriately and cleanly capture and pass build data for this new process. (#441)
• Most sql_graph_entity related functions got a #[doc(hidden)] attribute to clean up their docs. (#441)
• RUST_LOG env vars get correctly passed in pgx test. (#441)
• Some unnecessary dependencies and features were pruned. (#436)
• pgx-pg-sys now only generates bindings it needs, instead of bindings for all postgres verisons. (#455)
• Readme improvements. (#460, #463, #471)
• The development workspace has changed. The pgx-parent crate was removed, the pgx-examples directory members are now workspace members, and the workspaces uses resolver = "2". (#456)
• The pgx-examples no longer point outside the pgx directory then traverse back in. (#462)
• Several new headers are exposed via pgx_pg_sys:
  • tsearch/ts_public.h and tssearch/ts_utils.h (#464)
  • buffile.h (#476)
  • optimizer/appendinfo.h (#479)
• Improved the error messages if a user is missing initdb or tar (#484)
• Improved our Nix support to allow singleStep = false, improving rebuild speed, we also tweaked the Nix CI samples. (#474, #485)
• Fixed pgx-tests failing on doc.rs. (#468)

v0.4.0-beta.0

This release reworks the SQL generation process present in 0.2.x and 0.3.x, resulting in approximately 50% faster linking times for most commands.

The usability of #[pgx(sql = some_func)] has also improved, as we are now invoking these functions from the shared object itself, instead of some separate binary.

Upgrading

This release requires some manual intervention for all users, as we have updated the SQL generation process. We discovered a new (significantly faster and more safe) way to generate SQL that doesn't require the sql-generator binary or linker scripts.

Additionally, only users of #[pg_extern(sql = some_func)] (and thus using the pgx::datum::sql_entity_graph) should be aware that pgx::datum::sql_entity_graph was merged with pgx_utils::sql_entity_graph and is available at pgx::utils::sql_entity_graph now.

Steps to upgrade:

Please make sure to run cargo install cargo-pgx --version 0.4.0-beta.0 and update all the pgx extension Cargo.toml's pgx* versions to 0.4.0-beta.0.

Then:

• Remove .cargo/pgx-linker-script.sh, src/bin/sql-generator.rs.

  rm .cargo/pgx-linker-script.sh src/bin/sql-generator.rs

• Replace .cargo/config with:

  [build]
  # Postgres symbols won't be available until runtime
  rustflags = ["-C", "link-args=-Wl,-undefined,dynamic_lookup"]

• In your Cargo.toml:

    [lib]
  + crate-type = ["cdylib"]
  - crate-type = ["cdylib", "rlib"]

Nix user? Make sure to nix flake lock --update-input pgx as the pgx flake had it's lib.buildPgxExtension updated as well.

What's Changed

• pgx::datum::sql_entity_graph and pgx_utils::sql_entity_graph are merged and present in pgx_utils::sql_entity_graph together. (#441)
• pgx_utils is now exported from pgx at pgx::utils, this is primarily to expose pgx_utils::sql_entity_graph without requiring users have pgx_utils in their Cargo.toml. (#441)
• The sql-generator binary, .cargo/pgx-linker-script.sh and related .cargo/config settings are no longer needed. The old .cargo/config from 0.1.2x is back. (#441)
• pgx_graph_magic!() now expands with __pgx_source_only_sql_mappings and __pgx_typeid_sql_mappings symbols, allowing cargo-pgx to get the appropriate SQL mappings. (#441)
• cargo pgx schema now: (#441)
  1. Uses --message-format=json on some cargo invocations and captures that output.
  2. Processes that JSON to find the OUT_DIR used by the relevant pgx_pg_sys build.
  3. It uses that OUT_DIR data to read, generate, then build (with rustc) a 'stub' shared object. (This gets cached!)
  4. dlopens that shared object.
  5. dlopens the extension.
  6. Calls the pgx_utils::sql_entity_graph::PgxSql builders as the sql-generator used to.
• cargo pgx test and cargo pgx install had several changes to make them more appropriately and cleanly capture and pass build data for this new process. (#441)
• cargo pgx schema, cargo pgx install, and cargo pgx package all had outputs changed. (#441)
• Most sql_graph_entity related functions got a #[doc(hidden)] attribute to clean up their docs. (#441)
• RUST_LOG env vars get correctly passed in pgx test. (#441)
• Some unnecessary dependencies and features were pruned. (#436)
• pgx-pg-sys now only generates bindings it needs, instead of bindings for all postgres verisons. (#455)
• Readme improvements. (#460)
• The development workspace has changed. The pgx-parent crate was removed, the pgx-examples directory members are now workspace members, and the workspaces uses resolver = "2". (#456)

v0.3.3

Upgrading

Please make sure to run cargo install cargo-pgx and update all the pgx extension Cargo.toml's pgx* versions to 0.3.3.

What's Changed

• Resolved the use of a deprecated item in Clap. (#444)
• Fix: The @FUNCTION_NAME@ string fragments inside #[pg_extern(sql = "...@FUNCTION_NAME@...")] (from #410) would not get replaced by the generated wrapper name, as they used to in pgxsql doc comments. (#443)