dev-next

This is the latest development build, which gets updated on every successful build of the master branch.

v0.6.0

This flowctl release includes some exciting new features, and also a few breaking changes to command line options.

The biggest news is that flowctl now supports Python derivations in flowctl preview! We'll be rolling out support for Python derivations on private data planes shortly. Python derivations are not currently supported on our public data planes.

Additionally, the catalog list and catalog history commands now have a --models argument, which can be used in conjunction with --output json|yaml to include the models in the output. This can't be used with the default table output.

This update introduces some breaking changes to flowctl options as we migrate functionality to our GraphQL API. This will help us scale and streamline our workflows for the future.

We tried to limit breaking changes to options that aren't widely used. However, if you use flowctl in automated contexts, it would be wise to check your scripts for usage of any of the following before updating to the latest version of flowctl.

flowctl catalog list:

• --captures, --collections, --materializations, and --tests are now standalone flags rather than accepting boolean arguments. For example, instead of --captures=true, use --captures.
• Each flowctl catalog list command may only include one of the --captures, --collections, --materializations, or --tests options. Run flowctl catalog list multiple times if you need to list multiple types of resources.
• The --data-plane-name argument can only be provided at most once, where previously you could provide multiple values. Run flowctl catalog list multiple times if you need to list specs for multiple specific data planes.
• For those cosmopolitan users who have access to more than 5 different catalog namespaces, you'll now be required to pass an explicit --prefix argument.

flowctl catalog status:

• The format of --output json|yaml has changed, with renamed and moved fields. However, the table output should remain the same.

v0.5.24

Minor updates to support new reduce annotations when running locally.

v0.5.22

What's Changed

This release introduces support for the new redact annotation, which enables blocking or hashing portions of documents very early in capture process.

It also adds a new flowctl discover subcommand, which enables CLI-driven capture creation workflows.

New Contributors

• @danielnelson made their first contribution in #2403

Full Changelog: v0.5.21...v0.5.22

v0.5.21

Includes some minor updates related to flowctl catalog status

v0.5.20

Fixes an issue where flowctl would timeout trying to connect to some data planes.

v0.5.19

Changes the default value of the --default-data-plane argument to point to our shiny new public data plane.

v0.5.18

Updates flowctl to automatically encrypt all endpoint configurations whenever you run catalog publish|test or draft author. It will now use the same mechanism as our UI to encrypt any sensitive fields in the endpoint configs. Note that the local files will not be modified by this. Only the drafted specifications are modified. If you'd like to have your local endpoint config files encrypted, you do so by running pull-specs --overwrite after publishing, or draft develop --overwrite after running draft author.

v0.5.16

Adds support for groupBy in materializations, plus a couple of internal improvements.

v0.5.15

Includes some updates to flowctl catalog status!