Skip to content

Releases: sequinstream/sequin

Release v0.11.0

26 Jun 01:02
Compare
Choose a tag to compare

v0.11.0

There is (1) breaking change in this release and it affects NATS users only.

(1) ⚠️ Breaking change for NATS users only

NATS subject pattern change

We changed the default NATS subject naming pattern to bring it in line with our other sinks.

The default NATS subject naming pattern has changed from:

Old pattern (v0.10.x and earlier):

sequin.changes.<database>.<schema>.<table>.<action>
sequin.rows.<database>.<schema>.<table>

New pattern (v0.11.0+):

sequin.<database>.<schema>.<table>.<action>

Example

If you were streaming changes from a products table in the public schema of a database called shop, subjects have changed from:

Before:

  • sequin.changes.shop.public.products.insert
  • sequin.changes.shop.public.products.update
  • sequin.changes.shop.public.products.delete

After:

  • sequin.shop.public.products.insert
  • sequin.shop.public.products.update
  • sequin.shop.public.products.delete

Migration for NATS users

Option 1: Update your NATS subscriptions

Update your NATS subscribers to use the new subject pattern:

# Old subscription
nats sub "sequin.changes.>"

# New subscription  
nats sub "sequin.>"

Option 2: Use a routing function to maintain the old pattern

If you prefer to keep the old subject pattern, you can use a routing function in your NATS sink:

def route(action, _record, _changes, metadata) do
  %{
    subject: "sequin.changes.#{metadata.database_name}.#{metadata.table_schema}.#{metadata.table_name}.#{action}",
  }
end

This allows you to customize the subject pattern to match your existing infrastructure without changing your NATS subscribers.

Need help?

Join our Discord or Slack communities if you need assistance with this migration.

Features

Improvements

  • Collate noisy logs into single, debounced log lines.
  • Fixed logs around receiving first message from slot. Now, we will always only log one time.

Full Changelog: v0.10.8...v0.11.0

Release v0.10.8

25 Jun 01:08
Compare
Choose a tag to compare

Release notes for v0.10.8

What's Changed

  • add Meilisearch sink by @fcannizzaro in #1823
  • 💄 Surface better error when a publication is in an invalid state by @acco in #1844
  • 🗑️ Cleanup legacy_event consumer features by @RTLS in #1860
  • June 20 changelog by @thisisgoldman in #1858
  • ⚡️ [gcp pubsub] Improve throughput by fixing batching and ordering key by @acco in #1862

New Contributors

Full Changelog: v0.10.7...v0.10.8

Release v0.10.7

24 Jun 03:06
Compare
Choose a tag to compare

Release notes for v0.10.7

⚠️ Legacy webhook payloads are deprecated

For accounts created before 2024-11-06 and using Webhook sinks:

We have deprecated an older payload version which was active for webhook sinks in accounts created earlier than 2024-11-06. If this describes your webhook sink, the only change required to safely migrate is to set batch = false in the sink configuration card of your webhook sinks.

With batch = false set, you can safely migrate to v0.10.7 without changes to your delivered payloads.

What's Changed

  • 🗑️ Remove :legacy_event transforms in http push pipeline by @RTLS in #1859

Full Changelog: v0.10.6...v0.10.7

Release v0.10.6

22 Jun 16:27
Compare
Choose a tag to compare

Release notes for v0.10.6

What's Changed

  • ⚙️ Set low timeout on filtered messages batcher by @RTLS in #1853
  • ✨ Fail backfills without PKs by @RTLS in #1851
  • 🐛 Export all functions on sinks by @RTLS in #1850
  • 🐛 Fix group_id=nil regression for tables w/o pks by @RTLS in #1852
  • docs: update remaining references to records by @thisisgoldman in #1839
  • ✅ Comprehensive end-to-end YAML tests by @RTLS in #1854
  • ⛓️ :link instead of :monitor SPS <-> SMS by @RTLS in #1856

Full Changelog: v0.10.5...v0.10.6

Release v0.10.5

19 Jun 17:57
Compare
Choose a tag to compare

Release notes for v0.10.5

What's Changed

  • ✨ Add process metrics graphs to default grafana by @RTLS in #1840
  • 🤫 do not tcp connect for endpoint health checks by @RTLS in #1841
  • Do not add delivered wal_cursors to idempotency metadata (noisy) by @acco in #1845
  • 🔧 Fix esbuild-svelte warning by @davoclavo in #1847
  • 💄 Improve health summary language by @RTLS in #1848
  • 🎨 Adjust sink type icon grid to make room for more sinks by @davoclavo in #1846
  • 🐛 Opening GCP pub/sub create form by @acco in #1849

Full Changelog: v0.10.4...v0.10.5

Release v0.10.4

18 Jun 00:33
Compare
Choose a tag to compare

Release notes for v0.10.4

What's Changed

Full Changelog: v0.10.3...v0.10.4

Release v0.10.3

17 Jun 02:21
Compare
Choose a tag to compare

Release notes for v0.10.3

What's Changed

  • feat: allow users to configure postgres pool size by @davoclavo in #1811
  • ✅ Fix http e2e replication test by @RTLS in #1820
  • ♻️ Move flushing out of handle_data by @acco in #1803
  • ♻️ Buffer messages as opposed to disconnecting by @acco in #1804
  • ⚡️ Don't cancel/reschedule timer if flush imminent by @acco in #1816
  • ♻️ Assert that we receive at most one bufferable message by @acco in #1818
  • Add S2 sink by @acco in #1795
  • ✅ e2e tests with publication fan-in by @RTLS in #1825
  • Minor edits to consistency-model.mdx by @thisisgoldman in #1785
  • 🩹 Fix Counter.inc to allow non-one counts by @RTLS in #1826
  • 🧠 Reduce memory usage of consistency check worker by @RTLS in #1828
  • ⚡ Improve default performance by @acco in #1829
  • ♻️ Refactor ProcessMetrics to use frames/exclusive metrics by @acco in #1831

Full Changelog: v0.10.2...v0.10.3

Release v0.10.2

15 Jun 03:56
Compare
Choose a tag to compare

Release notes for v0.10.2

Full Changelog: v0.10.1...v0.10.2

Release v0.10.1

14 Jun 02:43
Compare
Choose a tag to compare

Release notes for v0.10.1

Full Changelog: v0.10.0...v0.10.1

Release v0.10.0

14 Jun 02:15
Compare
Choose a tag to compare

IAC (YAML and API) changes

To support multiple tables in one sink the specification for YAML and API requests has changed.

schema: and table: are now source:

schema and table

sinks:
  - name: kafka-users-sink
    table: public.users
  - name: kafka-public-sink
     schema: public

And you want to maintain single schema or table functionality, you must change this to:

sinks:
  - name: kafka-users-sink
    source:
      include_tables: ["public.users"]
  - name: kafka-public-sink
    source:
      include_schemas: ["public"]

group_column_names is now under tables

Further, group_column_names is now specified per table and exists under the tables key:

sinks:
  - name: kafka-users-sink
    group_column_names: ["account_id"]

now becomes:

sinks:
  - name: kafka-users-sink
    tables:
      - name: users
        group_column_names: ["account_id"]

⚠️ Column filters are deprecated

Column filters are deprecated in v0.10.0 and you will be unable to migrate your Sequin instance from v0.9 or earlier to v0.10.0 or later if any of your sinks have column filters. Please replace your column filters with filter functions before upgrading your Sequin instance.

Migrating to filter functions

Let's say you have a column filter that compares a status column of type text to "active" like:

Screenshot 2025-06-13 at 7 22 52 PM

You can replace this column filter with the following filter function:

def filter(action, record, changes, metadata) do
  record["status"] == "active"
end

You can read more about filter functions in our docs.

Join our Discord or Slack communities if you would like the Sequin team to help with this migration.

Migration failure

If you attempt to migrate a Sequin instance that has column filters, you will receive an error such as:

** (Postgrex.Error) ERROR P0001 (raise_exception) Cannot migrate: Found 1 sink(s) with column filters that need to be removed first: products-sink

What's Changed

Full Changelog: v0.9.4...v0.10.0