Skip to content

Commit 5521171

Browse files
Jeremyyang920claude
andcommitted
Deprecate QRep and XMin mirror types
Mark the Query Replication (QRep) and XMin mirror types as deprecated; CDC is the recommended, actively-maintained mirror type. They remain fully functional and no code is currently being removed. - ui/app/api/mirror-types: flag QRep and XMin cards as deprecated. - ui/app/mirrors/create/mirrorcards: render a "Deprecated" badge on those cards in the create-mirror flow. - README: add a mirror-type deprecation notice; soften "no code is being removed" to "currently" across the connector/mirror notices. - CONTRIBUTING.md, docs/deprecated-connectors.md: same "currently" wording. Refs #4465 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 1287d2a commit 5521171

5 files changed

Lines changed: 31 additions & 6 deletions

File tree

CONTRIBUTING.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,6 @@ Thanks for your interest in contributing to PeerDB! Bug reports, feature request
44

55
## Deprecated connectors
66

7-
Several destination connectors (Snowflake, BigQuery, ElasticSearch, Kafka including Confluent and Redpanda, Azure Event Hubs, Google Pub/Sub, and S3) are deprecated and no longer actively maintained. They remain fully functional, and no code is being removed. (BigQuery is deprecated only as a destination — it remains a supported source.)
7+
Several destination connectors (Snowflake, BigQuery, ElasticSearch, Kafka including Confluent and Redpanda, Azure Event Hubs, Google Pub/Sub, and S3) are deprecated and no longer actively maintained. They remain fully functional, and no code is currently being removed. (BigQuery is deprecated only as a destination — it remains a supported source.)
88

99
If you depend on one of these connectors, see the [deprecated connectors migration guide](docs/deprecated-connectors.md) for how to pin to a release or fork the relevant connector code.

README.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,9 @@ At PeerDB, we are building a fast, simple and the most cost effective way to str
1717

1818
We support different modes of streaming - log based (CDC), cursor based (timestamp or integer) and XMIN based. Performance wise, we are 10x faster than existing tools. Features wise, we support native Postgres features such as comprehensive set of data-types incl. jsonb/arrays/geospatial, efficiently streaming TOAST columns, schema changes and so on.
1919

20-
> **Connector deprecation notice:** The Snowflake, BigQuery, ElasticSearch, Kafka (including Confluent and Redpanda variants), Azure Event Hubs, Google Pub/Sub, and S3 **destination** connectors are deprecated and no longer actively maintained. They remain fully functional in the current and all prior releases, and no code is being removed. The actively-maintained paths going forward are Postgres → ClickHouse, Postgres → ClickHouse Cloud, and Postgres → Postgres. This deprecation applies to the **destination** role only — BigQuery remains a supported **source**. See the [connector status matrix](#status) and the [deprecated connectors migration guide](docs/deprecated-connectors.md) for how to pin to a release or fork the relevant code.
20+
> **Mirror type deprecation notice:** CDC is the recommended and actively-maintained mirror type. The **Query Replication (QRep)** and **XMIN** mirror types are deprecated and no longer actively maintained; they remain fully functional, and no code is currently being removed.
21+
22+
> **Connector deprecation notice:** The Snowflake, BigQuery, ElasticSearch, Kafka (including Confluent and Redpanda variants), Azure Event Hubs, Google Pub/Sub, and S3 **destination** connectors are deprecated and no longer actively maintained. They remain fully functional in the current and all prior releases, and no code is currently being removed. The actively-maintained paths going forward are Postgres → ClickHouse, Postgres → ClickHouse Cloud, and Postgres → Postgres. This deprecation applies to the **destination** role only — BigQuery remains a supported **source**. See the [connector status matrix](#status) and the [deprecated connectors migration guide](docs/deprecated-connectors.md) for how to pin to a release or fork the relevant code.
2123
2224
## Get started
2325

@@ -67,7 +69,7 @@ PeerDB is an ETL/ELT tool built for PostgreSQL. We implement multiple Postgres n
6769
6870
**For reliability**, we have mechanisms in place for fault tolerance - state management, automatic retries, handling idempotency and consistency and so on (<https://blog.peerdb.io/using-temporal-to-scale-data-synchronization-at-peerdb>). Configurable batching and parallelism prevent out of memory (OOMs) and crashes.
6971
70-
**From a feature richness standpoint**, we support efficient syncing of tables with large (TOAST) columns. We support multiple streaming modes - log based (CDC), cursor based (timestamp or integer) and XMIN based. We provide rich data-type mapping and plan to support every possible type (incl. Custom types) that Postgres supports to the best extent possible on the target data-store.
72+
**From a feature richness standpoint**, we support efficient syncing of tables with large (TOAST) columns. We support multiple streaming modes - log based (CDC), cursor based (timestamp or integer) and XMIN based (the cursor-based / Query Replication and XMIN modes are deprecated; CDC is recommended). We provide rich data-type mapping and plan to support every possible type (incl. Custom types) that Postgres supports to the best extent possible on the target data-store.
7173
7274
### Now available natively in ClickHouse Cloud (Generally Available)
7375

docs/deprecated-connectors.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ The following **destination** connectors are deprecated and no longer actively m
1010
- Google Pub/Sub
1111
- S3
1212

13-
These connectors remain **fully functional** in the current and all prior releases. No code is being removed, and they will **not** be automatically removed from existing deployments. The actively-maintained paths going forward are Postgres → ClickHouse, Postgres → ClickHouse Cloud, and Postgres → Postgres. Sources (Postgres, MySQL, MongoDB) are not affected by this deprecation.
13+
These connectors remain **fully functional** in the current and all prior releases. No code is currently being removed, and they will **not** be automatically removed from existing deployments. The actively-maintained paths going forward are Postgres → ClickHouse, Postgres → ClickHouse Cloud, and Postgres → Postgres. Sources (Postgres, MySQL, MongoDB) are not affected by this deprecation.
1414

1515
> **BigQuery is deprecated only as a destination.** BigQuery remains a **supported source** for QRep mirrors and is not deprecated in that role. The migration guidance below applies to BigQuery's use as a *destination*.
1616
@@ -27,7 +27,7 @@ Pin to the latest release tag before any future removal so that you have the mos
2727

2828
## How to fork
2929

30-
Because no code is being removed, you can fork PeerDB and carry the relevant connector code yourself. The connectors are self-contained enough that forking and maintaining a single connector is practical:
30+
Because the connector code is still present, you can fork PeerDB and carry the relevant connector code yourself. The connectors are self-contained enough that forking and maintaining a single connector is practical:
3131

3232
1. Fork the repository.
3333
2. Keep the connector's flow code, its UI form and helper, and (optionally) its end-to-end tests so you can continue validating it.

ui/app/api/mirror-types/route.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,15 @@ export async function GET(_: NextRequest) {
1515
description:
1616
'Query Replication allows you to specify a set of rows to be synced via a SELECT query. Useful for replicating views and tables without primary keys.',
1717
link: 'https://docs.peerdb.io/usecases/Streaming%20Query%20Replication/overview',
18+
deprecated: true,
1819
},
1920
];
2021
const xminCard = {
2122
title: MirrorType.XMin,
2223
description:
2324
'XMIN mode uses the xmin system column of PostgreSQL as a watermark column for replication.',
2425
link: 'https://docs.peerdb.io/sql/commands/create-mirror#xmin-query-replication',
26+
deprecated: true,
2527
};
2628

2729
if (!GetPeerDBClickHouseMode()) {

ui/app/mirrors/create/mirrorcards.tsx

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
'use client';
22
import { MirrorType } from '@/app/dto/MirrorsDTO';
33
import { fetcher } from '@/app/utils/swr';
4+
import { Badge } from '@/lib/Badge';
45
import { Label } from '@/lib/Label';
56
import { RowWithRadiobutton } from '@/lib/Layout';
67
import { ProgressCircle } from '@/lib/ProgressCircle';
@@ -22,6 +23,7 @@ export default function MirrorCards({
2223
title: MirrorType;
2324
description: string;
2425
link: string;
26+
deprecated?: boolean;
2527
}[]
2628
>('/api/mirror-types', fetcher);
2729

@@ -50,7 +52,26 @@ export default function MirrorCards({
5052
<RowWithRadiobutton
5153
label={
5254
<Label>
53-
<div style={{ fontWeight: 'bold' }}>{card.title}</div>
55+
<div
56+
style={{
57+
fontWeight: 'bold',
58+
display: 'flex',
59+
alignItems: 'center',
60+
gap: '0.5rem',
61+
}}
62+
>
63+
{card.title}
64+
{card.deprecated && (
65+
<Badge variant='destructive'>
66+
<Label
67+
as='label'
68+
style={{ fontSize: 13, padding: 0 }}
69+
>
70+
Deprecated
71+
</Label>
72+
</Badge>
73+
)}
74+
</div>
5475
</Label>
5576
}
5677
action={<RadioButton value={card.title} />}

0 commit comments

Comments
 (0)