Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
name: Bug report
description: Report a problem with PeerDB
title: "[Bug]: "
labels: ["bug"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to file a bug report! Please fill out the fields below so we can reproduce and triage the issue.

- type: textarea
id: description
attributes:
label: Description
description: A clear and concise description of what the bug is, including the expected behavior and what actually happened.
placeholder: When I run a CDC mirror to ..., I expected ... but instead ...
validations:
required: true

- type: textarea
id: steps_to_reproduce
attributes:
label: Steps to reproduce
description: Step-by-step instructions to reproduce the behavior.
placeholder: |
1. Create a peer with ...
2. Create a mirror with ...
3. Observe ...
validations:
required: true

- type: input
id: version
attributes:
label: PeerDB version
description: The PeerDB version (or Docker image tag / commit SHA) you are running.
placeholder: e.g. v0.30.0 or stable-dev
validations:
required: true

- type: dropdown
id: source_connector
attributes:
label: Source connector
description: If this issue relates to a source connector, select it here. Leave blank if not applicable.
options:
- Postgres
- MySQL
- MongoDB
- BigQuery
- Other
validations:
required: false

- type: markdown
attributes:
value: |
### ⚠️ Deprecated destination connectors

The following destination connectors are **deprecated**: **Snowflake**, **BigQuery**, **ElasticSearch**, **Kafka**, **Redpanda**, **Confluent**, **Azure Event Hubs**, **Google Pub/Sub**, and **S3**.

These destinations are no longer actively maintained, but remain functional. Bugs affecting only these destinations are unlikely to be prioritized. Note: BigQuery is deprecated **only as a destination** — it remains a supported source. See the [deprecated connectors documentation](https://github.com/PeerDB-io/peerdb/blob/main/docs/deprecated-connectors.md) for details and migration guidance.

- type: dropdown
id: destination_connector
attributes:
label: Destination connector
description: If this issue relates to a destination connector, select it here (see the deprecation notice above). Leave blank if not applicable.
options:
- ClickHouse
- ClickHouse Cloud
- Postgres
- S3
- Snowflake
- BigQuery
- ElasticSearch
- Kafka
- Redpanda
- Confluent
- Azure Event Hubs
- Google Pub/Sub
- Other
validations:
required: false
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
blank_issues_enabled: false
Comment thread
Jeremyyang920 marked this conversation as resolved.
77 changes: 77 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
name: Feature request
description: Suggest a feature or improvement for PeerDB
title: "[Feature]: "
labels: ["enhancement"]
body:
- type: markdown
attributes:
value: |
Thanks for suggesting an improvement! Please describe the problem you are trying to solve and what you would like to happen.

- type: textarea
id: problem
attributes:
label: Problem / motivation
description: What problem are you trying to solve, and what is the use case?
placeholder: When replicating ..., I need ... but currently ...
validations:
required: true

- type: textarea
id: proposal
attributes:
label: Proposed solution
description: Describe what you would like to happen.
validations:
required: true

- type: textarea
id: alternatives
attributes:
label: Alternatives considered
description: Any alternative solutions or workarounds you have considered.
validations:
required: false

- type: dropdown
id: source_connector
attributes:
label: Source connector
description: If this request relates to a source connector, select it here. Leave blank if not applicable.
options:
- Postgres
- MySQL
- MongoDB
- BigQuery
- Other
validations:
required: false

- type: markdown
attributes:
value: |
### ⚠️ Deprecated destination connectors

The following destination connectors are **deprecated** and no longer actively maintained: **Snowflake**, **BigQuery**, **ElasticSearch**, **Kafka**, **Redpanda**, **Confluent**, **Azure Event Hubs**, **Google Pub/Sub**, and **S3**. New features for these destinations are unlikely to be prioritized. Note: BigQuery is deprecated **only as a destination** — it remains a supported source. See the [deprecated connectors documentation](https://github.com/PeerDB-io/peerdb/blob/main/docs/deprecated-connectors.md).

- type: dropdown
id: destination_connector
attributes:
label: Destination connector
description: If this request relates to a destination connector, select it here (see the deprecation notice above). Leave blank if not applicable.
options:
- ClickHouse
- ClickHouse Cloud
- Postgres
- S3
- Snowflake
- BigQuery
- ElasticSearch
- Kafka
- Redpanda
- Confluent
- Azure Event Hubs
- Google Pub/Sub
- Other
validations:
required: false
53 changes: 53 additions & 0 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Configuration for actions/labeler@v5.
# Applies the `deprecated` label to PRs that touch deprecated destination connectors:
# Snowflake, BigQuery, ElasticSearch, Kafka (incl. Confluent/Redpanda), Azure Event Hubs, Google Pub/Sub, S3.
# ClickHouse and Postgres (the actively-maintained destinations) are intentionally excluded.
deprecated:
- changed-files:
- any-glob-to-any-file:
# Destination-only connectors — the whole connector is deprecated.
- flow/connectors/snowflake/**
- flow/connectors/elasticsearch/**
- flow/connectors/eventhub/**
- flow/connectors/kafka/**
- flow/connectors/pubsub/**
- flow/connectors/s3/**
# End-to-end tests
- flow/e2e/snowflake*.go
- flow/e2e/elasticsearch*.go
- flow/e2e/eventhub*.go
- flow/e2e/kafka*.go
- flow/e2e/pubsub*.go
- flow/e2e/s3_*.go
# UI peer forms
- ui/components/PeerForms/SnowflakeForm.tsx
- ui/components/PeerForms/ElasticsearchConfigForm.tsx
- ui/components/PeerForms/KafkaConfig.tsx
- ui/components/PeerForms/PubSubConfig.tsx
- ui/components/PeerForms/S3Form.tsx
- ui/components/PeerForms/Eventhubs/**
# UI peer-create helpers. The literal `[` in the [peerType] dynamic
# route segment is escaped as `[[]` so minimatch treats it literally
# rather than as a character class.
- ui/app/peers/create/[[]peerType]/helpers/sf.ts
- ui/app/peers/create/[[]peerType]/helpers/es.ts
- ui/app/peers/create/[[]peerType]/helpers/eh.ts
- ui/app/peers/create/[[]peerType]/helpers/ka.ts
- ui/app/peers/create/[[]peerType]/helpers/ps.ts
- ui/app/peers/create/[[]peerType]/helpers/s3.ts
# BigQuery is dual-role: only its DESTINATION is deprecated, while its
# source is actively maintained and lives in the same directory. List
# the destination-specific files explicitly so BigQuery *source*
# changes (source.go, source_schema.go, qrep_object_pull.go,
# bigquery_source_test.go) are never auto-labeled as deprecated.
# (A `!`-negation / `all-globs-to-all-files` block was tried first but
# silently disables the whole labeler config in actions/labeler v5.)
- flow/connectors/bigquery/bigquery.go
- flow/connectors/bigquery/merge_stmt_generator.go
- flow/connectors/bigquery/qrep.go
- flow/connectors/bigquery/qrep_avro_sync.go
Comment thread
Jeremyyang920 marked this conversation as resolved.
- flow/connectors/bigquery/clustering.go
- flow/e2e/bigquery_test.go
- flow/e2e/bigquery_qrep_test.go
- ui/components/PeerForms/BigqueryConfig.tsx
- ui/app/peers/create/[[]peerType]/helpers/bq.ts
147 changes: 147 additions & 0 deletions .github/workflows/deprecated-connector.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
name: Deprecated connector

# - Labels PRs that touch deprecated destination connectors (actions/labeler).
# - On bug reports, applies the `deprecated` label only when the selected
# destination connector (issue-form dropdown) is a deprecated one.
# - Posts a single informational notice on any issue/PR carrying the
# `deprecated` label. It never closes the issue or PR.
#
# NOTE: the `deprecated` label must be created once in the repository settings
# (Settings -> Labels) — labels are not defined in this repo.

on:
pull_request_target:
types: [opened, synchronize, reopened]
pull_request:
types: [labeled]
issues:
types: [opened, edited, labeled]

permissions:
contents: read

jobs:
# Apply path-based labels (incl. `deprecated`) to PRs per .github/labeler.yml.
label:
if: github.event_name == 'pull_request_target'
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- uses: actions/labeler@8558fd74291d67161a8a78ce36a881fa63b766a9 # v5.0.0
with:
sync-labels: false

# On a new/edited bug report, apply `deprecated` only when the chosen
# destination connector (issue-form dropdown) is a deprecated one. Issues
# that select an actively-maintained destination are left untouched.
triage-issue:
if: github.event_name == 'issues' && (github.event.action == 'opened' || github.event.action == 'edited')
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
with:
script: |
const issue = context.payload.issue;
const body = issue.body || '';
// Extract the "Destination connector" dropdown value from the form body.
const match = body.match(/###\s*Destination connector\s*\r?\n+\s*(.+)/);
const selection = match ? match[1].trim() : '';
const deprecated = new Set([
'Snowflake', 'BigQuery', 'ElasticSearch', 'Kafka', 'Redpanda',
'Confluent', 'Azure Event Hubs', 'Google Pub/Sub', 'S3',
]);
if (!deprecated.has(selection)) {
core.info(`Destination "${selection || '(none)'}" is not a deprecated connector; not labeling.`);
return;
}
if ((issue.labels || []).some((l) => (l.name || l) === 'deprecated')) {
core.info('Issue already labeled deprecated.');
return;
}
await github.rest.issues.addLabels({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue.number,
labels: ['deprecated'],
});
core.info(`Labeled issue #${issue.number} deprecated (destination: ${selection}).`);

# Post a one-time notice when the `deprecated` label is present on an issue/PR.
# Depends on the prior jobs so it observes labels they applied within this same
# run (a GITHUB_TOKEN label write does not itself emit a `labeled` event).
# `if: always()` keeps it running when an upstream job is skipped.
notice:
needs: [label, triage-issue]
if: always()
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
with:
script: |
const marker = '<!-- deprecated-connector-notice -->';
const number =
context.eventName === 'issues'
? context.payload.issue?.number
: context.payload.pull_request?.number;
if (!number) {
core.info('No issue/PR number in payload; nothing to do.');
return;
}

// Re-fetch labels so we observe any applied by the label/triage jobs
// earlier in this run (rather than the stale event payload).
const { data: labels } = await github.rest.issues.listLabelsOnIssue({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: number,
});
if (!labels.some((l) => l.name === 'deprecated')) {
core.info('No deprecated label present; nothing to do.');
return;
}

// Guard against duplicate comments using the hidden marker.
const comments = await github.paginate(github.rest.issues.listComments, {
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: number,
});
if (comments.some((c) => c.body && c.body.includes(marker))) {
core.info('Deprecated-connector notice already posted; skipping.');
return;
}

const body = [
marker,
'### ⚠️ Deprecated destination connector',
'',
'This issue or pull request relates to a **deprecated** destination connector ' +
'(Snowflake, BigQuery, ElasticSearch, Kafka, Redpanda, Confluent, Azure Event Hubs, Google Pub/Sub, or S3).',
'',
'These destinations are **no longer actively maintained**, but remain functional. ' +
'We are unlikely to prioritize new work here.',
'',
'> Note: BigQuery is deprecated **only as a destination** — it remains a supported source.',
'',
'If you depend on one of these connectors, we recommend:',
'',
'- **Pin** to a known-good PeerDB version so behavior stays stable.',
'- **Fork** the repository if you need to carry your own changes.',
'',
'See the [deprecated connectors documentation](https://github.com/PeerDB-io/peerdb/blob/main/docs/deprecated-connectors.md) for details and migration guidance.',
].join('\n');

await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: number,
body,
});
core.info(`Posted deprecated-connector notice on #${number}.`);
9 changes: 9 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Contributing to PeerDB

Thanks for your interest in contributing to PeerDB! Bug reports, feature requests, and pull requests are all welcome. If you have a question, feel free to drop by our [Slack](https://slack.peerdb.io/).

## Deprecated connectors

Several destination connectors (Snowflake, BigQuery, ElasticSearch, Kafka including Confluent and Redpanda, Azure Event Hubs, Google Pub/Sub, and S3) are deprecated and no longer actively maintained. They remain fully functional, and no code is currently being removed. (BigQuery is deprecated only as a destination — it remains a supported source.)

If you depend on one of these connectors, see the [deprecated connectors migration guide](docs/deprecated-connectors.md) for how to pin to a release or fork the relevant connector code.
Loading
Loading