Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion flow/e2e/test_data/mysql-debug/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
# plus mysqld-debug, then route only the final foreground server to mysqld-debug. The stock
# entrypoint's config checks, datadir initialization, and temporary init server keep using the
# release binary because the debug binary can trip InnoDB assertions on that path.
ARG MYSQL_VERSION=8.0.46

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build-breaking: The yum repo URL on lines 26-28 is hardcoded to mysql-8.0-community:

'[mysql80-community]'
'name=MySQL 8.0 Community Server'
'baseurl=https://repo.mysql.com/yum/mysql-8.0-community/el/9/$basearch/'

But the base image is now mysql:8.4.9. When the Dockerfile queries the RPM version from the 8.4.9 base image (line 23) and then tries to install packages at that version from the 8.0 repo, it will fail — the mysql-8.0-community repo only contains 8.0.x packages (up to 8.0.45).

The repo needs to be updated to mysql-8.4-community (confirmed it exists at https://repo.mysql.com/yum/mysql-8.4-community/el/9/x86_64/ with debug packages through 8.4.8).

Suggested fix:

'[mysql84-community]'
'name=MySQL 8.4 Community Server'
'baseurl=https://repo.mysql.com/yum/mysql-8.4-community/el/9/$basearch/'

Additionally, the 8.4.9 debug RPM may not yet be published in the repo (latest available is 8.4.8 as of this review). You may want to pin to 8.4.8 until 8.4.9 packages land, or verify the 8.4.9 RPMs are available before merging.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug (build-breaking): Bumping to 8.4.9 here will break the Docker build. Lines 26-28 (not shown in diff) hardcode the yum repo to mysql-8.0-community. When the base image is 8.4, line 23 extracts an 8.4.x RPM version, then lines 33-41 try to install those packages from the 8.0 repo -- which does not carry them. The repo config (lines 26-28) needs to be updated to mysql-8.4-lts-community.

ARG MYSQL_VERSION=8.4.9@sha256:c36050afdca850f23cef85703f84c7531a5ae155a11b5ee1c60acb09937c4084

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incomplete version bump: The ARG default is overridden by the bake file (docker-bake.hcl line 2) and workflow (mysql-debug-docker.yml lines 9, 24), both still at 8.0.46. The e2e test (clickhouse_mysql_test.go:1491) also references mysql-debug:8.0.46. This Dockerfile change is a no-op in CI. To complete the bump, also update: (1) docker-bake.hcl line 2, (2) mysql-debug-docker.yml lines 9 and 24, (3) clickhouse_mysql_test.go line 1491. Also update the yum repo URL on line 28 from mysql-8.0-community to mysql-8.4-lts-community.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High severity -- inconsistent version bump with latent breakage

Renovate updated only the Dockerfile ARG default from 8.0.46 to 8.4.9, but three other files still hardcode 8.0.46:

  1. docker-bake.hcl line 2: default = 8.0.46
  2. mysql-debug-docker.yml lines 9/24: defaults to 8.0.46
  3. clickhouse_mysql_test.go line 1558: Image ghcr.io/peerdb-io/mysql-debug:8.0.46

Since CI passes the bake/workflow default as a build arg, the actual Docker build still uses mysql:8.0.46 -- so this Dockerfile change is effectively a no-op today. However, it creates a confusing inconsistency.

More importantly, if someone later aligns the other files to 8.4, two things break:

  1. This Dockerfile itself -- lines 28-29 hardcode the mysql-8.0-community repo URL. The 8.0 repo will not have 8.4.x packages, so microdnf install will fail.

  2. MySQL 8.4 has critical breaking changes for CDC:

    • All SLAVE/MASTER replication SQL syntax is removed (must use REPLICA/SOURCE equivalents)
    • mysql_native_password plugin is disabled by default
    • binlog_transaction_dependency_tracking is removed
    • --binlog-format=ROW and SET GLOBAL binlog_format=row are deprecated/removed in 8.4
    • GTID mode is always-on in 8.4, breaking the dynamic toggling in mysql.go:116-140

Recommendation: Either revert the MySQL portion of this PR (keep the Dockerfile at 8.0.46 to match the other files), or commit to a full 8.4 migration that also updates the bake file, workflow, test references, repo URL, and validates e2e test compatibility.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug (build-breaking): Bumping to MySQL 8.4.9 will break the Docker build. Lines 26-28 below hardcode the yum repo to mysql-8.0-community:

[mysql80-community]
name=MySQL 8.0 Community Server
baseurl=https://repo.mysql.com/yum/mysql-8.0-community/el/9/...

MySQL 8.4 RPMs are published under a different repository path (e.g. mysql-8.4-lts-community). The microdnf install on lines 34-41 will fail because the 8.4.9 packages do not exist in the 8.0 repo.

The fix should update the baseurl to the correct 8.4 repo path. Ideally, derive the repo path dynamically from the major.minor version so it moves automatically with future bumps (verify the exact repo path -- Oracle uses mysql-8.4-lts-community for 8.4 LTS vs mysql-8.0-community for 8.0).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build breakage: The MySQL version jumps from 8.0 to 8.4, but the yum repo URL on line 29 is still hardcoded to mysql-8.0-community:

baseurl=https://repo.mysql.com/yum/mysql-8.0-community/el/9/$basearch/

When building with the new default MYSQL_VERSION=8.4.9, the RPM version extracted from mysql-community-server-minimal will be an 8.4.9 version string. The subsequent microdnf install tries to fetch mysql-community-server-debug-8.4.9-* from the 8.0 repo, which only has 8.0.x packages. The build will fail.

The repo URL needs to change to mysql-8.4-community:

baseurl=https://repo.mysql.com/yum/mysql-8.4-community/el/9/$basearch/

(Verified that mysql-community-server-debug packages exist in the mysql-8.4-community repo up to at least 8.4.7.)

Additionally, the CI workflow .github/workflows/mysql-debug-docker.yml still defaults MYSQL_DEBUG_VERSION to 8.0.46 (lines 9 and 24), which is now inconsistent with this Dockerfile ARG default. In CI the build-arg override means builds will still use 8.0.46 and succeed, but local builds using the Dockerfile default will hit the wrong repo.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: hardcoded MySQL 8.0 yum repo will break with 8.4 base image

Lines 26-28 define a yum repo pointing to mysql-8.0-community:

'[mysql80-community]'
'name=MySQL 8.0 Community Server'
'baseurl=https://repo.mysql.com/yum/mysql-8.0-community/el/9/$basearch/'

With the base image now mysql:8.4.9, the RPM version extracted at line 23 (mysql_rpm_version) will be 8.4.9-.... The microdnf install at lines 34-43 will then try to find mysql-community-server-debug-8.4.9-... in the 8.0 repo — those packages don't exist there, so the Docker build will fail.

The repo name and URL need to be updated to match MySQL 8.4, e.g. mysql-8.4-lts-community / https://repo.mysql.com/yum/mysql-8.4-lts-community/el/9/$basearch/.

Additionally, flow/e2e/clickhouse_mysql_test.go:1558 hardcodes ghcr.io/peerdb-io/mysql-debug:8.0.46 — this image tag reference will also need updating to match the new version.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug (build-breaking): The base image is bumped from MySQL 8.0.46 to 8.4.9, but lines 26-28 below still hardcode the MySQL 8.0 yum repo:

[mysql80-community]
name=MySQL 8.0 Community Server
baseurl=https://repo.mysql.com/yum/mysql-8.0-community/el/9/$basearch/

This will break the Docker build. Line 23 extracts the RPM version from mysql-community-server-minimal (which will be 8.4.9-... in the new base image), then lines 34-42 try to install packages at that version from the 8.0 repo -- which only has 8.0.x packages. The microdnf install will fail with a no-package-matching error.

Fix: Update lines 26-28 to point at the MySQL 8.4 LTS repo:

[mysql-8.4-lts-community]
name=MySQL 8.4 LTS Community Server
baseurl=https://repo.mysql.com/yum/mysql-8.4-lts-community/el/9/$basearch/

Also note that .github/workflows/mysql-debug-docker.yml still defaults MYSQL_DEBUG_VERSION to 8.0.46 (lines 9 and 24), so the workflow default should be updated to 8.4.9 to stay consistent with this Dockerfile ARG default.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High severity — build will break when this default is actually used.

Bumping the base image to mysql:8.4.9 without updating the rest of the Dockerfile will cause a build failure. Three issues:

  1. Hardcoded MySQL 8.0 YUM repo URL (lines 26-28): The repo config still points to mysql-8.0-community:

    baseurl=https://repo.mysql.com/yum/mysql-8.0-community/el/9/$basearch/
    

    Line 23 extracts the RPM version from the base image's mysql-community-server-minimal (which will be 8.4.9-…), then microdnf install at lines 34-41 tries to fetch those 8.4.x packages from the 8.0 repo — where they don't exist. This needs to be updated to the MySQL 8.4 LTS repo (e.g. mysql-8.4-lts-community).

  2. CI workflow still defaults to 8.0.46: .github/workflows/mysql-debug-docker.yml has MYSQL_DEBUG_VERSION: ${{ github.event.inputs.mysql_version || '8.0.46' }} and passes it as --build-arg, overriding this Dockerfile default. So the version change here is effectively dead code on CI, creating a confusing inconsistency. The workflow default should be updated to match.

  3. Test hardcodes old image tag: flow/e2e/clickhouse_mysql_test.go:1616 references ghcr.io/peerdb-io/mysql-debug:8.0.46. Even after this PR merges, the test will keep pulling the old 8.0 image. This reference needs updating too.

Additionally, MySQL 8.4 has notable behavioral changes relevant to CDC usage:

  • mysql_native_password is disabled by default — connections using this auth plugin will fail unless --mysql-native-password=ON is set.
  • binlog_format is deprecated (ROW is the only supported format) — the --binlog-format=ROW flag in the test (line 1626) will emit a deprecation warning but still work.
  • Several replication-related system variables were removed (expire_logs_days, log_bin_use_v1_row_events, transaction_write_set_extraction, all slave_*/master_* variables replaced by replica_*/source_*).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: MySQL 8.0 to 8.4 creates build-breaking inconsistency

This bumps the Dockerfile default from 8.0.46 to 8.4.9, but several related files were not updated:

  1. Hardcoded 8.0 yum repo (lines 27-28): The repo URL only has 8.0.x packages. When the base image is mysql:8.4.9, the extracted mysql_rpm_version will be 8.4.9-... and microdnf install will fail because those packages do not exist in the 8.0 repo. Building this Dockerfile with the default ARG is now broken.

  2. CI overrides not updated: Both .github/workflows/mysql-debug-docker.yml (line 24) and docker-bake.hcl (line 2) still default to 8.0.46. Renovate only updated the Dockerfile ARG default, so CI will still build with 8.0.46 via --build-arg. The Dockerfile default is misleading.

  3. Test hardcode: clickhouse_mysql_test.go:1616 still references ghcr.io/peerdb-io/mysql-debug:8.0.46.

If the intent is to stay on MySQL 8.0 for the debug image (which seems likely given the 8.0-specific repo configuration), this Dockerfile change should be reverted.

If upgrading to 8.4, several additional changes are needed:

  • Update the yum repo URL from mysql-8.0-community to the 8.4 equivalent
  • Update .github/workflows/mysql-debug-docker.yml and docker-bake.hcl defaults
  • Update the test image reference in clickhouse_mysql_test.go
  • Note: mysql_native_password is disabled by default in 8.4, and the binlog_format system variable is removed (8.4 only supports ROW format), which will break set global binlog_format=row in flow/e2e/mysql.go:93 and the SELECT @@binlog_format validation queries in flow/pkg/mysql/validation.go

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug (build-breaking): The MySQL version jumps from 8.0 to 8.4, but the yum repo configured below (line 29) is still hardcoded to mysql-8.0-community:

baseurl=https://repo.mysql.com/yum/mysql-8.0-community/el/9/$basearch/

The build extracts the RPM version from the 8.4.9 base image mysql-community-server-minimal package (yielding something like 8.4.9-1.el9.x86_64), then tries to install mysql-community-server-8.4.9-... from the 8.0 repo -- which will not have 8.4 packages. This will fail at microdnf install.

The repo URL needs to be updated to the MySQL 8.4 LTS repo (likely mysql-8.4-lts-community), and the section name/description on lines 26-27 should be updated accordingly.

Additionally, MySQL 8.4 has significant breaking changes vs 8.0 worth verifying for e2e tests:

  • mysql_native_password is disabled by default -- connections may fail without --mysql-native-password=ON
  • Old replication commands (SHOW SLAVE STATUS, SHOW MASTER STATUS, CHANGE MASTER TO) are removed (syntax errors) -- Go connector already handles this
  • Various InnoDB defaults changed, may affect mysqld-debug behavior for binlog incident injection
  • binlog_transaction_dependency_tracking system variable is removed

The PeerDB Go CDC connector code is already 8.4-compatible (flow/connectors/mysql/mysql.go:415-425, flow/pkg/mysql/validation.go:274-284), but this Dockerfile needs fixing before it can build.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug (build-breaking): The yum repo configured on lines 26-32 still points to the MySQL 8.0 community repo (baseurl=.../mysql-8.0-community/...), but the base image is now MySQL 8.4.9.

The build extracts the RPM version from mysql-community-server-minimal (which will be 8.4.9-...) and then tries to install mysql-community-server-debug from the 8.0 repo. The 8.0 repo does not carry 8.4.x packages, so microdnf install will fail with a package-not-found error.

The repo config needs to be updated to point to the 8.4 repo:

  • [mysql80-community] should be [mysql84-community]
  • name=MySQL 8.0 Community Server should be name=MySQL 8.4 Community Server
  • baseurl should use mysql-8.4-community instead of mysql-8.0-community

(Or better yet, derive the repo path from the major.minor version dynamically so future bumps do not hit this again.)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: version bump is a no-op. This ARG default is always overridden at build time. See the PR comment for details on which files need matching updates.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: yum repo URL at line 28 still points to mysql-8.0-community after this bump to MySQL 8.4

With this change, mysql_rpm_version (line 23) will resolve to an 8.4.x RPM version string. But the yum repo configured at line 28 points to mysql-8.0-community, which only hosts 8.0.x packages. The microdnf install at lines 34-41 will fail because packages like mysql-community-server-debug-8.4.9 don't exist in the 8.0 repo.

The repo section (lines 26-31) needs to be updated for 8.4. The MySQL 8.4 LTS repo path is mysql-8.4-lts-community:

  • Line 26: mysql80-community -> mysql84-lts-community
  • Line 27: MySQL 8.0 Community Server -> MySQL 8.4 LTS Community Server
  • Line 28: mysql-8.0-community -> mysql-8.4-lts-community

Additionally: MySQL 8.0 to 8.4 is a significant version jump with CDC-relevant changes:

  • mysql_native_password is disabled by default - test containers may need --mysql-native-password=ON
  • binlog_format variable is deprecated (ROW is the only supported format - good for PeerDB)
  • gtid_mode defaults to ON (good for PeerDB CDC)
  • Tagged GTIDs (UUID:TAG:NUMBER) are a new format that binlog parsers should handle

The PeerDB connector code already handles 8.4 properly (version-aware SQL for SHOW BINARY LOG STATUS, SHOW REPLICA STATUS, binlog_expire_logs_seconds), so the main concerns are: (1) this Dockerfile build breakage, and (2) potential test auth issues from mysql_native_password being off by default.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MySQL 8.0 to 8.4 bump is inconsistent with surrounding files and will break standalone builds.

This ARG default was bumped to 8.4.9, but three other files still reference 8.0.46:

  1. docker-bake.hcl line 2 (overrides the Dockerfile ARG in CI)
  2. mysql-debug-docker.yml lines 9 and 24
  3. clickhouse_mysql_test.go line 1738

Because docker-bake.hcl overrides MYSQL_VERSION at build time, CI still builds with 8.0.46 -- this Dockerfile change is effectively a no-op in the pipeline.

Worse, if someone builds the Dockerfile standalone (using this new default), it will fail: the yum repo URL on line 28 is hardcoded to mysql-8.0-community, which does not carry 8.4 packages. The RPM version extracted from the 8.4 base image will not be found in that repo.

If an upgrade to MySQL 8.4 is intended, all four files need updating and the yum repo URL needs to change to the 8.4-lts-community path. PeerDB Go code already handles 8.4 (SHOW BINARY LOG STATUS, SHOW REPLICA STATUS, binlog_expire_logs_seconds), so the application side is ready -- it is only the build infrastructure that needs alignment.

If the intent is to stay on 8.0 for now, this Dockerfile change should be reverted or Renovate should be configured to pin MySQL to 8.0.x.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build-breaking: This bumps the base image to MySQL 8.4.9, but the yum repo URL on line 29 is still hardcoded to mysql-8.0-community. The build extracts the RPM version from the 8.4.9 base image (line 23), then tries to install mysql-community-server at that version from the 8.0 repo -- but 8.4.x packages do not exist there. The correct repo for MySQL 8.4 is mysql-8.4-community. The repo name (line 27) and section header (line 26) also need updating. Additionally, docker-bake.hcl line 2 still defaults to 8.0.46, and clickhouse_mysql_test.go line 1738 still references the 8.0.46 image tag.

FROM mysql:${MYSQL_VERSION}
ARG MYSQL_VERSION

Expand Down
6 changes: 3 additions & 3 deletions stacks/flow.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# syntax=docker/dockerfile:1.24@sha256:87999aa3d42bdc6bea60565083ee17e86d1f3339802f543c0d03998580f9cb89
# syntax=docker/dockerfile:1.25@sha256:0adf442eae370b6087e08edc7c50b552d80ddf261576f4ebd6421006b2461f12

FROM golang:1.26-alpine@sha256:f1ddd9fe14fffc091dd98cb4bfa999f32c5fc77d2f2305ea9f0e2595c5437c14 AS builder
FROM golang:1.26-alpine@sha256:3ad57304ad93bbec8548a0437ad9e06a455660655d9af011d58b993f6f615648 AS builder
# Allow build flags to be passed in at build time, for example debug flags

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

golang digest + Alpine 3.23->3.24: Low risk. Golang is same version (1.26), new digest only. Alpine 3.24 has no breaking changes for PeerDB packages (ca-certificates, geos, openssl, postgresql-client, curl). Looks good.

ARG DEBUG_BUILD
ENV DEBUG_BUILD=${DEBUG_BUILD}
Expand Down Expand Up @@ -30,7 +30,7 @@ RUN --mount=type=cache,target="/root/.cache/go-build" if [[ "$DEBUG_BUILD" = "1"
go install github.com/go-delve/delve/cmd/dlv@latest; \
fi

FROM alpine:3.23@sha256:5b10f432ef3da1b8d4c7eb6c487f2f5a8f096bc91145e68878dd4a5019afde11 AS flow-base
FROM alpine:3.24@sha256:28bd5fe8b56d1bd048e5babf5b10710ebe0bae67db86916198a6eec434943f8b AS flow-base
ENV TZ=UTC
ADD --checksum=sha256:e5bb2084ccf45087bda1c9bffdea0eb15ee67f0b91646106e466714f9de3c7e3 https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem /usr/local/share/ca-certificates/global-aws-rds-bundle.pem
RUN apk add --no-cache ca-certificates geos && \
Expand Down
2 changes: 1 addition & 1 deletion stacks/peerdb-server.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ RUN --mount=type=cache,target=/usr/local/cargo/registry \
mkdir -p /root/target && \
cp target/${BUILD_MODE}/peerdb-server /root/target/

FROM alpine:3.23@sha256:5b10f432ef3da1b8d4c7eb6c487f2f5a8f096bc91145e68878dd4a5019afde11
FROM alpine:3.24@sha256:28bd5fe8b56d1bd048e5babf5b10710ebe0bae67db86916198a6eec434943f8b

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alpine 3.23->3.24: Low risk. Same assessment as flow.Dockerfile -- no breaking changes for packages used here (ca-certificates, postgresql-client, curl, iputils). Looks good.

ENV TZ=UTC
RUN apk add --no-cache ca-certificates postgresql-client curl iputils && \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug (build-breaking): Alpine 3.24 removed the unversioned postgresql-client meta-package. Only versioned variants exist now (postgresql16-client, postgresql17-client, postgresql18-client).

This apk add will fail with ERROR: unable to select packages: postgresql-client. Needs to be changed to a versioned package name, e.g. postgresql17-client.

adduser -s /bin/sh -D peerdb && \
Expand Down
4 changes: 2 additions & 2 deletions stacks/peerdb-ui.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# syntax=docker/dockerfile:1.24@sha256:87999aa3d42bdc6bea60565083ee17e86d1f3339802f543c0d03998580f9cb89
# syntax=docker/dockerfile:1.25@sha256:0adf442eae370b6087e08edc7c50b552d80ddf261576f4ebd6421006b2461f12

# Base stage
FROM node:24-alpine@sha256:21f403ab171f2dc89bad4dd69d7721bfd15f084ccb46cdd225f31f2bc59b5c9a AS base
FROM node:24-alpine@sha256:a0b9bf06e4e6193cf7a0f58816cc935ff8c2a908f81e6f1a95432d679c54fbfd AS base
ENV TZ=UTC
ENV NPM_CONFIG_UPDATE_NOTIFIER=false
RUN apk add --no-cache openssl && \
Expand Down
Loading