Skip to content

Add support of named and digest scylladb image reference with google cr #2491

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
/scylla-operator-tests
/gen-release-notes
/gen-api-reference
/get-scylla-version

# Test binary, build with `go test -c`
*.test
Expand Down
11 changes: 8 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,11 @@ define version-ldflags
endef
GO_LD_FLAGS ?=-ldflags '$(strip $(call version-ldflags,$(GO_PACKAGE)/pkg/version) $(GO_LD_EXTRA_FLAGS))'

GET_SCYLLADB_VERSION_SCRIPT ?= $(GO) run ./cmd/get-scylla-version/get-scylla-version.go
SCYLLADB_VERSION_FROM_CONFIG := $(shell yq e '.operator.scyllaDBVersion' ./assets/config/config.yaml)
GET_SCYLLADB_VERSION_SCRIPT_RESULT := $(shell $(GET_SCYLLADB_VERSION_SCRIPT) --image scylla --version $(SCYLLADB_VERSION_FROM_CONFIG))
SCYLLADB_SEM_VER := $(firstword $(GET_SCYLLADB_VERSION_SCRIPT_RESULT))

# TODO: look into how to make these local to the targets
export DOCKER_BUILDKIT :=1
export GOVERSION :=$(shell go version)
Expand Down Expand Up @@ -463,7 +468,7 @@ endef
# $1 - values.yaml
define update-scylla-helm-versions
$(YQ) eval-all -i -P '\
select(fi==0).scyllaImage.tag = ( select(fi==1) | .operator.scyllaDBVersion ) | \
select(fi==0).scyllaImage.tag = ( select(fi==1) | "$(SCYLLADB_SEM_VER)" ) | \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind elaborating why this is changed?

Copy link
Contributor Author

@grzywin grzywin Feb 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we execute make update command it is updating the manifests. Without this change the ScyllaDB version is taken directly from config.yaml file which at this moment I changed to digest. So in manifest we will something like below which won't work.

spec:
  version: sha256:5b53a7c60d9f9555bb87791ff29b2e633c6f472aec00de7afaf4db1addc6d594

Note: Just to clarify. My solution is not changing almost any logic directly in operator. I just added logic to handle digest and convert it to 'supported' version before it reaches the operator/manifests.

Copy link
Collaborator

@zimnx zimnx Feb 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: Just to clarify. My solution is not changing almost any logic directly in operator. I just added logic to handle digest and convert it to 'supported' version before it reaches the operator/manifests.

Lets do other way around. Having manipulations in tests and scripts doesn't bring any value to the user. If Operator would support all valid formats of image references, then any transformation in test/scripts wouldn't be needed, and users could use all variants of image references.

Copy link
Collaborator

@zimnx zimnx Feb 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think as part of this feature, we could also restructure config.yaml to accept full Scylla image ref instead just version. Places where split between repository and other part is required would be handled by the scripts around assets.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I thought the goal of this change was to allow us to use the digest/tag/'latest', so that we have more control over the exact version we're testing, rather than to please the users.

select(fi==0).agentImage.tag = ( select(fi==1) | .operator.scyllaDBManagerAgentVersion ) | \
select(fi==0)' \
'$(1)' './assets/config/config.yaml'
Expand All @@ -472,7 +477,7 @@ endef
# $1 - values.yaml
define update-scylla-manager-helm-versions
$(YQ) eval-all -i -P '\
select(fi==0).scylla.scyllaImage.tag = ( select(fi==1) | .operator.scyllaDBVersion ) | \
select(fi==0).scylla.scyllaImage.tag = ( select(fi==1) | "$(SCYLLADB_SEM_VER)" ) | \
select(fi==0).scylla.agentImage.tag = ( select(fi==1) | .operator.scyllaDBManagerAgentVersion ) | \
select(fi==0).image.tag = ( select(fi==1) | .operator.scyllaDBManagerVersion ) | \
select(fi==0)' \
Expand Down Expand Up @@ -545,7 +550,7 @@ verify-deploy:
# $2 - ScyllaCluster document index
define replace-scyllacluster-versions
$(YQ) eval-all -i -P '\
select(fi==0 and di==$(2)).spec.version = ( select(fi==1) | .operator.scyllaDBVersion ) | \
select(fi==0 and di==$(2)).spec.version = ( select(fi==1) | "$(SCYLLADB_SEM_VER)" ) | \
select(fi==0 and di==$(2)).spec.agentVersion = ( select(fi==1) | .operator.scyllaDBManagerAgentVersion ) | \
select(fi==0)' \
'$(1)' './assets/config/config.yaml'
Expand Down
6 changes: 3 additions & 3 deletions assets/config/config.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
operator:
scyllaDBVersion: "6.2.0"
scyllaDBVersion: "sha256:5b53a7c60d9f9555bb87791ff29b2e633c6f472aec00de7afaf4db1addc6d594"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unclear what version it is. Use tag+sha format

# scyllaDBEnterpriseVersionNeedingConsistentClusterManagementOverride sets enterprise version
# that requires consistent_cluster_management workaround for restore.
# In the future, enterprise versions should be run as a different config instance in its own run.
Expand All @@ -13,6 +13,6 @@ operator:
prometheusVersion: "v2.54.1"
operatorTests:
scyllaDBVersions:
updateFrom: "6.2.0-rc2"
upgradeFrom: "6.1.2"
updateFrom: "sha256:cf00b8bfcce4d27e8527cad913e83db89af842e6cfb64ff4736bce83594b33d1"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unclear what it's being tested, use tag+sha format

upgradeFrom: "sha256:0c1c6b49916230f62bdb0c2539b811b1a99d2c43d06ed810fad6f51015dea3ae"
nodeSetupImage: "quay.io/scylladb/scylla-operator-images:node-setup-v0.0.3@sha256:c6b3de240cc5c884d5c617485bae35c51572cdfd39b6431d2e1f759c7d7feea1"
37 changes: 37 additions & 0 deletions cmd/get-scylla-version/get-scylla-version.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
// Copyright (C) 2025 ScyllaDB

package main

import (
"flag"
"fmt"
"os"

scyllasemver "github.com/scylladb/scylla-operator/pkg/semver"
"k8s.io/klog/v2"
)

func main() {
klog.InitFlags(flag.CommandLine)
err := flag.Set("logtostderr", "true")
if err != nil {
panic(err)
}
defer klog.Flush()

imageName := flag.String("image", "", "Image name to check")
version := flag.String("version", "", "Version to check")

flag.Parse()

if *imageName == "" || *version == "" {
klog.Fatalf("Usage: %s --image <imageName> --version <version>", os.Args[0])
}

semVersion, fullVersion, err := scyllasemver.GetImageVersionAndDigest(*imageName, *version)
if err != nil {
klog.Fatalf("Error getting image version: %v", err)
}

fmt.Printf("%s %s\n", semVersion, fullVersion)
}
9 changes: 9 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ require (
github.com/gocql/gocql v1.7.0
github.com/godbus/dbus/v5 v5.1.0
github.com/google/go-cmp v0.7.0
github.com/google/go-containerregistry v0.20.2
github.com/grafana/grafana-api-golang-client v0.27.0
github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed
github.com/magiconair/properties v1.8.9
Expand Down Expand Up @@ -90,9 +91,13 @@ require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/cloudflare/circl v1.6.0 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.16.3 // indirect
github.com/containers/storage v1.57.1 // indirect
github.com/cyphar/filepath-securejoin v0.4.1 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/docker/cli v27.5.1+incompatible // indirect
github.com/docker/distribution v2.8.3+incompatible // indirect
github.com/docker/docker-credential-helpers v0.8.2 // indirect
github.com/emicklei/go-restful/v3 v3.12.1 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/fatih/color v1.18.0 // indirect
Expand Down Expand Up @@ -140,6 +145,7 @@ require (
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
github.com/moby/spdystream v0.5.0 // indirect
github.com/moby/term v0.5.2 // indirect
Expand All @@ -150,6 +156,7 @@ require (
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f // indirect
github.com/oklog/ulid v1.3.1 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/peterbourgon/diskv v2.0.1+incompatible // indirect
github.com/pjbgf/sha1cd v0.3.2 // indirect
Expand All @@ -162,7 +169,9 @@ require (
github.com/scylladb/termtables v0.0.0-20191203121021-c4c0b6d42ff4 // indirect
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
github.com/shurcooL/graphql v0.0.0-20230722043721-ed46e5a46466 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/skeema/knownhosts v1.3.1 // indirect
github.com/vbatts/tar-split v0.11.7 // indirect
github.com/x448/float16 v0.8.4 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
github.com/xlab/treeprint v1.2.0 // indirect
Expand Down
21 changes: 21 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UF
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cloudflare/circl v1.6.0 h1:cr5JKic4HI+LkINy2lg3W2jF8sHCVTBncJr5gIIq7qk=
github.com/cloudflare/circl v1.6.0/go.mod h1:uddAzsPgqdMAYatqJ0lsjX1oECcQLIlRpzZh3pJrofs=
github.com/containerd/stargz-snapshotter/estargz v0.16.3 h1:7evrXtoh1mSbGj/pfRccTampEyKpjpOnS3CyiV1Ebr8=
github.com/containerd/stargz-snapshotter/estargz v0.16.3/go.mod h1:uyr4BfYfOj3G9WBVE8cOlQmXAbPN9VEQpBBeJIuOipU=
github.com/containers/image/v5 v5.34.0 h1:HPqQaDUsox/3mC1pbOyLAIQEp0JhQqiUZ+6JiFIZLDI=
github.com/containers/image/v5 v5.34.0/go.mod h1:/WnvUSEfdqC/ahMRd4YJDBLrpYWkGl018rB77iB3FDo=
github.com/containers/storage v1.57.1 h1:hKPoFsuBcB3qTzBxa4IFpZMRzUuL5Xhv/BE44W0XHx8=
Expand All @@ -85,6 +87,12 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/docker/cli v27.5.1+incompatible h1:JB9cieUT9YNiMITtIsguaN55PLOHhBSz3LKVc6cqWaY=
github.com/docker/cli v27.5.1+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk=
github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker-credential-helpers v0.8.2 h1:bX3YxiGzFP5sOXWc3bTPEXdEaZSeVMrFgOr3T+zrFAo=
github.com/docker/docker-credential-helpers v0.8.2/go.mod h1:P3ci7E3lwkZg6XiHdRKft1KckHiO9a2rNtyFbZ/ry9M=
github.com/elazarl/goproxy v1.4.0 h1:4GyuSbFa+s26+3rmYNSuUVsx+HgPrV1bk1jXI0l9wjM=
github.com/elazarl/goproxy v1.4.0/go.mod h1:X/5W/t+gzDyLfHW4DrMdpjqYjpXsURlBt9lpBDxZZZQ=
github.com/emicklei/go-restful/v3 v3.12.1 h1:PJMDIM/ak7btuL8Ex0iYET9hxM3CI2sjZtzpL63nKAU=
Expand Down Expand Up @@ -163,6 +171,8 @@ github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/go-containerregistry v0.20.2 h1:B1wPJ1SN/S7pB+ZAimcciVD+r+yV/l/DSArMxlbwseo=
github.com/google/go-containerregistry v0.20.2/go.mod h1:z38EKdKh4h7IP2gSfUUqEvalZBqs6AoLeWfUy34nQC8=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
Expand Down Expand Up @@ -227,6 +237,8 @@ github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWE
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc=
github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/mitchellh/go-wordwrap v1.0.1 h1:TLuKupo69TCn6TQSyGxwI1EblZZEsQ0vMlAFQflz0v0=
github.com/mitchellh/go-wordwrap v1.0.1/go.mod h1:R62XHJLzvMFRBbcrT7m7WgmE1eOyTSsCt+hzestvNj0=
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
Expand Down Expand Up @@ -260,6 +272,8 @@ github.com/onsi/gomega v1.36.2 h1:koNYke6TVk6ZmnyHrCXba/T/MoLBXFjeC1PtvYgw0A8=
github.com/onsi/gomega v1.36.2/go.mod h1:DdwyADRjrc825LhMEkD76cHR5+pUnjhUN8GlHlRPHzY=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug=
github.com/opencontainers/image-spec v1.1.0/go.mod h1:W4s4sFTMaBeK1BQLXbG4AdM2szdn85PY75RI83NrTrM=
github.com/opentracing/opentracing-go v1.2.0 h1:uEJPy/1a5RIPAJ0Ov+OIO8OxWu77jEv+1B0VhjKrZUs=
github.com/opentracing/opentracing-go v1.2.0/go.mod h1:GxEUsuufX4nBwe+T+Wl9TAgYrxe9dPLANfrWvHYVTgc=
github.com/peterbourgon/diskv v2.0.1+incompatible h1:UBdAOUP5p4RWqPBg048CAvpKN+vxiaj6gdUUzhl4XmI=
Expand Down Expand Up @@ -316,6 +330,8 @@ github.com/shurcooL/githubv4 v0.0.0-20240727222349-48295856cce7/go.mod h1:zqMwyH
github.com/shurcooL/graphql v0.0.0-20230722043721-ed46e5a46466 h1:17JxqqJY66GmZVHkmAsGEkcIu0oCe3AM420QDgGwZx0=
github.com/shurcooL/graphql v0.0.0-20230722043721-ed46e5a46466/go.mod h1:9dIRpgIY7hVhoqfe0/FcYp0bpInZaT7dc3BYOprrIUE=
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/skeema/knownhosts v1.3.1 h1:X2osQ+RAjK76shCbvhHHHVl3ZlgDm8apHEHFqRjnBY8=
github.com/skeema/knownhosts v1.3.1/go.mod h1:r7KTdC8l4uxWRyK2TpQZ/1o5HaSzh06ePQNxPwTcfiY=
github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
Expand All @@ -338,6 +354,8 @@ github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXl
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/vbatts/tar-split v0.11.7 h1:ixZ93pO/GmvaZw4Vq9OwmfZK/kc2zKdPfu0B+gYqs3U=
github.com/vbatts/tar-split v0.11.7/go.mod h1:eF6B6i6ftWQcDqEn3/iGFRFRo8cBIMSJVOpnNdfTMFA=
github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=
github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
Expand Down Expand Up @@ -475,6 +493,9 @@ gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gotest.tools v2.2.0+incompatible h1:VsBPFP1AI068pPrMxtb/S8Zkgf9xEmTLJjfM+P5UIEo=
gotest.tools/v3 v3.5.1 h1:EENdUnS3pdur5nybKYIh2Vfgc8IUNBjxDPSjtiJcOzU=
gotest.tools/v3 v3.5.1/go.mod h1:isy3WKz7GK6uNw/sbHzfKBLvlvXwUyV06n6brMxxopU=
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
k8s.io/api v0.32.2 h1:bZrMLEkgizC24G9eViHGOPbW+aRo9duEISRIJKfdJuw=
k8s.io/api v0.32.2/go.mod h1:hKlhk4x1sJyYnHENsrdCWw31FEmCijNGPJO5WzHiJ6Y=
Expand Down
52 changes: 41 additions & 11 deletions pkg/cmd/tests/options.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,13 @@ import (
"regexp"
"strings"

"github.com/blang/semver"
"github.com/onsi/ginkgo/v2"
configassets "github.com/scylladb/scylla-operator/assets/config"
scyllav1 "github.com/scylladb/scylla-operator/pkg/api/scylla/v1"
"github.com/scylladb/scylla-operator/pkg/genericclioptions"
"github.com/scylladb/scylla-operator/pkg/helpers/slices"
scyllasemver "github.com/scylladb/scylla-operator/pkg/semver"
"github.com/scylladb/scylla-operator/test/e2e/framework"
"github.com/spf13/cobra"
apierrors "k8s.io/apimachinery/pkg/util/errors"
Expand All @@ -27,7 +29,7 @@ const (
digestRegexp = `[A-Za-z][A-Za-z0-9]*(?:[-_+.][A-Za-z][A-Za-z0-9]*)*[:][[:xdigit:]]{32,}`
)

var tagWithOptionalDigestRegexp = regexp.MustCompile("^" + referenceTagRegexp + "(?:@" + digestRegexp + ")?$")
var tagOrDigestRegexp = regexp.MustCompile("^(?:" + referenceTagRegexp + "(?:@" + digestRegexp + ")?|" + digestRegexp + ")$")

type IngressControllerOptions struct {
Address string
Expand Down Expand Up @@ -191,37 +193,65 @@ func (o *TestFrameworkOptions) Validate(args []string) error {
errors = append(errors, fmt.Errorf("gcs-service-account-key-path and s3-credentials-file-path can't be set simultanously"))
}

if !tagWithOptionalDigestRegexp.MatchString(o.ScyllaDBVersion) {
if !tagOrDigestRegexp.MatchString(o.ScyllaDBVersion) {
errors = append(errors, fmt.Errorf(
"invalid scylladb-version format: %q. Expected format: <tag>[@<digest>]",
"invalid scylladb-version format: %q. Expected format: <tag>, <tag>@<digest>, or <digest>",
o.ScyllaDBVersion,
))
}

if !tagWithOptionalDigestRegexp.MatchString(o.ScyllaDBUpdateFrom) {
if _, err := semver.Parse(o.ScyllaDBVersion); err != nil {
semScyllaDBVersion, fullScyllaDBVersion, err := scyllasemver.GetImageVersionAndDigest("scylla", o.ScyllaDBVersion)
if err != nil {
return fmt.Errorf("failed to resolve ScyllaDB version: %w", err)
}
fmt.Println("Extracted", semScyllaDBVersion, "from ScyllaDB version:", fullScyllaDBVersion)

o.ScyllaDBVersion = semScyllaDBVersion
}

if !tagOrDigestRegexp.MatchString(o.ScyllaDBUpdateFrom) {
Comment on lines +203 to +213
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't all of this be part of Operator not the tests?
We shouldn't manipulate the input parameters there to transform it to format accepted by the Operator, but Operator should accept any valid image reference and support it.

errors = append(errors, fmt.Errorf(
"invalid scylladb-update-from-version format: %q. Expected format: <tag>[@<digest>]",
"invalid scylladb-update-from-version format: %q. Expected format: <tag>, <tag>@<digest>, or <digest>",
o.ScyllaDBUpdateFrom,
))
}

if !tagWithOptionalDigestRegexp.MatchString(o.ScyllaDBUpgradeFrom) {
if _, err := semver.Parse(o.ScyllaDBUpdateFrom); err != nil {
semScyllaDBUpdateFromVersion, fullScyllaDBUpdateFromVersion, err := scyllasemver.GetImageVersionAndDigest("scylla", o.ScyllaDBUpdateFrom)
if err != nil {
return fmt.Errorf("failed to resolve ScyllaDB update from version: %w", err)
}
fmt.Println("Extracted", semScyllaDBUpdateFromVersion, "from ScyllaDB 'update from' version:", fullScyllaDBUpdateFromVersion)
o.ScyllaDBUpdateFrom = semScyllaDBUpdateFromVersion
}

if !tagOrDigestRegexp.MatchString(o.ScyllaDBUpgradeFrom) {
errors = append(errors, fmt.Errorf(
"invalid scylladb-upgrade-from-version format: %q. Expected format: <tag>[@<digest>]",
"invalid scylladb-upgrade-from-version format: %q. Expected format: <tag>, <tag>@<digest>, or <digest>",
o.ScyllaDBUpgradeFrom,
))
}

if !tagWithOptionalDigestRegexp.MatchString(o.ScyllaDBManagerVersion) {
if _, err := semver.Parse(o.ScyllaDBUpgradeFrom); err != nil {
semScyllaDBUpgradeFromVersion, fullScyllaDBUpgradeFromVersion, err := scyllasemver.GetImageVersionAndDigest("scylla", o.ScyllaDBUpgradeFrom)
if err != nil {
return fmt.Errorf("failed to resolve ScyllaDB upgrade from version: %w", err)
}
fmt.Println("Extracted", semScyllaDBUpgradeFromVersion, "from ScyllaDB 'upgrade from' version:", fullScyllaDBUpgradeFromVersion)
o.ScyllaDBUpgradeFrom = semScyllaDBUpgradeFromVersion
}

if !tagOrDigestRegexp.MatchString(o.ScyllaDBManagerVersion) {
errors = append(errors, fmt.Errorf(
"invalid scylladb-manager-version format: %q. Expected format: <tag>[@<digest>]",
"invalid scylladb-manager-version format: %q. Expected format: <tag>, <tag>@<digest>, or <digest>",
o.ScyllaDBManagerVersion,
))
}

if !tagWithOptionalDigestRegexp.MatchString(o.ScyllaDBManagerAgentVersion) {
if !tagOrDigestRegexp.MatchString(o.ScyllaDBManagerAgentVersion) {
errors = append(errors, fmt.Errorf(
"invalid scylladb-manager-agent-version format: %q. Expected format: <tag>[@<digest>]",
"invalid scylladb-manager-agent-version format: %q. Expected format: <tag>, <tag>@<digest>, or <digest>",
o.ScyllaDBManagerAgentVersion,
))
}
Expand Down
34 changes: 28 additions & 6 deletions pkg/controller/scylladbdatacenter/sync_statefulsets.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import (
"github.com/scylladb/scylla-operator/pkg/pointer"
"github.com/scylladb/scylla-operator/pkg/resourceapply"
"github.com/scylladb/scylla-operator/pkg/scyllaclient"
scyllasemver "github.com/scylladb/scylla-operator/pkg/semver"
"github.com/scylladb/scylla-operator/pkg/util/hash"
"github.com/scylladb/scylla-operator/pkg/util/parallel"
appsv1 "k8s.io/api/apps/v1"
Expand Down Expand Up @@ -976,15 +977,36 @@ func (sdcc *Controller) syncStatefulSets(
existingVersionString, existingVersionLabelPresent := existing.Labels[naming.ScyllaVersionLabel]

if requiredVersionLabelPresent && existingVersionLabelPresent {
requiredVersion, err := semver.Parse(requiredVersionString)
if err != nil {
return progressingConditions, err
parsedSemRequiredVersion, parseErr := semver.Parse(requiredVersionString)
if parseErr != nil {
semRequiredVersion, _, getVersionErr := scyllasemver.GetImageVersionAndDigest("scylla", requiredVersionString)
if getVersionErr != nil {
return progressingConditions, getVersionErr
}

parsedSemRequiredVersion, err = semver.Parse(semRequiredVersion)
if err != nil {
return progressingConditions, err
}
}
existingVersion, err := semver.Parse(existingVersionString)
if err != nil {
return progressingConditions, err

requiredVersion := parsedSemRequiredVersion

parsedSemExistingVersion, parseErr := semver.Parse(existingVersionString)
if parseErr != nil {
semExistingVersion, _, getVersionErr := scyllasemver.GetImageVersionAndDigest("scylla", existingVersionString)
if getVersionErr != nil {
return progressingConditions, getVersionErr
}

parsedSemExistingVersion, err = semver.Parse(semExistingVersion)
if err != nil {
return progressingConditions, err
}
}

existingVersion := parsedSemExistingVersion

if requiredVersion.Major != existingVersion.Major ||
requiredVersion.Minor != existingVersion.Minor {
// We need to run hooks for version upgrades.
Expand Down
Loading