Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support for custom tag and build opts through env #62

Open
wants to merge 32 commits into
base: main
Choose a base branch
from
Open

feat: support for custom tag and build opts through env #62

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
6fae4f6
fixed develop build
vikramdevtron Nov 19, 2024
90ec3a6
Merge pull request #49 from devtron-labs/fix-develop-builds
vikramdevtron Nov 19, 2024
b8de2c1
main sync develop
vikramdevtron Nov 22, 2024
8c8c007
Merge pull request #57 from devtron-labs/main-sync-develop-nov22
vikramdevtron Nov 22, 2024
3eec9fe
added func for sourcebashrc
ayu-devtron Nov 22, 2024
a156321
enabling support for envs in build artifact
ayu-devtron Nov 22, 2024
ef3fdcd
compile fix
ayu-devtron Nov 22, 2024
20f53f8
handle empty check
ayu-devtron Nov 22, 2024
621af28
handle not configure env variable
ayu-devtron Nov 22, 2024
60563a9
if check
ayu-devtron Nov 22, 2024
46b4d63
handle not changed val
ayu-devtron Nov 22, 2024
08db920
remove empty check
ayu-devtron Nov 22, 2024
32f1a6b
bug fix
ayu-devtron Nov 22, 2024
4cc6b42
add multi tag support build and push non-buildx
ayu-devtron Nov 25, 2024
8608a13
build full url for custom tag
ayu-devtron Nov 25, 2024
71388e4
support for buildx
ayu-devtron Nov 25, 2024
ab2b1da
fix log
ayu-devtron Nov 25, 2024
bd23206
improve code for buildx support
ayu-devtron Nov 26, 2024
c3a63aa
using full image url for buildx
ayu-devtron Nov 26, 2024
0cbad7a
use GetGitRepoNameFromGitRepoUrl for getting project name from git url
prakash100198 Nov 26, 2024
f2df4a2
remove unnecessary logs and functions
ayu-devtron Nov 27, 2024
3e6023e
remove unused func
prakash100198 Nov 27, 2024
368fc70
Merge pull request #63 from devtron-labs/git-url-parser-fix
prakash100198 Nov 27, 2024
ab13066
Merge branch 'main' into oss-sync-27-nov-2024
prakash100198 Nov 27, 2024
76c4a4a
Merge pull request #64 from devtron-labs/oss-sync-27-nov-2024
prakash100198 Nov 27, 2024
bb7d0c4
go mod vendor update
vikramdevtron Nov 27, 2024
c666c49
build fix
vikramdevtron Nov 27, 2024
8edd543
remove github.com/devtron-labs/kubelink from kubeoink
prakash100198 Nov 27, 2024
f18b768
Merge pull request #67 from devtron-labs/fix-develop-build
prakash100198 Nov 27, 2024
cd27adb
Merge branch 'main' into main-sync-develop-dec2
vikramdevtron Dec 2, 2024
d726745
Merge pull request #69 from devtron-labs/main-sync-develop-dec2
vikramdevtron Dec 2, 2024
ee0dcab
Merge branch 'develop' into docker-env-build
badal773 Dec 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion chart-sync/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ go 1.22.4
toolchain go1.22.6

replace (
github.com/devtron-labs/common-lib => github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241121092824-79be1ba0e3da
github.com/devtron-labs/common-lib => github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241127070215-76c4a4abfeca
helm.sh/helm/v3 v3.14.3 => github.com/devtron-labs/helm/v3 v3.14.1-0.20240401080259-90238cf69e42
)

Expand Down Expand Up @@ -123,3 +123,5 @@ require (
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
sigs.k8s.io/yaml v1.3.0 // indirect
)

replace github.com/devtron-labs/chart-sync => github.com/devtron-labs/devtron-services/chart-sync v0.0.0-20241024131256-7d3ba621dece
4 changes: 2 additions & 2 deletions chart-sync/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ github.com/creack/pty v1.1.18/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241121092824-79be1ba0e3da h1:wbqNhTahQp+c/htUfGxLtuzA0dddOcvdPtlSViKBJ70=
github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241121092824-79be1ba0e3da/go.mod h1:NJSMdv+zTUK3p7rML12RZSeAUKHeLaoY3sR/oK0xhwo=
github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241127070215-76c4a4abfeca h1:WXh3S0n1Bu26lWEB05CvH9Lc3NzUlqa9gbWeVCVGkhE=
github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241127070215-76c4a4abfeca/go.mod h1:NJSMdv+zTUK3p7rML12RZSeAUKHeLaoY3sR/oK0xhwo=
github.com/devtron-labs/helm/v3 v3.14.1-0.20240401080259-90238cf69e42 h1:pJmK44QaSztOiZe0iQHNf0sdy5KwkAeceydyhOG4RaY=
github.com/devtron-labs/helm/v3 v3.14.1-0.20240401080259-90238cf69e42/go.mod h1:v6myVbyseSBJTzhmeE39UcPLNv6cQK6qss3dvgAySaE=
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78=
Expand Down
4 changes: 2 additions & 2 deletions chart-sync/vendor/modules.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ github.com/containerd/platforms
# github.com/davecgh/go-spew v1.1.1
## explicit
github.com/davecgh/go-spew/spew
# github.com/devtron-labs/common-lib v0.0.0 => github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241121092824-79be1ba0e3da
# github.com/devtron-labs/common-lib v0.0.0 => github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241127070215-76c4a4abfeca
## explicit; go 1.21
github.com/devtron-labs/common-lib/helmLib/registry
github.com/devtron-labs/common-lib/utils/http
Expand Down Expand Up @@ -785,4 +785,4 @@ sigs.k8s.io/structured-merge-diff/v4/value
# sigs.k8s.io/yaml v1.3.0
## explicit; go 1.12
sigs.k8s.io/yaml
# github.com/devtron-labs/common-lib => github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241121092824-79be1ba0e3da
# github.com/devtron-labs/common-lib => github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241127070215-76c4a4abfeca
32 changes: 27 additions & 5 deletions ci-runner/executor/stage/ciStages.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -387,7 +387,7 @@ func (impl *CiStage) runBuildArtifact(ciCdRequest *helper.CiCdTriggerEvent, metr
// build
start := time.Now()
metrics.BuildStartTime = start
dest, err := impl.dockerHelper.BuildArtifact(ciCdRequest.CommonWorkflowRequest) // TODO make it skipable
dest, err := impl.dockerHelper.BuildArtifact(ciCdRequest.CommonWorkflowRequest, scriptEnvs, preCiStageOutVariable) // TODO make it skipable
metrics.BuildDuration = time.Since(start).Seconds()
if err != nil {
log.Println("Error in building artifact", "err", err)
Expand All @@ -410,7 +410,7 @@ func (impl *CiStage) runBuildArtifact(ciCdRequest *helper.CiCdTriggerEvent, metr
return dest, err
}

func (impl *CiStage) extractDigest(ciCdRequest *helper.CiCdTriggerEvent, dest string, metrics *helper.CIMetrics, artifactUploaded bool) (string, error) {
func (impl *CiStage) extractDigest(ciCdRequest *helper.CiCdTriggerEvent, dest string, metrics *helper.CIMetrics, artifactUploaded bool, scriptEnvs map[string]string, preCiStageOutVariable map[int]map[string]*helper.VariableObject) (string, error) {

var digest string
var err error
Expand All @@ -423,7 +423,7 @@ func (impl *CiStage) extractDigest(ciCdRequest *helper.CiCdTriggerEvent, dest st
} else {
// push to dest
log.Println(util.DEVTRON, "Docker push Artifact", "dest", dest)
err = impl.pushArtifact(ciCdRequest, dest, digest, metrics, artifactUploaded)
err = impl.pushArtifact(ciCdRequest, dest, digest, metrics, artifactUploaded, scriptEnvs, preCiStageOutVariable)
if err != nil {
return err
}
Expand Down Expand Up @@ -496,7 +496,7 @@ func (impl *CiStage) getImageDestAndDigest(ciCdRequest *helper.CiCdTriggerEvent,
if err != nil {
return "", "", err
}
digest, err := impl.extractDigest(ciCdRequest, dest, metrics, artifactUploaded)
digest, err := impl.extractDigest(ciCdRequest, dest, metrics, artifactUploaded, scriptEnvs, preCiStageOutVariable)
if err != nil {
log.Println("Error in extracting digest", "err", err)
return "", "", err
Expand Down Expand Up @@ -564,9 +564,10 @@ func sendCDFailureEvent(ciRequest *helper.CommonWorkflowRequest, err *helper.CdS
}
}

func (impl *CiStage) pushArtifact(ciCdRequest *helper.CiCdTriggerEvent, dest string, digest string, metrics *helper.CIMetrics, artifactUploaded bool) error {
func (impl *CiStage) pushArtifact(ciCdRequest *helper.CiCdTriggerEvent, dest string, digest string, metrics *helper.CIMetrics, artifactUploaded bool, scriptEnvs map[string]string, preCiStageOutVariable map[int]map[string]*helper.VariableObject) error {
imageRetryCountValue := ciCdRequest.CommonWorkflowRequest.ImageRetryCount
imageRetryIntervalValue := ciCdRequest.CommonWorkflowRequest.ImageRetryInterval

var err error
for i := 0; i < imageRetryCountValue+1; i++ {
if i != 0 {
Expand All @@ -585,6 +586,27 @@ func (impl *CiStage) pushArtifact(ciCdRequest *helper.CiCdTriggerEvent, dest str
WithFailureMessage(workFlow.PushFailed.String()).
WithArtifactUploaded(artifactUploaded)
}
multiDockerTagsValue := helper.GetMultiDockerTagsValue(scriptEnvs, preCiStageOutVariable)
if len(multiDockerTagsValue) > 0 {
tags := strings.Split(multiDockerTagsValue, `,`)
for _, tmpDockerTag := range tags {
if !strings.Contains(tmpDockerTag, ":") {
fullImageUrl, err := helper.BuildDockerImagePathForCustomTag(ciCdRequest.CommonWorkflowRequest, tmpDockerTag)
if err != nil {
log.Println("Error in building docker image", "fullImageUrl", fullImageUrl, "err", err)
return err
}
tmpDockerTag = fullImageUrl
}
log.Println(" -----> custom-tag push " + tmpDockerTag)
ciContext := cicxt.BuildCiContext(context.Background(), ciCdRequest.CommonWorkflowRequest.EnableSecretMasking)
err = impl.dockerHelper.PushArtifact(ciContext, tmpDockerTag)
if err != nil {
log.Println("Error in pushing artifact", "artifact", tmpDockerTag, "err", err)
}
}

}
return err
}

Expand Down
4 changes: 2 additions & 2 deletions ci-runner/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ go 1.21

toolchain go1.21.8

replace github.com/devtron-labs/common-lib => github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241121092824-79be1ba0e3da
replace github.com/devtron-labs/common-lib => github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241127070215-76c4a4abfeca

require (
github.com/Knetic/govaluate v3.0.0+incompatible
Expand Down Expand Up @@ -125,4 +125,4 @@ require (
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
sigs.k8s.io/yaml v1.3.0 // indirect
)
)
4 changes: 2 additions & 2 deletions ci-runner/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241121092824-79be1ba0e3da h1:wbqNhTahQp+c/htUfGxLtuzA0dddOcvdPtlSViKBJ70=
github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241121092824-79be1ba0e3da/go.mod h1:NJSMdv+zTUK3p7rML12RZSeAUKHeLaoY3sR/oK0xhwo=
github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241127070215-76c4a4abfeca h1:WXh3S0n1Bu26lWEB05CvH9Lc3NzUlqa9gbWeVCVGkhE=
github.com/devtron-labs/devtron-services/common-lib v0.0.0-20241127070215-76c4a4abfeca/go.mod h1:NJSMdv+zTUK3p7rML12RZSeAUKHeLaoY3sR/oK0xhwo=
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/docker/cli v24.0.6+incompatible h1:fF+XCQCgJjjQNIMjzaSmiKJSCcfcXb3TWTcc7GAneOY=
Expand Down
Loading
Loading