Pass `-P-consume-incrementals` unless a PR specifically includes marker file `consume-incrementals` by jglick · Pull Request #991 · jenkins-infra/pipeline-library

jglick · 2026-02-09T15:40:25Z

Block usage of Incrementals dependencies not just during releases as jenkins-infra/jenkins-maven-cd-action#3 does, but “shifting left” into branch builds (master) as well as all PR builds by default including those used by both Dependabot and Renovate, so that the consume-incrementals profile would be active only locally (due to .mvn/maven.config) and in PR builds that specifically request it. Should offer a first line of defense against problems like jenkins-infra/helpdesk#4990 (comment).

Required by jenkins-infra/helpdesk#4998. Probably harmless to merge first, but useless because as seen in jenkinsci/jenkins-infra-test-plugin#212 the Incrementals deps are loaded anyway, pending jenkins-infra/helpdesk#4998.

lemeurherve · 2026-02-10T19:54:35Z

https://github.com/search?q=org%3Ajenkins-infra%20ARTIFACT_CACHING_PROXY_SERVERID&type=code cannot find where it is defined

I don't trust GitHub search.

Recommanding https://github.com/gabrie30/ghorg to search across a local copy of @jenkins-infra and/or @jenkinsci¹

$ rg ARTIFACT_CACHING_PROXY_SERVERID
jenkins-infra/dist/profile/templates/jenkinscontroller/casc/clouds-kubernetes.yaml.erb
64:                  key: "ARTIFACT_CACHING_PROXY_SERVERID"

jenkins-infra/dist/profile/templates/jenkinscontroller/casc/clouds-azurevm.yaml.erb
39:          [System.Environment]::SetEnvironmentVariable("ARTIFACT_CACHING_PROXY_SERVERID", "<%= @jcasc['artifact_caching_proxy']['servers'][cloudsetup['acpServerId']]['url'] %>", "Machine")
40:          $Env:ARTIFACT_CACHING_PROXY_SERVERID = "<%= @jcasc['artifact_caching_proxy']['servers'][cloudsetup['acpServerId']]['url'] %>"
124:            export ARTIFACT_CACHING_PROXY_SERVERID='<%= @jcasc['artifact_caching_proxy']['servers'][cloudsetup['acpServerId']]['url'] %>'
148:            Environment="ARTIFACT_CACHING_PROXY_SERVERID=^${ARTIFACT_CACHING_PROXY_SERVERID}"

jenkins-infra/dist/profile/templates/jenkinscontroller/casc/clouds-ec2.yaml.erb
82:            - key: "ARTIFACT_CACHING_PROXY_SERVERID"

pipeline-library/test/groovy/InfraStepTests.groovy
194:    env.ARTIFACT_CACHING_PROXY_SERVERID = 'https://foo:1313'
205:    assertTrue(assertMethodCallContainsPattern('echo', "INFO: using artifact caching proxy server '${env.ARTIFACT_CACHING_PROXY_SERVERID}'."))
207:    assertTrue(assertMethodCallContainsPattern('configFile', "fileId=${env.ARTIFACT_CACHING_PROXY_SERVERID}"))
226:    assertTrue(assertMethodCallContainsPattern('echo', "WARNING: artifact caching proxy is enabled but the provided 'ARTIFACT_CACHING_PROXY_SERVERID' setup on the agent is empty, will use repo.jenkins-ci.org."))
254:    env.ARTIFACT_CACHING_PROXY_SERVERID = 'https://foo:1313'
268:    assertTrue(assertMethodCallContainsPattern('configFile', "fileId=${env.ARTIFACT_CACHING_PROXY_SERVERID}"))
279:    env.ARTIFACT_CACHING_PROXY_SERVERID = 'https://foo:1313'
293:    assertFalse(assertMethodCallContainsPattern('configFile', "fileId=${env.ARTIFACT_CACHING_PROXY_SERVERID}"))
304:    env.ARTIFACT_CACHING_PROXY_SERVERID = 'https://foo:1313'
317:    assertTrue(assertMethodCallContainsPattern('configFile', "fileId=${env.ARTIFACT_CACHING_PROXY_SERVERID}"))
327:    env.ARTIFACT_CACHING_PROXY_SERVERID = 'https://foo:1313'
341:    assertTrue(assertMethodCallContainsPattern('echo', "WARNING: the artifact caching proxy server '${env.ARTIFACT_CACHING_PROXY_SERVERID}' isn't reachable, will use repo.jenkins-ci.org."))
343:    assertFalse(assertMethodCallContainsPattern('echo', "INFO: using artifact caching proxy server '${env.ARTIFACT_CACHING_PROXY_SERVERID}'."))
345:    assertFalse(assertMethodCallContainsPattern('configFile', "fileId=${env.ARTIFACT_CACHING_PROXY_SERVERID}"))
357:    env.ARTIFACT_CACHING_PROXY_SERVERID = 'https://foo:1313'

pipeline-library/vars/infra.groovy
185: * Artifact Caching Proxy server defined in agent's env.ARTIFACT_CACHING_PROXY_SERVERID variable.
191:  final String acpServerId = env.ARTIFACT_CACHING_PROXY_SERVERID
194:    echo "WARNING: artifact caching proxy is enabled but the provided 'ARTIFACT_CACHING_PROXY_SERVERID' setup on the agent is empty, will use repo.jenkins-ci.org."

pipeline-library/vars/infra.txt
44:     Artifact Caching Proxy server defined in agent's env.ARTIFACT_CACHING_PROXY_SERVERID variable.

took less than 4 minutes to clone its 2716 public repos (14.5GB) on my mac mini m4 ↩

lemeurherve · 2026-02-10T19:59:05Z

@lemeurherve given jenkinsci/jenkins-infra-test-plugin#207 (comment) can we make sure the proxy ignores the incrementals repo? Or is there some way to adjust the Maven config? Like
<mirrorOf>*,!incrementals</mirrorOf>

Most probably, here is the current config of ACP for our controllers on VMs (ci.jenkins.io, trusted.ci.jenkins.io & cert.ci.jenkins.io): https://github.com/jenkins-infra/jenkins-infra/blob/dcd24c38d6a61d2d0596f1dc6d84819a16e5ee75/dist/profile/templates/jenkinscontroller/casc/artifact-caching-proxy.yaml.erb

See blame for evolution and links to the corresponding helpdesk issues. (incrementals has been discussed previously AFAIR, but details are a bit fuzzy to me atm)

jglick · 2026-02-11T13:46:00Z

@lemeurherve @dduportal I think I need help here, since I do not really know how the ACP is set up, and jenkins-infra/jenkins-infra#4669 did not work—it is apparently still serving artifacts from the incrementals repo, which makes me think jenkins-infra/jenkins-infra#3350 was wrong in this regard at least: ACP should be caching artifacts from Jenkins releases as well as Central. It is actually fine if ACP caches incrementals (mainly for negative lookups, so we do not hit maven_metadata.xml for dependencies which do not actually exist yet which do not fail the build) but it needs to do so under a distinct repository id, like it did prior to jenkins-infra/jenkins-infra#3350 (AFAICT) so that you can turn this repo on or off.

timja · 2026-02-11T14:27:58Z

Context in here:

jenkins-infra/helpdesk#3969 (comment)

and Basil explaining consequence here:
jenkins-infra/helpdesk#3969 (comment)

vars/buildPlugin.groovy

dduportal · 2026-02-12T11:03:09Z

Context in here:

jenkins-infra/helpdesk#3969 (comment)

and Basil explaining consequence here: jenkins-infra/helpdesk#3969 (comment)

Please note that the gain of caching incrementals has been surprisingly huge in term of bandwidth consumed from repo.jenkins-ci.org.

If you are blocked by this, the config is here: https://github.com/jenkins-infra/helm-charts/blob/9d8ae214c001fa28598a7438556e0dcc73d828ca/charts/artifact-caching-proxy/templates/nginx-proxy-configmap.yaml#L133-L158

=> the fallback_jenkins_incrementals location should be removed (and https://github.com/jenkins-infra/helm-charts/blob/9d8ae214c001fa28598a7438556e0dcc73d828ca/charts/artifact-caching-proxy/templates/nginx-proxy-configmap.yaml#L134 must be adapted) since jenkins-infra/jenkins-infra#4669 has been set up.

Be careful. it WILL have side effect, I can't recall which ones.

…brary into disable-consume-incrementals

jglick · 2026-02-13T23:54:22Z

vars/buildPlugin.groovy

                    mavenOptions += "help:evaluate -Dexpression=changelist -Doutput=$changelistF"
                  }
                }
+                if (fileExists('consume-incrementals')) {


Alternately, we could just have a marker file incrementals which would gate both production and consumption of incrementals. Thus, when developing paired upstream/downstream PRs, you would need to create this file on both PRs, and delete it prior to merging. (Assuming CD, the upstream PR would deploy a release after merging. jenkins-infra/interesting-category-action#1 would make this friendlier.)

I'd rather avoid the production one unless its really needed.

Pass -P-consume-incrementals on origin repo builds

35c3f65

jglick requested a review from a team as a code owner February 9, 2026 15:40

This comment was marked as outdated.

Sign in to view

This was referenced Feb 9, 2026

Pretest consume-incrementals change: forked PR jenkinsci/jenkins-infra-test-plugin#205

Closed

Pretest consume-incrementals change: origin PR jenkinsci/jenkins-infra-test-plugin#206

Closed

jglick marked this pull request as draft February 9, 2026 15:54

This comment was marked as outdated.

Sign in to view

This was referenced Feb 9, 2026

Pretest consume-incrementals change on BOM: origin PR jenkinsci/jenkins-infra-test-plugin#207

Closed

Pretest consume-incrementals change on BOM: forked PR jenkinsci/jenkins-infra-test-plugin#208

Closed

This comment was marked as outdated.

Sign in to view

jglick mentioned this pull request Feb 10, 2026

Do not mirror the incrementals repo jenkins-infra/jenkins-infra#4669

Merged

This comment was marked as outdated.

Sign in to view

lemeurherve mentioned this pull request Feb 11, 2026

Bogus org.jenkins-ci.plugins:plugin:6.2126.v08295c851985 jenkins-infra/helpdesk#4990

Closed

timja reviewed Feb 11, 2026

View reviewed changes

vars/buildPlugin.groovy Outdated Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

jglick added 2 commits February 13, 2026 17:40

Merge branch 'master' of https://github.com/jenkins-infra/pipeline-li…

7ad5352

…brary into disable-consume-incrementals

Switching to marker file approach

75cc91a

This was referenced Feb 13, 2026

Pretest lib PR-991: no use of Incrementals jenkinsci/jenkins-infra-test-plugin#210

Draft

Pretest lib PR-991: declared use of Incrementals jenkinsci/jenkins-infra-test-plugin#211

Draft

jglick mentioned this pull request Feb 13, 2026

Pretest lib PR-991: undeclared use of Incrementals jenkinsci/jenkins-infra-test-plugin#212

Draft

Syntax error

e9d1b33

jglick changed the title ~~Pass -P-consume-incrementals on origin repo builds~~ Pass -P-consume-incrementals unless a PR specifically requests them with marked file consume-incrementals Feb 13, 2026

jglick changed the title ~~Pass -P-consume-incrementals unless a PR specifically requests them with marked file consume-incrementals~~ Pass -P-consume-incrementals unless a PR specifically includes marker file consume-incrementals Feb 13, 2026

jglick added 3 commits February 13, 2026 18:23

Wrong step name

ba892ab

github-checks apparently neglects to escape backticks

6c4de93

Typo

05af9b1

jglick mentioned this pull request Feb 13, 2026

Stop mirroring incrementals from ACP jenkins-infra/helpdesk#4998

Open

jglick commented Feb 13, 2026

View reviewed changes

Uh oh!

Conversation

jglick commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

lemeurherve commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Footnotes

Uh oh!

lemeurherve commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

jglick commented Feb 11, 2026

Uh oh!

timja commented Feb 11, 2026

Uh oh!

Uh oh!

dduportal commented Feb 12, 2026

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

jglick Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

timja Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jglick commented Feb 9, 2026 •

edited

Loading

lemeurherve commented Feb 10, 2026 •

edited

Loading

lemeurherve commented Feb 10, 2026 •

edited

Loading