das-up-to-nevents.py picks up golden runs from cvmfs

[smuzaffar]

@AdrianoDee, as mentioned in #50094 (comment) , looks like https://github.com/cms-sw/cmssw/blob/master/Configuration/PyReleaseValidation/scripts/das-up-to-nevents.py picks up golden run number from /cvmfs e.g. /cvmfs/cms-griddata.cern.ch/cat/metadata/DC/Collisions25//latest//Cert_Collisions2025_391658_398903_Golden.json for workflow 2025.0010001. As the files on cvmfs are not in bots control so any change in these files results in different run/input used by baseline and PR tests.

I was wondering how can we make sure that bot uses the same golden runs from cvmfs? I see there are dated directories . May be we can have a symlink which points to latest date and when we run baseline then we record the dated directory name along with the baseline results and let PR tests use the same dated directory . This hopefully will get rid of issues like

The workflows 2025.0010001, 2024.0050001, 2023.0020001, 2022.0030001 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

[cmsbuild]

cms-bot internal usage

[cmsbuild]

A new Issue was created by @smuzaffar.

@Dr15Jones, @ftenchini, @makortel, @mandrenguyen, @sextonkennedy, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[smuzaffar]

FYI @AdrianoDee

[makortel]

assign pdmv

[cmsbuild]

New categories assigned: pdmv

@AdrianoDee,@antoniovagnerini,@DickyChant,@miquork you have been requested to review this Pull request/Issue and eventually sign? Thanks

[AdrianoDee]

Hi @smuzaffar: for my understanding, this happens since in the time that occurred between the "baseline" and the "PR" runs, the golden json file changed?

[smuzaffar]

that is also my guess @AdrianoDee .
From the file time stamp I see that files were change at Feb 9 23:35 so ideally both CMSSW_16_1_X_2026-02-09-2300 IB's baseline (which was run around 7am today) and PR tests should have used the same /cvmfs/cms-griddata.cern.ch/cat/metadata/DC/Collisions25//latest contents.

[smuzaffar]

@AdrianoDee , though both IB baseline and PR elvals should have seen the same contents of same /cvmfs/cms-griddata.cern.ch/cat/metadata/DC/Collisions25//latest but for baseline das-up-to-nevents.py selected lumirange [a] while for PR tests it use [b].

[a] https://cmssdt.cern.ch/SDT/jenkins-artifacts/ib-baseline-tests/CMSSW_16_1_X_2026-02-09-2300/el8_amd64_gcc13/AuthenticAMD/matrix-results/2025.0010001_RunJetMET02025C_10k/step1_lumiRanges.log

{"392295": [[39, 43]]}

[b] https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-547a8e/51214/runTheMatrix-results/2025.0010001_RunJetMET02025C_10k/step1_lumiRanges.log

{"392295": [[1, 4]]}

[smuzaffar]

OK, looks like issue is with the dasgoclient results which were changed 8 hours ago for the query dasgoclient --query='file,lumi,run dataset=/JetMET0/Run2025C-v1/RAW '

previously our cached had https://github.com/cms-sw/cms-sw.github.io/blob/135bab0c3085664f61d4f394f3839441afdf1822/das_queries/42/42c9caff17a9007ec022e68773d36c9b44c2e49d3da7aac4a7f625095b8d8d82 and 8 hours ago this changed and now our cache has https://github.com/cms-sw/cms-sw.github.io/blob/1bd529e77095928b5886d6b39bd4e218ccf6a1d4/das_queries/42/42c9caff17a9007ec022e68773d36c9b44c2e49d3da7aac4a7f625095b8d8d82