v3.260514

The main purpose of this release is to try the ORA-00001 failed block-publication retry mechanism in production.

What's Changed

• Stop mounting CVMFS inside containers. Completes #9235 by @belforte in #9308
• update DebugFailedBlockPublication.py for #9309 by @belforte in #9310
• save blocks which fail to publish also in JSON format by @belforte in #9312
• Add DbsApi wrapper to catch and retry ORA-00001, failed block publication. by @nausikt in #9315

Full Changelog: v3.260428...v3.260514

v3.260428

Most urgent is fix for Publisher (limit at 100K lumis per block).
It is also good to deploy the fix for $FRONTIER_ID ( #9306 )

What's Changed

• add lsof and strace to containers by @belforte in #9301
• Revert add jitter delays to Publisher DBS API calls to prevent throttling. by @nausikt in #9304
• maxLumisPerBlock = 100k. Fix #9302 by @belforte in #9305
• CRAB_Reqname, not ReqName. Anyhow use grep -i. Fix #9306 by @belforte in #9307

Full Changelog: v3.260418...v3.260428

Validation

• https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14639785 - OK

v3.260418

Hotfix for the current chaos: O(112k) publication files are stuck in Acquire, forming a baseline of terminally failed migrations. [#9299]

What's Changed

• Fix a few logging typos in gitlab cicd by @oljemark in #9298
• Mitigate terminally failed migration baseline by adding jitter delays between our Publisher DBS API calls. by @nausikt in #9300

Full Changelog: v3.260415...v3.260418

v3.260415

Main reason of the tag is to deploy the possibility for sites to require ASO via Rucio
as requested by Italian T2's

What's Changed

• Use python3 for wmc httpd by @belforte in #9291
• Add manual pipeline by @belforte in #9293
• deprecate (sidecar) crab-k8s-overlays, switching CI to mono-repo approach by @nausikt in #9279
• Allow sites to force ASO via Rucio. Fix #9294 by @belforte in #9297

Full Changelog: v3.260407...v3.260415

Validation

• validation pipeline https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14533059 OK

v3.260407

A technical release to try out the new CI test pipeline.
The only change to the code is the very minor fix for Rucio ASO.

What's Changed

• Tune claude-generated script by @belforte in #9289
• refactor test pipeline - version 2 by @belforte in #9287
• during Rucio ASO never wait >8h for running PJ. Fix $9079 by @belforte in #9290

Full Changelog: v3.260330...v3.260407

Validation

• automatic pipeline https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14467823 : OK

v3.260330

The purpose of this tag is to release Alma9-based CRABServer, TaskWorker for further testing.

What's Changed

• Use local disk to read groups info by @belforte in #9278
• Migration to Alma9 with py3.9 on CRABServer & TW by @nausikt in #9265
• Use timeless importlib for module importing by @nausikt in #9253
• add script from Claude by @belforte in #9286

Full Changelog: v3.260226...v3.260330

v3.260226

Quick patch for v3.260221 in which RenewRemoteProxies is not working.
It was the only place where we still had import htcondor instead if import htcondor2 as htcondor (same for classad)

What's Changed

• use v2 HTCondor API. so it works with HTC 25.x and later by @belforte in #9277

Full Changelog: v3.260221...v3.260226

v3.260221

Main reason is to capture change to min. requires CMSSW version for enabling token in job wrapper.
Other changes are mostly for build pipeline and cleanup and should not affect user submissions.
Yet we are still running v3.251015-stable so we better check this well and update production !

What's Changed

• drop GIT_SSL_CAINFO, GIT_EXEC_PATH to use system git instead. by @nausikt in #9263
• Add again dagman config file by @belforte in #9240
• remove obsolete scripts/Monitor by @belforte in #9266
• rename submission classads CUDA* to new HTCondor vendor agnostic, DESIRED_GPU*, refactored acceleratedParams interface. by @nausikt in #9258
• remove config examples now in puppet. by @belforte in #9267
• Fix python2/3 compatibility for Status Tracking with different CMSSW versions by @aspiringmind-code in #9268
• remove LDAP dependency by @belforte in #9269
• add script GetHighPriorityUsersFromCric.py for #9235 by @belforte in #9274
• only enable token in jobs for CMSSW >=15. For #9273 by @belforte in #9275

Full Changelog: v3.260116...v3.260221

Validation

• CICD pipeline https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14100592 - failed because some tasks were picked by tw04
• running again after stopping tw04 : pipeline https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14113733 OK
• pipeline with StatusTracking https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14118490 - failed most likely because tests started running while tasks had not started yet
• pipeline with Status Tracking only https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14121173 - failed due to #9213 not being fully solved yet
• I submitted manually all ST tasks for CMSSW_13 from my account and after a few resubmit all completed successfully inclluding publication - OK

v3.260116

The purpose of this tag is to revert resubmission refactoring, wrap-up changes made Since Oct 15. Put them into test after a while.

What's Changed

• workaround for remote proxy renewal failed in delegation mode. by @nausikt in #9256
• Revert "Refactoring resubmission (#9228)" by @aspiringmind-code in #9255
• add resubmissions study script by @belforte in #9259
• [PEP632] deprecate distutils module in py310 by @nausikt in #9260
• [PEP389] introducing of argparse and the deprecation of optparse, since py2.7+, py3.2+ by @nausikt in #9261

Full Changelog: v3.251216...v3.260116

v3.251216

Hotfixes, Follow-up patches of panic releases v3.251214, v3.251215. Actually nothing much goes wrongs with the new codes, Fatal cause by the corrupted image. Other than that is to make atomicReplaceAcrossFS py2 compatible with old CMSSW envs that use python2, downgrade htcondor.

What's Changed

• hotfix, forgot to deprecate retires, delays args from wrapper. by @nausikt in #9247
• make atomicReplaceAcrossFS a py2 compatible. by @nausikt in #9248
• downgrade htcondor to 25.0.3 by @nausikt in #9249

Full Changelog:

• v3.251213...v3.251214
• v3.251214...v3.251215
• v3.251215...v3.251216