Remove alpaka procModfier from workflows in which it is no longer useful

[VourMa]

In #48921, Patatrack quadraplet pixel tracks were made the default in CMSSW, so the alpaka modifier was rendered useless in a quite a few workflows which were using it to enable Patatrack pixel tracking. This PR removes the alpaka procModifier from these workflows. One workflow is completely removed, as it was duplicate of another, and that leads to a small change in the workflow numbering.

The PR was validated by making sure that all the modified workflows succeed.

FYI @rovere and @waredjeb: Since the alpaka procModifier is used in Phase 2 HLT to enable the HGCal heterogeneous reconstruction, I wanted to ask whether it is useful to keep it in any of the modified workflows.

[cmsbuild]

cms-bot internal usage

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-49755/47340

• There are other open Pull requests which might conflict with changes you have proposed:
  • File Configuration/PyReleaseValidation/python/relval_Run4.py modified in PR(s): GNN Track-Tracksters Linking in TICLv5 #49652
  • File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): GNN Track-Tracksters Linking in TICLv5 #49652

[cmsbuild]

A new Pull Request was created by @VourMa for master.

It involves the following packages:

• Configuration/PyReleaseValidation (pdmv)

@AdrianoDee, @DickyChant, @antoniovagnerini, @cmsbuild, @miquork can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @fabiocos, @makortel, @slomeo this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[mmusich]

assign hlt

• given the nature of the content

[cmsbuild]

New categories assigned: hlt

@Martin-Grunewald,@mmusich you have been requested to review this Pull request/Issue and eventually sign? Thanks

[mmusich]

@VourMa

Thanks for the work on this, but I don’t think we should proceed with these changes.

By removing alpaka, we effectively lose the heterogeneous HGCAL workflow. alpaka is not something that was used only for Pixel tracking; it encodes an architectural meaning in the workflows. If we drop it, that meaning doesn’t stay abstractly in the system — it simply disappears, and at that point we’re not just simplifying, we’re changing the workflow model. So the question becomes: why are we intentionally changing the workflow? If there isn’t a clear and strong justification, I think it’s better to keep alpaka present.

In my view, the alpaka process modifier also has an important role in the development lifecycle: it’s very useful as a transitional mechanism for heterogeneous features that are mature enough to go into release, but not yet ready to become the default. It provides a structured “staging area” where we can deploy and evaluate heterogeneous components in realistic conditions without forcing them as baseline immediately. I can easily imagine, for instance, something like Pixel vertexing on GPU going through an alpaka-gated modifier first, and then eventually becoming the default once validated and stable enough.

For these reasons, I think removing alpaka here is a step backwards in terms of workflow expressiveness and development flexibility, so I would not support the change as it stands.

[VourMa]

Hi @mmusich, these are valid concerns that I thought a bit about. Let me reply inline and we can discuss further:

By removing alpaka, we effectively lose the heterogeneous HGCAL workflow. alpaka is not something that was used only for Pixel tracking; it encodes an architectural meaning in the workflows. If we drop it, that meaning doesn’t stay abstractly in the system — it simply disappears, and at that point we’re not just simplifying, we’re changing the workflow model. So the question becomes: why are we intentionally changing the workflow? If there isn’t a clear and strong justification, I think it’s better to keep alpaka present.

The alpaka procModifier is not removed from all the workflows - sorry if that was the message passed by the PR description, and I can change it if you think it is better to do so.
The alpaka procModifier is removed only from the workflows in which was introduced only to have Patatrack pixel tracks running. Patatrack pixel tracking is now the default, hence the procModifier in these specific workflows is redundant. Note that the alpaka procModifier is already absent in newer "heterogeneous" workflows, e.g. 0.7511, 0.774 or 0.775 (even though these mention "alpaka" explicitly in the README description). Given the above, this is an effort to homogenize the procModifier sequences in similar workflows. One could go the other way (introducing alpaka in the workflows I just mentioned) but this sounds like an unnecessary complication to me.
On the other hand, the alpaka procModifier has remained in the general "alpaka" HLT workflow (0.751) and the "heterogeneous" NGT scouting workflow (0.771). These workflows can still be used to test the HGCal workflow, and maybe a dedicated workflow for that would be even better/cleaner.
Finally, if we want a workflow that properly tests all of the heterogeneous developments together, we should create one and clearly label it as such, as it is currently not there. I would be happy to do it, once we internally decide which procModifiers it should include (hence I would do it in a separate PR).

In my view, the alpaka process modifier also has an important role in the development lifecycle: it’s very useful as a transitional mechanism for heterogeneous features that are mature enough to go into release, but not yet ready to become the default. It provides a structured “staging area” where we can deploy and evaluate heterogeneous components in realistic conditions without forcing them as baseline immediately. I can easily imagine, for instance, something like Pixel vertexing on GPU going through an alpaka-gated modifier first, and then eventually becoming the default once validated and stable enough.

I agree. As I tried to explain above, the alpaka procModifier is still there, both as a functionality and as a part of non-dedicated workflows, to be useful as the testing ground for new developments, so any new workflows can keep using it in the way that you describe. In my view, the changes in this PR remove the alpaka modifier from dedicated workflows exactly because the relevant development behind it (Patatrack) became the default, so it fits the development model you described.