New version of RawPrime (SiStripApproximateCluster) #49013

silviodonato · 2025-09-28T16:38:12Z

PR description:

This PR adds an improved version of RawPrime (SiStripApproximateCluster).
The new version has been developed by @saswatinandan , the results are described in CMS DP-2025/031, EPS talk, NGT presentation (see previous PR).
On top of this, I applied some changes to allow CMSSW to run both versions of RawPrime using the same code (at the cost of adding a version_ flag in the SiStripApproximateCluster dataformat) + some other minor improvements, mainly about code readability.

In summary:

by default version=1 is used and it is expect to give no changes at all
in version=2:
- avgCharge is stored using 6 bits instead of 8 (ie. with a precision of 255/64 = 4 ADC)
  - the remaining two bits are used to encode filter_ and peakFilter_ flags
- barycenter_ will save the distance between the cluster barycenter instead of the cluster barycenter position
  - isSaturated_ flag is encoded in the first barycenter_ bit
- the other 15 bits are used to store the distance between clusters with a precision of 1/21 strips ((2*768)/2^15 ) instead of 1/10th of strip
- new function getBarycenter(...) and getAvgCharge(...) have been added to return float instead of int
- the code of creating the amplitudes_ of a SiStripCluster have been improved to give exactly the same value as SiStripCluster::charge().
- SiStripApproximateClusterCollection:
  - deletion of beginIndices_
  - save the distance between detId_ instead of their value.

Note: The datamember isSaturated_, peakFilter_ and filter_ are not used anymore in V2 and should be removed when V1 will not be used anymore.

PR validation:

I prepared a ConfDb area for testing /users/sdonato/2025Sept/RawSecond/HLT/V11 which creates two SiStripApproxClusters collections hltSiStripClusters2ApproxClusters and hltSiStripClusters2ApproxClustersV2.
It also decompresses these collections to standard si strip clusters hltSiStripClustersFromRawPrime and hltSiStripClustersFromRawPrimeV2.

You can download the configuration from hltGetConfiguration /users/sdonato/2025Sept/RawSecond/HLT/V11 --globaltag auto:run3_hlt_HIon --input file:...aRAWfile.root --max-events 100 --eras Run3_2025 --prescale none --l1-emulator uGT --l1 L1Menu_Collisions2025_v1_3_0_xml
and then edit the configuration with

process.hltSiStripClusters2ApproxClusters.version = cms.uint32( 1 )
process.hltSiStripClusters2ApproxClustersV2.version = cms.uint32( 2 )
process.hltSiStripClusters2ApproxClusters.collectionVersion = cms.uint32( 1 )
process.hltSiStripClusters2ApproxClustersV2.collectionVersion = cms.uint32( 2 )
process.hltSiStripClustersFromRawPrime.collectionVersion = cms.uint32( 1 )
process.hltSiStripClustersFromRawPrimeV2.collectionVersion = cms.uint32( 2 )

process.hltOutputPhysicsHIPhysicsRawSecond.outputCommands.append("keep SiStripClusteredmNewDetSetVector_*_*_*")
process.hltOutputPhysicsHIPhysicsRawSecond.outputCommands.append("keep *_hltSiStripClusters2ApproxClusters_*_*")

to enable the new versions and save the decompressed clusters.

Using this script I made comparisons between clusters collections:

original clusters vs compressed-decompressed v1 clusters [red]
original clusters vs compressed-decompressed v2 clusters [blue]

in terms of diff of

barycenter
average charge

Using another customization I computed the data reduction obtained running over 4000 events. It is 8.2% for the full event size (18% considering only siStripApproximateCluster). A further 3.3% of reduction can be achieved applying a cluster charge cut, as shown in the presentation (no PR needed for this, only HLT configuration change).
Test done using LZMA4, run=388350 lumi=11

process.hltOutputPhysicsHIPhysicsRawPrime.compressionAlgorithm = "LZMA"
process.hltOutputPhysicsHIPhysicsRawPrime.compressionLevel = 4
process.hltOutputPhysicsHIPhysicsRawSecond.compressionAlgorithm = "LZMA"
process.hltOutputPhysicsHIPhysicsRawSecond.compressionLevel = 4

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

This PR is urgent, as we would like to have a backport to CMSSW_15_1_X for the upcoming HIon collisions. As discussed with TSG (@cms-sw/hlt-l2 ) and HIon (@icali @mandrenguyen) we would like to have a stream online using this new dataformat in the upcoming HIon run for testing (with the idea to use it in 2026 if the test is successful).

mmusich · 2025-09-28T20:33:39Z

@cmsbuild ping

cmsbuild · 2025-09-28T20:34:05Z

cms-bot internal usage

cmsbuild · 2025-09-28T20:35:44Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-49013/46205

Found files with invalid states:
- DataFormats/SiStripCluster/interface/SiStripApproximateClusterCollection_v1.h:
  - Added: 3f4f47e
  - Deleted: f465968
- RecoLocalTracker/SiStripClusterizer/test/step2_L1REPACK_HLT_rawp.py:
  - Added: 3f4f47e
  - Deleted: f465968
- RecoLocalTracker/SiStripClusterizer/test/README.md:
  - Added: 3f4f47e
  - Deleted: f465968
- DataFormats/SiStripCluster/src/SiStripApproximateClusterCollection_v1.cc:
  - Added: 3f4f47e
  - Deleted: f465968
There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/SiStripCluster/interface/SiStripApproximateCluster.h modified in PR(s): update on strip approximate cluster #47367
- File DataFormats/SiStripCluster/interface/SiStripCluster.h modified in PR(s): SiStripClusterizer, an Alpaka port of the CUDA ClustersFromRawProducerGPU #47629, Modification of existing rawprime version of SiStripApproxCluster #49015
- File DataFormats/SiStripCluster/src/SiStripApproximateCluster.cc modified in PR(s): update on strip approximate cluster #47367
- File DataFormats/SiStripCluster/src/SiStripCluster.cc modified in PR(s): update on strip approximate cluster #47367, Modification of existing rawprime version of SiStripApproxCluster #49015
- File DataFormats/SiStripCluster/src/classes_def.xml modified in PR(s): update on strip approximate cluster #47367, Modification of existing rawprime version of SiStripApproxCluster #49015
- File RecoLocalTracker/SiStripClusterizer/plugins/SiStripApprox2Clusters.cc modified in PR(s): Modification of existing rawprime version of SiStripApproxCluster #49015

cmsbuild · 2025-09-28T20:36:03Z

A new Pull Request was created by @silviodonato for master.

It involves the following packages:

DataFormats/SiStripCluster (reconstruction)
RecoLocalTracker/SiStripClusterizer (reconstruction)

@cmsbuild, @jfernan2, @mandrenguyen can you please review it and eventually sign? Thanks.
@GiacomoSguazzoni, @VinInn, @VourMa, @alesaggio, @echabert, @elusian, @felicepantaleo, @gbenelli, @gpetruc, @jlidrych, @missirol, @mmasciov, @mmusich, @mtosi, @robervalwalsh, @rovere, @threus, @yduhm this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

mmusich · 2025-09-28T20:39:39Z

test parameters:

addpkg = DQM/Integration
workflows = 161,161.02,161.03,161.1,161.2,161.3,161.4,162,162.02,162.03,162.1,162.2,162.3,162.4

mmusich · 2025-09-28T20:40:33Z

@cmsbuild, please test

saswatinandan · 2025-09-28T20:41:27Z

Currently something is missing here. Difference of detId are not considered and the variable beginIndices are still present in the collection. The correspnding codes DataFormats/SiStripCluster/interface/SiStripApproximateClusterCollection_v1.h, DataFormats/SiStripCluster/src/SiStripApproximateClusterCollection_v1.cc are deleted.

cmsbuild · 2025-09-28T22:06:41Z

-1

Failed Tests: UnitTests RelVals
Size: This PR adds an extra 36KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8a5440/48317/summary.html
COMMIT: f465968
CMSSW: CMSSW_16_0_X_2025-09-28-0000/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/49013/48317/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test TestDQMOnlineClient-sistrip_approx_dqm_sourceclient had ERRORS

RelVals

161.1

----- Begin Fatal Exception 29-Sep-2025 00:00:31 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 4 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Calling method for module PrimaryVertexResolution/'hltVerticesPFFilterResolution'
Exception Message:
RefCore: A request to resolve a reference to a product of type 'std::vector<reco::Track>' with ProductID '2:1474'
can not be satisfied because the product cannot be found.
Probably the branch containing the product is not stored in the input file.
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

161.0

----- Begin Fatal Exception 29-Sep-2025 00:00:05 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Calling method for module PrimaryVertexResolution/'hltVerticesPFFilterResolution'
Exception Message:
RefCore: A request to resolve a reference to a product of type 'std::vector<reco::Track>' with ProductID '2:1474'
can not be satisfied because the product cannot be found.
Probably the branch containing the product is not stored in the input file.
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

161.3

----- Begin Fatal Exception 29-Sep-2025 00:00:04 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 3 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Calling method for module PrimaryVertexResolution/'hltVerticesPFFilterResolution'
Exception Message:
RefCore: A request to resolve a reference to a product of type 'std::vector<reco::Track>' with ProductID '2:1474'
can not be satisfied because the product cannot be found.
Probably the branch containing the product is not stored in the input file.
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

…inor improvements in readibility

nothingface0 · 2025-09-29T09:44:29Z

DataFormats/SiStripCluster/interface/SiStripApproximateCluster.h

+        return barycenter_ * 0.1;  // in the old format barycenter_ is in tenths of strips
+      case 2: {
+        // Drop the first bit (encoding the saturation info)
+        double barycenter_decoded = (barycenter_ & 0b0111'1111'1111'1111);


Shouldn't those bitmasks be a constant somewhere, so it's more readable?

ok, I've changed it

Great, thanks! :)

cmsbuild · 2025-09-29T10:12:49Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-49013/46216

Found files with invalid states:
- DataFormats/SiStripCluster/interface/SiStripApproximateClusterCollection_v1.h:
  - Added: 3f4f47e
  - Deleted: 89eef23
- RecoLocalTracker/SiStripClusterizer/test/step2_L1REPACK_HLT_rawp.py:
  - Added: 3f4f47e
  - Deleted: 89eef23
- RecoLocalTracker/SiStripClusterizer/test/README.md:
  - Added: 3f4f47e
  - Deleted: 89eef23
- DataFormats/SiStripCluster/src/SiStripApproximateClusterCollection_v1.cc:
  - Added: 3f4f47e
  - Deleted: 89eef23
There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/SiStripCluster/interface/SiStripApproximateCluster.h modified in PR(s): update on strip approximate cluster #47367
- File DataFormats/SiStripCluster/interface/SiStripCluster.h modified in PR(s): SiStripClusterizer, an Alpaka port of the CUDA ClustersFromRawProducerGPU #47629, Modification of existing rawprime version of SiStripApproxCluster #49015
- File DataFormats/SiStripCluster/src/SiStripApproximateCluster.cc modified in PR(s): update on strip approximate cluster #47367
- File DataFormats/SiStripCluster/src/SiStripCluster.cc modified in PR(s): update on strip approximate cluster #47367, Modification of existing rawprime version of SiStripApproxCluster #49015
- File DataFormats/SiStripCluster/src/classes.h modified in PR(s): Modification of existing rawprime version of SiStripApproxCluster #49015
- File DataFormats/SiStripCluster/src/classes_def.xml modified in PR(s): update on strip approximate cluster #47367, Modification of existing rawprime version of SiStripApproxCluster #49015
- File RecoLocalTracker/SiStripClusterizer/plugins/SiStripApprox2Clusters.cc modified in PR(s): Modification of existing rawprime version of SiStripApproxCluster #49015

cmsbuild · 2025-09-29T10:13:12Z

Pull request #49013 was updated. @cmsbuild, @jfernan2, @mandrenguyen can you please check and sign again.

silviodonato · 2025-09-29T10:16:00Z

please test with cms-data/DQM-Integration#13

cmsbuild · 2025-09-29T12:32:47Z

-1

Failed Tests: RelVals
Size: This PR adds an extra 40KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8a5440/48335/summary.html
COMMIT: cb97fa0
CMSSW: CMSSW_16_0_X_2025-09-28-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/49013/48335/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

2025.0000001DAS Error
161.1

----- Begin Fatal Exception 29-Sep-2025 14:26:48 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 4 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Calling method for module PrimaryVertexResolution/'hltVerticesPFFilterResolution'
Exception Message:
RefCore: A request to resolve a reference to a product of type 'std::vector<reco::Track>' with ProductID '2:1474'
can not be satisfied because the product cannot be found.
Probably the branch containing the product is not stored in the input file.
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

161.0

----- Begin Fatal Exception 29-Sep-2025 14:26:24 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Calling method for module PrimaryVertexResolution/'hltVerticesPFFilterResolution'
Exception Message:
RefCore: A request to resolve a reference to a product of type 'std::vector<reco::Track>' with ProductID '2:1474'
can not be satisfied because the product cannot be found.
Probably the branch containing the product is not stored in the input file.
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

161.3
161.2
161.4
2024.0050001
161.02
2025.0010001
2024.0060001

mmusich · 2025-09-29T12:37:22Z

-1

unrelated failures, see #49018. The next IB should be clean (at lease for the non-DAS related ones)

silviodonato · 2025-09-30T08:51:54Z

please test with cms-data/DQM-Integration#13

cmsbuild · 2025-09-30T13:05:08Z

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8a5440/48366/summary.html
COMMIT: cb97fa0
CMSSW: CMSSW_16_0_X_2025-09-29-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/49013/48366/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8a5440/48366/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8a5440/48366/git-merge-result

Comparison Summary

Summary:

You potentially added 18 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 63
DQMHistoTests: Total histograms compared: 5531720
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 5531700
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 62 files compared)
Checked 282 log files, 248 edm output root files, 63 DQM output files
TriggerResults: no differences found

mandrenguyen · 2025-09-30T18:45:08Z

Closed in favor of #49015

adding code for test purposeÃ

3f4f47e

saswatinandan mentioned this pull request Sep 28, 2025

Rawp v3 #48992

Closed

cmsbuild mentioned this pull request Sep 28, 2025

Modification of existing rawprime version of SiStripApproxCluster #49015

Merged

cmsbuild added this to the CMSSW_16_0_X milestone Sep 28, 2025

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending code-checks-pending trk changes-dataformats labels Sep 28, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 28, 2025

cmsbuild added tests-started and removed tests-pending labels Sep 28, 2025

cmsbuild added tests-rejected and removed tests-started labels Sep 28, 2025

use a single code to run on both v1 and v2 RawPrime, using version. M…

89eef23

…inor improvements in readibility

silviodonato force-pushed the RawPrimeV2 branch from f465968 to 89eef23 Compare September 29, 2025 00:50

cmsbuild added tests-pending and removed tests-rejected code-checks-approved labels Sep 29, 2025

cmsbuild added requires-external tests-started and removed tests-rejected labels Sep 29, 2025

nothingface0 reviewed Sep 29, 2025

View reviewed changes

use constants instead of magic numbers

cb97fa0

cmsbuild added tests-pending code-checks-pending and removed tests-started requires-external code-checks-approved labels Sep 29, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 29, 2025

cmsbuild added tests-started requires-external and removed tests-pending labels Sep 29, 2025

cmsbuild added tests-rejected and removed tests-started labels Sep 29, 2025

cmsbuild added tests-started and removed tests-rejected labels Sep 30, 2025

cmsbuild added tests-approved and removed tests-started labels Sep 30, 2025

mandrenguyen closed this Sep 30, 2025

New version of RawPrime (SiStripApproximateCluster) #49013

New version of RawPrime (SiStripApproximateCluster) #49013

Uh oh!

Conversation

silviodonato commented Sep 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Uh oh!

mmusich commented Sep 28, 2025

Uh oh!

cmsbuild commented Sep 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Sep 28, 2025

Uh oh!

cmsbuild commented Sep 28, 2025

Uh oh!

mmusich commented Sep 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mmusich commented Sep 28, 2025

Uh oh!

saswatinandan commented Sep 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Sep 28, 2025

Unit Tests

RelVals

Uh oh!

nothingface0 Sep 29, 2025

Choose a reason for hiding this comment

Uh oh!

silviodonato Sep 29, 2025

Choose a reason for hiding this comment

Uh oh!

nothingface0 Sep 29, 2025

Choose a reason for hiding this comment

Uh oh!

cmsbuild commented Sep 29, 2025

Uh oh!

cmsbuild commented Sep 29, 2025

Uh oh!

silviodonato commented Sep 29, 2025

Uh oh!

cmsbuild commented Sep 29, 2025

RelVals

Uh oh!

mmusich commented Sep 29, 2025

Uh oh!

silviodonato commented Sep 30, 2025

Uh oh!

cmsbuild commented Sep 30, 2025

Comparison Summary

Uh oh!

mandrenguyen commented Sep 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

silviodonato commented Sep 28, 2025 •

edited

Loading

cmsbuild commented Sep 28, 2025 •

edited

Loading

mmusich commented Sep 28, 2025 •

edited

Loading

saswatinandan commented Sep 28, 2025 •

edited

Loading