Rework Muon candidate selection in few tracker ALCARECO #46574

mmusich · 2024-10-31T21:03:39Z

PR description:

This PR is a partial response to #46493.
In #46493 (comment) it was suggested to replace few of the MuonSelector (ObjectSelectorBase<SingleElementCollectionSelector<edm::View<reco::Muon>, ...> ) instances with modules that express the structure of the selection in C++ code instead, as this would allow to reduce the memory footprint in Run 3 prompt reco jobs for Muon* PDs.
This is implemented in commits 8e3de65 and 2d97abd that introduce ad-hoc classes AlignmentGoodIdMuonSelector and AlignmentRelCombIsoMuonSelector for this purpose.
Additionally in #46493 (comment) it was suggested that the duplication of selection identical modules configuration has both memory and runtime cost. This is addressed at 8e49f10 by introducing a new common sequence seqALCARECOTkAlRelCombIsoMuons that ulimately takes in input a reco::MuonCollection (with label muons) and returns a skimmed reco::MuonCollection called TkAlRelCombIsoMuonSelector which is then used to feed the rest of the selection chain.

PR validation:

runTheMatrix.py -l 1001.0,1001.2,1001.3,1001.4,1002.4

runs.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

N/A

cmsbuild · 2024-10-31T21:04:04Z

cms-bot internal usage

cmsbuild · 2024-10-31T21:05:19Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46574/42463

cmsbuild · 2024-10-31T21:05:44Z

A new Pull Request was created by @mmusich for master.

It involves the following packages:

Alignment/CommonAlignmentProducer (alca)
Calibration/TkAlCaRecoProducers (alca)

@atpathak, @cmsbuild, @consuegs, @perrotta can you please review it and eventually sign? Thanks.
@adewit, @mmusich, @pakhotin, @rsreds, @threus, @tlampen, @tocheng, @yuanchao this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

mmusich · 2024-10-31T21:06:40Z

test parameters:

workflows = 1001.0,1001.2,1001.3,1001.4,1002.4

mmusich · 2024-10-31T21:06:56Z

type performance-improvements

mmusich · 2024-10-31T21:14:03Z

@cmsbuild, please test

cmsbuild · 2024-11-01T01:06:50Z

-1

Failed Tests: RelVals-INPUT
Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3f8e67/42523/summary.html
COMMIT: 8e49f10
CMSSW: CMSSW_14_2_X_2024-10-31-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46574/42523/install.sh to create a dev area with all the needed externals and cmssw changes.

DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals-INPUT

2024.0000012024.000001_RunJetMET02024D_10k/step1_dasquery.log
2024.0010012024.001001_RunZeroBias2024D_10k/step1_dasquery.log
2024.1000012024.100001_RunJetMET02024C_10k/step1_dasquery.log

Expand to see more relval errors ...

2024.101001
2024.000001
2024.001001
2024.100001
2024.101001

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 10 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3683524
DQMHistoTests: Total failures: 425
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3683078
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.094 KiB( 49 files compared)
DQMHistoSizes: changed ( 1002.4,... ): -0.023 KiB MessageLogger/Errors
DQMHistoSizes: changed ( 1002.4,... ): -0.023 KiB MessageLogger/Warnings
Checked 221 log files, 175 edm output root files, 50 DQM output files
TriggerResults: found differences in 2 / 48 workflows

Dr15Jones

Thanks for this!

Alignment/CommonAlignmentProducer/python/TkAlMuonSelectors_cfi.py

…e of the selection in C++ code - replace in 4 ALCARECO producers separated configuration with a common block: TkAlGoodIdMuonSelector+TkAlRelCombIsoMuonSelector

cmsbuild · 2024-11-01T09:37:54Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46574/42467

cmsbuild · 2024-11-01T09:38:20Z

Pull request #46574 was updated. @atpathak, @cmsbuild, @consuegs, @perrotta can you please check and sign again.

mmusich · 2024-11-01T09:43:56Z

test parameters:

enable_tests = threading
workflows_threading = 1000.0, 1001.0,1001.2,1001.3,1001.4,1002.4

mmusich · 2024-11-01T09:44:03Z

@cmsbuild, please test

cmsbuild · 2024-11-01T15:00:58Z

-1

Failed Tests: RelVals
Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3f8e67/42532/summary.html
COMMIT: b916211
CMSSW: CMSSW_14_2_X_2024-10-31-2300/el8_amd64_gcc12
Additional Tests: THREADING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46574/42532/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

The relvals timed out after 4 hours.

mmusich · 2024-11-01T15:52:59Z

@cmsbuild, please test

let's re-try, in my local checkout area I get:

1000.0_RunMinBias2011A Step0-PASSED Step1-PASSED Step2-PASSED Step3-PASSED Step4-PASSED  - time date Fri Nov  1 16:27:15 2024-date Fri Nov  1 16:22:40 2024; exit: 0 0 0 0 0
1001.0_RunMinBias2011A Step0-PASSED Step1-PASSED Step2-PASSED Step3-PASSED Step4-PASSED Step5-PASSED Step6-PASSED Step7-PASSED Step8-PASSED Step9-PASSED Step10-PASSED  - time date Fri Nov  1 16:27:59 2024-date Fri Nov  1 16:22:40 2024; exit: 0 0 0 0 0 0 0 0 0 0 0
1001.2_RunZeroBias2017F Step0-PASSED Step1-PASSED Step2-PASSED Step3-PASSED Step4-PASSED Step5-PASSED Step6-PASSED  - time date Fri Nov  1 16:28:38 2024-date Fri Nov  1 16:22:41 2024; exit: 0 0 0 0 0 0 0
1001.3_RunSingleMuon2022B Step0-PASSED Step1-PASSED Step2-PASSED Step3-PASSED  - time date Fri Nov  1 16:27:57 2024-date Fri Nov  1 16:22:41 2024; exit: 0 0 0 0
1001.4_RunDoubleMuon2022B Step0-PASSED Step1-PASSED Step2-PASSED Step3-PASSED Step4-PASSED Step5-PASSED  - time date Fri Nov  1 16:28:01 2024-date Fri Nov  1 16:22:42 2024; exit: 0 0 0 0 0 0
1002.4_RunDoubleMuon2022B Step0-PASSED Step1-PASSED Step2-PASSED  - time date Fri Nov  1 16:27:35 2024-date Fri Nov  1 16:22:42 2024; exit: 0 0 0
6 6 6 5 4 3 2 1 1 1 1 tests passed, 0 0 0 0 0 0 0 0 0 0 0 failed

cmsbuild · 2024-11-01T19:07:19Z

+1

Size: This PR adds an extra 12KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3f8e67/42540/summary.html
COMMIT: b916211
CMSSW: CMSSW_14_2_X_2024-11-01-1100/el8_amd64_gcc12
Additional Tests: THREADING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46574/42540/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 3 lines to the logs
Reco comparison results: 7 differences found in the comparisons
DQMHistoTests: Total files compared: 46
DQMHistoTests: Total histograms compared: 3569567
DQMHistoTests: Total failures: 520
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3569026
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.047 KiB( 45 files compared)
DQMHistoSizes: changed ( 1000.0 ): -0.023 KiB MessageLogger/Errors
DQMHistoSizes: changed ( 1000.0 ): -0.023 KiB MessageLogger/Warnings
Checked 201 log files, 171 edm output root files, 46 DQM output files
TriggerResults: found differences in 1 / 44 workflows

perrotta · 2024-11-04T13:12:08Z

+alca

cmsbuild · 2024-11-04T13:12:33Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @antoniovilela, @rappoccio, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2024-11-04T13:28:18Z

+1

mmusich added 2 commits October 31, 2024 17:07

Introduce AlignmentGoodIdMuonSelector

8e3de65

Introduce AlignmentRelCombIsoMuonSelector

2d97abd

cmsbuild added this to the CMSSW_14_2_X milestone Oct 31, 2024

cmsbuild added alca-pending pending-signatures tests-pending orp-pending code-checks-pending trk labels Oct 31, 2024

mmusich changed the title ~~Rework~~ Rework Muon candidate selection in few tracker ALCARECO Oct 31, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 31, 2024

mmusich mentioned this pull request Oct 31, 2024

[Run3 PromptReco] 390 MB in cut/expression parser #46493

Open

cmsbuild added the performance-improvements label Oct 31, 2024

cmsbuild added tests-started and removed tests-pending labels Oct 31, 2024

cmsbuild added tests-rejected and removed tests-started labels Nov 1, 2024

Dr15Jones reviewed Nov 1, 2024

View reviewed changes

Alignment/CommonAlignmentProducer/python/TkAlMuonSelectors_cfi.py Outdated Show resolved Hide resolved

TkAlMuonSelectors_cfi: replace MuonSelector that express the structur…

b916211

…e of the selection in C++ code - replace in 4 ALCARECO producers separated configuration with a common block: TkAlGoodIdMuonSelector+TkAlRelCombIsoMuonSelector

mmusich force-pushed the mm_dev_ali_goo_id_muon_selector branch from 8e49f10 to b916211 Compare November 1, 2024 09:33

cmsbuild removed tests-rejected code-checks-approved labels Nov 1, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 1, 2024

cmsbuild added tests-started and removed tests-pending labels Nov 1, 2024

cmsbuild added tests-rejected and removed tests-started labels Nov 1, 2024

cmsbuild added tests-started and removed tests-rejected labels Nov 1, 2024

cmsbuild added tests-approved and removed tests-started labels Nov 1, 2024

mmusich marked this pull request as ready for review November 2, 2024 10:13

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Nov 4, 2024

cmsbuild added orp-approved and removed orp-pending labels Nov 4, 2024

cmsbuild merged commit d08538a into cms-sw:master Nov 4, 2024
13 checks passed

mmusich deleted the mm_dev_ali_goo_id_muon_selector branch November 4, 2024 13:29

cmsbuild mentioned this pull request Nov 4, 2024

Fixing Wf Names for ParkingDouble* #46601

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rework Muon candidate selection in few tracker ALCARECO #46574

Rework Muon candidate selection in few tracker ALCARECO #46574

mmusich commented Oct 31, 2024 •

edited

Loading

cmsbuild commented Oct 31, 2024 •

edited

Loading

cmsbuild commented Oct 31, 2024

cmsbuild commented Oct 31, 2024

mmusich commented Oct 31, 2024

mmusich commented Oct 31, 2024

mmusich commented Oct 31, 2024

cmsbuild commented Nov 1, 2024

Dr15Jones left a comment

cmsbuild commented Nov 1, 2024

cmsbuild commented Nov 1, 2024

mmusich commented Nov 1, 2024

mmusich commented Nov 1, 2024

cmsbuild commented Nov 1, 2024

mmusich commented Nov 1, 2024

cmsbuild commented Nov 1, 2024

perrotta commented Nov 4, 2024

cmsbuild commented Nov 4, 2024

mandrenguyen commented Nov 4, 2024

Rework Muon candidate selection in few tracker ALCARECO #46574

Rework Muon candidate selection in few tracker ALCARECO #46574

Conversation

mmusich commented Oct 31, 2024 • edited Loading

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Oct 31, 2024 • edited Loading

cmsbuild commented Oct 31, 2024

cmsbuild commented Oct 31, 2024

mmusich commented Oct 31, 2024

mmusich commented Oct 31, 2024

mmusich commented Oct 31, 2024

cmsbuild commented Nov 1, 2024

RelVals-INPUT

Comparison Summary

Dr15Jones left a comment

Choose a reason for hiding this comment

cmsbuild commented Nov 1, 2024

cmsbuild commented Nov 1, 2024

mmusich commented Nov 1, 2024

mmusich commented Nov 1, 2024

cmsbuild commented Nov 1, 2024

RelVals

mmusich commented Nov 1, 2024

cmsbuild commented Nov 1, 2024

Comparison Summary

perrotta commented Nov 4, 2024

cmsbuild commented Nov 4, 2024

mandrenguyen commented Nov 4, 2024

mmusich commented Oct 31, 2024 •

edited

Loading

cmsbuild commented Oct 31, 2024 •

edited

Loading