MTD Geometry: update DDD filtering for better memory performance

[fabiocos]

PR description:

This PR addresses #46512 , by:

• remove vetoing no more present volumes (since v2);
• separate namespace and name filtering, as suggested by @makortel, eliminating the double string creation if only namespace filtering is needed;
• use locally string_view for comparisons to reduce memory usage.

PR validation:

Geometry/MTD* unit tests successfully run, producing geometry dumps consistent with the reference.

[cmsbuild]

cms-bot internal usage

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46531/42394

[cmsbuild]

A new Pull Request was created by @fabiocos for master.

It involves the following packages:

• Geometry/MTDNumberingBuilder (geometry, upgrade)

@Dr15Jones, @Moanwar, @bsunanda, @civanch, @cmsbuild, @kpedro88, @makortel, @mdhildreth, @srimanob, @subirsarkar can you please review it and eventually sign? Thanks.
@bsunanda, @martinamalberti this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[fabiocos]

please test

[cmsbuild]

+1

Size: This PR adds an extra 20KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c26682/42427/summary.html
COMMIT: d4baa1c
CMSSW: CMSSW_14_2_X_2024-10-28-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46531/42427/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 12 differences found in the comparisons
• DQMHistoTests: Total files compared: 46
• DQMHistoTests: Total histograms compared: 3569177
• DQMHistoTests: Total failures: 380
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3568777
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
• Checked 201 log files, 171 edm output root files, 46 DQM output files
• TriggerResults: no differences found

[fabiocos]

enable profiling

[fabiocos]

please test

[makortel]

From the logs of workflow 29634.0 step 3, the number of memory allocations goes down from 244090020 to 232478470, i.e. a reduction of 11.6 millions.

[fabiocos]

as far as I can see in my standalone unit test on cmsdev45, there is no obvious significant impact on the speed of execution, in any case the caching of the strings is preserved to prevent the previous performance problems, that were fixed one year ago.

[bsunanda]

+geometry

[fabiocos]

@smuzaffar are two days normal for a profiling request? As pointed out by @makortel the memory benefit seems anyway visible by the simple memory report in log files, so in case I could abort the test

[cmsbuild]

-1

Failed Tests: UnitTests RelVals-INPUT
Size: This PR adds an extra 12KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c26682/42637/summary.html
COMMIT: d4baa1c
CMSSW: CMSSW_14_2_X_2024-11-06-2300/el8_amd64_gcc12
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46531/42637/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test GeometryMTDNumberingBuilderTestDriver had ERRORS

RelVals-INPUT

• 2024.1010012024.101001_RunBTagMu2024C_10k/step1_dasquery.log
• 2024.1020012024.102001_RunJetMET02024C_10k/step1_dasquery.log
• 2024.2010012024.201001_RunBTagMu2024D_10k/step1_dasquery.log

Expand to see more relval errors ...

• 2024.202001
• 2024.101001
• 2024.102001
• 2024.201001
• 2024.202001

Comparison Summary

Summary:

• You potentially removed 2 lines from the logs
• Reco comparison results: 7 differences found in the comparisons
• DQMHistoTests: Total files compared: 46
• DQMHistoTests: Total histograms compared: 3343138
• DQMHistoTests: Total failures: 413
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3342705
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
• Checked 195 log files, 172 edm output root files, 46 DQM output files
• TriggerResults: no differences found

[fabiocos]

the unit test was broken by other concurrent developments

[fabiocos]

please test with #46608

[cmsbuild]

-1

Failed Tests: RelVals-INPUT
Size: This PR adds an extra 12KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c26682/42650/summary.html
COMMIT: d4baa1c
CMSSW: CMSSW_14_2_X_2024-11-07-1100/el8_amd64_gcc12
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46531/42650/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

• 2024.0020012024.002001_RunJetMET02024B_10k/step1_dasquery.log
• 2024.002001DAS Error

Comparison Summary

Summary:

• You potentially removed 1 lines from the logs
• Reco comparison results: 9 differences found in the comparisons
• DQMHistoTests: Total files compared: 46
• DQMHistoTests: Total histograms compared: 3343138
• DQMHistoTests: Total failures: 417
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3342701
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
• Checked 195 log files, 172 edm output root files, 46 DQM output files
• TriggerResults: no differences found

[fabiocos]

@cms-sw/upgrade-l2 any comment? Or can we finally converge on this PR, as the errors in tests are unrelated, and the benefit in memory was seen?

[Moanwar]

+Upgrade

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @rappoccio, @antoniovilela, @sextonkennedy, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

[mandrenguyen]

ignore tests-rejected with ib-failure

[mandrenguyen]

+1