Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add project id's to list so that more E3SM jobs at NERSC can have provenance data saved #6678

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

ndkeen
Copy link
Contributor

@ndkeen ndkeen commented Oct 10, 2024

There are now quite a few E3SM jobs running at NERSC and we are not saving info.
This change adds several project ID's that have recently run larger-node/longer-time E3SM jobs.

[bfb]

@ndkeen ndkeen added Machine Files BFB PR leaves answers BFB provenance pm-gpu Perlmutter machine at NERSC (GPU nodes) pm-cpu Perlmutter at NERSC (CPU-only nodes) labels Oct 10, 2024
@ndkeen ndkeen requested a review from rljacob October 10, 2024 23:15
Copy link

PR Preview Action v1.4.8
🚀 Deployed preview to https://E3SM-Project.github.io/E3SM/pr-preview/pr-6678/
on branch gh-pages at 2024-10-10 23:17 UTC

@rljacob rljacob requested a review from sarats October 11, 2024 03:28
@rljacob rljacob self-assigned this Oct 11, 2024
@rljacob rljacob added this to the maint-3.0 milestone Oct 11, 2024
@sarats
Copy link
Member

sarats commented Oct 12, 2024

As I mentioned in #5753, I would like to assess the impact of inclusion of any non-core ERCAP project.

For that purpose, I would like to get the name of the project, PI contact info and planned runs. If there are plans for running large number of small or single node runs or small-scale ensembles, I don't wish to archive them in PACE and would like them to provide assurance and understand disabling archiving for such runs (./xmlchange SAVE_TIMING=FALSE)

Either this information can be listed here or on Confluence.

One additional aspect to consider is the performance archive location and ability for users in some of these projects to write to that location.

The limiting constraint for archiving any and all runs is the limited storage capacity on the PACE server (for database and raw data).

@ndkeen
Copy link
Contributor Author

ndkeen commented Oct 12, 2024

OK, so you are going to take this Sarat?
I think it would be fine to add these, and remove any if they are too frequent.
It's true there would be a problem if they could not write to the dir.

@rljacob
Copy link
Member

rljacob commented Oct 15, 2024

Should we just add the ones mentioned in #5753 now and other's later after a conversation with the PI?

@ndkeen do you have PI contact info for all the projects you're adding?

@sarats
Copy link
Member

sarats commented Oct 15, 2024

FYI, I discussed this topic today with the EXEC during our call and they noted that this is a low-priority.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BFB PR leaves answers BFB Machine Files pm-cpu Perlmutter at NERSC (CPU-only nodes) pm-gpu Perlmutter machine at NERSC (GPU nodes) provenance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants