Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

n1850.ne30_tn14.hybrid.20240822 #30

Open
mvertens opened this issue Aug 23, 2024 · 8 comments
Open

n1850.ne30_tn14.hybrid.20240822 #30

mvertens opened this issue Aug 23, 2024 · 8 comments
Labels
N1850 piControl - fully coupled

Comments

@mvertens
Copy link
Contributor

mvertens commented Aug 23, 2024

Purpose:
Exactly like n1850.ne30_tn14.hybrid.20240808 EXCEPT

  • for an updated code with a bug fix oslo_aero (corrected dust emissions)
  • updates to the BLOM for hybrid coordinates
  • new initial conditions for BLOM and CICE
  • NOTE: a memory leak is present in the run and hits agains the node limit when running past 11 years.
    See memory leak in n1850 case NorESM#550

Description:

  • create_newcase commnad
    ./create_newcase --case /cluster/projects/nn9560k/mvertens/cases/n1850.ne30_tn14.hybrid.20240822 --compset N1850 --res ne30pg3_tn14 --project nn9039k --run-unsupported

  • user_nl_cam

 interpolate_nlat   = 192
 interpolate_nlon   = 288
 interpolate_output = .true.
 history_aerosol = .true. 
 zmconv_c0_lnd  =  0.0075D0
 zmconv_c0_ocn  =  0.0300D0
 zmconv_ke =  5.0E-6
 zmconv_ke_lnd =  1.0E-5
 clim_modal_aero_top_press =  1.D-4
 bndtvg = '/cluster/shared/noresm/inputdata/atm/cam/ggas/noaamisc.r8.nc'
  • user_nl_blom
    ICFILE = "/cluster/shared/noresm/inputdata/restart/NOIIAJRAOC20TR_TL319_tn14_ppm_20240816/rest/1775-01-01-00000/NOIIAJRAOC20TR_TL319_tn14_ppm_20240816.blom.r.1775-01-01-00000.nc"

  • user_nl_cice
    ice_ic = "/cluster/shared/noresm/inputdata/restart/NOIIAJRAOC20TR_TL319_tn14_ppm_20240816/rest/1775-01-01-00000/NOIIAJRAOC20TR_TL319_tn14_ppm_20240816.cice.r.1775-01-01-00000.nc"

  • xmlchange commands

./xmlchange NTASKS=3200
./xmlchange NTASKS_OCN=256
./xmlchange ROOTPE=0
./xmlchange ROOTPE_OCN=3200
./xmlchange BLOM_VCOORD=cntiso_hybrid,BLOM_TURBULENT_CLOSURE=
./xmlchange STOP_OPTION=nyears
./xmlchange STOP_N=20
./xmlchange REST_N=1
./xmlchange REST_OPTION=nyears
./xmlchange RESUBMIT=1
./xmlchange --subgroup case.run JOB_WALLCLOCK_TIME=48:00:00
./xmlchange --subgroup case.st_archive JOB_WALLCLOCK_TIME=03:00:00

Compset long name
1850_CAM%DEV%LT%NORESM%CAMoslo_CLM51%SP_CICE_BLOM%ECO_MOSART_DGLC%NOEVOLVE_SWAV_SESP


Model grid long name
a%ne30np4.pg3_l%ne30pg4.pg3_oi%tnx1v4_r%r05_w%null_z%null_g%gris4_m%tnx1v4

Case directory:

  • Locally on betzy: /cluster/projects/nn9560k/cases/mvertens/n1850.ne30_tn14.hybrid.20240822
  • On github: TODO

Code version on github (either NorESMhub or user's fork):


Diagnostics:


Output:

  • Raw output: /nird/datalake/NS9560K/noresm3/cases/ n1850.ne30_tn14.hybrid.20240822
  • CAM Climos: TODO

Contacts:
@mvertens, @adagj


Extra details:

@mvertens mvertens added the N1850 piControl - fully coupled label Aug 23, 2024
@mvertens mvertens changed the title Enter casename (shortcompsetname.grid.codeversion.somekindofinfo.yyyymmdd) n1850.ne30_tn14.hybrid.20240822 Aug 23, 2024
@monsieuralok
Copy link

@mvertens I have tested NF1850 case with yours tag; it is also happening there; Memory start to increase just after every mid month; if I remember well; CAM is reading somedata mid months probably there could be allocation of memory I will check it.

@mvertens
Copy link
Contributor Author

@monsieuralok - I have discovered that this memory leak also occurs in CESM simulations. @jedwards4b is tracking this down in CAM right now and believes that it in tphysac. I'll update the issue once I know more.

@jedwards4b
Copy link

Just to clarify - I'm not sure we are seeing the same leak - mine appears to be a smaller increment and not associated with the 15th of the month as yours is.

@mvertens
Copy link
Contributor Author

@jedwards4b - I agree. It would be good to see what the source of the CESM memory leak is at this point and see if it impacts what I am seeing.

@jedwards4b
Copy link

ESCOMP/CAM#1136

@monsieuralok
Copy link

@mvertens I guess this solved memory leak issue issue; variable lam and phi are not deallocated in file :- components/cam/src/chemistry/oslo_aero/src_cam/mo_drydep.F90

probably same in @jedwards4b

components/cam/src/chemistry/mozart/mo_drydep.F90

but, it has solved my memory leak issue atleast in NorESM but, I need to double check.

Case folder on Betzy:- /cluster/projects/nn9560k/alok/NorESM-mariana/cases/NF1850.ne30_tn14.hybrid.20240822_N43

You look for SourceMods and only file mo_drydep.F90

@mvertens
Copy link
Contributor Author

mvertens commented Sep 2, 2024

@monsieuralok - since lam and phi are allocatable variables in mo_drydep.F90
real(r8), allocatable :: lam(:), phi(:)
my understanding is that they will be automatically deallocated upon exit of the subroutine. If they were pointers that were allocated, this would be a different case. Do you agree with this?
That said - in looking at your mediator log file in /cluster/work/users/agu002/archive/NF1850.ne30_tn14.hybrid.20240822_N43/logs your memory profile does indeed look better - and the memory leak I saw happened earlier in the year. But just to make sure - can you please run longer than that to confirm your hypothesis. I would be very surprised if not deallocating allocatable variables solves this.

@mvertens
Copy link
Contributor Author

mvertens commented Sep 2, 2024

@monsieuralok - to clarify - lam and phi are allocatable local variables - not module variables - and so according to the fortran standard they should be deallocated upon exiting the subroutine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
N1850 piControl - fully coupled
Projects
None yet
Development

No branches or pull requests

3 participants