Skip to content

Conference Notes for 2021 09 07

Jump to bottom Edit New page
Robert McLay edited this page Sep 7, 2021 · 1 revision

Agenda for Lmod Mtg 2021-09-07 at 9:00am (UTC-5)

Attendees(12)

  • Robert McLay (TACC)
  • Ward Poelmans (VUB)
  • Joey Dumont (Government of Canada)
  • Ward Poelmans (Vrije Universiteit Brussel)
  • Shelly Johnson (Univ. of Michigan)
  • Bennet Fauber (University of Michigan)
  • Bart Oldeman (Compute Canada)
  • Gowtham (Michigan Tech)
  • Raghu Reddy (NOAA/GDIT)
  • Bennet Fauber (Univ. of Michigan)
  • Shahzeb Shiddiqui (NERSC)
  • Kenneth Hoste(UGent)

Notes

  • next meetings will be 30min later (14:30 UTC)
    • next meeting Nov 9th

Lmod 8.5.13

  • support for rc shell from Plan 9
  • fix for zsh/ksh

Configuration file for Lmod

(comment by Ward)

  Currently we build our own Lmod RPM mostly because we want to set a couple of
  options using the `configure` script. It would be nice if we could have a
  proper configuration file for Lmod. Something like `/etc/lmod` where we can drop
  the `SitePackage.lua` and define constants that take precedence over the ones
  in `myGlobals.lua` (which are set by the configure script).
  This would allow us to simply use the Lmod from EPEL and let our configuration
  managment system write out the Lmod config files.
  • proper (runtime) configuration file for Lmod, to allow using Lmod RPM from EPEL
  • currently only possible via $LMOD_* environment variables

Demo on Lmod hooks

  • slides: https://github.com/TACC/Lmod/blob/master/my_docs/21/lmod_zoom_mtg_2021_09_07/presentation.pdf

  • location of SitePackage.lua can be set via $LMOD_PACKAGE_PATH, but this may change with additional mechanisms if there's a separate configuration file supported as well

  • Lmod runs in a sandbox, you don't have access to all functions (controlled via sandbox_registration)

  • in load hook, you can discriminate between "user loads" and other loads (incl. when a module loads another module)

  • Bart briefly discussed how they'll leveraging the Lmod load hook to give the user warnings about loading licensed software they don't have access to, setting module properties, Lmod family, controlling visibility of module files (like hide CUDA versions which are not compatible with the CUDA driver on the system)

    • SitePackage_properties.lua dynamically sets properties, like which modules provide AI software

  • the "t" passed to the "load_hook" is a table with information about which module is being loaded

  • distinction between basic/advanced hooks in the slides is relatively arbitrary

  • avail_hook can be used to "group" modules that are locaed in different directories

  • isVisibleHook gets full node, short name, location of module file in a table

    • can be used to dynamically control visibility of modules in avail/spider

  • listHook can be used to lie to users about which modules are currently loaded (to hide loaded modules in output of 'module list')

  • HookArray can be used to ensure only modules which actually get loaded are logged (not the ones that don't end up actually happening because load is aborted due to an error)

Q&A

  • Raghu: users tend to load "all possible modules they ever need" in their startup (.rc) files

    • Robert: XALT can track what is actually being used, Lmod load hook can only track which modules get loaded

  • Raghu: creating modules for Intel oneAPI (see email)

    • problem with sourcing a script from a module file is that the effects can be reversed when the module is UNloaded
    • Bennet: another way could be to start a subshell, and source the script there, unloading is then equivalent to exiting that subshell (similar to conda environment)
    • sh_to_modulefile script to convert shell script to module file mostly works, but it's not 100% perfect
      • Robert: this is a hack, doesn't take into account dynamic behaviour of the original script (like reacting to some environment variables), resulting module file is static
    • Robert: Tmod has support for sourcing scripts, with support for unloading/reverting, but it's unclear how it works or how well it works

  • Shahzeb: way to auto-document available modules in the system?

  • topic for next conf call: how to debug module files

Add a custom footer
Add a custom sidebar
Clone this wiki locally