PyAlbany

Python and Albany

Albany is using Python for 2 purposes:

• testing purpose,
• PyAlbany: a python interface for Albany (2021 PyAlbany Tutorial/Demo ).

For testing purpose, Albany requires at least:

• python 2.7
• scipy 1.2.1
• numpy 1.15.1

For PyAlbany, SWIG and Mpi4Py are required too, more information on the installation of PyAlbany can be found in the PyAlbany README.

Testbed

To install the required python packages, it is recommended to do as follows:

pip3 install numpy --trusted-host=pypi.org --trusted-host=files.pythonhosted.org --user
pip3 install scipy --trusted-host=pypi.org --trusted-host=files.pythonhosted.org --user

For IBM machines, older version of numpy is required such as 1.19.2:

pip3 install numpy==1.19.2 --trusted-host=pypi.org --trusted-host=files.pythonhosted.org --user

To use matplotlib with a non-GUI backend on the testbed clusters to create plot stored as png images, it is recommended:

• to install matplotlib as follows:

pip3 install --user pillow --global-option="build_ext" --global-option="--disable-jpeg"
pip3 install --user matplotlib

• to initialize matplotlib as follows:

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt

Cori

A stable version of PyAlbany is installed and available on Cori.

Configuration

The following steps have to be followed to use it:

1. Set the environment variables:

module purge
source /opt/modules/default/init/sh
module rm craype craype-mic-knl craype-haswell PrgEnv-intel PrgEnv-cray PrgEnv-gnu intel cce gcc cray-parallel-netcdf cray-hdf5-parallel pmi cray-mpich2 cray-mpich cray-netcdf cray-hdf5 cray-netcdf-hdf5parallel cray-libsci papi cmake cray-petsc esmf zlib craype-hugepages2M darshan
module load craype PrgEnv-intel cray-mpich
module swap cray-mpich cray-mpich/7.7.10
module load PrgEnv-intel/6.0.5
module rm intel
module load intel/18.0.1.163
module swap craype craype/2.5.18
module rm pmi
module load pmi/5.0.14
module rm cray-netcdf-hdf5parallel
module load cray-netcdf-hdf5parallel/4.6.3.2 cray-hdf5-parallel/1.10.5.2 cray-parallel-netcdf/1.11.1.1
module rm git
module load git
module rm cmake
module load cmake/3.18.2
module unloadn cray-libsci
export MPICH_ENV_DISPLAY=1
export MPICH_VERSION_DISPLAY=1
export OMP_STACKSIZE=128M
export OMP_PROC_BIND=spread
export OMP_PLACES=threads
export HDF5_USE_FILE_LOCKING=FALSE
export MPICH_GNI_DYNAMIC_CONN=disabled
export FORT_BUFFERED=yes
export MPICH_MEMORY_REPORT=1
export CRAYPE_LINK_TYPE=dynamic
export CRAY_CPU_TARGET=haswell

2. Create a conda environment (!!! This needs to be done for the first time only !!!):

conda create -n my_pytrilinos_env python=3.8
source activate my_pytrilinos_env
MPICC="cc -shared" pip install --no-binary=mpi4py mpi4py==3.0.3
MPICC="cc -shared" pip install --no-binary=numpy numpy==1.20.2

3. Activate the conda environment (if not activated):

conda activate my_pytrilinos_env

4. Export the last environment variables:

export PYTHONPATH="/global/homes/k/knliege/shared/trilinos/lib/python3.8/site-packages":$PYTHONPATH
export PYTHONPATH="/global/homes/k/knliege/shared/albany/lib/python3.8/site-packages":$PYTHONPATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/global/homes/k/knliege/shared/trilinos/lib/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/global/homes/k/knliege/shared/albany/lib/

Step 2 is required only the first time that PyAlbany is used.

Verification

To verify that your session is correctly set, run:

python -c "import PyAlbany; print(PyAlbany.__version__)"

This line should print 1.0.0 or higher.

Usage

You can now use PyAlbany on Cori.

As an example, you can run the following commands where Albany_SRC is your Albany source directory:

salloc -N 1 -C haswell -q interactive -t 00:20:00
cd ${Albany_SRC}/PyAlbany/examples/UQ/
srun -n 1 python MC_example.py