Run

ORNL CADES

0. Get an interactive job from a queue with P100 GPUs

salloc -A ccsd -p gpu_p100 --gpus-per-task=1 -N 2 -n4 -c4 -G4 --mem=0G -t 01:00:00 /bin/bash

1. Load installed python modules and programming environment

source subdir/name-env/bin/activate
module load PE-gnu/3.0

2. Run example

srun -n 4 python run_config_input.py

ORNL SUMMIT

0. Get an interactive job (V100 GPUs)

bsub -W 30 -nnodes 1 -P PROJECT -q debug -Is $SHELL

1. Load installed python modules and programming environment

module purge
ml DefApps
ml gcc

module use -a /gpfs/alpine/world-shared/csc143/jyc/summit/sw/modulefiles
ml envs/py38
ml boost/1.78.0
ml rdkit/devel
ml adios2/devel
ml papi

2. Run example

export LD_PRELOAD=/sw/summit/gcc/9.1.0-alpha+20190716/lib64/libstdc++.so:/sw/summit/gcc/9.1.0-alpha+20190716/lib64/libgomp.so
export PYTHONPATH=$PWD:$PYTHONPATH

# 1 node, 1 GPU
jsrun -n1 -a1 -c1 -g1 --smpiargs="off" python example.py
# 4 node, 24 GPU
jsrun -n24 -a1 -g1 -c7 -r6 -b rs --smpiargs="off" python example.py

3. (Optional) In case you want to run IBM's Machine Learning CE with python 3.9, use the following module command:

module purge
ml DefApps
ml gcc

module use -a /gpfs/alpine/world-shared/csc143/jyc/summit/sw/modulefiles
ml open-ce-olcf-ext/devel

The above command will load Summit's module (open-ce-olcf) and HydraGNN specific libraries, such as PyG, Adios2, etc.