Note: pyhf v0.6.1 on BlueWaters for CPU usage

[matthewfeickert]

ssh to a Bluewater login node and then

feickert@h2ologin3:~> qsub -I -l gres=shifter -l nodes=1:ppn=32:xe -l walltime=01:00:00
INFO: Job submitted to account: bbdz
qsub: waiting for job 12215424.bw to start
qsub: job 12215424.bw ready

In Torque Shifter prologue batchID: 12215424
Number of nodes: 1
Starting munge and nscd service on compute nodes
uid=57956(feickert) gid=17389(ILL_bbdz) groups=1001(hpss_user),1000(ie_user),999(login_user),17389(ILL_bbdz)
Successfully started munge and nscd service on compute nodes
----------------------------------------
Begin Torque Prologue on nid27639
at Thu May 13 16:03:41 CDT 2021
Job Id:			12215424.bw
Username:		feickert
Group:			ILL_bbdz
Job name:		STDIN
Requested resources:	gres=shifter,nodes=1:ppn=32:xe,walltime=01:00:00,neednodes=1:ppn=32:xe
Queue:			normal
Account:		bbdz
End Torque Prologue:  12.055 elapsed
----------------------------------------
feickert@nid27639:~> module load shifter
feickert@nid27639:~> shifterimg pull neubauergroup/bluewaters-pyhf:0.6.1 # This takes a long long time (over 30 minutes)
2021-05-13T16:29:00 Pulling Image: docker:neubauergroup/bluewaters-pyhf:0.6.1, status: READY
feickert@nid27639:~> shifterimg images | grep pyhf
bluewaters docker     READY    6350bbba36   2021-05-13T16:29:40 neubauergroup/bluewaters-pyhf:0.6.1
bluewaters docker     READY    6350bbba36   2021-05-13T16:29:40 neubauergroup/bluewaters-pyhf:latest
feickert@nid27639:~> aprun -b -N 1 -- shifter --image=neubauergroup/bluewaters-pyhf:0.6.1 -- /bin/bash -c "python3 --version --version && pyhf --version && pyhf --help"
mount: warning: ufs seems to be mounted read-only.
mount: warning: dsl/opt seems to be mounted read-only.
Python 3.8.8 (default, May 13 2021, 20:51:44) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
pyhf, version 0.6.1
Usage: pyhf [OPTIONS] COMMAND [ARGS]...

  Top-level CLI entrypoint.

Options:
  --version           Show the version and exit.
  --cite, --citation  Print the bibtex citation
                      for this software

  -h, --help          Show this message and exit.

Commands:
  cls          Compute CLs value(s) for a given...
  combine      Combine two workspaces into a...
  completions  Generate shell completion code.
  contrib      Contrib experimental operations.
  digest       Use hashing algorithm to...
  fit          Perform a maximum likelihood fit...
  inspect      Inspect a pyhf JSON document.
  json2xml     Convert pyhf JSON back to XML +...
  patchset     Operations involving patchsets.
  prune        Prune components from the...
  rename       Rename components of the...
  sort         Sort the workspace.
  xml2json     Entrypoint XML: The top-level
               XML...
Application 99844056 resources: utime ~4s, stime ~2s, Rss ~59812, inblocks ~115075, outblocks ~54076

[matthewfeickert]

For reasons unknown to me yet @BenGalewsky tried to pull on Bluewater and got an error

shifterimg pull  neubauergroup/bluewaters-pyhf:latest 
2021-05-13T17:59:25 Pulling Image: docker: status: FAILURE

[matthewfeickert]

For reasons unknown to me yet @BenGalewsky tried to pull on Bluewater and got an error

shifterimg pull  neubauergroup/bluewaters-pyhf:latest 
2021-05-13T17:59:25 Pulling Image: docker: status: FAILURE

This seems to be resolved, for reasons unknown, so it may come back.