Issue with containers - .command.sh: line 2: python: command not found #33

Erin-Chee · 2024-07-30T15:53:41Z

Operating System

CentOS 7

Other Linux

No response

Workflow Version

v1.2.0-g36f41e7

Workflow Execution

Command line (Cluster)

Other workflow execution

No response

EPI2ME Version

No response

CLI command run

#!/bin/bash
#SBATCH -p defq
#SBATCH -c 2

export SINGULARITY_BIND="/mnt/storage/nobackup/proj/scbsu/nec181/testing/EPI2ME/wf-alignment/"
export SINGULARITYENV_PREPEND_PATH="/home/epi2melabs/conda/bin/"

module load Java/17.0.6

nextflow run epi2me-labs/wf-alignment
--fastq wf-alignment-demo/fastq
--references wf-alignment-demo/references
-profile singularity

Workflow Execution - CLI Execution Profile

singularity

What happened?

Hi, I've been encountering errors related to container usage (I think) when trying to run wf-alignment with the demo data.

To try and diagnose the issue, I ran
singularity shell ontresearch-wf-alignment-shaa9faef16822c5aa48366a4c45b401c9233a6c0f7.img
python --version
but it returned
bash: python: command not found

I had gotten a similar error when running wf-human-variation but resolved it by setting:
export SINGULARITYENV_PREPEND_PATH="/home/epi2melabs/conda/bin/"
before starting the pipeline. However, this solution hasn't worked in this case.

I also tried starting an interactive shell session with ontresearch-wf-common-sha338caea0a2532dc0ea8f46638ccc322bb8f9af48.img but encountered the following error:
ERROR: /bin/sh does not exist in container

How should I fix this? Any help is much appreciated. Thanks!

Relevant log output

N E X T F L O W   ~  version 24.04.3

Launching `https://github.com/epi2me-labs/wf-alignment` [compassionate_brattain] DSL2 - revision: 36f41e7966 [master]


||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-alignment v1.2.0-g36f41e7
--------------------------------------------------------------------------------
Core Nextflow options
  revision       : master
  runName        : compassionate_brattain
  containerEngine: singularity
  container      : [withLabel:wfalignment:/mnt/storage/nobackup/proj/scbsu/nec181/testing/EPI2ME/wf-alignment/work/singularity/ontresearch-wf-alignment/, withLabel:wf_common:/mnt/storage/nobackup/proj/scb
su/nec181/testing/EPI2ME/wf-alignment/work/singularity/ontresearch-wf-common/]
  launchDir      : /mnt/storage/nobackup/proj/scbsu/nec181/testing/EPI2ME/wf-alignment
  workDir        : /mnt/storage/nobackup/proj/scbsu/nec181/testing/EPI2ME/wf-alignment/work
  projectDir     : /mnt/nfs/home/nec181/.nextflow/assets/epi2me-labs/wf-alignment
  userName       : nec181
  profile        : singularity
  configFiles    : /mnt/nfs/home/nec181/.nextflow/assets/epi2me-labs/wf-alignment/nextflow.config

Input Options
  fastq          : wf-alignment-demo/fastq
  references     : wf-alignment-demo/references

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-alignment for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x


--------------------------------------------------------------------------------
This is epi2me-labs/wf-alignment v1.2.0-g36f41e7.
--------------------------------------------------------------------------------
Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files.
[-        ] fastcat -

[-        ] fastcat                        -
[-        ] move_or_compress_fq_file       -
[-        ] pipeline:getParams             -
[-        ] pipeline:getVersions           -
[-        ] pip…:process_references:fx2tab -
[-        ] pip…process_references:combine -
[-        ] pipeline:makeMMIndex           -

[-        ] fastcat                        -
[-        ] move_or_compress_fq_file       -
[-        ] pipeline:getParams             -
[-        ] pipeline:getVersions           -
[-        ] pip…:process_references:fx2tab -
[-        ] pip…process_references:combine -
[-        ] pipeline:makeMMIndex           -
[-        ] pipeline:alignReads            -
[-        ] pipeline:bamstats              -
[-        ] pipeline:addStepsColumn        -
[-        ] pipeline:readDepthPerRef       -
[-        ] pipeline:collectFilesInDir     -

[-        ] fastcat                        -
[-        ] move_or_compress_fq_file       -
[-        ] pipeline:getParams             -
[-        ] pipeline:getVersions           -
[-        ] pip…:process_references:fx2tab -
[-        ] pip…process_references:combine -
[-        ] pipeline:makeMMIndex           -
[-        ] pipeline:alignReads            -
[-        ] pipeline:bamstats              -
[-        ] pipeline:addStepsColumn        -
[-        ] pipeline:readDepthPerRef       -
[-        ] pipeline:collectFilesInDir     -
[-        ] pipeline:makeReport            -
[-        ] pipeline:configure_igv         -
[-        ] output                         -
[-        ] fastcat                        -
[-        ] move_or_compress_fq_file       -
[-        ] pipeline:getParams             -
[-        ] pipeline:getVersions           -
[-        ] pip…:process_references:fx2tab -
[-        ] pip…process_references:combine -
[-        ] pipeline:makeMMIndex           -
[-        ] pipeline:alignReads            -
[-        ] pipeline:bamstats              -
[-        ] pipeline:addStepsColumn        -
[-        ] pipeline:readDepthPerRef       -
[-        ] pipeline:collectFilesInDir     -

[-        ] fastcat                        -
[-        ] move_or_compress_fq_file       -
[-        ] pipeline:getParams             -
[-        ] pipeline:getVersions           -
[-        ] pip…:process_references:fx2tab -
[-        ] pip…process_references:combine -
[-        ] pipeline:makeMMIndex           -
[-        ] pipeline:alignReads            -
[-        ] pipeline:bamstats              -
[-        ] pipeline:addStepsColumn        -
[-        ] pipeline:readDepthPerRef       -
[-        ] pipeline:collectFilesInDir     -
[-        ] pipeline:makeReport            -
[-        ] pipeline:configure_igv         -
[-        ] output                         -

[-        ] fastcat                        -
[-        ] move_or_compress_fq_file       | 0 of 3
[-        ] pipeline:getParams             | 0 of 1
[-        ] pipeline:getVersions           | 0 of 1
[-        ] pip…:process_references:fx2tab | 0 of 3
[-        ] pip…process_references:combine | 0 of 1
[-        ] pipeline:makeMMIndex           -
[-        ] pipeline:alignReads            -
[-        ] pipeline:bamstats              -
[-        ] pipeline:addStepsColumn        -
[-        ] pipeline:readDepthPerRef       -
[-        ] pipeline:collectFilesInDir     -
[-        ] pipeline:makeReport            -
[-        ] pipeline:configure_igv         -
[-        ] output                         -

executor >  local (2)
[-        ] fastcat                        -
[-        ] move_or_compress_fq_file       | 0 of 3
[e2/51519b] pipeline:getParams             | 0 of 1
[ab/fcc7a1] pipeline:getVersions           | 0 of 1
[-        ] pip…:process_references:fx2tab | 0 of 3
[-        ] pip…process_references:combine | 0 of 1
[-        ] pipeline:makeMMIndex           -
[-        ] pipeline:alignReads            -
[-        ] pipeline:bamstats              -
[-        ] pipeline:addStepsColumn        -
[-        ] pipeline:readDepthPerRef       -
[-        ] pipeline:collectFilesInDir     -
[-        ] pipeline:makeReport            -
[-        ] pipeline:configure_igv         -
[-        ] output                         -

executor >  local (2)
[-        ] fastcat                        -
[-        ] move_or_compress_fq_file       | 0 of 3
[e2/51519b] pipeline:getParams             | 0 of 1
[ab/fcc7a1] pipeline:getVersions           | 0 of 1
[-        ] pip…:process_references:fx2tab | 0 of 3
[-        ] pip…process_references:combine | 0 of 1
[-        ] pipeline:makeMMIndex           -
[-        ] pipeline:alignReads            -
[-        ] pipeline:bamstats              -
[-        ] pipeline:addStepsColumn        -
[-        ] pipeline:readDepthPerRef       -
[-        ] pipeline:collectFilesInDir     -
[-        ] pipeline:makeReport            -
[-        ] pipeline:configure_igv         -
[-        ] output                         -

executor >  local (4)
[-        ] fastcat                        -
[fb/ad59ce] move_or_compress_fq_file (1)   | 0 of 3
[e2/51519b] pipeline:getParams             | 1 of 1 ✔
[ab/fcc7a1] pipeline:getVersions           | 0 of 1
[df/1aaa27] pip…cess_references:fx2tab (2) | 0 of 3
[-        ] pip…process_references:combine | 0 of 1
[-        ] pipeline:makeMMIndex           -
[-        ] pipeline:alignReads            -
[-        ] pipeline:bamstats              -
[-        ] pipeline:addStepsColumn        -
[-        ] pipeline:readDepthPerRef       -
[-        ] pipeline:collectFilesInDir     -
[-        ] pipeline:makeReport            -
[-        ] pipeline:configure_igv         -
[-        ] output                         | 0 of 1
ERROR ~ Error executing process > 'pipeline:getVersions'

Caused by:
  Process `pipeline:getVersions` terminated with an error exit status (127)


Command executed:

  python --version | tr -s ' ' ',' | tr '[:upper:]' '[:lower:]' > versions.txt
  seqkit version | sed 's/ /,/' >> versions.txt
  minimap2 --version | sed 's/^/minimap2,/' >> versions.txt
  samtools --version | (head -n 1 && exit 0) | sed 's/ /,/' >> versions.txt
  fastcat --version | sed 's/^/fastcat,/' >> versions.txt
  mosdepth --version | sed 's/ /,/' >> versions.txt
  ezcharts --version | sed 's/ /,/' >> versions.txt
  python -c "import pysam; print(f'pysam,{pysam.__version__}')" >> versions.txt
  bgzip --version | head -n1 | sed -E 's/(.*) /\1,/' >> versions.txt

Command exit status:
  127

Command output:
  (empty)

Command error:
  WARNING: passwd file doesn't exist in container, not updating
  WARNING: group file doesn't exist in container, not updating
  .command.sh: line 2: python: command not found

Work dir:
  /mnt/storage/nobackup/proj/scbsu/nec181/testing/EPI2ME/wf-alignment/work/ab/fcc7a164d38ce8b15b4a998434231c

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details
WARN: Killing running tasks (2)

executor >  local (4)
[-        ] fastcat                        -
[fb/ad59ce] move_or_compress_fq_file (1)   | 0 of 2
[e2/51519b] pipeline:getParams             | 1 of 1 ✔
[ab/fcc7a1] pipeline:getVersions           | 1 of 1, failed: 1 ✘
[df/1aaa27] pip…cess_references:fx2tab (2) | 0 of 2
[-        ] pip…process_references:combine | 0 of 1
[-        ] pipeline:makeMMIndex           -
[-        ] pipeline:alignReads            -
[-        ] pipeline:bamstats              -
[-        ] pipeline:addStepsColumn        -
[-        ] pipeline:readDepthPerRef       -
[-        ] pipeline:collectFilesInDir     -
[-        ] pipeline:makeReport            -
[-        ] pipeline:configure_igv         -
[-        ] output                         | 0 of 1
ERROR ~ Error executing process > 'pipeline:getVersions'

Caused by:
  Process `pipeline:getVersions` terminated with an error exit status (127)


Command executed:

  python --version | tr -s ' ' ',' | tr '[:upper:]' '[:lower:]' > versions.txt
  seqkit version | sed 's/ /,/' >> versions.txt
  minimap2 --version | sed 's/^/minimap2,/' >> versions.txt
  samtools --version | (head -n 1 && exit 0) | sed 's/ /,/' >> versions.txt
  fastcat --version | sed 's/^/fastcat,/' >> versions.txt
  mosdepth --version | sed 's/ /,/' >> versions.txt
  ezcharts --version | sed 's/ /,/' >> versions.txt
  python -c "import pysam; print(f'pysam,{pysam.__version__}')" >> versions.txt
  bgzip --version | head -n1 | sed -E 's/(.*) /\1,/' >> versions.txt

Command exit status:
  127

Command output:
  (empty)

Command error:
  WARNING: passwd file doesn't exist in container, not updating
  WARNING: group file doesn't exist in container, not updating
  .command.sh: line 2: python: command not found

Work dir:
  /mnt/storage/nobackup/proj/scbsu/nec181/testing/EPI2ME/wf-alignment/work/ab/fcc7a164d38ce8b15b4a998434231c

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details

Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

no

Other demo data information

No response

The text was updated successfully, but these errors were encountered:

SamStudio8 · 2024-07-30T18:07:34Z

@Erin-Chee Are you able to provide a debug output for your command to find python?

singularity --debug exec ontresearch-wf-alignment-shaa9faef16822c5aa48366a4c45b401c9233a6c0f7.img which python

SamStudio8 · 2024-07-30T18:10:35Z

@Erin-Chee Actually, I should have asked more simply, does this work if you provide --no-home?

singularity exec --no-home ontresearch-wf-alignment-shaa9faef16822c5aa48366a4c45b401c9233a6c0f7.img which python

Erin-Chee · 2024-07-31T09:41:52Z

Hi @SamStudio8 thanks for the reply.

I ran singularity exec with --no-home, but there was no output for which python.

I tried it with --debug and this was the output:

DEBUG [U=599395,P=11349] persistentPreRun() DEBUG [U=599395,P=11349] persistentPreRun() DEBUG [U=599395,P=11349] handleConfDir() DEBUG [U=599395,P=11349] launchContainer() DEBUG [U=599395,P=11349] setUmask() DEBUG [U=599395,P=11349] Init() DEBUG [U=599395,P=11349] Init() DEBUG [U=599395,P=11349] Init() DEBUG [U=599395,P=11349] Init() DEBUG [U=599395,P=11349] Init() DEBUG [U=599395,P=11349] checkEncryptionKey() DEBUG [U=599395,P=11349] setCgroups() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() VERBOSE [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] SetContainerEnv() VERBOSE [U=599395,P=11349] SetContainerEnv() VERBOSE [U=599395,P=11349] SetContainerEnv() DEBUG [U=599395,P=11349] init() DEBUG [U=599395,P=11349] Exec() VERBOSE [U=0,P=11349] print() VERBOSE [U=0,P=11349] init() DEBUG [U=0,P=11349] load_overlay_module() DEBUG [U=0,P=11349] load_overlay_module() VERBOSE [U=0,P=11349] is_suid() VERBOSE [U=0,P=11349] priv_drop() DEBUG [U=599395,P=11349] read_engine_config() DEBUG [U=599395,P=11349] init() DEBUG [U=599395,P=11349] cleanenv() DEBUG [U=599395,P=11349] cleanenv() DEBUG [U=599395,P=11349] cleanenv() DEBUG [U=599395,P=11349] cleanenv() DEBUG [U=599395,P=11349] init() VERBOSE [U=599395,P=11364] priv_drop() DEBUG [U=599395,P=11364] set_parent_death_signal() VERBOSE [U=599395,P=11364] init() DEBUG [U=599395,P=11364] startup() VERBOSE [U=599395,P=11364] startup() DEBUG [U=599395,P=11364] StageOne() DEBUG [U=599395,P=11364] prepareAutofs() DEBUG [U=599395,P=11364] prepareAutofs() DEBUG [U=599395,P=11364] prepareAutofs() DEBUG [U=599395,P=11364] prepareAutofs() DEBUG [U=599395,P=11364] prepareAutofs() DEBUG [U=599395,P=11364] Init() DEBUG [U=599395,P=11364] Init() DEBUG [U=599395,P=11364] Init() DEBUG [U=599395,P=11364] Init() DEBUG [U=599395,P=11364] Init() DEBUG [U=599395,P=11364] setSessionLayer() DEBUG [U=599395,P=11364] setSessionLayer() VERBOSE [U=599395,P=11349] wait_child() DEBUG [U=599395,P=11349] cleanup_fd() DEBUG [U=599395,P=11349] cleanup_fd() DEBUG [U=599395,P=11349] cleanup_fd() DEBUG [U=599395,P=11349] cleanup_fd() DEBUG [U=599395,P=11349] cleanup_fd() DEBUG [U=599395,P=11349] cleanup_fd() DEBUG [U=599395,P=11349] init() DEBUG [U=599395,P=11349] init() DEBUG [U=599395,P=11349] init() VERBOSE [U=599395,P=11349] priv_escalate() VERBOSE [U=0,P=11349] priv_escalate() VERBOSE [U=0,P=11349] init() DEBUG [U=0,P=11370] set_parent_death_signal() VERBOSE [U=0,P=11370] create_namespace() VERBOSE [U=0,P=11349] enter_namespace() DEBUG [U=0,P=11349] enter_namespace() VERBOSE [U=0,P=11370] create_namespace() DEBUG [U=0,P=11349] set_master_privileges() DEBUG [U=0,P=11349] apply_privileges() DEBUG [U=0,P=11349] apply_privileges() DEBUG [U=0,P=11349] apply_privileges() DEBUG [U=0,P=11349] apply_privileges() DEBUG [U=0,P=11349] apply_privileges() DEBUG [U=0,P=11349] apply_privileges() DEBUG [U=0,P=11371] set_rpc_privileges() DEBUG [U=0,P=11371] apply_privileges() DEBUG [U=0,P=11371] apply_privileges() DEBUG [U=0,P=11371] apply_privileges() DEBUG [U=0,P=11371] apply_privileges() DEBUG [U=0,P=11371] apply_privileges() DEBUG [U=0,P=11371] apply_privileges() DEBUG [U=599395,P=11371] set_parent_death_signal() VERBOSE [U=599395,P=11371] init() DEBUG [U=599395,P=11349] startup() VERBOSE [U=599395,P=11349] startup() DEBUG [U=599395,P=11371] startup() VERBOSE [U=599395,P=11371] startup() DEBUG [U=599395,P=11349] setupSessionLayout() DEBUG [U=599395,P=11349] setupOverlayLayout() DEBUG [U=599395,P=11349] addRootfsMount() DEBUG [U=599395,P=11349] addRootfsMount() DEBUG [U=599395,P=11349] addRootfsMount() DEBUG [U=599395,P=11349] addKernelMount() DEBUG [U=599395,P=11349] addKernelMount() VERBOSE [U=599395,P=11349] addKernelMount() DEBUG [U=599395,P=11349] addKernelMount() DEBUG [U=599395,P=11349] addKernelMount() VERBOSE [U=599395,P=11349] addKernelMount() DEBUG [U=599395,P=11349] addDevMount() DEBUG [U=599395,P=11349] addDevMount() VERBOSE [U=599395,P=11349] addDevMount() DEBUG [U=599395,P=11349] addHostMount() VERBOSE [U=599395,P=11349] addBindsMount() VERBOSE [U=599395,P=11349] addBindsMount() DEBUG [U=599395,P=11349] addHomeStagingDir() DEBUG [U=599395,P=11349] addHomeMount() DEBUG [U=599395,P=11349] addTmpMount() VERBOSE [U=599395,P=11349] addTmpMount() VERBOSE [U=599395,P=11349] addTmpMount() DEBUG [U=599395,P=11349] addScratchMount() DEBUG [U=599395,P=11349] addLibsMount() DEBUG [U=599395,P=11349] addFilesMount() DEBUG [U=599395,P=11349] addResolvConfMount() VERBOSE [U=599395,P=11349] addResolvConfMount() DEBUG [U=599395,P=11349] addHostnameMount() DEBUG [U=599395,P=11349] create() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountImage() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] setPropagationMount() VERBOSE [U=599395,P=11349] Passwd() WARNING [U=599395,P=11349] addIdentityMount() VERBOSE [U=599395,P=11349] Group() WARNING [U=599395,P=11349] addIdentityMount() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] mountGeneric() DEBUG [U=599395,P=11349] addCwdMount() VERBOSE [U=599395,P=11349] addCwdMount() DEBUG [U=599395,P=11349] create() DEBUG [U=599395,P=11371] Chroot() DEBUG [U=599395,P=11371] Chroot() DEBUG [U=599395,P=11371] Chroot() DEBUG [U=599395,P=11371] Chroot() DEBUG [U=599395,P=11371] Chroot() DEBUG [U=599395,P=11371] Chroot() DEBUG [U=599395,P=11349] create() VERBOSE [U=0,P=11370] wait_child() DEBUG [U=0,P=11370] init() DEBUG [U=0,P=11370] apply_privileges() DEBUG [U=0,P=11370] apply_privileges() DEBUG [U=0,P=11370] apply_privileges() DEBUG [U=0,P=11370] apply_privileges() DEBUG [U=0,P=11370] apply_privileges() DEBUG [U=0,P=11370] apply_privileges() DEBUG [U=599395,P=11370] set_parent_death_signal() DEBUG [U=599395,P=11370] startup() VERBOSE [U=599395,P=11370] startup() DEBUG [U=599395,P=11370] StageTwo() DEBUG [U=599395,P=11370] StartProcess() DEBUG [U=599395,P=11370] func1() DEBUG [U=599395,P=11370] func1() DEBUG [U=599395,P=11370] sylogBuiltin() DEBUG [U=599395,P=11349] PostStartProcess() DEBUG [U=599395,P=11349] Master() Singularity version: 3.11.5-1.el7
Parsing configuration file /etc/singularity/singularity.conf
/mnt/nfs/home/nec181/.singularity already exists. Not creating.
Using native runtime launcher.
Saving umask 0077 for propagation into container
Image format detection
Check for sandbox image format
sandbox format initializer returned: not a directory image
Check for sif image format
sif image format detected
Checking for encrypted system partition
Recording rootless XDG_RUNTIME_DIR / DBUS_SESSION_BUS_ADDRESS
Forwarding LC_PAPER environment variable
Forwarding MANPATH environment variable
Forwarding XDG_SESSION_ID environment variable
Forwarding LC_ADDRESS environment variable
Forwarding HOSTNAME environment variable
Forwarding LC_MONETARY environment variable
Forwarding TERM environment variable
Forwarding SHELL environment variable
Forwarding LMOD_ROOT environment variable
Forwarding HISTSIZE environment variable
Forwarding MODULEPATH_ROOT environment variable
Forwarding TMPDIR environment variable
Forwarding SSH_CLIENT environment variable
Forwarding LMOD_PKG environment variable
Forwarding I_MPI_PMI_LIBRARY environment variable
Forwarding LC_NUMERIC environment variable
Forwarding OLDPWD environment variable
Forwarding LMOD_VERSION environment variable
Forwarding SSH_TTY environment variable
Forwarding USER environment variable
Forwarding LMOD_sys environment variable
Forwarding LS_COLORS environment variable
Forwarding LC_TELEPHONE environment variable
Forwarding TMOUT environment variable
Forwarding MAIL environment variable
Forwarding LC_IDENTIFICATION environment variable
Forwarding PWD environment variable
Forwarding LANG environment variable
Forwarding MODULEPATH environment variable
Forwarding LC_MEASUREMENT environment variable
Forwarding LMOD_CMD environment variable
Forwarding HISTCONTROL environment variable
Forwarding SHLVL environment variable
Forwarding BASH_ENV environment variable
Forwarding LOGNAME environment variable
Forwarding SSH_CONNECTION environment variable
Forwarding MODULESHOME environment variable
Forwarding LMOD_SETTARG_FULL_SUPPORT environment variable
Forwarding LESSOPEN environment variable
Forwarding XDG_RUNTIME_DIR environment variable
Forwarding LMOD_DIR environment variable
Forwarding LC_TIME environment variable
Forwarding LC_NAME environment variable
Forwarding BASH_FUNC_module() environment variable
Forwarding BASH_FUNC_ml() environment variable
Forwarding _ environment variable
Not forwarding SINGULARITY_DEBUG environment variable
Forwarding USER_PATH environment variable
Setting HOME=/mnt/nfs/home/nec181
Setting PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Use starter binary /usr/libexec/singularity/bin/starter-suid
Setting GOGC=off for starter
Set messagelevel to: 5
Starter initialization
Trying to load overlay kernel module
Overlay seems supported by the kernel
Check if we are running as setuid
Drop root privileges
Read engine configuration
CleanupHost not requested
Clearing env var LOAD_OVERLAY_MODULE=1
Keeping env var SINGULARITY_MESSAGELEVEL=5
Clearing env var ENGINE_CONFIG1={"engineName":"singularity","containerID":"","engineConfig":{"jsonConfig":{"unixSocketPair":[0,0],"image":"/mnt/nfs/home/nec181/scbsu/nec181/testing/EPI2ME/wf-alignment/work/singularity/ontresearch-wf-alignment-shaa9faef16822c5aa48366a4c45b401c9233a6c0f7.img","homedir":"/mnt/nfs/home/nec181","homeDest":"/mnt/nfs/home/nec181","network":"bridge","cwd":"/mnt/nfs/home/nec181/scbsu/nec181/testing/EPI2ME/wf-alignment/work/singularity","configurationFile":"/etc/singu(TRUNCATED...)DEBUG [U=599395,P=11349] cleanenv() Clearing env var ENGINE_CONFIG_CHUNKS=1
Keeping env var GOGC=off
Wait completion of stage1
Drop root privileges permanently
Set parent death signal to 9
Spawn stage 1
singularity runtime engine selected
Execute stage 1
Entering stage 1
Found "/proc/sys/fs/binfmt_misc" as autofs mount point
Could not keep file descriptor for bind path /etc/localtime: no mount point
Could not keep file descriptor for bind path /etc/hosts: no mount point
Could not keep file descriptor for home directory /mnt/nfs/home/nec181: no mount point
Could not keep file descriptor for current working directory /mnt/nfs/home/nec181/scbsu/nec181/testing/EPI2ME/wf-alignment/work/singularity: no mount point
Image format detection
Check for sandbox image format
sandbox format initializer returned: not a directory image
Check for sif image format
sif image format detected
Overlay seems supported and allowed by kernel
Attempting to use overlayfs (enable overlay = try)
stage 1 exited with status 0
Setting FD_CLOEXEC on starter fd 3
Close file descriptor 4
Close file descriptor 5
Close file descriptor 6
Close file descriptor 7
Close file descriptor 8
Set child signal mask
Create socketpair for master communication channel
Create RPC socketpair for communication between stage 2 and RPC server
Get root privileges
Change filesystem uid to 599395
Spawn master process
Set parent death signal to 9
Create mount namespace
Entering in mount namespace
Opening namespace file ns/mnt
Create mount namespace
Set master privileges
Effective capabilities: 0x00000000000000c0
Permitted capabilities: 0x0000001fffffffff
Bounding capabilities: 0x0000001fffffffff
Inheritable capabilities: 0x0000001fffffffff
Ambient capabilities: 0x0000000000000000
Set user ID to 599395
Set RPC privileges
Effective capabilities: 0x0000000000200000
Permitted capabilities: 0x0000001fffffffff
Bounding capabilities: 0x0000000008204000
Inheritable capabilities: 0x0000000000000000
Ambient capabilities: 0x0000000000000000
Set user ID to 599395
Set parent death signal to 9
Spawn RPC server
singularity runtime engine selected
Execute master process
singularity runtime engine selected
Serve RPC requests
Using Layer system: overlay
Creating overlay SESSIONDIR layout
Mount rootfs in read-only mode
Image type is 4096
Mounting block [squashfs] image: /mnt/nfs/home/nec181/scbsu/nec181/testing/EPI2ME/wf-alignment/work/singularity/ontresearch-wf-alignment-shaa9faef16822c5aa48366a4c45b401c9233a6c0f7.img
Checking configuration file for 'mount proc'
Adding proc to mount list
Default mount: /proc:/proc
Checking configuration file for 'mount sys'
Adding sysfs to mount list
Default mount: /sys:/sys
Checking configuration file for 'mount dev'
Adding dev to mount list
Default mount: /dev:/dev
Not mounting host file systems per configuration
Found 'bind path' = /etc/localtime, /etc/localtime
Found 'bind path' = /etc/hosts, /etc/hosts
Staging home directory (/mnt/nfs/home/nec181) at /var/lib/singularity/mnt/session/mnt/nfs/home/nec181
Adding home directory mount [/var/lib/singularity/mnt/session/mnt/nfs/home/nec181:/mnt/nfs/home/nec181] to list using layer: overlay
Checking for 'mount tmp' in configuration file
Default mount: /tmp:/tmp
Default mount: /var/tmp:/var/tmp
Not mounting scratch directory: Not requested
Checking for 'user bind control' in configuration file
Checking for 'user bind control' in configuration file
Adding /etc/resolv.conf to mount list
Default mount: /etc/resolv.conf:/etc/resolv.conf
Skipping hostname mount, not virtualizing UTS namespace on user request
Mount all
Mounting tmpfs to /var/lib/singularity/mnt/session
Mounting loop device /dev/loop0 to /var/lib/singularity/mnt/session/rootfs of type squashfs
Mounting overlay to /var/lib/singularity/mnt/session/final
Remounting /var/lib/singularity/mnt/session/final
Set RPC mount propagation flag to SLAVE
Checking for template passwd file: /var/lib/singularity/mnt/session/rootfs/etc/passwd
passwd file doesn't exist in container, not updating
Checking for template group file: /var/lib/singularity/mnt/session/rootfs/etc/group
group file doesn't exist in container, not updating
Mounting /dev to /var/lib/singularity/mnt/session/final/dev
Mounting /etc/localtime to /var/lib/singularity/mnt/session/final/etc/localtime
Remounting /var/lib/singularity/mnt/session/final/etc/localtime
Mounting /etc/hosts to /var/lib/singularity/mnt/session/final/etc/hosts
Remounting /var/lib/singularity/mnt/session/final/etc/hosts
Mounting /proc to /var/lib/singularity/mnt/session/final/proc
Remounting /var/lib/singularity/mnt/session/final/proc
Mounting sysfs to /var/lib/singularity/mnt/session/final/sys
Mounting /mnt/nfs/home/nec181 to /var/lib/singularity/mnt/session/mnt/nfs/home/nec181
Remounting /var/lib/singularity/mnt/session/mnt/nfs/home/nec181
Mounting /var/lib/singularity/mnt/session/mnt/nfs/home/nec181 to /var/lib/singularity/mnt/session/final/mnt/nfs/home/nec181
Mounting /tmp to /var/lib/singularity/mnt/session/final/tmp
Remounting /var/lib/singularity/mnt/session/final/tmp
Mounting /var/tmp to /var/lib/singularity/mnt/session/final/var/tmp
Remounting /var/lib/singularity/mnt/session/final/var/tmp
Mounting /var/lib/singularity/mnt/session/etc/resolv.conf to /var/lib/singularity/mnt/session/final/etc/resolv.conf
Using /mnt/nfs/home/nec181/scbsu/nec181/testing/EPI2ME/wf-alignment/work/singularity as current working directory
/mnt/nfs/home/nec181/scbsu/nec181/testing/EPI2ME/wf-alignment/work/singularity found within container
Chroot into /var/lib/singularity/mnt/session/final
Hold reference to host / directory
Called pivot_root on /var/lib/singularity/mnt/session/final
Change current directory to host / directory
Apply slave mount propagation for host / directory
Called unmount(/, syscall.MNT_DETACH)
Changing directory to / to avoid getpwd issues
Chdir into / to avoid errors
rpc server exited with status 0
Set container privileges
Effective capabilities: 0x0000000000000000
Permitted capabilities: 0x0000000000000000
Bounding capabilities: 0x0000000000000000
Inheritable capabilities: 0x0000000000000000
Ambient capabilities: 0x0000000000000000
Set user ID to 599395
Set parent death signal to 9
singularity runtime engine selected
Execute stage 2
Entering stage 2
Setting umask in container to 0077
Not exporting "BASH_FUNC_ml()" to container environment: invalid key
Not exporting "BASH_FUNC_module()" to container environment: invalid key
Running action command exec
Post start process
Child exited with exit status 1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue with containers - .command.sh: line 2: python: command not found #33

Issue with containers - .command.sh: line 2: python: command not found #33

Erin-Chee commented Jul 30, 2024

SamStudio8 commented Jul 30, 2024

SamStudio8 commented Jul 30, 2024

Erin-Chee commented Jul 31, 2024

Issue with containers - .command.sh: line 2: python: command not found #33

Issue with containers - .command.sh: line 2: python: command not found #33

Comments

Erin-Chee commented Jul 30, 2024

Operating System

Other Linux

Workflow Version

Workflow Execution

Other workflow execution

EPI2ME Version

CLI command run

Workflow Execution - CLI Execution Profile

What happened?

Relevant log output

Application activity log entry

Were you able to successfully run the latest version of the workflow with the demo data?

Other demo data information

SamStudio8 commented Jul 30, 2024

SamStudio8 commented Jul 30, 2024

Erin-Chee commented Jul 31, 2024