Kernel compatibility for singularity run

[ciszew]

Hello
Thank you for this fantastic pipeline. I have been using/validating mcmicro using local computer and it works so well that i decided to run it on our institution HPC to accommodate more users and speed up the process.
Unfortunately i have a hard time getting it too work, when running exemplar-001 using:
nextflow run labsyspharm/mcmicro --in exemplar-001 -profile singularity
Im getting couple errors:
'''Error executing process > 'registration:ashlar'

Caused by:
Process registration:ashlar terminated with an error exit status (127)

Command executed:

ashlar 'exemplar-001-cycle-06.ome.tiff' 'exemplar-001-cycle-07.ome.tiff' 'exemplar-001-cycle-08.ome.tiff' -m 30 --ffp exemplar-001-cycle-06-ffp.tif exemplar-001-cycle-07-ffp.tif exemplar-001-cycle-08-ffp.tif --dfp exemplar-001-cycle-06-dfp.tif exemplar-001-cycle-07-dfp.tif exemplar-001-cycle-08-dfp.tif -o mcmicro_test1.ome.tif

Command exit status:
127

Command output:
(empty)

Command error:
/bin/touch: line 26: /scratch/log/touch.log: Permission denied
WARNING: Skipping user bind, non existent bind point (directory) in container: '/gpfs/data/hdid-share/Codex/mcmicro_test1'
WARNING: Not mounting requested bind point (already mounted in container): /home/cciszews/singularity/work/68/8501e83be0faecbec02242fe10840e
FATAL: kernel too old
'''
So if i understand this correctly at least one issue is caused by incompatible/old kernel on my HPC (which is Red Hat Enterprise Linux 6 Update 7 from 2016, singularity version is also dated at 2.6.0).
I was wondering if you have any suggestions how this can be fixed?
If this is indeed dated linux kernel problem do you know what kernel version docker/singularity containers would work on?

[ArtemSokolov]

Thanks for the kind words, @ciszew.

I'm also noticing a Permission denied error. Are you able to run Singularity containers outside of MCMICRO on your HPC? For example, can you try the following:

singularity exec -C docker://labsyspharm/ashlar:1.17.0 ashlar -h

which runs the ASHLAR container isolated from the host system (the -C flag). A successful execution should display the ASHLAR help page.

Another thing to try is:

singularity exec -C -B "$PWD" docker://labsyspharm/ashlar:1.17.0 echo "Hello World" > test.txt

which should mount the present working directory (-B "$PWD") to be visible inside the container and write test.txt into that directory.

As far as versions, our automated tests use Singularity v3.6.4 (https://github.com/labsyspharm/mcmicro/blob/master/.github/workflows/ci.yml#L87). Not entirely sure about the kernel version.

[ciszew]

Thank you quick replay. Unfortunately both commands results with the same error pointing again to outdated kernel on our HPC:
'''

(base) [cciszews@cri16cn054 singularity]$ singularity exec -C docker://labsyspharm/ashlar:1.17.0 ashlar -h
Docker image path: index.docker.io/labsyspharm/ashlar:1.17.0
Cache folder set to /home/cciszews/.singularity/docker
Creating container runtime...
Exploding layer: sha256:d19f32bd9e4106d487f1a703fc2f09c8edadd92db4405d477978e8e466ab290d.tar.gz
Exploding layer: sha256:b0efcaefb024e93c2e3a43ad8ba40a9004a03eeb26be35da21cc3cd34aa0efb1.tar.gz
Exploding layer: sha256:d6a1f121ae324f4bc287cce78156090a68176202aeb0a7921a10baf12d2a3324.tar.gz
Exploding layer: sha256:5725980b01ede4e17c35421f952a76404137f0197a949f413ceb42cf9dc6b8f9.tar.gz
Exploding layer: sha256:934f8f3ec0b72852b2f29c8b90446f48a6d8de577b9b006f395bc5959af20016.tar.gz
Exploding layer: sha256:b2b1c3ea2696d5da8669347ad9869a69f95759e7165945d5980435f1b2b7efe9.tar.gz
Exploding layer: sha256:42dfd903a398ef9bcd92719935762c3b923334a22be7b7ce2d1fdc465cb3149f.tar.gz
FATAL: kernel too old
(base) [cciszews@cri16cn054 singularity]$ singularity exec -C -B "$PWD" docker://labsyspharm/ashlar:1.17.0 echo "Hello World" > test.txt
WARNING: Skipping user bind, non existent bind point (directory) in container: '/home/cciszews/singularity'
FATAL: kernel too old
'''
There are 4 compute nodes on our HPC with newer version of RHEL (Kernel version: 3.10.0-957.12.1.el7.x86_64) which may solve the problem but im not even sure i can get access to them and if i can use singularity on them unfortunately.

[ArtemSokolov]

Unfortunately, you will need to ask your HPC administrators to upgrade the kernel. Singularity uses the kernel of the host system, so there is not much that can be done from the container side. Anything from 2021 or newer should work.