-with-docker is very slow compared to host

[yamad]

I have a pipeline with a large docker image. When I try to run my nextflow workflow using the docker image, I get very slow--almost 10x slower--performance. For example, the first step uses cutadapt. If I install cutadapt into a python environment on the host machine, then the process finishes in ~35 mins:

source venv/bin/activate
pip install cutadapt
nextflow run
...
35 min

On the other hand, if I try to use the cutadapt built into my docker image, it runs in >8.5 hours:

nextflow run -with-docker pipeline:latest
...
8 hours 35 min

Is there a way to speed up runs that use the -with-docker option? Thanks!

[pditommaso]

Hard to say but Nextflow is hardly involved since what is doing is just to create a Bash script running the target command into the container.

You can troubleshoot the problematic task by running bash .command.run in the task work directory, and figure out what's wrong inspecting the generated docker command.

[yamad]

Following up after a long hiatus on this problem, turns out the container was only getting 1 cpu. The .command.run file was helpful in tracking this down, as the docker command is given the --cpus 1 argument if cpus is not explicitly set. Thanks!