Support foreign workflow systems

[jsbrittain]

Provide an interface to non-Snakemake workflow manager rules / modules. We will focus on nextflow as another common workflow system. Options include:

• Install nextflow into a conda environment and launch form there (questions over how effective this will be, especially when run on HPC).
• Use Snakemake's native foreign interface support:
  • Integrating foreign workflow management systems
  • Snakemake 6.2 and later allows to hand over execution steps to other workflow management systems. By this, it is possible to make use of workflows written for other systems, while performing any further pre- or postprocessing within Snakemake. Such a handover is indicated with the handover directive.
  • Here, the workflow is executed as usual until this rule is reached. Then, Snakemake passes all resources to the nextflow workflow management system, which generates certain files. The rule is executed as a local rule, meaning that it would not be submitted to a cluster or cloud system by Snakemake. Instead, the invoked other workflow management system is responsible for that. E.g., in case of Nextflow, submission behavior can be configured via a nextflow.conf file or environment variables. After the step is done, Snakemake continues execution with the output files produced by the foreign workflow.

A useful trial workflow is the vocpl pipeline, which we have implemented in both nextflow and Snakemake.