Merge pull request #113 from genomic-medicine-sweden/dev

Release 2.0.0
genomic-medicine-sweden · Apr 24, 2024 · 9aea9e8 · 9aea9e8
2 parents 69df13c + 179bcc2
commit 9aea9e8
Show file tree

Hide file tree

Showing 70 changed files with 2,349 additions and 410 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -28,7 +28,7 @@ jobs:
           - "latest-everything"
         parameters:
           - "-profile test,docker"
-          - "-profile test,docker --switch_subsample_region false --switch_downsample false --switch_build_tracks false --switch_stringtie false --switch_vep false --switch_drop_ae false --switch_drop_as false"
+          - "-profile test,docker --skip_subsample_region true --skip_downsample true --skip_build_tracks true --skip_stringtie true --skip_vep true --skip_drop_ae true --skip_drop_as true"
     steps:
       - name: Check out pipeline code
         uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4

diff --git a/.github/workflows/download_pipeline.yml b/.github/workflows/download_pipeline.yml
@@ -30,6 +30,9 @@ jobs:
       - name: Install Nextflow
         uses: nf-core/setup-nextflow@v1
 
+      - name: Disk space cleanup
+        uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1
+
       - uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5
         with:
           python-version: "3.11"

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -3,17 +3,66 @@
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+### Parameters
+
+| Old parameter | New parameter |
+| ------------- | ------------- |
+
+:::note
+Parameter has been updated if both old and new parameter information is present.
+Parameter has been added if just the new parameter information is present.
+Parameter has been removed if new parameter information isn't present.
+:::
+
+## 2.0.1 - [XXXX-XX-XX]
+
+### `Added`
+
+### `Fixed`
+
+### `Parameters`
+
+## 2.0.0 - Santa [2024-04-19]
+
 ### `Added`
 
 - Added automatic tests to test the pipeline with all switches set to false [#100](https://github.com/genomic-medicine-sweden/tomte/pull/100)
 - Added better documentation on subworkflow input [#101](https://github.com/genomic-medicine-sweden/tomte/pull/101)
+- Added option to add extra arguments to DROP aberrant expression and aberrant splicing [#104](https://github.com/genomic-medicine-sweden/tomte/pull/104)
+- Added a function to branch references into compressed/uncompressed [#107](https://github.com/genomic-medicine-sweden/tomte/pull/107)
+- Added nf-core modules gawk and filter vep to create a clinical vcf [#109](https://github.com/genomic-medicine-sweden/tomte/pull/109)
 
 ### `Fixed`
 
 - Subsample and downsample switches [#97](https://github.com/genomic-medicine-sweden/tomte/pull/97)
 - Now all reference files come with meta to avoid confusion [#101](https://github.com/genomic-medicine-sweden/tomte/pull/101)
 - GATK4_ASEREADCOUNTER and GATK4_SPLITNCIGARREADS have been updated [#101](https://github.com/genomic-medicine-sweden/tomte/pull/101)
 - Updated GATK4_ASEREADCOUNTER, now bam and vcf will be given as one channel [#103](https://github.com/genomic-medicine-sweden/tomte/pull/103)
+- Prepare reference subworkflow has been reformated and simplified [#105](https://github.com/genomic-medicine-sweden/tomte/pull/105)
+- FastQC have been updated to correctly allocate memory [#106](https://github.com/genomic-medicine-sweden/tomte/pull/106)
+- vep_filters is now extracted from gene_panel_clinical_filter [#109](https://github.com/genomic-medicine-sweden/tomte/pull/109)
+- Updated modules bcftools/stats, ensemblvep/vep, fastp, gatk4/bedtointervallist, samtools/faidx [#110](https://github.com/genomic-medicine-sweden/tomte/pull/110)
+
+### `Parameters`
+
+- Removed `--vep_filters`, it will now be automatically extracted from the `--gene_panel_clinical_filter`[#109](https://github.com/genomic-medicine-sweden/tomte/pull/109)
+  | Old parameter | New parameter |
+  | --------------- | ------------- |
+  | `--vep_filters` | |
+
+- Updated parameter names to make their use easier and more clear, changing the names from `switch` to `skip` and their default value from `true` to `false` [#108](https://github.com/genomic-medicine-sweden/tomte/pull/108)
+
+| Old parameter               | New parameter             |
+| --------------------------- | ------------------------- |
+| `--switch_subsample_region` | `--skip_subsample_region` |
+| `--switch_downsample`       | `--skip_downsample`       |
+| `--switch_build_tracks`     | `--skip_build_tracks`     |
+| `--switch_stringtie`        | `--skip_stringtie`        |
+| `--switch_vep`              | `--skip_vep`              |
+| `--switch_drop_ae`          | `--skip_drop_ae`          |
+| `--switch_drop_as`          | `--skip_drop_as`          |
+
+:::note Parameter has been updated if both old and new parameter information is present. Parameter has been added if just the new parameter information is present. Parameter has been removed if new parameter information isn't present. :::
 
 ## 1.1.0 - Rudolph [2024-03-11]
 

diff --git a/assets/multiqc_config.yml b/assets/multiqc_config.yml
@@ -1,5 +1,5 @@
 report_comment: >
-  This report has been generated by the <a href="https://github.com/genomic-medicine-sweden/tomte/releases/tag/1.1.0" target="_blank">genomic-medicine-sweden/tomte</a>
+  This report has been generated by the <a href="https://github.com/genomic-medicine-sweden/tomte/releases/tag/2.0.0" target="_blank">genomic-medicine-sweden/tomte</a>
   analysis pipeline. For information about how to interpret these results, please see the
   <a href="https://github.com/genomic-medicine-sweden/tomte/blob/master/docs/output.md" target="_blank">documentation</a>.
 report_section_order:

diff --git a/conf/modules/analyse_transcripts.config b/conf/modules/analyse_transcripts.config
@@ -16,7 +16,7 @@
 
 process {
     withName: '.*ANALYSE_TRANSCRIPTS:DROP_SAMPLE_ANNOT' {
-        ext.when = { params.switch_drop_ae | params.switch_drop_as }
+        ext.when = { (!params.skip_drop_ae) | (!params.skip_drop_as) }
         publishDir = [
             path: { "${params.outdir}/analyse_transcripts/drop" },
             mode: params.publish_dir_mode,
@@ -25,7 +25,7 @@ process {
     }
 
     withName: '.*ANALYSE_TRANSCRIPTS:DROP_CONFIG_RUN_AE' {
-        ext.when = { params.switch_drop_ae }
+        ext.when = { (!params.skip_drop_ae) }
         publishDir = [
             path: { "${params.outdir}/analyse_transcripts/drop/AE" },
             mode: params.publish_dir_mode,
@@ -34,7 +34,7 @@ process {
     }
 
     withName: '.*ANALYSE_TRANSCRIPTS:DROP_CONFIG_RUN_AS' {
-        ext.when = { params.switch_drop_as }
+        ext.when = { (!params.skip_drop_as) }
         publishDir = [
             path: { "${params.outdir}/analyse_transcripts/drop/AS" },
             mode: params.publish_dir_mode,
@@ -43,7 +43,7 @@ process {
     }
 
     withName: '.*ANALYSE_TRANSCRIPTS:DROP_FILTER_RESULTS' {
-        ext.when = { params.switch_drop_ae | params.switch_drop_as }
+        ext.when = { (!params.skip_drop_ae) | (!params.skip_drop_as) }
         publishDir = [
             path: { "${params.outdir}/analyse_transcripts/drop" },
             mode: params.publish_dir_mode,
@@ -52,7 +52,7 @@ process {
     }
 
     withName: '.*ANALYSE_TRANSCRIPTS:STRINGTIE_STRINGTIE' {
-        ext.when = { params.switch_stringtie }
+        ext.when = { (!params.skip_stringtie) }
         publishDir = [
             path: { "${params.outdir}/analyse_transcripts" },
             mode: params.publish_dir_mode,
@@ -61,7 +61,7 @@ process {
     }
 
     withName: '.*ANALYSE_TRANSCRIPTS:GFFCOMPARE' {
-        ext.when = { params.switch_stringtie }
+        ext.when = { (!params.skip_stringtie) }
         publishDir = [
             path: { "${params.outdir}/analyse_transcripts" },
             mode: params.publish_dir_mode,

diff --git a/conf/modules/annotate_snv.config b/conf/modules/annotate_snv.config
@@ -16,7 +16,7 @@
 
 process {
     withName: '.*ANNOTATE_SNV:ENSEMBLVEP_VEP' {
-        ext.when = { params.switch_vep }
+        ext.when = { (!params.skip_vep) }
         ext.prefix = { "${vcf.simpleName}_vep" }
         ext.args   = { [
             '--dir_plugins vep_cache/Plugins',
@@ -37,7 +37,41 @@ process {
             saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
         ]
     }
-    withName: '.*ANNOTATE_SNV:TABIX_VEP' {
+
+    withName: '.*ANNOTATE_SNV:RENAME_FILES' {
+        ext.when = { (!params.skip_vep) }
+        ext.prefix = { "${meta.id}_vep_${meta.set}" }
+    }
+
+    withName: '.*ANNOTATE_SNV:TABIX_TABIX' {
+        ext.when = { (!params.skip_vep) }
+        ext.prefix = { "${meta.id}_vep_${meta.set}" }
+        publishDir = [
+            path: { "${params.outdir}/annotate_vep" },
+            mode: params.publish_dir_mode,
+            saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
+        ]
+    }
+
+    withName: '.*ANNOTATE_SNV:GAWK' {
+        ext.when = { (!params.skip_vep) }
+        ext.args2 = { '\'!/^#/ {print $4}\'' }
+    }
+
+    withName: '.*ANNOTATE_SNV:ENSEMBLVEP_FILTERVEP' {
+        ext.when = { (!params.skip_vep && params.gene_panel_clinical_filter) }
+        ext.prefix = { "${meta.id}_vep_${meta.set}" }
+        ext.args   = { "--filter \"HGNC_ID in ${feature_file}\"" }
+        publishDir = [
+            path: { "${params.outdir}/annotate_vep" },
+            mode: params.publish_dir_mode,
+            saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
+        ]
+    }
+
+    withName: '.*ANNOTATE_SNV:TABIX_BGZIPTABIX' {
+        ext.when = { (!params.skip_vep && params.gene_panel_clinical_filter) }
+        ext.prefix = { "${meta.id}_vep_${meta.set}" }
         publishDir = [
             path: { "${params.outdir}/annotate_vep" },
             mode: params.publish_dir_mode,

diff --git a/conf/modules/igv_tracks.config b/conf/modules/igv_tracks.config
@@ -16,16 +16,16 @@
 
 process {
     withName: '.*IGV_TRACKS:UCSC_WIGTOBIGWIG' {
-        ext.when = { params.switch_build_tracks }
+        ext.when = { (!params.skip_build_tracks) }
         ext.args = { '-clip' }
     }
 
     withName: '.*IGV_TRACKS:JUNCTION_TRACK' {
-        ext.when = { params.switch_build_tracks }
+        ext.when = { (!params.skip_build_tracks) }
     }
 
     withName: '.*IGV_TRACKS:TABIX_BGZIPTABIX' {
-        ext.when = { params.switch_build_tracks }
+        ext.when = { (!params.skip_build_tracks) }
         ext.args2  = { '--preset bed' }
         ext.prefix = { "${meta.id}_junction" }
         publishDir = [

diff --git a/conf/modules/prepare_references.config b/conf/modules/prepare_references.config
@@ -50,6 +50,17 @@ process {
         ]
     }
 
+    withName: '.*PREPARE_REFERENCES:GUNZIP_GTF' {
+        ext.when = { params.gtf.endsWith(".gz") }
+        publishDir = [
+            path: { "${params.outdir}/references" },
+            mode: params.publish_dir_mode,
+            saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
+            enabled: params.save_reference,
+            pattern: "*gtf"
+        ]
+    }
+
     withName: '.*PREPARE_REFERENCES:GET_RRNA_TRANSCRIPTS' {
         publishDir = [
             path: { "${params.outdir}/references" },
@@ -88,14 +99,14 @@ process {
         ]
     }
 
-    withName: '.*PREPARE_REFERENCES:GUNZIP_GTF' {
-        ext.when = { params.gtf && params.gtf.endsWith(".gz") }
+    withName: '.*PREPARE_REFERENCES:GUNZIP_TRFASTA' {
+        ext.when = { params.transcript_fasta && params.transcript_fasta.endsWith(".gz") }
         publishDir = [
             path: { "${params.outdir}/references" },
             mode: params.publish_dir_mode,
             saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
             enabled: params.save_reference,
-            pattern: "*gtf"
+            pattern: "*.{fasta,fa}"
         ]
     }
 
@@ -123,23 +134,12 @@ process {
     }
 
     withName: '.*PREPARE_REFERENCES:UNTAR_VEP_CACHE' {
-        ext.when = { (params.vep_cache && params.vep_cache.endsWith("tar.gz")) }
+        ext.when = { (params.vep_cache.endsWith("tar.gz")) }
         publishDir = [
             enabled: false
         ]
     }
 
-    withName: '.*PREPARE_REFERENCES:GUNZIP_TRFASTA' {
-        ext.when = { params.transcript_fasta && params.transcript_fasta.endsWith(".gz") }
-        publishDir = [
-            path: { "${params.outdir}/references" },
-            mode: params.publish_dir_mode,
-            saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
-            enabled: params.save_reference,
-            pattern: "*.{fasta,fa}"
-        ]
-    }
-
     withName: '.*PREPARE_REFERENCES:GFFREAD' {
         ext.when = { !params.transcript_fasta }
         ext.args = { '-w' }
@@ -151,8 +151,8 @@ process {
         ]
     }
 
-    withName: '.*PREPARE_REFERENCES:SALMON_INDEX' {
-        ext.when = { !params.salmon_index }
+    withName: '.*PREPARE_REFERENCES:UNTAR_SALMON_INDEX' {
+        ext.when = { params.salmon_index.endsWith(".gz") }
         publishDir = [
             path: { "${params.outdir}/references" },
             mode: params.publish_dir_mode,
@@ -161,8 +161,8 @@ process {
         ]
     }
 
-    withName: '.*PREPARE_REFERENCES:UNTAR_SALMON_INDEX' {
-        ext.when = { params.salmon_index.endsWith(".gz") }
+    withName: '.*PREPARE_REFERENCES:SALMON_INDEX' {
+        ext.when = { !params.salmon_index }
         publishDir = [
             path: { "${params.outdir}/references" },
             mode: params.publish_dir_mode,

diff --git a/conf/test.config b/conf/test.config
@@ -19,6 +19,9 @@ params {
     max_memory = '6.GB'
     max_time   = '6.h'
 
+    // Skip when GITHUB actions
+    skip_drop_ae = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true
+    skip_drop_as = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true
 
     // Input data
     input = "${projectDir}/test_data/samplesheet_chr21.csv"
@@ -36,7 +39,6 @@ params {
 
     // VEP
     vep_cache            = "https://raw.githubusercontent.com/nf-core/test-datasets/raredisease/reference/vep_cache_and_plugins.tar.gz"
-    vep_filters          = "https://raw.githubusercontent.com/nf-core/test-datasets/raredisease/reference/hgnc.txt"
     vep_cache_version    = 107
     vep_plugin_files     = "https://raw.githubusercontent.com/nf-core/test-datasets/raredisease/reference/vep_files.csv"
 

diff --git a/docs/output.md b/docs/output.md
@@ -212,8 +212,10 @@ to "gatk". Involves several steps: [`SplitN Cigar Reads`](https://gatk.broadinst
 <summary>Output files</summary>
 
 - `annotate_vep`
-  - `*ase_vep.vcf.gz`: annotated vcf
-  - `*ase_vep.vcf.gz.tbi`: index for annotated vcf
+  - `*vep_research.vcf.gz`: annotated vcf conatining all snvs
+  - `*vep_research.vcf.gz.tbi`: index for annotated vcf containing all snvs
+  - `*vep_clinical.vcf.gz`: annotated vcf filtered according to genes provided by gene_panel_clinical_filter
+  - `*vep_clinical.vcf.gz.tbi`: index for annotated vcf filtered according to genes provided by gene_panel_clinical_filter
 
 </details>