Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

de_analysis.R: Creating counts dataframe changes counts variable from matrix to 'named' integer #95

Open
damioresegun opened this issue Jun 21, 2024 · 8 comments

Comments

@damioresegun
Copy link

damioresegun commented Jun 21, 2024

Operating System

Other Linux (please specify below)

Other Linux

Red Hat Enterprise Linux release 8.6

Workflow Version

v1.1.1-g999fb4e

Workflow Execution

Command line (Cluster)

Other workflow execution

No response

EPI2ME Version

No response

CLI command run

nextflow run epi2me-labs/wf-transcriptomes --fastq fastqs_dir --sample_sheet sample_sheet.csv --de_analysis --ref_genome ref_genomic.fna --transcriptome_source precomputed --ref_transcriptome ref_rna.fna --ref_annotation ref.gtf --minimap2_index_opts '-k 15' --threads 32 --cdna_kit SQK-PCS111 --pychopper_opts '-U -y' -profile singularity

Workflow Execution - CLI Execution Profile

singularity

What happened?

Observed:

The workflow started with no issues and proceeded to the de_analysis.R step. However, this results in an error.
Error in data.frame(gene_id = txdf$GENEID, feature_id = txdf$TXNAME, cts) : arguments imply differing number of rows: 0, 42347 Execution halted -- see logfile and below this for full error message.

Expected/Solution:

After trying a bunch of things, I decided to 'brute-force' it and look through the de_analysis.R script and manually run the commands myself to figure out what's happened.

Basically in line 70 this command: cts <- cts[rownames(cts) %in% txdf$TXNAME, ] broke the workflow for me.
This line caused a change in the cts from being a matrix to being just a named integer list. That resulted in the subsequent commands:

    txdf <- txdf[match(rownames(cts), txdf$TXNAME), ]
    rownames(txdf) <- NULL
    
    # Create counts data frame:
    counts<-data.frame(gene_id=txdf$GENEID, feature_id=txdf$TXNAME, cts)

to be unsuccessful.
My understanding is that when subsetting a matrix like line 70 is attempting, sometimes the result is coerced to a vector if the subset operation results in a single row or column. So I fixed this by adding the drop=FALSE to the command.
I checked it manually and this retained cts as a matrix.

So I propose a change to line 70 that looks like this: cts <- cts[rownames(cts) %in% txdf$TXNAME, ,drop=FALSE ]

However at this point, I'm unsure how to proceed as the hot-fix I figured out manually can't be applied to the workflow as I currently run it. So I would appreciate any pointers on how to change the script locally and run it with the workflow (I'm assuming that you may be too busy to push out an update).

Observed Error:

	Workflow execution completed unsuccessfully!
      The exit status of the task that caused the workflow execution to fail was: 1.
      
      The full error message was:
      
      Error executing process > 'pipeline:differential_expression:deAnalysis'
      
      Caused by:
        Process `pipeline:differential_expression:deAnalysis` terminated with an error exit status (1)
      
      Command executed:
      
        mkdir merged
        mkdir de_analysis
        de_analysis.R annotation.gtf 3 1 10 3
      
      Command exit status:
        1
      
      Command output:
        Loading counts, conditions and parameters.
        Checking annotation file type.
        Annotation file type is gtf.
        Checking annotation file for presence of transcript_id versions.
        Annotation file transcript_ids include versions.
        Loading annotation database.
      
      Command error:
        Loading counts, conditions and parameters.
        Checking annotation file type.
        Annotation file type is gtf.
        Checking annotation file for presence of transcript_id versions.
        Annotation file transcript_ids include versions.
        Loading annotation database.
        Import genomic features from the file as a GRanges object ... OK
        Prepare the 'metadata' data frame ... OK
        Make the TxDb object ... OK
        Warning messages:
        1: In for (i in seq_along(specs)) { :
          closing unused connection 3 (annotation.gtf)
        2: In for (i in seq_along(defined)) { :
          closing unused connection 4 (annotation.gtf)
        3: In .get_cds_IDX(mcols0$type, mcols0$phase) :
          The "phase" metadata column contains non-NA values for features of type
          stop_codon. This information was ignored.
        4: In .find_exon_cds(exons, cds) :
          The following transcripts have exons that contain more than one CDS
          (only the first CDS was kept for each exon): NM_001113112.2,
          NM_001300762.1, NM_001301325.1, NM_203549.2
        'select()' returned 1:many mapping between keys and columns
        Error in data.frame(gene_id = txdf$GENEID, feature_id = txdf$TXNAME, cts) : 
          arguments imply differing number of rows: 0, 42347
        Execution halted
      
      Work dir:
        /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/0b/6227a68f2b4edb98144b9dd06e4ffa

Relevant log output

Jun-20 18:22:00.361 [main] DEBUG nextflow.cli.Launcher - $> nextflow run epi2me-labs/wf-transcriptomes --fastq /uoa/home/s07do3/sharedscratch/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/fastqs --sample_sheet /uoa/home/s07do3/sharedscratch/CGEBMP424A/wf-transcriptome_approach/testing/sample_sheet.csv --de_analysis --ref_genome /uoa/home/s07do3/sharedscratch/CGEBMP424A/Reference/GCF_000004195.4_UCB_Xtro_10.0_genomic.fna --transcriptome_source precomputed --ref_transcriptome /uoa/home/s07do3/sharedscratch/CGEBMP424A/Reference/GCF_000004195.4_UCB_Xtro_10.0_rna.fna --ref_annotation /uoa/home/s07do3/sharedscratch/CGEBMP424A/Reference/GCF_000004195.4_UCB_Xtro_10.0_genomic.gtf --minimap2_index_opts '-k 15' --threads 32 --cdna_kit SQK-PCS111 --pychopper_opts '-U -y' -profile singularity
Jun-20 18:22:00.742 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 23.10.1
Jun-20 18:22:00.777 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/uoa/home/s07do3/.nextflow/plugins; core-plugins: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected]
Jun-20 18:22:00.811 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Jun-20 18:22:00.814 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Jun-20 18:22:00.822 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Jun-20 18:22:00.872 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Jun-20 18:22:00.915 [main] DEBUG nextflow.scm.ProviderConfig - Using SCM config path: /uoa/home/s07do3/.nextflow/scm
Jun-20 18:22:02.116 [main] DEBUG nextflow.scm.AssetManager - Git config: /uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/.git/config; branch: master; remote: origin; url: https://github.com/epi2me-labs/wf-transcriptomes.git
Jun-20 18:22:02.155 [main] DEBUG nextflow.scm.RepositoryFactory - Found Git repository result: [RepositoryFactory]
Jun-20 18:22:02.168 [main] DEBUG nextflow.scm.AssetManager - Git config: /uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/.git/config; branch: master; remote: origin; url: https://github.com/epi2me-labs/wf-transcriptomes.git
Jun-20 18:22:06.085 [main] INFO  nextflow.scm.AssetManager - NOTE: Your local project version looks outdated - a different revision is available in the remote repository [9353e23cc9]
Jun-20 18:22:06.098 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/nextflow.config
Jun-20 18:22:06.100 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/nextflow.config
Jun-20 18:22:06.123 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `singularity`
Jun-20 18:22:06.271 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [standard, conda, singularity, awsbatch, local]
Jun-20 18:22:06.427 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
Jun-20 18:22:06.430 [main] INFO  nextflow.cli.CmdRun - Launching `https://github.com/epi2me-labs/wf-transcriptomes` [awesome_keller] DSL2 - revision: 999fb4e44a [master]
Jun-20 18:22:06.433 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Jun-20 18:22:06.436 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
Jun-20 18:22:06.443 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /uoa/home/s07do3/.nextflow/secrets/store.json
Jun-20 18:22:06.449 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@29d070c7] - activable => nextflow.secret.LocalSecretsProvider@29d070c7
Jun-20 18:22:06.577 [main] DEBUG nextflow.Session - Session UUID: d4f2a079-5f49-4cce-bbb1-b1a510d06231
Jun-20 18:22:06.578 [main] DEBUG nextflow.Session - Run name: awesome_keller
Jun-20 18:22:06.581 [main] DEBUG nextflow.Session - Executor pool size: 80
Jun-20 18:22:06.600 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
Jun-20 18:22:06.616 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=240; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Jun-20 18:22:06.947 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 23.10.1 build 5891
  Created: 12-01-2024 22:01 UTC (22:01 BST)
  System: Linux 4.18.0-372.9.1.el8.x86_64
  Runtime: Groovy 3.0.19 on OpenJDK 64-Bit Server VM 11.0.15+10
  Encoding: UTF-8 (UTF-8)
  Process: [email protected] [172.22.1.1]
  CPUs: 80 - Mem: 188.2 GB (117.9 GB) - Swap: 0 (0)
Jun-20 18:22:07.012 [main] DEBUG nextflow.Session - Work-dir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work [fhgfs]
Jun-20 18:22:07.077 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Jun-20 18:22:07.102 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Jun-20 18:22:07.185 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Jun-20 18:22:07.245 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 81; maxThreads: 1000
Jun-20 18:22:07.475 [main] DEBUG nextflow.Session - Session start
Jun-20 18:22:07.510 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/output/execution/trace.txt
Jun-20 18:22:07.541 [main] DEBUG nextflow.Session - Using default localLib path: /uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/lib
Jun-20 18:22:07.549 [main] DEBUG nextflow.Session - Adding to the classpath library: /uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/lib
Jun-20 18:22:07.550 [main] DEBUG nextflow.Session - Adding to the classpath library: /uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/lib/nfcore_external_java_deps.jar
Jun-20 18:22:10.481 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Jun-20 18:22:11.573 [main] INFO  nextflow.Nextflow - 
�[0;92m||||||||||   �[0m�[2m_____ ____ ___ ____  __  __ _____      _       _
�[0;92m||||||||||  �[0m�[2m| ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
�[0;33m|||||       �[0m�[2m|  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
�[0;33m|||||       �[0m�[2m| |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
�[0;94m||||||||||  �[0m�[2m|_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
�[0;94m||||||||||  �[0m�[1mwf-transcriptomes v1.1.1-g999fb4e�[0m
�[2m--------------------------------------------------------------------------------�[0m
�[1mCore Nextflow options�[0m
  �[0;34mrevision            : �[0;32mmaster�[0m
  �[0;34mrunName             : �[0;32mawesome_keller�[0m
  �[0;34mcontainerEngine     : �[0;32msingularity�[0m
  �[0;34mcontainer           : �[0;32m[withLabel:isoforms:ontresearch/wf-transcriptomes:shae7c9f184996a384e99be68e790f0612f0c732867, withLabel:wf_common:ontresearch/wf-common:sha1c5febff9f75143710826498b093d9769a5edbb9]�[0m
  �[0;34mlaunchDir           : �[0;32m/uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign�[0m
  �[0;34mworkDir             : �[0;32m/uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work�[0m
  �[0;34mprojectDir          : �[0;32m/uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes�[0m
  �[0;34muserName            : �[0;32ms07do3�[0m
  �[0;34mprofile             : �[0;32msingularity�[0m
  �[0;34mconfigFiles         : �[0;32m/uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/nextflow.config�[0m

�[1mInput Options�[0m
  �[0;34mfastq               : �[0;32m/uoa/home/s07do3/sharedscratch/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/fastqs�[0m
  �[0;34mtranscriptome_source: �[0;32mprecomputed�[0m
  �[0;34mref_genome          : �[0;32m/uoa/home/s07do3/sharedscratch/CGEBMP424A/Reference/GCF_000004195.4_UCB_Xtro_10.0_genomic.fna�[0m
  �[0;34mref_transcriptome   : �[0;32m/uoa/home/s07do3/sharedscratch/CGEBMP424A/Reference/GCF_000004195.4_UCB_Xtro_10.0_rna.fna�[0m
  �[0;34mref_annotation      : �[0;32m/uoa/home/s07do3/sharedscratch/CGEBMP424A/Reference/GCF_000004195.4_UCB_Xtro_10.0_genomic.gtf�[0m

�[1mSample Options�[0m
  �[0;34msample_sheet        : �[0;32m/uoa/home/s07do3/sharedscratch/CGEBMP424A/wf-transcriptome_approach/testing/sample_sheet.csv�[0m

�[1mOptions for reference-based workflow�[0m
  �[0;34mminimap2_index_opts : �[0;32m-k 15�[0m

�[1mDifferential Expression Options�[0m
  �[0;34mde_analysis         : �[0;32mtrue�[0m

�[1mAdvanced Options�[0m
  �[0;34mthreads             : �[0;32m32�[0m
  �[0;34mcdna_kit            : �[0;32mSQK-PCS111�[0m
  �[0;34mpychopper_opts      : �[0;32m-U -y�[0m

!! Only displaying parameters that differ from the pipeline defaults !!
�[2m--------------------------------------------------------------------------------�[0m
If you use epi2me-labs/wf-transcriptomes for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x


�[2m--------------------------------------------------------------------------------�[0m
This is epi2me-labs/wf-transcriptomes v1.1.1-g999fb4e.
�[2m--------------------------------------------------------------------------------�[0m
Jun-20 18:22:12.111 [main] INFO  nextflow.Nextflow - Reference Transcriptome provided will be used for differential expression.
Jun-20 18:22:12.122 [main] INFO  nextflow.Nextflow - Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files.
Jun-20 18:22:12.457 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_common` matches labels `ingress,wf_common` for process with name fastcat
Jun-20 18:22:12.463 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.465 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.475 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Jun-20 18:22:12.488 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=80; memory=188.2 GB; capacity=80; pollInterval=100ms; dumpInterval=5m
Jun-20 18:22:12.492 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)
Jun-20 18:22:12.628 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:preprocess_ref_annotation
Jun-20 18:22:12.630 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.630 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.640 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:collectFastqIngressResultsInDir
Jun-20 18:22:12.641 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.643 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.647 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:getVersions
Jun-20 18:22:12.648 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.649 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.653 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:getParams
Jun-20 18:22:12.655 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.656 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.692 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:preprocess_reads
Jun-20 18:22:12.695 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.696 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.734 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:preprocess_ref_transcriptome
Jun-20 18:22:12.736 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.737 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.741 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:differential_expression:checkSampleSheetCondition
Jun-20 18:22:12.743 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.746 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.776 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:differential_expression:build_minimap_index_transcriptome
Jun-20 18:22:12.778 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.779 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.797 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:differential_expression:map_transcriptome
Jun-20 18:22:12.798 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.800 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.809 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:differential_expression:count_transcripts
Jun-20 18:22:12.811 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.815 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.828 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:differential_expression:mergeCounts
Jun-20 18:22:12.831 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.832 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.837 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:differential_expression:mergeTPM
Jun-20 18:22:12.839 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.841 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.846 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:differential_expression:deAnalysis
Jun-20 18:22:12.848 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.850 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.901 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:differential_expression:plotResults
Jun-20 18:22:12.904 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.907 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.931 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:makeReport
Jun-20 18:22:12.933 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.934 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.963 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name output
Jun-20 18:22:12.964 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-20 18:22:12.966 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-20 18:22:12.969 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: preprocess_ref_transcriptome, checkSampleSheetCondition, move_or_compress_fq_file, mergeTPM, pipeline:getParams, deAnalysis, checkBamHeaders, mergeBams, output, collectFastqIngressResultsInDir, makeReport, getVersions, assemble_transcripts, get_transcriptome, merge_transcriptomes, preprocess_reads, bamstats, pipeline:differential_expression:count_transcripts, pipeline:differential_expression:deAnalysis, fastcat, plotResults, pipeline:differential_expression:mergeCounts, map_reads, getParams, pipeline:preprocess_ref_transcriptome, preprocess_ref_annotation, sortBam, merge_gff_bundles, validate_sample_sheet, pipeline:differential_expression:plotResults, jaffal, pipeline:makeReport, pipeline:differential_expression:checkSampleSheetCondition, decompress_transcriptome, map_transcriptome, build_minimap_index, pipeline:differential_expression:mergeTPM, run_gffcompare, pipeline:collectFastqIngressResultsInDir, count_transcripts, pipeline:getVersions, pipeline:preprocess_reads, pipeline:differential_expression:build_minimap_index_transcriptome, mergeCounts, pipeline:differential_expression:map_transcriptome, decompress_ref, split_bam, build_minimap_index_transcriptome, decompress_annotation, pipeline:preprocess_ref_annotation, catSortBams
Jun-20 18:22:12.972 [main] DEBUG nextflow.Session - Igniting dataflow network (21)
Jun-20 18:22:12.975 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > fastcat
Jun-20 18:22:13.016 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:preprocess_ref_annotation
Jun-20 18:22:13.017 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:collectFastqIngressResultsInDir
Jun-20 18:22:13.020 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:getVersions
Jun-20 18:22:13.022 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:getParams
Jun-20 18:22:13.024 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:preprocess_reads
Jun-20 18:22:13.029 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:preprocess_ref_transcriptome
Jun-20 18:22:13.032 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:differential_expression:checkSampleSheetCondition
Jun-20 18:22:13.034 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:differential_expression:build_minimap_index_transcriptome
Jun-20 18:22:13.043 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:differential_expression:map_transcriptome
Jun-20 18:22:13.045 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:differential_expression:count_transcripts
Jun-20 18:22:13.050 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:differential_expression:mergeCounts
Jun-20 18:22:13.054 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:differential_expression:mergeTPM
Jun-20 18:22:13.055 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:differential_expression:deAnalysis
Jun-20 18:22:13.056 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:differential_expression:plotResults
Jun-20 18:22:13.059 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:makeReport
Jun-20 18:22:13.059 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > output
Jun-20 18:22:13.063 [main] DEBUG nextflow.script.ScriptRunner - Parsed script files:
  Script_6c9ad54f8f06decd: /uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/./subworkflows/reference_assembly.nf
  Script_52f11d1ddf9ca2a2: /uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/./lib/ingress.nf
  Script_eb93bcc3448c341a: /uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/main.nf
  Script_05eaa6b24b9dd568: /uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/./subworkflows/JAFFAL/gene_fusions.nf
  Script_b280fdb3f732bae1: /uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/./subworkflows/differential_expression.nf
Jun-20 18:22:13.065 [main] DEBUG nextflow.script.ScriptRunner - > Awaiting termination 
Jun-20 18:22:13.068 [main] DEBUG nextflow.Session - Session await
Jun-20 18:22:13.568 [Actor Thread 80] WARN  nextflow.container.SingularityCache - Singularity cache directory has not been defined -- Remote image will be stored in the path: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/singularity -- Use the environment variable NXF_SINGULARITY_CACHEDIR to specify a different location
Jun-20 18:22:13.575 [Actor Thread 80] INFO  nextflow.container.SingularityCache - Pulling Singularity image docker://ontresearch/wf-transcriptomes:shae7c9f184996a384e99be68e790f0612f0c732867 [cache /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/singularity/ontresearch-wf-transcriptomes-shae7c9f184996a384e99be68e790f0612f0c732867.img]
Jun-20 18:22:13.575 [Actor Thread 81] INFO  nextflow.container.SingularityCache - Pulling Singularity image docker://ontresearch/wf-common:sha1c5febff9f75143710826498b093d9769a5edbb9 [cache /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/singularity/ontresearch-wf-common-sha1c5febff9f75143710826498b093d9769a5edbb9.img]
Jun-20 18:22:17.416 [Actor Thread 81] DEBUG nextflow.container.SingularityCache - Singularity pull complete image=docker://ontresearch/wf-common:sha1c5febff9f75143710826498b093d9769a5edbb9 path=/uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/singularity/ontresearch-wf-common-sha1c5febff9f75143710826498b093d9769a5edbb9.img
Jun-20 18:22:17.599 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-20 18:22:17.631 [Task submitter] INFO  nextflow.Session - [4b/0ea32e] Submitted process > fastcat (1)
Jun-20 18:22:20.691 [Actor Thread 80] DEBUG nextflow.container.SingularityCache - Singularity pull complete image=docker://ontresearch/wf-transcriptomes:shae7c9f184996a384e99be68e790f0612f0c732867 path=/uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/singularity/ontresearch-wf-transcriptomes-shae7c9f184996a384e99be68e790f0612f0c732867.img
Jun-20 18:22:20.708 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-20 18:22:20.709 [Task submitter] INFO  nextflow.Session - [35/ad3f28] Submitted process > pipeline:getVersions
Jun-20 18:22:20.717 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-20 18:22:20.718 [Task submitter] INFO  nextflow.Session - [92/fd1a55] Submitted process > pipeline:getParams
Jun-20 18:22:20.719 [Actor Thread 75] DEBUG nextflow.util.CacheHelper - Hash asset file sha-256: /uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/bin/workflow-glue
Jun-20 18:22:20.726 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-20 18:22:20.727 [Task submitter] INFO  nextflow.Session - [33/5db996] Submitted process > pipeline:preprocess_ref_transcriptome (1)
Jun-20 18:22:20.734 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-20 18:22:20.734 [Task submitter] INFO  nextflow.Session - [5b/fea2bd] Submitted process > pipeline:preprocess_ref_annotation
Jun-20 18:22:20.769 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-20 18:22:20.771 [Task submitter] INFO  nextflow.Session - [26/e01c67] Submitted process > pipeline:differential_expression:checkSampleSheetCondition
Jun-20 18:22:23.800 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 2; name: pipeline:getParams; status: COMPLETED; exit: 0; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/92/fd1a552227f7b62399439713090505]
Jun-20 18:22:24.622 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 3; name: pipeline:preprocess_ref_transcriptome (1); status: COMPLETED; exit: 0; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/33/5db996db68f5b279537b4cd0f79a20]
Jun-20 18:22:24.738 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-20 18:22:24.739 [Task submitter] INFO  nextflow.Session - [00/621ad1] Submitted process > pipeline:differential_expression:build_minimap_index_transcriptome (1)
Jun-20 18:22:26.203 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: pipeline:preprocess_ref_annotation; status: COMPLETED; exit: 0; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/5b/fea2bdb0025cae43df7d62d18aa332]
Jun-20 18:22:33.546 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 7; name: pipeline:differential_expression:build_minimap_index_transcriptome (1); status: COMPLETED; exit: 0; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/00/621ad1cf7830d6487193c66466c375]
Jun-20 18:23:00.593 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 6; name: pipeline:differential_expression:checkSampleSheetCondition; status: COMPLETED; exit: 0; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/26/e01c67aba55626d6450cdb96b146d0]
Jun-20 18:23:00.894 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 5; name: pipeline:getVersions; status: COMPLETED; exit: 0; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/35/ad3f28b7b3ac40ed908de878e878dc]
Jun-20 18:27:12.669 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 4; name: fastcat (1); status: RUNNING; exit: -; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/4b/0ea32eae3461e0eb73ceecbfb80838]
Jun-20 18:32:12.733 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 4; name: fastcat (1); status: RUNNING; exit: -; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/4b/0ea32eae3461e0eb73ceecbfb80838]
Jun-20 18:37:10.365 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 4; name: fastcat (1); status: COMPLETED; exit: 0; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/4b/0ea32eae3461e0eb73ceecbfb80838]
Jun-20 18:37:10.495 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-20 18:37:10.496 [Task submitter] INFO  nextflow.Session - [f2/36c195] Submitted process > pipeline:differential_expression:map_transcriptome (1)
Jun-20 18:37:10.524 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-20 18:37:10.526 [Task submitter] INFO  nextflow.Session - [09/31d701] Submitted process > pipeline:collectFastqIngressResultsInDir (1)
Jun-20 18:37:10.536 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-20 18:37:10.538 [Task submitter] INFO  nextflow.Session - [eb/9878ed] Submitted process > pipeline:preprocess_reads (1)
Jun-20 18:37:12.768 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 3 -- submitted tasks are shown below
~> TaskHandler[id: 10; name: pipeline:differential_expression:map_transcriptome (1); status: RUNNING; exit: -; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/f2/36c19574561e0bb6ffeaf8bfe09d65]
~> TaskHandler[id: 9; name: pipeline:collectFastqIngressResultsInDir (1); status: RUNNING; exit: -; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/09/31d70164b27ccd5172c8553b8a7105]
~> TaskHandler[id: 8; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/eb/9878ed77221ebb907aa564df28282f]
Jun-20 18:37:14.240 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 9; name: pipeline:collectFastqIngressResultsInDir (1); status: COMPLETED; exit: 0; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/09/31d70164b27ccd5172c8553b8a7105]
Jun-20 18:42:12.802 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 2 -- submitted tasks are shown below
~> TaskHandler[id: 10; name: pipeline:differential_expression:map_transcriptome (1); status: RUNNING; exit: -; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/f2/36c19574561e0bb6ffeaf8bfe09d65]
~> TaskHandler[id: 8; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/eb/9878ed77221ebb907aa564df28282f]
Jun-20 18:47:12.811 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 2 -- submitted tasks are shown below
~> TaskHandler[id: 10; name: pipeline:differential_expression:map_transcriptome (1); status: RUNNING; exit: -; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/f2/36c19574561e0bb6ffeaf8bfe09d65]
~> TaskHandler[id: 8; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/eb/9878ed77221ebb907aa564df28282f]
Jun-20 18:52:12.810 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 2 -- submitted tasks are shown below
~> TaskHandler[id: 10; name: pipeline:differential_expression:map_transcriptome (1); status: RUNNING; exit: -; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/f2/36c19574561e0bb6ffeaf8bfe09d65]
~> TaskHandler[id: 8; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/eb/9878ed77221ebb907aa564df28282f]
Jun-20 18:52:25.316 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 10; name: pipeline:differential_expression:map_transcriptome (1); status: COMPLETED; exit: 0; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/f2/36c19574561e0bb6ffeaf8bfe09d65]
Jun-20 18:52:25.419 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-20 18:52:25.421 [Task submitter] INFO  nextflow.Session - [29/485319] Submitted process > pipeline:differential_expression:count_transcripts (1)
Jun-20 18:57:12.879 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 2 -- submitted tasks are shown below
~> TaskHandler[id: 8; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/eb/9878ed77221ebb907aa564df28282f]
~> TaskHandler[id: 11; name: pipeline:differential_expression:count_transcripts (1); status: RUNNING; exit: -; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/29/48531922a4f9904eb6f9e13fdef6db]
Jun-20 18:59:04.937 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 11; name: pipeline:differential_expression:count_transcripts (1); status: COMPLETED; exit: 0; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/29/48531922a4f9904eb6f9e13fdef6db]
Jun-20 18:59:04.982 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-20 18:59:04.984 [Task submitter] INFO  nextflow.Session - [29/fd8826] Submitted process > pipeline:differential_expression:mergeTPM
Jun-20 18:59:04.994 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-20 18:59:04.996 [Task submitter] INFO  nextflow.Session - [84/6e2cb8] Submitted process > pipeline:differential_expression:mergeCounts
Jun-20 18:59:45.541 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 13; name: pipeline:differential_expression:mergeCounts; status: COMPLETED; exit: 0; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/84/6e2cb8fd2f2b9b0597d13691ed8161]
Jun-20 18:59:45.632 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 12; name: pipeline:differential_expression:mergeTPM; status: COMPLETED; exit: 0; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/29/fd882653eb13ae3ac5c823fabaa5b0]
Jun-20 18:59:45.649 [Actor Thread 71] DEBUG nextflow.util.CacheHelper - Hash asset file sha-256: /uoa/home/s07do3/.nextflow/assets/epi2me-labs/wf-transcriptomes/bin/de_analysis.R
Jun-20 18:59:45.692 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-20 18:59:45.693 [Task submitter] INFO  nextflow.Session - [0b/6227a6] Submitted process > pipeline:differential_expression:deAnalysis
Jun-20 19:01:25.599 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 14; name: pipeline:differential_expression:deAnalysis; status: COMPLETED; exit: 1; error: -; workDir: /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/0b/6227a68f2b4edb98144b9dd06e4ffa]
Jun-20 19:01:25.624 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=pipeline:differential_expression:deAnalysis; work-dir=/uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/0b/6227a68f2b4edb98144b9dd06e4ffa
  error [nextflow.exception.ProcessFailedException]: Process `pipeline:differential_expression:deAnalysis` terminated with an error exit status (1)
Jun-20 19:01:25.682 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'pipeline:differential_expression:deAnalysis'

Caused by:
  Process `pipeline:differential_expression:deAnalysis` terminated with an error exit status (1)

Command executed:

  mkdir merged
  mkdir de_analysis
  de_analysis.R annotation.gtf 3 1 10 3

Command exit status:
  1

Command output:
  Loading counts, conditions and parameters.
  Checking annotation file type.
  Annotation file type is gtf.
  Checking annotation file for presence of transcript_id versions.
  Annotation file transcript_ids include versions.
  Loading annotation database.

Command error:
  Loading counts, conditions and parameters.
  Checking annotation file type.
  Annotation file type is gtf.
  Checking annotation file for presence of transcript_id versions.
  Annotation file transcript_ids include versions.
  Loading annotation database.
  Import genomic features from the file as a GRanges object ... OK
  Prepare the 'metadata' data frame ... OK
  Make the TxDb object ... OK
  Warning messages:
  1: In for (i in seq_along(specs)) { :
    closing unused connection 3 (annotation.gtf)
  2: In for (i in seq_along(defined)) { :
    closing unused connection 4 (annotation.gtf)
  3: In .get_cds_IDX(mcols0$type, mcols0$phase) :
    The "phase" metadata column contains non-NA values for features of type
    stop_codon. This information was ignored.
  4: In .find_exon_cds(exons, cds) :
    The following transcripts have exons that contain more than one CDS
    (only the first CDS was kept for each exon): NM_001113112.2,
    NM_001300762.1, NM_001301325.1, NM_203549.2
  'select()' returned 1:many mapping between keys and columns
  Error in data.frame(gene_id = txdf$GENEID, feature_id = txdf$TXNAME, cts) : 
    arguments imply differing number of rows: 0, 42347
  Execution halted

Work dir:
  /uoa/scratch/users/s07do3/CGEBMP424A/wf-transcriptome_approach/doradoV6_data_run1_noalign/work/0b/6227a68f2b4edb98144b9dd06e4ffa

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
Jun-20 19:01:25.688 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process `pipeline:differential_expression:deAnalysis` terminated with an error exit status (1)
Jun-20 19:01:25.724 [Task monitor] DEBUG nextflow.Session - The following nodes are still active:
[process] pipeline:makeReport
  status=ACTIVE
  port 0: (value) bound ; channel: versions
  port 1: (value) bound ; channel: params.json
  port 2: (queue) OPEN  ; channel: pychopper_report/*
  port 3: (value) bound ; channel: jaffal_csv/*
  port 4: (value) bound ; channel: per_read_stats/?.gz
  port 5: (value) bound ; channel: aln_stats/*
  port 6: (value) bound ; channel: gffcmp_dir/*
  port 7: (value) bound ; channel: gff_annotation/*
  port 8: (value) bound ; channel: de_report/*
  port 9: (value) bound ; channel: seqkit/*
  port 10: (value) bound ; channel: isoforms_table/*
  port 11: (cntrl) -     ; channel: $

[process] output
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (cntrl) -     ; channel: $

Jun-20 19:01:25.882 [main] DEBUG nextflow.Session - Session await > all processes finished
Jun-20 19:01:25.886 [main] DEBUG nextflow.Session - Session await > all barriers passed
Jun-20 19:01:25.899 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
Jun-20 19:01:25.909 [Actor Thread 76] DEBUG nextflow.file.SortFileCollector - FileCollector temp dir not removed: null
Jun-20 19:01:25.949 [main] WARN  n.processor.TaskPollingMonitor - Killing running tasks (1)
Jun-20 19:01:25.984 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=12; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=1; succeedDuration=12h 28m 13s; failedDuration=6m 39s; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=6; peakCpus=38; peakMemory=24 GB; ]
Jun-20 19:01:25.985 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file
Jun-20 19:01:26.045 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report
Jun-20 19:01:26.902 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline
Jun-20 19:01:27.072 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Jun-20 19:01:27.141 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

yes

Other demo data information

No response

@sarahjeeeze
Copy link
Contributor

Hi, thanks for this feedback - We will update it in the workflow as soon as possible in the meantime you can run the local version that you have editted the script in by running it with the main script whereever that is located for you locally eg.

nextflow run main.nf --fastq fastqs_dir --sample_sheet sample_sheet.csv --de_analysis --ref_genome ref_genomic.fna --transcriptome_source precomputed --ref_transcriptome ref_rna.fna --ref_annotation ref.gtf --minimap2_index_opts '-k 15' --threads 32 --cdna_kit SQK-PCS111 --pychopper_opts '-U -y' -profile singularity

@sarahjeeeze
Copy link
Contributor

also is your ref_annotation and transcript from a publicly available source? Just to help us recreate the error.

@damioresegun
Copy link
Author

damioresegun commented Jun 24, 2024

Hello,
Thanks for getting back to me. Yes, I've got that going today. Came across a subsequent error:

Error in dmDSdata(counts = counts, samples = coldata) : 
    all(samples$sample_id %in% colnames(counts)) is not TRUE
	  Calls: dmDSdata -> stopifnot
	  Execution halted

I'm investigating now to see what the issue with it is.

I have half a mind to manually carry out the de_analysis.R to catch any errors and make changes as needed.
I'll keep a note of the issues I come across as I go.

Yes, my annotation and transcript are the X_tropicalis reference from the ftp. I'm using the genomic.fna and rna_from_genomic respectively for reference and transcripts. I've also tried using both the gff and gtf files.

@damioresegun
Copy link
Author

Okay, I've done more testing and figured out something that's my fault.
Basically the input fastq folder contained all the barcode fastqs rather than nested barcode folders which in turn contained the barcode fastq.
This meant that downstream, the counts table only showed a sum of counts for each gene for the 9 barcode fastqs in the same folder.

So I have changed this and importantly I've run it with an unchanged de_analysis.R to determine if my original error is related to my mistake. So far so good. It seems like it is indeed related!

The workflow completed successfully!

@sarahjeeeze
Copy link
Contributor

ah great, let us know if any other questions or issues.

@NatachaClairet
Copy link

Hello,
I am having the exact same problems as you did, but I am not sure to understand fully your solution.

Okay, I've done more testing and figured out something that's my fault. Basically the input fastq folder contained all the barcode fastqs rather than nested barcode folders which in turn contained the barcode fastq. This meant that downstream, the counts table only showed a sum of counts for each gene for the 9 barcode fastqs in the same folder.

I also put all the fastq files in the same input folder and got the same errors as you. Seeing your message, I tried adding 6 subfolders in my input folder, and named them with the barcodes corresponding to my 6 conditions . Finally, in each of my subfolders I put my fastq.gz files starting with the corresponding barcode. By doing so, I get an error informing me that my barcode format is wrong:

“Invalid sample sheet: values in 'barcode' column are incorrect format”.

Did I enter the wrong barcodes?
I have set the constant parts of my fastq.gz files as barcodes “FAZ77022_pass_8f5fe01f_bd82bd75”.

@sarahjeeeze
Copy link
Contributor

Hi, the barcode column should only be in the format eg. barcode_01 , you can use alias column to rename sample_01 and the condition column to indicate the conditions see the read me for an example and instructions

barcode,alias,condition
barcode01,sample01,control
barcode02,sample02,control
barcode03,sample03,control
barcode04,sample04,treated
barcode05,sample05,treated
barcode06,sample06,treated

@NatachaClairet
Copy link

NatachaClairet commented Oct 16, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

3 participants