Error: Missing process or function when running the pipeline #29
Description
Hi, thanks for developing and mantaining YAMP!
I've been trying to run YAMP from a fresh installation using singularity but I keep getting this error, regardless of whether I use the base or the test profiles using the test samples
Missing process or function call([/mnt/lustre/groups/maier/maide581/bin/YAMP/conf/multiqc_config.yaml, /mnt/lustre/groups/maier/maide581/bin/YAMP/work/workflow_summary_mqc.yaml, DataflowVariable(value=null), DataflowVariable(value=null), DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?]])
I have configured the nextflow.config file to work with singularity as specified in the wiki as follows:
singularity {
singularity.enabled = true
singularity.cacheDir = '/mnt/lustre/groups/maier/maide581/bin/YAMP/singularity/'
process.executor = 'slurm'
}
The complete nextflow log is
May-07 16:31:09.476 [main] DEBUG nextflow.cli.Launcher - $> nextflow run YAMP.nf --reads1 /mnt/lustre/groups/maier/maide581/bin/YAMP/data/test_data/random_ncbi_reads_with_duplicated_and_contaminants_R1.fastq.gz --reads2 /mnt/lustre/groups/maier/maide581/bin/YAMP/data/test_data/random_ncbi_reads_with_duplicated_and_contaminants_R1.fastq.gz --prefix TST --outdir ./output_folder --mode complete -profile test,singularity
May-07 16:31:09.623 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 23.10.1
May-07 16:31:09.638 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/mnt/lustre/home/maier/maide581/.nextflow/plugins; core-plugins: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected]
May-07 16:31:09.647 [main] INFO o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
May-07 16:31:09.648 [main] INFO o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
May-07 16:31:09.650 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
May-07 16:31:09.660 [main] INFO org.pf4j.AbstractPluginManager - No plugins
May-07 16:31:09.674 [main] DEBUG nextflow.config.ConfigBuilder - Found config local: /mnt/lustre/groups/maier/maide581/bin/YAMP/nextflow.config
May-07 16:31:09.675 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /mnt/lustre/groups/maier/maide581/bin/YAMP/nextflow.config
May-07 16:31:09.697 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `test,singularity`
May-07 16:31:10.860 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [test, test_single_end_QC, test_single_end_characterisation, test_paired_end_complete, test_paired_end_characterisation, test_paired_end_QC_nodedup, test_single_end_complete, docker, test_paired_end_QC, create, awsbatch, singularity, test_single_end_QC_nodedup, base]
May-07 16:31:10.931 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
May-07 16:31:10.949 [main] INFO nextflow.cli.CmdRun - Launching `YAMP.nf` [nice_pauling] DSL2 - revision: 95d5c279c2
May-07 16:31:10.950 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
May-07 16:31:10.950 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
May-07 16:31:10.957 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /mnt/lustre/home/maier/maide581/.nextflow/secrets/store.json
May-07 16:31:10.960 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@3f183caa] - activable => nextflow.secret.LocalSecretsProvider@3f183caa
May-07 16:31:11.002 [main] DEBUG nextflow.Session - Session UUID: 1d3230df-46db-4daa-9f68-cec2caa6f111
May-07 16:31:11.002 [main] DEBUG nextflow.Session - Run name: nice_pauling
May-07 16:31:11.002 [main] DEBUG nextflow.Session - Executor pool size: 16
May-07 16:31:11.013 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
May-07 16:31:11.037 [main] DEBUG nextflow.cli.CmdRun -
Version: 23.10.1 build 5891
Created: 12-01-2024 22:01 UTC (23:01 CEST)
System: Linux 5.14.0-362.24.1.el9_3.0.1.x86_64
Runtime: Groovy 3.0.19 on OpenJDK 64-Bit Server VM 21-internal-adhoc.conda.src
Encoding: UTF-8 (UTF-8)
Process: 284017@login1 [127.0.0.1]
CPUs: 16 - Mem: 503.2 GB (482.2 GB) - Swap: 0 (0) - Virtual threads ON
May-07 16:31:11.074 [main] DEBUG nextflow.Session - Work-dir: /mnt/lustre/groups/maier/maide581/bin/YAMP/work [lustre]
May-07 16:31:11.094 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
May-07 16:31:11.108 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
May-07 16:31:11.152 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
May-07 16:31:11.162 [main] DEBUG nextflow.util.CustomPoolFactory - Creating virtual thread pool
May-07 16:31:11.344 [main] DEBUG nextflow.Session - Session start
May-07 16:31:11.831 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
May-07 16:31:12.550 [main] INFO nextflow.Nextflow - ---------------------------------------------
YET ANOTHER METAGENOMIC PIPELINE (YAMP)
---------------------------------------------
Analysis introspection:
May-07 16:31:12.555 [main] INFO nextflow.Nextflow - Starting time : Tue May 07 16:31:12 CEST 2024
Environment :
Pipeline Name : YAMP
Pipeline Version : 0.9.6.0
Config Profile : test,singularity
Resumed : false
Nextflow version : 23.10.1 build 5891 (12-01-2024 22:01 UTC)
Java version : 21-internal
Java Virtual Machine : OpenJDK 64-Bit Server VM(21-internal-adhoc.conda.src)
Operating system : Linux amd64 v5.14.0-362.24.1.el9_3.0.1.x86_64
User name : maide581
Container Engine : singularity
Container : [:]
BBmap : https://depot.galaxyproject.org/singularity/bbmap:38.87--h1296035_0
FastQC : https://depot.galaxyproject.org/singularity/fastqc:0.11.9--0
biobakery : biobakery/workflows:3.0.0.a.6.metaphlanv3.0.7
qiime : qiime2/core:2020.8
MultiQC : https://depot.galaxyproject.org/singularity/multiqc:1.9--py_1
Running parameters :
Reads : [/mnt/lustre/groups/maier/maide581/bin/YAMP/data/test_data/random_ncbi_reads_with_duplicated_and_contaminants_R1.fastq.gz, /mnt/lustre/groups/maier/maide581/bin/YAMP/data/test_data/random_ncbi_reads_with_duplicated_and_contaminants_R1.fastq.gz]
Prefix : TST
Running mode : complete
Layout : Paired-End
Performing de-duplication : true
Synthetic contaminants :
Artefacts : /mnt/lustre/groups/maier/maide581/bin/YAMP/assets/data/sequencing_artifacts.fa.gz
Phix174ill : /mnt/lustre/groups/maier/maide581/bin/YAMP/assets/data/phix174_ill.ref.fa.gz
Adapters : /mnt/lustre/groups/maier/maide581/bin/YAMP/assets/data/adapters.fa
Trimming parameters :
Input quality offset : ASCII+33
Min phred score : 10
Min length : 60
kmer lenght : 23
Shorter kmer : 11
Max Hamming distance : 1
Decontamination parameters :
Contaminant (pan)genome : /mnt/lustre/groups/maier/maide581/bin/YAMP/assets/demo/genome.fa
Min alignment identity : 0.95
Max indel length : 3
Max alignment band : 0.16
MetaPhlAn parameters :
MetaPhlAn database : /mnt/lustre/groups/maier/maide581/bin/YAMP/assets/data/metaphlan_databases/
Bowtie2 options : very-sensitive
HUMAnN parameters :
Chocophlan database : /mnt/lustre/groups/maier/maide581/bin/YAMP/assets/demo/chocophlan
Uniref database : /mnt/lustre/groups/maier/maide581/bin/YAMP/assets/demo/uniref
Folders :
Output dir : ./output_folder
Working dir : /mnt/lustre/groups/maier/maide581/bin/YAMP/work
Script dir : /mnt/lustre/groups/maier/maide581/bin/YAMP
Lunching dir : /mnt/lustre/groups/maier/maide581/bin/YAMP
May-07 16:31:12.557 [main] INFO nextflow.Nextflow -
May-07 16:31:12.687 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:get_software_versions` matches process get_software_versions
May-07 16:31:12.690 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
May-07 16:31:12.690 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
May-07 16:31:12.695 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
May-07 16:31:12.701 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=16; memory=503.2 GB; capacity=16; pollInterval=100ms; dumpInterval=5m
May-07 16:31:12.703 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)
May-07 16:31:12.774 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:dedup` matches process dedup
May-07 16:31:12.777 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
May-07 16:31:12.777 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
May-07 16:31:12.801 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:remove_synthetic_contaminants` matches process remove_synthetic_contaminants
May-07 16:31:12.812 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
May-07 16:31:12.812 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
May-07 16:31:12.817 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:trim` matches process trim
May-07 16:31:12.818 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
May-07 16:31:12.818 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
May-07 16:31:12.822 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:index_foreign_genome` matches process index_foreign_genome
May-07 16:31:12.823 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
May-07 16:31:12.823 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
May-07 16:31:12.830 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:decontaminate` matches process decontaminate
May-07 16:31:12.831 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
May-07 16:31:12.831 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
May-07 16:31:12.836 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:quality_assessment` matches process quality_assessment
May-07 16:31:12.837 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
May-07 16:31:12.837 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
May-07 16:31:12.841 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:profile_taxa` matches process profile_taxa
May-07 16:31:12.842 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
May-07 16:31:12.842 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
May-07 16:31:12.847 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:profile_function` matches process profile_function
May-07 16:31:12.847 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
May-07 16:31:12.847 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
May-07 16:31:12.851 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:alpha_diversity` matches process alpha_diversity
May-07 16:31:12.852 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
May-07 16:31:12.852 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
May-07 16:31:12.879 [main] DEBUG nextflow.script.ScriptRunner - Parsed script files:
Script_dcbca6b588780e9b: /mnt/lustre/groups/maier/maide581/bin/YAMP/./modules/house_keeping.nf
Script_15717f98cab785a5: /mnt/lustre/groups/maier/maide581/bin/YAMP/./modules/quality_control.nf
Script_ee9478e9ccceff0e: /mnt/lustre/groups/maier/maide581/bin/YAMP/YAMP.nf
Script_0bf69ef226688eaa: /mnt/lustre/groups/maier/maide581/bin/YAMP/./modules/community_characterisation.nf
May-07 16:31:12.880 [main] DEBUG nextflow.Session - Session aborted -- Cause: Missing process or function call([/mnt/lustre/groups/maier/maide581/bin/YAMP/conf/multiqc_config.yaml, /mnt/lustre/groups/maier/maide581/bin/YAMP/work/workflow_summary_mqc.yaml, DataflowVariable(value=null), DataflowVariable(value=null), DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?]])
May-07 16:31:12.893 [main] DEBUG nextflow.Session - The following nodes are still active:
[operator] combine
[operator] combine
[operator] combine
[operator] combine
[operator] mix
[operator] collect
[operator] ifEmpty
[operator] ifEmpty
[operator] ifEmpty
[operator] ifEmpty
[operator] ifEmpty
[operator] ifEmpty
[operator] ifEmpty
[operator] ifEmpty
[operator] ifEmpty
[operator] ifEmpty
May-07 16:31:12.898 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
May-07 16:31:12.903 [main] ERROR nextflow.cli.Launcher - @unknown
nextflow.exception.MissingProcessException: Missing process or function call([/mnt/lustre/groups/maier/maide581/bin/YAMP/conf/multiqc_config.yaml, /mnt/lustre/groups/maier/maide581/bin/YAMP/work/workflow_summary_mqc.yaml, DataflowVariable(value=null), DataflowVariable(value=null), DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?], DataflowBroadcast around DataflowStream[?]])
at nextflow.script.WorkflowDef.run(WorkflowDef.groovy:191)
at nextflow.script.BindableDef.invoke_a(BindableDef.groovy:51)
at nextflow.script.IterableDef$invoke_a.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
at nextflow.script.BaseScript.run0(BaseScript.groovy:183)
at nextflow.script.BaseScript.run(BaseScript.groovy:192)
at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:236)
at nextflow.script.ScriptRunner.run(ScriptRunner.groovy:242)
at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:137)
at nextflow.cli.CmdRun.run(CmdRun.groovy:372)
at nextflow.cli.Launcher.run(Launcher.groovy:500)
at nextflow.cli.Launcher.main(Launcher.groovy:672)
Caused by: groovy.lang.MissingMethodException: No signature of method: ch.qos.logback.classic.Logger.call() is applicable for argument types: (sun.nio.fs.UnixPath, sun.nio.fs.UnixPath, groovyx.gpars.dataflow.DataflowVariable...) values: [/mnt/lustre/groups/maier/maide581/bin/YAMP/conf/multiqc_config.yaml, ...]
Possible solutions: wait(), any()
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:70)
at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:46)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at Script_ee9478e9ccceff0e$_runScript_closure3$_closure4.doCall(Script_ee9478e9ccceff0e:516)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1030)
at groovy.lang.Closure.call(Closure.java:427)
at groovy.lang.Closure.call(Closure.java:406)
at nextflow.script.WorkflowDef.run0(WorkflowDef.groovy:204)
at nextflow.script.WorkflowDef.run(WorkflowDef.groovy:188)
... 13 common frames omitted