Issue using corncob and metagenomeseq

[bdpessem]

I encountered an issue when running qadabra, it has to do with two of the tools that are used by qadabra; corncob and metagenomeseq. I checked both log files (see underneath), but it was not clear to me if the error has to do with my input files and how to resolve?
Tue Feb 13 15:11:19 2024]
Error in rule metagenomeseq:
jobid: 15
input: /Users/bpessemier/my_qadabra3/data/metaphlan4species_readstats_transposed.biom, /Users/bpessemier/my_qadabra3/data/metadata_Psoriasis.tsv
output: results/skin_microbiome/tools/metagenomeseq/differentials.tsv, results/skin_microbiome/tools/metagenomeseq/results.rds
log: log/skin_microbiome/metagenomeseq.log (check log file(s) for error details)
conda-env: /Users/bpessemier/my_qadabra3/.snakemake/conda/656a891cb8f6dd51f64ad50a1c6b0f26_
RuleException:
CalledProcessError in file /Users/bpessemier/anaconda3/envs/qadabra_env/lib/python3.9/site-packages/qadabra/workflow/rules/diffab.smk, line 131:
Command 'source /Users/bpessemier/anaconda3/envs/qadabra_env/bin/activate '/Users/bpessemier/my_qadabra3/.snakemake/conda/656a891cb8f6dd51f64ad50a1c6b0f26_'; set -euo pipefail; Rscript --vanilla /Users/bpessemier/my_qadabra3/.snakemake/scripts/tmpyrssflqd.metagenomeseq.R' returned non-zero exit status 1.
File "/Users/bpessemier/anaconda3/envs/qadabra_env/lib/python3.9/site-packages/qadabra/workflow/rules/diffab.smk", line 131, in rule_metagenomeseq
File "/Users/bpessemier/anaconda3/envs/qadabra_env/lib/python3.9/concurrent/futures/thread.py", line 58, in run
[Tue Feb 13 15:11:39 2024]
Finished job 7.
25 of 163 steps (15%) done
[Tue Feb 13 15:12:05 2024]
Error in rule corncob:
jobid: 17
input: /Users/bpessemier/my_qadabra3/data/metaphlan4species_readstats_transposed.biom, /Users/bpessemier/my_qadabra3/data/metadata_Psoriasis.tsv
output: results/skin_microbiome/tools/corncob/differentials.tsv, results/skin_microbiome/tools/corncob/results.rds
log: log/skin_microbiome/corncob.log (check log file(s) for error details)
conda-env: /Users/bpessemier/my_qadabra3/.snakemake/conda/656a891cb8f6dd51f64ad50a1c6b0f26
RuleException:
CalledProcessError in file /Users/bpessemier/anaconda3/envs/qadabra_env/lib/python3.9/site-packages/qadabra/workflow/rules/diffab.smk, line 147:
Command 'source /Users/bpessemier/anaconda3/envs/qadabra_env/bin/activate '/Users/bpessemier/my_qadabra3/.snakemake/conda/656a891cb8f6dd51f64ad50a1c6b0f26'; set -euo pipefail; Rscript --vanilla /Users/bpessemier/my_qadabra3/.snakemake/scripts/tmpfdrzbei1.corncob.R' returned non-zero exit status 1.
File "/Users/bpessemier/anaconda3/envs/qadabra_env/lib/python3.9/site-packages/qadabra/workflow/rules/diffab.smk", line 147, in __rule_corncob
File "/Users/bpessemier/anaconda3/envs/qadabra_env/lib/python3.9/concurrent/futures/thread.py", line 58, in run
Removing output files of failed job corncob since they might be corrupted:
results/skin_microbiome/tools/corncob/results.rds
[Tue Feb 13 15:13:08 2024]
Finished job 9.
26 of 163 steps (16%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2024-02-13T150501.551322.snakemake.log

This was the log output of corncob;
[1] "Loading table..."
[1] "Loading metadata..."
[1] "Harmonizing table and metadata samples..."
[1] "Converting to phyloseq..."
[1] "Creating design formula..."
~Pool
[1] "Running corncob..."
[1] "Saved RDS!"
[1] "Aggregating models..."
Error: $ operator is invalid for atomic vectors
Execution halted

This were the the logoutput of metagenomeseq;
[1] "Loading table..."
[1] "Loading metadata..."
[1] "Harmonizing table and metadata samples..."
Default value being used.
[1] "Creating design formula..."
[1] "Running metagenomeSeq..."
it= 0, nll=210.47, log10(eps+1)=Inf, stillActive=738
it= 1, nll=210.34, log10(eps+1)=Inf, stillActive=253
it= 2, nll=206.62, log10(eps+1)=Inf, stillActive=242
it= 3, nll=204.02, log10(eps+1)=Inf, stillActive=194
it= 4, nll=202.55, log10(eps+1)=Inf, stillActive=142
it= 5, nll=202.81, log10(eps+1)=Inf, stillActive=74
it= 6, nll=203.25, log10(eps+1)=Inf, stillActive=49
it= 7, nll=203.77, log10(eps+1)=Inf, stillActive=38
it= 8, nll=203.71, log10(eps+1)=Inf, stillActive=38
it= 9, nll=203.66, log10(eps+1)=Inf, stillActive=38
Error in if (max(df.residual) == 0) stop("No residual degrees of freedom in linear model fits") :
missing value where TRUE/FALSE needed
Calls: -> .do_fitZig -> -> .ebayes
Execution halted

[gibsramen]

Thanks @bdpessem

Can you share a little bit about your data? i.e. number of samples and class distribution?

cc @yangchen2

[bdpessem]

Yes,
I have 64 samples coming from diseased pp and 83 samples coming from healthy pp.

[mossheadd]

I am also having the same issue with metagenomeseq, a similar problem with corncob (), and an extra problem with songbird (the songbird log is empty in my case).

This is my corncob log:
[1] "Loading table..."
[1] "Loading metadata..."
[1] "Harmonizing table and metadata samples..."
[1] "Converting to phyloseq..."
[1] "Creating design formula..."
~group
[1] "Running corncob..."
Error in corncob::differentialTest(formula = design.formula, formula_null = ~1, :
All models failed to converge!

       If you are seeing this, it is likely that your model is overspecified. This occurs when your sample size is not large enough to estimate all the parameters of your model. This is most commonly due to categorical variables that include many categories. 

       Alternatively, double-check your values for the arguments `link`, `phi.link`, and `method` to makes sure that they follow the specified options. 

       To confirm you have fixed the issue, try running a model for a single taxon with bbdml.

Execution halted

[yangchen2]

Hi @mossheadd, can you share some details about your biom table size and metadata file?

[mossheadd]

HI @yangchen2 my biom file has viral orthologs, 27474 rows and 24 columns that are sample IDs. My metadata file contains sample IDs and medium type, sediment and water.

[yangchen2]

@mossheadd Can you also share your add dataset command?

[mossheadd]

@yangchen2 below is my add dataset command
qadabra add-dataset \
--workflow-dest ~/Virus_medium_test_4/test_4
--table ~/Virus_medium_test_4/test_4/data/virus_ortho_medium_2.biom
--metadata ~/Virus_medium_test_4/test_4/data/Meta_ortho_medium_2.tsv
--name medium_test_4
--factor-name Medium
--target-level Sediment
--reference-level Water
--verbose