Error when running glasso(): The system is too ill-conditioned for this solver

[gwentea]

Dear Author:

Thank you for the great tool for ST analysis.
I was testing my own data following 01_multiple_references.ipynb but ran into error when running glasso(). The ST data was 10X Visium data subsetted to top 2000 HVGs.

Error message:

Traceback (most recent call last):
  File "/SGRNJ03/pipline_test/datascience/chenxuezhen/training/licong/sagenet/multi/script/model.py", line 35, in <module>
    glasso(adata_r1,n_jobs=5)
  File "/SGRNJ/Public/Software/conda_env/ds_scArches/lib/python3.9/site-packages/sagenet/utils.py", line 40, in glasso
    cov = GraphicalLassoCV(alphas=alphas, n_jobs=n_jobs).fit(data)
  File "/SGRNJ/Public/Software/conda_env/ds_scArches/lib/python3.9/site-packages/sklearn/covariance/_graph_lasso.py", line 986, in fit
    self.covariance_, self.precision_, self.n_iter_ = graphical_lasso(
  File "/SGRNJ/Public/Software/conda_env/ds_scArches/lib/python3.9/site-packages/sklearn/covariance/_graph_lasso.py", line 304, in graphical_lasso
    raise e
  File "/SGRNJ/Public/Software/conda_env/ds_scArches/lib/python3.9/site-packages/sklearn/covariance/_graph_lasso.py", line 278, in graphical_lasso
    raise FloatingPointError(
FloatingPointError: The system is too ill-conditioned for this solver. The system is too ill-conditioned for this solver

The problem remained unsolved even after I increased the number of iterations. I am not familiar with the algorithm involved so it would really be great if you could please give me some advice!
Thanks in advance!!

[EliHei2]

Dear Author:

Thank you for the great tool for ST analysis. I was testing my own data following 01_multiple_references.ipynb but ran into error when running glasso(). The ST data was 10X Visium data subsetted to top 2000 HVGs.

Error message:

Traceback (most recent call last):
  File "/SGRNJ03/pipline_test/datascience/chenxuezhen/training/licong/sagenet/multi/script/model.py", line 35, in <module>
    glasso(adata_r1,n_jobs=5)
  File "/SGRNJ/Public/Software/conda_env/ds_scArches/lib/python3.9/site-packages/sagenet/utils.py", line 40, in glasso
    cov = GraphicalLassoCV(alphas=alphas, n_jobs=n_jobs).fit(data)
  File "/SGRNJ/Public/Software/conda_env/ds_scArches/lib/python3.9/site-packages/sklearn/covariance/_graph_lasso.py", line 986, in fit
    self.covariance_, self.precision_, self.n_iter_ = graphical_lasso(
  File "/SGRNJ/Public/Software/conda_env/ds_scArches/lib/python3.9/site-packages/sklearn/covariance/_graph_lasso.py", line 304, in graphical_lasso
    raise e
  File "/SGRNJ/Public/Software/conda_env/ds_scArches/lib/python3.9/site-packages/sklearn/covariance/_graph_lasso.py", line 278, in graphical_lasso
    raise FloatingPointError(
FloatingPointError: The system is too ill-conditioned for this solver. The system is too ill-conditioned for this solver

The problem remained unsolved even after I increased the number of iterations. I am not familiar with the algorithm involved so it would really be great if you could please give me some advice! Thanks in advance!!

Hi @gwentea! thanks for your interest and posting the issue. This happens when the number of genes is close to or higher than the number of spots, would the problem stay if you subset to say 500 HVGs?

[gwentea]

Hi @EliHei2, thanks for your response! The number of spots is relatively low for my datasets (~300 - 500 spots for each dataset) so I took your advice and subset the data with a lower number of HVGs. But the problem remains unsolved even with 200 HVGs...

[EliHei2]

@gwentea this seems to be an internal sklearn problem, de-meaning or scaling data in such cases might help (see here), but I can't tell for sure as long as I have no access to the data, if you provide a sample raw counts matrix I can investigate it further.