Question on choosing K and whether to run bp_extract_signatures_iter() for CN signatures (W method)

[SenLiBio]

Hi Shixiang,

First of all, thank you very much for developing sigminer — it has been very helpful for my copy number signature analysis.

I am currently using copy number signatures with the W method in sigminer, and I would like to ask for your advice on two points:

1. How should I choose K when the aggregated score and the raw metrics do not fully agree?
2. Under what circumstances would you recommend running bp_extract_signatures_iter() after bp_extract_signatures()?

My current setup:
- Data type: copy number segments
- Signature method: sig_tally(..., method = "W")
- K range tested: 2:8
- I used the best-practice workflow with bp_extract_signatures()

From my current results, bp_res$suggested returned 7, but after checking the raw metrics, I am leaning toward K = 6.

However, after checking both the rank score and the raw statistics, I am leaning toward K = 6 as a more parsimonious solution.

K_selection_full_survey.pdf

K_selection_survey2_K6.pdf

My interpretation is:

• K = 7 is suggested because its aggregated score is slightly higher than K = 6
• but K = 6 seems more attractive because:
  • L2_error is slightly lower at K = 6 than at K = 7
  • silhouette is much better at K = 6 than at K = 7
  • signature_similarity_within_cluster is also higher at K = 6
  • sample_cosine_distance improves only marginally from K = 6 to K = 7
  • exposure_positive_correlation is lower at K = 6 than at K = 7

So my first question is:

Question 1

In a case like this, would you consider K = 6 a reasonable final choice, even though bp_res$suggested is 7?

More generally, when the aggregated score and the raw metrics point to slightly different conclusions, how would you recommend prioritizing them in practice?

About iteration

I also read the documentation for bp_extract_signatures_iter(), and my understanding is that it is mainly useful when some samples are not well reconstructed by the initial extraction.

However, in my current CN W-method workflow, get_sig_rec_similarity() returns the following warning:

Cannot calculate reconstructed profile without raw W and H for CN 'W'/'M' method.

So at the moment I am mainly relying on:

• bp_res$rank_score
• bp_get_stats(bp_res)
• survey plots
• signature/exposure visualizations

I also noticed that the pairwise correlations among the 6 signature profiles seem relatively high, which makes me wonder whether:

• K = 6 may still contain partially redundant signatures, or
• iteration would help identify a more stable consensus solution.

signature_similarity_heatmap.pdf

Question 2

Based on this kind of result, would you recommend running bp_extract_signatures_iter() for CN signatures with the W method?

More specifically, what would be the most practical criteria for deciding whether iteration is needed in this setting?

For example, would you recommend iteration mainly when:

• a subset of samples has clearly poor reconstruction-related statistics,
• the signatures appear partially redundant,
• or when there is evidence of heterogeneous subgroups not captured by the first extraction?

Also, for CN W method specifically, is there a preferred way to assess whether iteration is needed, given that get_sig_rec_similarity() is not available in this case?

Thanks again for your time and for developing this package.
Sen

[ShixiangWang]

I think choosing option 6 is reasonable, as it is at a critical point where multiple metrics (sihouetee and error) are appropriate. Copy number signatures may have higher inter-similarity, so it is necessary to determine through corresponding data analysis whether there is a need for merging/deleting signatures (for example, by drawing an activity proportion plot; if one signature has a very low proportion in all samples, it can actually be considered for removal).

The optimal k calculated is only a reference standard; the standard of fact still needs to be determined based on the data itself and the application perspective.

You have very good thoughts and viewpoints, bp_extract_signatures_iter() is worth trying (https://shixiangwang.github.io/sigminer/reference/bp.html#details-1), but since I haven't paid much attention to this package for a long time, I don't remember many details. Overall, sigminer incorporates a lot of ideas and implementations from various studies. You can use it as a reference to apply in your data and judge a reasonable result. In the future, I might consider exploring and redesigning a new signature analysis tool or a completely updated version of it.

[SenLiBio]

Hi Shixiang,

Many thanks for your reply — it has made things much clearer for me.

I wanted to share a few updates from my side. The results I showed previously were obtained using n_bootstrap = 20L and n_nmf_run = 50. I then ran the analysis again using n_bootstrap = 50L and n_nmf_run = 50. In this new run, the final score for K = 7 is 98, while the score for K = 6 is 91. In addition, the curves also suggested that K = 7 might be a better solution, so I used K = 7 for the downstream analysis.

However, when I examined the quality of the extracted signatures, I noticed that the signature instability remains quite high. In addition, even when choosing K = 3, the inter-signature similarity is still high. I was wondering whether this could be related to the relatively small cohort size in my dataset (144 samples).

I also have another question: do you think it would be interesting or meaningful to explore copy number signature analysis using single-cell copy number data? My thought is that each cell could be treated as a “sample,” and we can calculate a score for each cell population.

If this direction is also of interest to you, I would be very happy to discuss it further. I do have relevant single-cell copy number data, so there may also be a possibility for collaboration.

Best,
Sen