Skip to content

Avoid densifying matrix in ProjectCellEmbeddings#10337

Open
assaron wants to merge 2 commits into
satijalab:mainfrom
assaron:transfer-anchors-mem
Open

Avoid densifying matrix in ProjectCellEmbeddings#10337
assaron wants to merge 2 commits into
satijalab:mainfrom
assaron:transfer-anchors-mem

Conversation

@assaron

@assaron assaron commented Mar 18, 2026

Copy link
Copy Markdown
Contributor

This PR fixes an issue with large transient memory usage in FindTransferAnchors arising from generation of a dense matrix inside ProjectCellEmbeddings. I had this problem when working with Xenium 5K data and there was a similar report here: #7929 This PR removes explicit dense matrix used for the projection by moving the adjustments down the chain. The behavior doesn't change at all and the results match exactly (up to floating point arithmetic errors). It might also be a little faster.

Unfortunately, I was not able to come up with a good illustrative example on a standard dataset—apparently some other memory allocations dominate in such cases—so the example below is on synthetic data. Still, in my workflow it drastically improves memory usage (from 30GB+ to ~16GB, but I didn't check it carefully).

Example:

library(devtools)
library(Matrix)
library(Seurat)

set.seed(42)
n_genes       <- 20000
n_ref_cells   <- 200
n_query_cells <- 15000
n_dims        <- 30

gene_names  <- paste0("GENE",  seq_len(n_genes))
ref_names   <- paste0("ref_",   seq_len(n_ref_cells))
query_names <- paste0("query_", seq_len(n_query_cells))

# minimal reference object — only needs normalized data + PCA loadings
ref_counts <- rsparsematrix(n_genes, n_ref_cells, density = 0.1,
                            dimnames = list(gene_names, ref_names))
ref <- CreateSeuratObject(counts = ref_counts)
ref <- NormalizeData(ref, verbose = FALSE)

loadings <- matrix(rnorm(n_genes * n_dims), n_genes, n_dims,
                   dimnames = list(gene_names, paste0("PC_", seq_len(n_dims))))
ref[["pca"]] <- CreateDimReducObject(
  embeddings = matrix(0, n_ref_cells, n_dims,
                      dimnames = list(ref_names, paste0("PC_", seq_len(n_dims)))),
  loadings   = loadings,
  key        = "PC_",
  assay      = "RNA"
)

# sparse query matrix — 5% density, typical for scRNA-seq
query_data <- rsparsematrix(n_genes, n_query_cells, density = 0.05,
                            dimnames = list(gene_names, query_names))
query_data@x <- abs(query_data@x)   # non-negative counts


gc(reset = TRUE)
st <- system.time({
  proj <- ProjectCellEmbeddings(
    query                = query_data,
    reference            = ref,
    reduction            = "pca",
    dims                 = seq_len(n_dims),
    normalization.method = "LogNormalize",
    verbose              = FALSE
  )
}, gcFirst = FALSE)
gcRes <- gc()
print(gcRes)
peak_mb <- max(gcRes[, "max used"]) * 8 / 1024^2

print(st)
cat(sprintf("Peak memory: %.0f MB\n", peak_mb))
cat(sprintf("proj[1,1]:   %.7f\n",    proj[1, 1]))

For the current CRAN version the peak memory usage is 2.7GB, for the PR version it's 1.1GB.

Disclaimer: the code was generated with a help of Claude Code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants