Skip to content

Faster gmt2mat#13

Open
TylerSagendorf wants to merge 3 commits into
bigomics:mainfrom
TylerSagendorf:fast-gmt2mat
Open

Faster gmt2mat#13
TylerSagendorf wants to merge 3 commits into
bigomics:mainfrom
TylerSagendorf:fast-gmt2mat

Conversation

@TylerSagendorf

@TylerSagendorf TylerSagendorf commented Jun 10, 2026

Copy link
Copy Markdown

@ivokwee @ESCRI11 Hello, I've written a faster version of gmt2mat() if you are interested in using it.

Changes

  • Improve speed and memory usage of gmt2mat() by a factor of ~12.
  • Warn when use.multicore = TRUE in gmt2mat() and set the default to FALSE.
  • Replace parallel import with collapse.

Benchmarks

# Simulate gene sets
n_genes <- 1e4L
genes <- paste0("gene", seq_len(n_genes))

n_sets <- 5e4L
min_size <- 10L
max_size <- 1000L

set_sizes <- rep(min_size:max_size, length.out = n_sets)

gene_sets <- lapply(seq_len(n_sets), function(i) {
  set.seed(i)
  sample(x = genes, size = set_sizes[i])
})
names(gene_sets) <- paste0("set", seq_len(n_sets))

# gmt2mat_old contains the original gmt2mat code (not shown)
bench::mark(
  "old" = gmt2mat_old(gene_sets, use.multicore = FALSE),
  "new" = gmt2mat(gene_sets, use.multicore = FALSE),
  iterations = 10L,
  filter_gc = FALSE
)
  expression      min   median `itr/sec` mem_alloc `gc/sec` n_itr  n_gc total_time
  <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl> <int> <dbl>   <bch:tm>
1 old           17.4s    19.8s    0.0499   19.88GB    0.753    10   151      3.34m
2 new           1.43s     1.6s    0.603     1.69GB    0.784    10    13     16.59s

Replaces parallel import with collapse.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant