Skip to content

Fixed the sparse matrix issue #89

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 25 additions & 6 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,34 @@ Authors@R: c(person("Vladimir", "Kiselev",
role=c("aut")))
Description: A tool for unsupervised clustering and analysis of single cell RNA-Seq data.
License: GPL-3
Imports: graphics, stats, utils, methods, e1071, parallel, foreach,
doParallel, doRNG, shiny, ggplot2, pheatmap (>= 1.0.8),
ROCR, robustbase, rrcov, cluster, WriteXLS,
Rcpp (>= 0.11.1), SummarizedExperiment, SingleCellExperiment,
BiocGenerics, S4Vectors
Imports:
graphics,
stats,
utils,
methods,
e1071,
parallel,
foreach,
doParallel,
doRNG,
shiny,
ggplot2,
pheatmap (>= 1.0.8),
ROCR,
robustbase,
rrcov,
cluster,
WriteXLS,
Rcpp (>= 0.11.1),
SummarizedExperiment,
SingleCellExperiment,
BiocGenerics,
S4Vectors,
Matrix
Depends: R(>= 3.3)
LinkingTo: Rcpp, RcppArmadillo
LazyData: TRUE
RoxygenNote: 6.0.1
RoxygenNote: 6.1.1
Suggests: knitr, rmarkdown, mclust, scater
VignetteBuilder: knitr
biocViews: SingleCell, Software, Classification, Clustering, DimensionReduction,
Expand Down
2 changes: 2 additions & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ export(get_marker_genes)
export(get_outl_cells)
export(get_processed_dataset)
export(norm_laplacian)
export(organise_marker_genes)
export(reindex_clusters)
export(sc3)
export(sc3_calc_biology)
Expand Down Expand Up @@ -54,6 +55,7 @@ importFrom(ggplot2,ylim)
importFrom(graphics,plot)
importFrom(methods,as)
importFrom(methods,new)
importFrom(parallel,clusterCall)
importFrom(parallel,detectCores)
importFrom(parallel,makeCluster)
importFrom(parallel,stopCluster)
Expand Down
3 changes: 3 additions & 0 deletions R/CoreFunctions.R
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@
#' @importFrom Rcpp sourceCpp
#'
calculate_distance <- function(data, method) {
if(class(data) == "dgCMatrix"){
data = as.matrix(data)
}
return(if (method == "spearman") {
as.matrix(1 - cor(data, method = "spearman"))
} else if (method == "pearson") {
Expand Down
27 changes: 21 additions & 6 deletions R/CoreMethods.R
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ sc3_prepare.SingleCellExperiment <- function(object, gene_filter, pct_dropout_mi
f_data <- rowData(object)
f_data$sc3_gene_filter <- TRUE
if (gene_filter) {
dropouts <- rowSums(counts(object) == 0)/ncol(object)*100
dropouts <- Matrix::rowSums(counts(object) == 0)/ncol(object)*100
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am aware of the discussion in #53 , but I am still not sure if hardcoding it this way without checking the type of counts(object) is a good idea. For hdf5-backed object, the returned matrix can be of class "DelayedMatrix", which is not handled correctly by Matrix::rowSums(), just as an example.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks for the comment! That's why I didn't rush to merge it, the issue is quite big and requires some time to think and decide.

if(!is.null(isSpike(object))) {
f_data$sc3_gene_filter <- dropouts < pct_dropout_max & dropouts > pct_dropout_min & !isSpike(object)
} else {
Expand Down Expand Up @@ -291,7 +291,7 @@ setMethod("sc3_estimate_k", signature(object = "SingleCellExperiment"), sc3_esti
#'
#' @importFrom doRNG %dorng%
#' @importFrom foreach foreach %dopar%
#' @importFrom parallel makeCluster stopCluster
#' @importFrom parallel makeCluster stopCluster clusterCall
#' @importFrom doParallel registerDoParallel
sc3_calc_dists.SingleCellExperiment <- function(object) {
dataset <- get_processed_dataset(object)
Expand All @@ -317,6 +317,9 @@ sc3_calc_dists.SingleCellExperiment <- function(object) {
cl <- parallel::makeCluster(n_cores, outfile = "")
doParallel::registerDoParallel(cl, cores = n_cores)

## pass .libPaths to workers
parallel::clusterCall(cl, function(x) {.libPaths(.Library); .libPaths(x)}, .libPaths())

# calculate distances in parallel
dists <- foreach::foreach(i = distances) %dorng% {
try({
Expand Down Expand Up @@ -358,7 +361,7 @@ setMethod("sc3_calc_dists", signature(object = "SingleCellExperiment"), sc3_calc
#'
#' @importFrom doRNG %dorng%
#' @importFrom foreach foreach
#' @importFrom parallel makeCluster stopCluster
#' @importFrom parallel makeCluster stopCluster clusterCall
#' @importFrom doParallel registerDoParallel
sc3_calc_transfs.SingleCellExperiment <- function(object) {
dists <- metadata(object)$sc3$distances
Expand Down Expand Up @@ -388,6 +391,9 @@ sc3_calc_transfs.SingleCellExperiment <- function(object) {
cl <- parallel::makeCluster(n_cores, outfile = "")
doParallel::registerDoParallel(cl, cores = n_cores)

## pass .libPaths to workers
parallel::clusterCall(cl, function(x) {.libPaths(.Library); .libPaths(x)}, .libPaths())

# calculate the 6 distinct transformations in parallel
transfs <- foreach::foreach(i = 1:nrow(hash.table)) %dorng% {
try({
Expand Down Expand Up @@ -431,7 +437,7 @@ setMethod("sc3_calc_transfs", signature(object = "SingleCellExperiment"), sc3_ca
#'
#' @importFrom doRNG %dorng%
#' @importFrom foreach foreach
#' @importFrom parallel makeCluster stopCluster
#' @importFrom parallel makeCluster stopCluster clusterCall
#' @importFrom doParallel registerDoParallel
#' @importFrom utils setTxtProgressBar txtProgressBar
#' @importFrom stats kmeans
Expand Down Expand Up @@ -464,6 +470,9 @@ sc3_kmeans.SingleCellExperiment <- function(object, ks) {
cl <- parallel::makeCluster(n_cores, outfile = "")
doParallel::registerDoParallel(cl, cores = n_cores)

## pass .libPaths to workers
parallel::clusterCall(cl, function(x) {.libPaths(.Library); .libPaths(x)}, .libPaths())

pb <- utils::txtProgressBar(min = 1, max = nrow(hash.table), style = 3)

# calculate the 6 distinct transformations in parallel
Expand Down Expand Up @@ -515,7 +524,7 @@ setMethod("sc3_kmeans", signature(object = "SingleCellExperiment"), sc3_kmeans.S
#'
#' @importFrom doRNG %dorng%
#' @importFrom foreach foreach
#' @importFrom parallel makeCluster stopCluster
#' @importFrom parallel makeCluster stopCluster clusterCall
#' @importFrom doParallel registerDoParallel
#' @import cluster
#' @importFrom stats hclust dist as.dist
Expand Down Expand Up @@ -545,6 +554,9 @@ sc3_calc_consens.SingleCellExperiment <- function(object) {
cl <- parallel::makeCluster(n_cores, outfile = "")
doParallel::registerDoParallel(cl, cores = n_cores)

## pass .libPaths to workers
parallel::clusterCall(cl, function(x) {.libPaths(.Library); .libPaths(x)}, .libPaths())

cons <- foreach::foreach(i = ks) %dorng% {
try({
d <- k.means[grep(paste0("_", i, "_"), names(k.means))]
Expand Down Expand Up @@ -644,7 +656,7 @@ setMethod("sc3_calc_consens", signature(object = "SingleCellExperiment"), sc3_ca
#'
#' @importFrom doRNG %dorng%
#' @importFrom foreach foreach
#' @importFrom parallel makeCluster stopCluster
#' @importFrom parallel makeCluster stopCluster clusterCall
#' @importFrom doParallel registerDoParallel
#' @importFrom methods as
sc3_calc_biology.SingleCellExperiment <- function(object, ks, regime) {
Expand Down Expand Up @@ -695,6 +707,9 @@ sc3_calc_biology.SingleCellExperiment <- function(object, ks, regime) {
cl <- parallel::makeCluster(n_cores, outfile = "")
doParallel::registerDoParallel(cl, cores = n_cores)

## pass .libPaths to workers
parallel::clusterCall(cl, function(x) {.libPaths(.Library); .libPaths(x)}, .libPaths())

biol <- foreach::foreach(i = 1:nrow(hash.table)) %dorng% {
try({
get_biolgy(dataset, clusts[, paste0("sc3_", hash.table[i, 1], "_clusters")], hash.table[i, 2])
Expand Down
3 changes: 2 additions & 1 deletion man/prepare_for_svm.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 10 additions & 11 deletions man/sc3.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions man/sc3_calc_biology.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions man/sc3_calc_consens.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions man/sc3_calc_dists.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions man/sc3_calc_transfs.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions man/sc3_estimate_k.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion man/sc3_export_results_xls.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions man/sc3_interactive.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions man/sc3_kmeans.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions man/sc3_plot_cluster_stability.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions man/sc3_plot_consensus.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 2 additions & 4 deletions man/sc3_plot_de_genes.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions man/sc3_plot_expression.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 2 additions & 4 deletions man/sc3_plot_markers.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions man/sc3_plot_silhouette.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 3 additions & 4 deletions man/sc3_prepare.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions man/sc3_run_svm.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.