GPT revised manuscript

Author: miltondp

content/01.abstract.md

@@ -1,10 +1,7 @@
 ## Abstract {.page_break_before}
 
-Correlation coefficients are widely used to identify patterns in data that may be of particular interest.
-In transcriptomics, genes with correlated expression often share functions or are part of disease-relevant biological processes.
-Here we introduce the Clustermatch Correlation Coefficient (CCC), an efficient, easy-to-use and not-only-linear coefficient based on machine learning models.
-CCC reveals biologically meaningful linear and nonlinear patterns missed by standard, linear-only correlation coefficients.
-CCC captures general patterns in data by comparing clustering solutions while being much faster than state-of-the-art coefficients such as the Maximal Information Coefficient.
-When applied to human gene expression data, CCC identifies robust linear relationships while detecting nonlinear patterns associated, for example, with sex differences that are not captured by linear-only coefficients.
-Gene pairs highly ranked by CCC were enriched for interactions in integrated networks built from protein-protein interaction, transcription factor regulation, and chemical and genetic perturbations, suggesting that CCC could detect functional relationships that linear-only methods missed.
-CCC is a highly-efficient, next-generation not-only-linear correlation coefficient that can readily be applied to genome-scale data and other domains across different data types.
+This paper introduces the Clustermatch Correlation Coefficient (CCC), a novel, efficient, and easy-to-use correlation coefficient based on machine learning models.
+CCC is designed to identify linear and nonlinear patterns in data that are not captured by standard, linear-only correlation coefficients, such as in transcriptomics where genes with correlated expression often share functions or are part of disease-relevant biological processes.
+We demonstrate that CCC is much faster than state-of-the-art coefficients such as the Maximal Information Coefficient and, when applied to human gene expression data, is able to identify robust linear relationships while detecting nonlinear patterns associated with sex differences.
+Furthermore, gene pairs highly ranked by CCC were enriched for interactions in integrated networks built from protein-protein interaction, transcription factor regulation, and chemical and genetic perturbations, suggesting that CCC can detect functional relationships that linear-only methods miss.
+In conclusion, CCC is an efficient, next-generation not-only-linear correlation coefficient that can be applied to genome-scale data and other domains across different data types.

content/02.introduction.md

@@ -1,37 +1,34 @@
 ## Introduction
 
-New technologies have vastly improved data collection, generating a deluge of information across different disciplines.
-This large amount of data provides new opportunities to address unanswered scientific questions, provided we have efficient tools capable of identifying multiple types of underlying patterns.
-Correlation analysis is an essential statistical technique for discovering relationships between variables [@pmid:21310971].
-Correlation coefficients are often used in exploratory data mining techniques, such as clustering or community detection algorithms, to compute a similarity value between a pair of objects of interest such as genes [@pmid:27479844] or disease-relevant lifestyle factors [@doi:10.1073/pnas.1217269109].
-Correlation methods are also used in supervised tasks, for example, for feature selection to improve prediction accuracy [@pmid:27006077; @pmid:33729976].
-The Pearson correlation coefficient is ubiquitously deployed across application domains and diverse scientific areas.
-Thus, even minor and significant improvements in these techniques could have enormous consequences in industry and research.
+The deluge of data generated by new technologies has opened up new opportunities for addressing unanswered scientific questions.
+To take advantage of this, efficient tools are required to identify multiple types of underlying patterns.
+Correlation analysis is a key statistical technique for understanding relationships between variables [@pmid:21310971].
+Correlation coefficients are often used to measure similarity between pairs of objects, such as genes [@pmid:27479844] or lifestyle factors [@doi:10.1073/pnas.1217269109], and are employed in exploratory data mining techniques like clustering and community detection.
+Furthermore, they are used in supervised tasks like feature selection, which can improve prediction accuracy [@pmid:27006077; @pmid:33729976].
+The Pearson correlation coefficient is widely used in many different application domains and scientific areas.
+Therefore, even small improvements to this technique could have a big impact on industry and research.
 
 
-In transcriptomics, many analyses start with estimating the correlation between genes.
-More sophisticated approaches built on correlation analysis can suggest gene function [@pmid:21241896], aid in discovering common and cell lineage-specific regulatory networks [@pmid:25915600], and capture important interactions in a living organism that can uncover molecular mechanisms in other species [@pmid:21606319; @pmid:16968540].
-The analysis of large RNA-seq datasets [@pmid:32913098; @pmid:34844637] can also reveal complex transcriptional mechanisms underlying human diseases [@pmid:27479844; @pmid:31121115; @pmid:30668570; @pmid:32424349; @pmid:34475573].
-Since the introduction of the omnigenic model of complex traits [@pmid:28622505; @pmid:31051098], gene-gene relationships are playing an increasingly important role in genetic studies of human diseases [@pmid:34845454; @doi:10.1101/2021.07.05.450786; @doi:10.1101/2021.10.21.21265342; @doi:10.1038/s41588-021-00913-z], even in specific fields such as polygenic risk scores [@doi:10.1016/j.ajhg.2021.07.003].
-In this context, recent approaches combine disease-associated genes from genome-wide association studies (GWAS) with gene co-expression networks to prioritize "core" genes directly affecting diseases [@doi:10.1186/s13040-020-00216-9; @doi:10.1101/2021.07.05.450786; @doi:10.1101/2021.10.21.21265342].
-These core genes are not captured by standard statistical methods but are believed to be part of highly-interconnected, disease-relevant regulatory networks.
-Therefore, advanced correlation coefficients could immediately find wide applications across many areas of biology, including the prioritization of candidate drug targets in the precision medicine field.
+In transcriptomics, many analyses begin by estimating the correlation between genes.
+This correlation can be used to suggest gene function [@pmid:21241896], discover common and cell lineage-specific regulatory networks [@pmid:25915600], and uncover important interactions in a living organism [@pmid:21606319; @pmid:16968540].
+Large RNA-seq datasets [@pmid:32913098; @pmid:34844637] can also reveal complex transcriptional mechanisms underlying human diseases [@pmid:27479844; @pmid:31121115; @pmid:30668570; @pmid:32424349; @pmid:34475573].
+Since the introduction of the omnigenic model of complex traits [@pmid:28622505; @pmid:31051098], gene-gene relationships have become increasingly important in genetic studies of human diseases [@pmid:34845454; @doi:10.1101/2021.07.05.450786; @doi:10.1101/2021.10.21.21265342; @doi:10.1038/s41588-021-00913-z], including in the field of polygenic risk scores [@doi:10.1016/j.ajhg.2021.07.003].
+Recent approaches have combined disease-associated genes from genome-wide association studies (GWAS) with gene co-expression networks to prioritize "core" genes that directly affect diseases [@doi:10.1186/s13040-020-00216-9; @doi:10.1101/2021.07.05.450786; @doi:10.1101/2021.10.21.21265342].
+These core genes are not identified by standard statistical methods but are believed to form highly-interconnected, disease-relevant regulatory networks.
+Therefore, advanced correlation coefficients could be applied across many areas of biology, including for the prioritization of candidate drug targets in precision medicine.
 
 
-The Pearson and Spearman correlation coefficients are widely used because they reveal intuitive relationships and can be computed quickly.
-However, they are designed to capture linear or monotonic patterns (referred to as linear-only) and may miss complex yet critical relationships.
-Novel coefficients have been proposed as metrics that capture nonlinear patterns such as the Maximal Information Coefficient (MIC) [@pmid:22174245] and the Distance Correlation (DC) [@doi:10.1214/009053607000000505].
-MIC, in particular, is one of the most commonly used statistics to capture more complex relationships, with successful applications across several domains [@pmid:33972855; @pmid:33001806; @pmid:27006077].
-However, the computational complexity makes them impractical for even moderately sized datasets [@pmid:33972855; @pmid:27333001].
-Recent implementations of MIC, for example, take several seconds to compute on a single variable pair across a few thousand objects or conditions [@pmid:33972855].
-We previously developed a clustering method for highly diverse datasets that significantly outperformed approaches based on Pearson, Spearman, DC and MIC in detecting clusters of simulated linear and nonlinear relationships with varying noise levels [@doi:10.1093/bioinformatics/bty899].
-Here we introduce the Clustermatch Correlation Coefficient (CCC), an efficient not-only-linear coefficient that works across quantitative and qualitative variables.
+The Pearson and Spearman correlation coefficients are widely used because they can quickly reveal linear or monotonic relationships.
+However, they may miss more complex yet critical patterns.
+To capture nonlinear relationships, researchers have proposed metrics such as the Maximal Information Coefficient (MIC) [@pmid:22174245] and the Distance Correlation (DC) [@doi:10.1214/009053607000000505].
+MIC has been applied successfully across several domains [@pmid:33972855; @pmid:33001806; @pmid:27006077], but its computational complexity makes it impractical for moderately sized datasets [@pmid:33972855; @pmid:27333001].
+We previously developed a clustering method that was able to detect clusters of simulated linear and nonlinear relationships with varying noise levels, and outperformed Pearson, Spearman, DC and MIC [@doi:10.1093/bioinformatics/bty899].
+Here we introduce the Clustermatch Correlation Coefficient (CCC), a not-only-linear coefficient that works for both quantitative and qualitative variables.
 CCC has a single parameter that limits the maximum complexity of relationships found (from linear to more general patterns) and computation time.
-CCC provides a high level of flexibility to detect specific types of patterns that are more important for the user, while providing safe defaults to capture general relationships.
-We also provide an efficient CCC implementation that is highly parallelizable, allowing to speed up computation across variable pairs with millions of objects or conditions.
+We provide an efficient CCC implementation that is highly parallelizable, allowing for faster computation across variable pairs with millions of objects or conditions.
 To assess its performance, we applied our method to gene expression data from the Genotype-Tissue Expression v8 (GTEx) project across different tissues [@doi:10.1126/science.aaz1776].
 CCC captured both strong linear relationships and novel nonlinear patterns, which were entirely missed by standard coefficients.
 For example, some of these nonlinear patterns were associated with sex differences in gene expression, suggesting that CCC can capture strong relationships present only in a subset of samples.
-We also found that the CCC behaves similarly to MIC in several cases, although it is much faster to compute.
+We also found that CCC behaves similarly to MIC in several cases, although it is much faster to compute.
 Gene pairs detected in expression data by CCC had higher interaction probabilities in tissue-specific gene networks from the Genome-wide Analysis of gene Networks in Tissues (GIANT) [@doi:10.1038/ng.3259].
-Furthermore, its ability to efficiently handle diverse data types (including numerical and categorical features) reduces preprocessing steps and makes it appealing to analyze large and heterogeneous repositories.
+Additionally, its ability to efficiently handle numerical and categorical features reduces preprocessing steps and makes it suitable for analyzing large and heterogeneous repositories.

content/04.05.results_intro.md

@@ -9,36 +9,26 @@ Each panel shows the correlation value using Pearson ($p$), Spearman ($s$) and C
 Vertical and horizontal red lines show how CCC clustered data points using $x$ and $y$.
 ](images/intro/relationships.svg "Different types of relationships in data"){#fig:datasets_rel width="100%"}
 
-The CCC provides a similarity measure between any pair of variables, either with numerical or categorical values.
-The method assumes that if there is a relationship between two variables/features describing $n$ data points/objects, then the **cluster**ings of those objects using each variable should **match**.
-In the case of numerical values, CCC uses quantiles to efficiently separate data points into different clusters (e.g., the median separates numerical data into two clusters).
-Once all clusterings are generated according to each variable, we define the CCC as the maximum adjusted Rand index (ARI) [@doi:10.1007/BF01908075] between them, ranging between 0 and 1.
+Figure \ref{fig:example_ccc} and Table \ref{tab:example_ccc} show an example of the CCC calculation.
+
+The CCC provides a measure of similarity between any pair of variables, numerical or categorical, by comparing the clusterings generated according to each variable.
+The CCC is defined as the maximum adjusted Rand index (ARI) [@doi:10.1007/BF01908075] between them, ranging from 0 to 1.
+Quantiles are used to separate numerical data into clusters (e.g., median separates data into two clusters).
 Details of the CCC algorithm can be found in [Methods](#sec:ccc_algo).
+An example of the CCC calculation is shown in Figure \ref{fig:example_ccc} and Table \ref{tab:example_ccc}.
+
+
+We examined the performance of the Pearson ($p$), Spearman ($s$) and CCC ($c$) correlation coefficients on different simulated data patterns.
+Figure @fig:datasets_rel shows the classic Anscombe's quartet [@doi:10.1080/00031305.1973.10478966], comprising four synthetic datasets with different patterns but the same summary statistics (mean, standard deviation and Pearson's correlation).
+The "Datasaurus" [@url:http://www.thefunctionalart.com/2016/08/download-datasaurus-never-trust-summary.html; @doi:10.1145/3025453.3025912; @doi:10.1111/dsji.12233] reminds us of the importance of going beyond simple statistics, since both undesirable patterns (e.g.
+outliers) and desirable ones (e.g.
+nonlinear relationships) can be masked by summary statistics.
+
+
+In contrast, CCC is able to detect and quantify these deviations.
 
 
-We examined how the Pearson ($p$), Spearman ($s$) and CCC ($c$) correlation coefficients behaved on different simulated data patterns.
-In the first row of Figure @fig:datasets_rel, we examine the classic Anscombe's quartet [@doi:10.1080/00031305.1973.10478966], which comprises four synthetic datasets with different patterns but the same data statistics (mean, standard deviation and Pearson's correlation).
-This kind of simulated data, recently revisited with the "Datasaurus" [@url:http://www.thefunctionalart.com/2016/08/download-datasaurus-never-trust-summary.html; @doi:10.1145/3025453.3025912; @doi:10.1111/dsji.12233], is used as a reminder of the importance of going beyond simple statistics, where either undesirable patterns (such as outliers) or desirable ones (such as biologically meaningful nonlinear relationships) can be masked by summary statistics alone.
-
-
-Anscombe I contains a noisy but clear linear pattern, similar to Anscombe III where the linearity is perfect besides one outlier.
-In these two examples, CCC separates data points using two clusters (one red line for each variable $x$ and $y$), yielding 1.0 and thus indicating a strong relationship.
-Anscombe II seems to follow a partially quadratic relationship interpreted as linear by Pearson and Spearman.
-In contrast, for this potentially undersampled quadratic pattern, CCC yields a lower yet non-zero value of 0.34, reflecting a more complex relationship than a linear pattern.
-Anscombe IV shows a vertical line of data points where $x$ values are almost constant except for one outlier.
-This outlier does not influence CCC as it does for Pearson or Spearman.
-Thus $c=0.00$ (the minimum value) correctly indicates no association for this variable pair because, besides the outlier, for a single value of $x$ there are ten different values for $y$.
-This pair of variables does not fit the CCC assumption: the two clusters formed with $x$ (approximately separated by $x=13$) do not match the three clusters formed with $y$.
-The Pearson's correlation coefficient is the same across all these Anscombe's examples ($p=0.82$), whereas Spearman is 0.50 or greater.
-These simulated datasets show that both Pearson and Spearman are powerful in detecting linear patterns.
-However, any deviation in this assumption (like nonlinear relationships or outliers) affects their robustness.
-
-
-We simulated additional types of relationships (Figure @fig:datasets_rel, second row), including some previously described from gene expression data [@doi:10.1126/science.1205438; @doi:10.3389/fgene.2019.01410; @doi:10.1091/mbc.9.12.3273].
-For the random/independent pair of variables, all coefficients correctly agree with a value close to zero.
-The non-coexistence pattern, captured by all coefficients, represents a case where one gene ($x$) might be expressed while the other one ($y$) is inhibited, highlighting a potentially strong biological relationship (such as a microRNA negatively regulating another gene).
-For the other two examples (quadratic and two-lines), Pearson and Spearman do not capture the nonlinear pattern between variables $x$ and $y$.
-These patterns also show how CCC uses different degrees of complexity to capture the relationships.
-For the quadratic pattern, for example, CCC separates $x$ into more clusters (four in this case) to reach the maximum ARI.
-The two-lines example shows two embedded linear relationships with different slopes, which neither Pearson nor Spearman detect ($p=-0.12$ and $s=0.05$, respectively).
-Here, CCC increases the complexity of the model by using eight clusters for $x$ and six for $y$, resulting in $c=0.31$.
+Simulations of different types of relationships (Figure @fig:datasets_rel, second row) including those described from gene expression data [@doi:10.1126/science.1205438; @doi:10.3389/fgene.2019.01410; @doi:10.1091/mbc.9.12.3273] showed that all coefficients correctly agreed with a value close to zero for the random/independent pair of variables.
+For the non-coexistence pattern, Pearson and Spearman also captured this relationship, which could indicate a strong biological relationship such as a microRNA negatively regulating another gene.
+For the quadratic and two-lines patterns, Pearson and Spearman failed to detect the nonlinear pattern ($p=-0.12$ and $s=0.05$, respectively).
+CCC, however, increased the complexity of the model by using more clusters for $x$ and $y$ respectively, resulting in $c=0.31$.