Search and visualize biological pathways (SCP-5982)

[eweitz]

This enables researchers to explore how gene expression relates to disease mechanisms and gene regulatory networks.

Overview

Previously, users could only search for genes in a study. This was useful for narrow analyses, or when users knew which genes they wanted to search.

Now users can search for pathways, too. Pathways are searchable by title -- higher-level concepts like diseases, phenotypes, and biological processes. Upon selecting a pathway from the search autocomplete menu, a biological pathway diagram is drawn. These pathways comprise nodes (genes, metabolites, diseases, etc.) and directed interactions among them, in cellular context.

Hovering over a gene shows its expression metrics. These reuse "scaled mean expression" and "percent of cells expressing" metrics from dot plots, like #2172. Clicking a pathway node in the diagram navigates to a new pathway.
Descriptions and cell type annotations for the pathway are also shown.

Video

Here's how it looks!

Explore_biological_pathways__SCP_2025-04-24.mov

The pathways come from WikiPathways, and use Ideogram.js integrations. This feature is limited to human studies for now, as those diagrams are the highest-quality. Future work might use orthology projection to extend support to mouse and other organisms.

Test

Automated tests verify new behaviors. Optional manual tests:

1. Set pathway_expression_overlay feature flag to true.
2. Go to "Cellular and transcriptional diversity over the course of human lactation" study.
3. In search box, type "lac".
4. Confirm suggestions appear in "Genes" and "Pathways" groups in autocomplete.
5. Select first pathway suggestion.
6. Confirm "Mammary gland development: pregnancy and lactation - stage 3 of 4" pathway appears.
7. Wait, then confirm "CSN2" gene node is colored bold blue.
8. In right panel, change "Color by expression in" to "LC1".
9. Confirm "CSN2" gene node is colored bold blue.
10. In the pathway diagram, click the green "Involution" node at top.
11. Confirm "IL6ST" node is colored light red.
12. In search box, type in "CSN1S1".
13. Confirm a pathway option appears. Select it.
14. Confirm "CSN1S1" node is colored bold red.

Further context

This satisfies SCP-5982, SCP-5977, SCP-5945, and other tickets.

[codecov]

Codecov Report

Attention: Patch coverage is 63.49558% with 165 lines in your changes missing coverage. Please review.

Project coverage is 71.18%. Comparing base (d74fe21) to head (8a98465).
Report is 73 commits behind head on development.

Files with missing lines	Patch %	Lines
...p/javascript/components/explore/StudyGeneField.jsx	61.80%	55 Missing ⚠️
...pp/javascript/components/visualization/Pathway.jsx	67.90%	26 Missing ⚠️
app/javascript/lib/pathway-expression.js	36.84%	21 Missing and 3 partials ⚠️
...omponents/visualization/controls/LabelSelector.jsx	0.00%	21 Missing ⚠️
...vascript/components/explore/ExploreDisplayTabs.jsx	30.76%	18 Missing ⚠️
app/javascript/lib/search-utils.js	90.00%	10 Missing ⚠️
...ascript/components/visualization/DotPlotLegend.jsx	77.77%	4 Missing ⚠️
.../components/explore/ExploreDisplayPanelManager.jsx	33.33%	2 Missing ⚠️
...javascript/components/explore/ExploreTabRouter.jsx	75.00%	2 Missing ⚠️
...ents/visualization/controls/AnnotationSelector.jsx	0.00%	2 Missing ⚠️
... and 1 more

Additional details and impacted files

@@               Coverage Diff               @@
##           development    #2206      +/-   ##
===============================================
+ Coverage        70.82%   71.18%   +0.35%     
===============================================
  Files              331      335       +4     
  Lines            28311    28672     +361     
  Branches          2431     2520      +89     
===============================================
+ Hits             20052    20410     +358     
- Misses            8116     8120       +4     
+ Partials           143      142       -1     

Files with missing lines	Coverage Δ
app/javascript/components/explore/PlotTabs.jsx	94.44% <ø> (ø)
...pp/javascript/components/visualization/DotPlot.jsx	22.22% <ø> (+14.81%)	⬆️
test/js/visualization/pathway.test-data.js	100.00% <ø> (ø)
app/javascript/lib/cluster-utils.js	51.02% <90.90%> (+5.04%)	⬆️
.../components/explore/ExploreDisplayPanelManager.jsx	61.86% <33.33%> (-0.12%)	⬇️
...javascript/components/explore/ExploreTabRouter.jsx	87.05% <75.00%> (-1.55%)	⬇️
...ents/visualization/controls/AnnotationSelector.jsx	53.33% <0.00%> (-1.84%)	⬇️
...ascript/components/visualization/DotPlotLegend.jsx	80.95% <77.77%> (+14.28%)	⬆️
app/javascript/lib/search-utils.js	90.83% <90.00%> (-4.63%)	⬇️
...vascript/components/explore/ExploreDisplayTabs.jsx	64.75% <30.76%> (-2.12%)	⬇️
... and 4 more

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[bistline]

Really comprehensive feature! I'm excited to see this in action. This absolutely should get a feature announcement and also a socials post. Do you think we need a Zendesk wiki page to document the functionality as well?

[bistline]

At first I was worried that this would mean a lot of our older studies wouldn't be able to use this feature since they lack species data, but in actuality it's only 66 public studies that are all before accession SCP260.

[eweitz]

Thanks for the review! Good points on getting the word out. I opened a few tickets for that, and possible species-backfill.