Merge pull request #139 from mastodon-sc/lineage-motifs-multiple-projects

Compare lineage motifs from multiple projects by importing the motif to compare

Author: stefanhahmann

README.md

@@ -452,41 +452,53 @@ Tree2
 
 ## Lineage Motif Search
 
-* Menu Location: `Plugins > Lineage Analysis > Lineage Motif Search`
-* This command is capable of finding lineage motifs that are similar to a motif defined by a selection of the user.
-* The linage motif search operates on Mastodon's branch graph.
+* Menu Location: `Plugins > Lineage Analysis > Find lineage motifs`
+* This command is capable of finding lineage motifs that are similar to a motif defined by either a tracklet selected by
+  the user or a tracklet imported via a graph ml file.
+* The linage motif search can operate on Mastodon's branch graph or on the model graph. While using the model graph is
+  slightly more accurate, it is recommended to use the branch graph since it is a lot faster.
 * Lineage trees are considered similar if they share a similar structure and thus represent a similar cell division
-  pattern. The structure of a lineage tree is represented by the tree topology.
+  pattern. The tree topology represents the structure of a lineage tree.
   This tree topology consists of the actual branching pattern and the cell lifetimes,
-  i.e., the time points between two subsequent cell divisions.
-* The algorithm iterates over the branch graph
+  i.e., the time points between two cell divisions.
 
 ### Workflow
 
-1. The user selects a motif in the track scheme, which should be searched for in the lineage trees.
+1. The user selects a motif
+    1. By selecting it in the track scheme
+    2. By importing it from a graph ml file
 2. The algorithm iterates over each branch in the branch graph using an offset before the first division, which is the
    same as the duration before the first division of the selected motif.
-3. For each branch, the algorithm computes the tree edit distance between the selected motif and the branch
+3. For each branch, the algorithm computes the tree edit distance between the selected (imported) motif and the branch
    tree. For more details on the tree edit distance, see section [Zhang tree edit distance](#zhang-tree-edit-distance).
-4. The algorithm creates a new tag set and tags the spots belonging to the `n` most similar branches with a color
+4. The algorithm sorts the branches based on their computed distance (ascending order) to the selected (imported) motif.
+5. The algorithm creates a new tag set and tags the spots belonging to the `n` most similar branches with a color
    that is faded out from the original motif color. `n` is the number of motifs specified by the user. The lower the
    computed distance, the more similar the found motif is to the original motif.
 
 ### Usage
 
-1. The user selects a motif in the track scheme, which should be searched for in the lineage trees.
-    1. The motif must have exactly one root node, i.e. the selected spots all must be connected to one root spot.
+1. The user selects a motif, which should be searched for in the lineage trees.
+    1. Directly in the track scheme
+    2. By importing it from a graph ml file. The file can be chosen to choose the file in the dialog
+    3. In both cases: the motif must have exactly one root node, i.e., the selected spots all must be connected to one
+       root spot.
 2. Open the dialog. Set the parameters and click on `OK`.
-3. Visualize the results in the track scheme and/or the BigDataViewer using `View > Coloring > The generated tag set`.
+3. The algorithm computes the most similar motifs in the lineage trees and creates a new tag set with a tag and color
+   for each found motif.
+4. Visualize the results in the track scheme and/or the BigDataViewer using `View > Coloring > The generated tag set`.
 
 ### Parameters
 
 * Number of motifs
     * The number of motifs to search for in the lineage trees.
     * The given motif will always be included in the results.
-* Color
-    * A color that will be used to tag spots that are part of a motif. The actual colors will be faded out versions of
-      the given color. The more the color is faded, the less similar is the found motif to the original motif.
+* Color of the most similar motif
+    * A color that will be used to tag spots that are part of the most similar motif.
+* Color of the least similar motif
+    * A color that will be used to tag spots that are part of the least similar motif.
+    * The colors of the motifs in between will be interpolated on a gradient from the color of the most similar motif to
+      the color of the least similar motif.
 * Similarity measure:
     1. (default) ![normalized_zhang_distance.gif](doc/clustering/normalized_zhang_distance.gif)<sup>1,2</sup>
     2. ![per_branch_zhang_distance.gif](doc/clustering/per_branch_zhang_distance.gif)<sup>1</sup>
@@ -495,6 +507,19 @@ Tree2
     * <sup>1</sup>Local cost function: ![local_cost.gif](doc/clustering/local_cost.gif)
     * <sup>2</sup>Local cost function with
       normalization: ![local_cost_normalized.gif](doc/clustering/local_cost_normalized.gif)
+* Run on:
+    * The graph on which the motif search should be run
+        1. Branch graph (default):  faster, (sightly) less accurate
+        2. Model graph: much slower, (sightly) more accurate
+* Load motif from a file:
+    * Only available when choosing Find similar motifs based on imported motif
+    * The file must be a graph ml file containing a single tree with exactly one root node
+    * Such a file can be exported from any Mastodon project using after selecting a tracklet and
+      `File > Export > Export selected spots to GraphML (one file)`
+* Scaling of the search motif:
+    * Only available when choosing Find similar motifs based on imported motif
+    * The imported motif can be scaled in time to account for faster or slower cell cycles in the current project
+      compared to the project from which the motif had been exported.
 
 ### Example
 

pom.xml

@@ -179,6 +179,14 @@
             <scope>test</scope>
         </dependency>
 
+        <!-- Awaitility used in some unit tests -->
+        <dependency>
+            <groupId>org.awaitility</groupId>
+            <artifactId>awaitility</artifactId>
+            <version>4.3.0</version>
+            <scope>test</scope>
+        </dependency>
+
 
         <!-- hierarchical clustering, tests only -->
         <dependency>

src/main/java/org/mastodon/mamut/clustering/config/SimilarityMeasure.java

@@ -1,40 +1,12 @@
-/*-
- * #%L
- * mastodon-deep-lineage
- * %%
- * Copyright (C) 2022 - 2025 Stefan Hahmann
- * %%
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- * 
- * 1. Redistributions of source code must retain the above copyright notice,
- *    this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright notice,
- *    this list of conditions and the following disclaimer in the documentation
- *    and/or other materials provided with the distribution.
- * 
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- * #L%
- */
 package org.mastodon.mamut.clustering.config;
 
-import org.apache.commons.lang3.function.TriFunction;
+import java.util.NoSuchElementException;
+
 import org.mastodon.mamut.clustering.treesimilarity.TreeDistances;
 import org.mastodon.mamut.clustering.treesimilarity.ZhangUnorderedTreeEditDistance;
 import org.mastodon.mamut.clustering.treesimilarity.tree.Tree;
-
-import java.util.NoSuchElementException;
-import java.util.function.ToDoubleBiFunction;
+import org.mastodon.mamut.util.ToDoubleQuadFunction;
+import org.mastodon.mamut.util.ToDoubleTriFunction;
 
 public enum SimilarityMeasure implements HasName
 {
@@ -56,31 +28,32 @@ public enum SimilarityMeasure implements HasName
 
 	private final String name;
 
-	private final TriFunction< Tree< Double >, Tree< Double >, ToDoubleBiFunction< Double, Double >, Double > distanceFunction;
+	private final ToDoubleQuadFunction< Tree< Double >, Tree< Double >, ToDoubleTriFunction< Double, Double, Double >,
+			Double > distanceFunction;
 
-	private final ToDoubleBiFunction< Double, Double > costFunction;
+	private final ToDoubleTriFunction< Double, Double, Double > costFunctionWithScale;
 
-	SimilarityMeasure( final String name,
-			final TriFunction< Tree< Double >, Tree< Double >, ToDoubleBiFunction< Double, Double >, Double > distanceFunction,
-			final ToDoubleBiFunction< Double, Double > costFunction )
+	SimilarityMeasure( final String name, final ToDoubleQuadFunction< Tree< Double >, Tree< Double >,
+			ToDoubleTriFunction< Double, Double, Double >, Double > distanceFunction,
+			final ToDoubleTriFunction< Double, Double, Double > costFunctionWithScale )
 	{
 		this.name = name;
 		this.distanceFunction = distanceFunction;
-		this.costFunction = costFunction;
+		this.costFunctionWithScale = costFunctionWithScale;
 	}
 
-	public static SimilarityMeasure getByName(final String name)
+	public static SimilarityMeasure getByName( final String name )
 	{
-		for (final SimilarityMeasure measure : values())
-			if (measure.getName().equals(name))
+		for ( final SimilarityMeasure measure : values() )
+			if ( measure.getName().equals( name ) )
 				return measure;
 
 		throw new NoSuchElementException();
 	}
 
-	public double compute( final Tree< Double > tree1, final Tree< Double > tree2 )
+	public double compute( final Tree< Double > tree1, final Tree< Double > tree2, final Double scale )
 	{
-		return distanceFunction.apply( tree1, tree2, costFunction );
+		return distanceFunction.applyAsDouble( tree1, tree2, costFunctionWithScale, scale );
 	}
 
 	public String getName()

src/main/java/org/mastodon/mamut/clustering/treesimilarity/TreeDistances.java

@@ -30,9 +30,9 @@
 
 import org.mastodon.mamut.clustering.treesimilarity.tree.Tree;
 import org.mastodon.mamut.clustering.treesimilarity.tree.TreeUtils;
+import org.mastodon.mamut.util.ToDoubleTriFunction;
 
 import javax.annotation.Nullable;
-import java.util.function.ToDoubleBiFunction;
 
 /**
  * Utility class for calculating distances between trees.
@@ -51,14 +51,16 @@ private TreeDistances()
 	 *
 	 * @see <a href="https://gitlab.inria.fr/mosaic/treex/-/blob/master/test/test_analysis/test_zhang_labeled_trees.py?ref_type=heads#L99">treex library</a>
 	 */
-	public static final ToDoubleBiFunction< Double, Double > LOCAL_ABSOLUTE_COST_FUNCTION = TreeDistances::localAbsoluteCostFunction;
+	public static final ToDoubleTriFunction< Double, Double, Double > LOCAL_ABSOLUTE_COST_FUNCTION =
+			TreeDistances::localAbsoluteCostFunction;
 
 	/**
 	 * Cost function as used in Guignard et al. 2020. It returns the normalized absolute difference between two attributes or 1 if one attribute is {@code null}.
 	 *
 	 * @see <a href="https://www.science.org/doi/suppl/10.1126/science.aar5663/suppl_file/aar5663_guignard_sm.pdf">Guignard et al. (2020) Page 38-39</a>
 	 */
-	public static final ToDoubleBiFunction< Double, Double > LOCAL_NORMALIZED_COST_FUNCTION = TreeDistances::localNormalizedCostFunction;
+	public static final ToDoubleTriFunction< Double, Double, Double > LOCAL_NORMALIZED_COST_FUNCTION =
+			TreeDistances::localNormalizedCostFunction;
 
 	/**
 	 * Calculates the normalized Zhang edit distance between two labeled unordered trees.
@@ -73,14 +75,14 @@ private TreeDistances()
 	 * @return The normalized Zhang edit distance between tree1 and tree2.
 	 */
 	public static < T > double normalizedDistance( @Nullable final Tree< T > tree1, final @Nullable Tree< T > tree2,
-			final ToDoubleBiFunction< T, T > costFunction )
+			final ToDoubleTriFunction< T, T, T > costFunction, final T scale )
 	{
-		double denominator = ZhangUnorderedTreeEditDistance.distance( tree1, null, costFunction )
-				+ ZhangUnorderedTreeEditDistance.distance( null, tree2, costFunction );
+		double denominator = ZhangUnorderedTreeEditDistance.distance( tree1, null, costFunction, scale )
+				+ ZhangUnorderedTreeEditDistance.distance( null, tree2, costFunction, scale );
 		// NB: avoid division by zero. Two empty trees are considered equal and also two trees with zero edit distance are considered equal.
 		if ( denominator == 0 )
 			return 0;
-		return ZhangUnorderedTreeEditDistance.distance( tree1, tree2, costFunction ) / denominator;
+		return ZhangUnorderedTreeEditDistance.distance( tree1, tree2, costFunction, scale ) / denominator;
 	}
 
 	/**
@@ -96,38 +98,55 @@ public static < T > double normalizedDistance( @Nullable final Tree< T > tree1,
 	 * @return The average Zhang edit distance between tree1 and tree2.
 	 */
 	public static < T > double averageDistance( @Nullable final Tree< T > tree1, final @Nullable Tree< T > tree2,
-			final ToDoubleBiFunction< T, T > costFunction )
+			final ToDoubleTriFunction< T, T, T > costFunction, final T scale )
 	{
 		double denominator = ( double ) TreeUtils.size( tree1 ) + ( double ) TreeUtils.size( tree2 );
 		// NB: avoid division by zero. Two empty trees are considered equal.
 		if ( denominator == 0 )
 			return 0;
-		return ZhangUnorderedTreeEditDistance.distance( tree1, tree2, costFunction ) / denominator;
+		return ZhangUnorderedTreeEditDistance.distance( tree1, tree2, costFunction, scale ) / denominator;
 	}
 
 	/**
 	 * @see <a href="https://gitlab.inria.fr/mosaic/treex/-/blob/master/test/test_analysis/test_zhang_labeled_trees.py?ref_type=heads#L99">treex library</a>
 	 */
-	private static Double localAbsoluteCostFunction( final Double o1, final Double o2 )
+	private static Double localAbsoluteCostFunction( final Double o1, final Double o2, final Double scale )
 	{
 		if ( o2 == null )
 			return o1;
 		else if ( o1 == null )
-			return o2;
+			return scale == 1d ? o2 : o2 * scale;
 		else
-			return Math.abs( o1 - o2 );
+		{
+			if ( scale == 1d )
+				return Math.abs( o1 - o2 );
+			else
+				return Math.abs( o1 - o2 * scale );
+		}
 	}
 
 	/**
 	 * @see <a href="https://www.science.org/doi/suppl/10.1126/science.aar5663/suppl_file/aar5663_guignard_sm.pdf">Guignard et al. (2020) Page 38</a>
 	 */
-	private static Double localNormalizedCostFunction( final Double o1, final Double o2 )
+	private static Double localNormalizedCostFunction( final Double o1, final Double o2, final Double scale )
 	{
 		if ( o1 == null || o2 == null )
 			return 1d;
-		else if ( o1.equals( o2 ) ) // NB: avoid non-required division and possible division by zero
-			return 0d;
+
+		if ( scale == 1d )
+		{
+			if ( o1.equals( o2 ) )
+				return 0d;
+			else
+				return Math.abs( o1 - o2 ) / ( o1 + o2 );
+		}
 		else
-			return Math.abs( o1 - o2 ) / ( o1 + o2 );
+		{
+			double scaleTimesO2 = o2 * scale;
+			if ( o1.equals( scaleTimesO2 ) )
+				return 0d;
+			else
+				return Math.abs( o1 - scaleTimesO2 ) / ( o1 + scaleTimesO2 );
+		}
 	}
 }