Issues Exporting as MaMut File

[sandraoluoch]

Hello! I'm having trouble with the export options on Mastodon. My goal is to import tracking data into Mastodon, perform manual curation, and export the data with my changes either in .csv format or as an .xml file.

My workflow is as follows: I use the "Import MaMut" option to load tracking data into Mastodon. See below:

Once it's successfully imported, I manually curate some of the tracks. I'm able to save my changes as a Mastodon file, but I am struggling with export options. When I click on the "Export to MamuT file" option, I get the following error about missing .yaml files:

Is there any way to resolve this? Additionally, what would be the best way to export this data so I can have an .xml or .csv file with my updated changes? Kindly let me know.

Best,
Sandra

[tinevez]

Hello @sandraoluoch and thank you for the super clear issue.

I am not sure what is causing the error. Maybe it is linked to the features that were important from the MaMuT file and that we are trying to resave. Would you be ok to share the mastodon file with me so that I can investigate?

[sandraoluoch]

Hi @tinevez thank you for the quick response! It won't let me upload the file here directly, so I created this DropBox link that has the Mastodon file and the .csv file that has the tracking information before manual curation was done. The features I imported were x, y and z centroid coordinates, timepoint and track ID. Let me know what you think!

https://www.dropbox.com/scl/fo/wa7d3lnzsafjufo59cqb1/AGEZbcJ_H4Cg3BECV0nvxlo?rlkey=shqzky2x6arctec8cr84xm4ky&st=82kfl1rt&dl=0

[tinevez]

Thanks got it!

[tinevez]

And I can reproduce the bug:

selectedFile = downsampled_2x_fullmovie_bdv-mamut.xml
Exception in thread "AWT-EventQueue-0" org.jdom2.IllegalAddException: Cannot add duplicate attribute
	at org.jdom2.AttributeList.add(AttributeList.java:192)
	at org.jdom2.AttributeList.addAll(AttributeList.java:274)
	at org.jdom2.AttributeList.clearAndSet(AttributeList.java:336)
	at org.jdom2.Element.setAttributes(Element.java:1277)
	at org.mastodon.mamut.io.importer.trackmate.MamutExporter.spotToXml(MamutExporter.java:592)

I will need a couple of days to investigate fix it. Thanks for your patience.

[sandraoluoch]

Thank you so much @tinevez for taking the time to work on this! Let me know if there is any additional information I can provide.

Best,
Sandra

[tinevez]

Dear @sandraoluoch

Can you test the tentative fix I made?

Please download the following file: https://dl.pasteur.fr/fop/XxceSpvp/mastodon-1.0.0-beta-35-SNAPSHOT.jar
and put it in the jars/ folder of your Fiji installation. You will have to remove the existing mastodon-1.0.0-beta-34 for this to work properly.
I tested it on your data and could export to MaMuT. Opening the file in MaMuT, I received warnings about the MANUAL_COLOR features, probably because Mastodon did not import them properly, and could not therefore export them properly as well.
Tell me if this is an issue.

[sandraoluoch]

Hi @tinevez ,

I followed your instructions of downloading the file and placing it in the jars/ folder. I didn't have a mastodon-1.0.0-beta-34 file, but I had a mastodon-1.0.0-beta-32 file which I removed. However, now when I try to load the Mastodon file, I get the following error:

Additionally, I'm now not able to import original Mamut file like I was before. For context, I use Trackmate to import csv features onto the raw image, then I save out the image as an .xml file, convert it to MaMut, and open it in Mastodon. After doing some curation, I save the file as a Mastodon file. In terms of troubleshooting, what could I do? Perhaps delete and reinstall Mastodon? Weird it's suddenly throwing this error even for importing. Let me know,

Best,
Sandra

[tinevez]

Hello.

Yep this is the error you get with a Fiji that is not up to date. Can you update it and make sure it says 'up to date', then retry copying the jar file?

[sandraoluoch]

Good morning,

I am able to open and export the Mastodon file as an .xml file following your instructions. Thank you!

However, I am now having trouble opening/displaying the raw image with my tracking data... When I attempt to open the Mastodon file, I am shown this error that it can't find the raw image. It then asks if I want to open it in Dummy Images:

(the error window)

(the console showing it can't find the file)

(screenshot of my folder containing the raw image file (selected). This is the same filepath listed in the mamut file I originally import for curation)

Once opening the Mastodon file via Dummy Images, I'm able to see the tracking data with my changes. I'm able to export as an .xml file with no issues otherwise:

(the Mastodon file opened using Dummy Images)

But I go into the "Fix File Path" section as the error window suggested, and I see that the current filepath to the raw image is the correct one already:

(what the current filepath is when I click on the "Fix Image Path". This is the filepath to the raw BDV image)

I'm wondering if you faced the same issue? Out of curiosity, I went back and tried to import the mamut file like I did before, and now I get an error that it can't open. It's almost like the file is now corrupted and won't open:

(mamut file I am trying to import)

(the console error)

I also try and open the .bdv file in Fiji and get this error:

(the BDV file I'm trying to open)

(the Fiji error)

I hope this makes sense. Do you have any idea why I can no longer open the raw image and why Mastodon and Fiji can't locate/read it all of a sudden? Kindly let me know. While I wait for your response, I'm going to try resaving the raw image and seeing if this new, resaved version works.

Thanks so much,
Sandra

[sandraoluoch]

Update: I think the issue is I'm not able to open the .bdv file. I export as normal and no errors pop up, but now I can't open the .bdv file. The files exist, despite it saying the can't be found...

This is the error I get when opening the .bdv file on Fiji. Again, it can't seem to locate it even when I drag and drop the file:

And opening in Mamut file Mastodon that contains filepath to .bdv file:

[tinevez]

It seems like it cannot find the path to the BDV file. Maybe it's not a Mastodon issue.

I would open the downsampled_2x_fullmovie_bdv.xml file with a text editor, and fix the path to the image. Apparently what is entered in the base path is faulty. Can you correct it so that it is a proper path to the image?

[sandraoluoch]

Good morning @tinevez

Still working to resolve this issue... How do I go about fixing the base path?

I've noticed that now whenever I try to open any .xml file (files that used to open with no issues), I get the same error as above. That base path never changes, even when I try to open other .xml files from other locations. The current base path is \\allen\aics\assay-dev\users\Sandi\emt-tracking\emt-data-010725\raw\3500006851\B3_P1\movie where all the .xml, .h5 and .mastodon files are. So right now the base path is just a folder. Maybe it's throwing an error because it's not an .xml file path?

As for changing the filepath to the raw image, I opened downsampled_2x_fullmovie_bdv.xml in VSCode but there is nowhere to add/edit a filepath. However, the mamut.xml file (downsampled_2x_fullmovie_bdv-mamut.xml) has a section for the filepath which I edit before importing. I have it set the .bdv file and I was able to open it in Mastodon until yesterday, but Ihaven't made any changes so that's why I'm suspecting the issue is either with Mastodon or BigDataViewer? Let me know what you think.