List of broken Data source tools

[B0r1sD]

Hi friends of the European instance!

We noticed that a lot of Data Source tools are not working correctly on the Belgian instance. As our instance is based on the European one, I also tested these tools at usegalaxy.eu where they don't seem to work either. The following list are all the Data Source tools found under our 'Get Data' section and seem to be broken on both .be and .eu (and some also on .org.au but only tested a few so far):

List of broken Data source tools

• Biomart and Biomart test server
  Case in point: XML not updated since 9 years: https://github.com/martenson/galaxy/blob/81cf234576bc1f52f93a11b193f6cc9c26130c28/tools/data_source/biomart.xml
• Gramene
• CBI Rice Mart
• EuPathDB
• Flymine
• HbVar
• GDCWebApp
• Get Microbial Data
• MetabolicMine
• modENCODE fly
• modENCODE modMine
• modENCODE worm
• NCBI Datasets Genome
• SRA
• Ratmine
• Wormbase
• YeastMine
• ZebrafishMine
• gdcwebapp

I believe the solution for some tools might just be a matter of updating the redirect URL, and others will require contacting the developers of the database. Some might be deprecated indefinitly, that's where I request your input and see where we can help out.

Context for newer Galaxy colleagues: Data Source tools are a special type of tools that redirect you to a database, which has an option to send the data back directly to the Galaxy history of the user. Read more about them here: https://docs.galaxyproject.org/en/latest/dev/data_source.html. Example of a broken tool (notice the URL suffix which includes the Galaxy instance from where you were redirected):

• on .be: https://ricemart.cbi.edu.cn/biomart/martview/?GALAXY_URL=https%3A//usegalaxy.be/tool_runner/biomart
• and on .eu: https://ricemart.cbi.edu.cn/biomart/martview/?GALAXY_URL=https%3A//usegalaxy.eu/tool_runner/biomart

Those type of tools are not installed via the toolshed but via an entry in the following file: https://github.com/usegalaxy-eu/infrastructure-playbook/blob/6f504b51e219b69e99eeca54cafb8c5f3df7a74d/templates/galaxy/config/tool_conf.xml.j2, which is the same with us.

.org does not make use of such data source tools, as they might of cleaned up such tools a long time ago when they started failing (they only have the UCSC Tablebrowser tool that works as described earlier), see galaxyproject/galaxy#5302 (comment) and especially galaxyproject/tools-iuc#1292. They are included by default in the general Galaxy codebase so might need to elevate there as well: https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/config/sample/tool_conf.xml.sample.

Let me know what you think!

All the best
Boris from https://usegalaxy.be

[mvdbeek]

Case in point: XML not updated since 9 years:

this is not an indication of it not working. The data source tools are extremely simple and usually don't need any updates, all that's happening is that the remote site is POSTing the location of the data Galaxy is supposed to fetch, there's very little that needs updating.

.org does not make use of such data source tools, as they might of cleaned up such tools a long time ago when they started failing

a couple of weeks ago, and I don't think we evaluated at all whether the data source tools are working.

The difficulty with these is of course finding how this is integrated on the remote side -- it might all work, but we don't know where the send to galaxy link is.

[B0r1sD]

this is not an indication of it not working. The data source tools are extremely simple and usually don't need any updates, all that's happening is that the remote site is POSTing the location of the data Galaxy is supposed to fetch, there's very little that needs updating.

In hindsight that is indeed a bad indicator to base a point on; thanks for clarifying!

a couple of weeks ago, and I don't think we evaluated at all whether the data source tools are working.

The difficulty with these is of course finding how this is integrated on the remote side -- it might all work, but we don't know where the send to galaxy link is.

As discussed in the community call, we (BE) will start by providing the list from above in order of priority, so we can start by going over the more important ones one-by-one to see if it's something on the Galaxy side or remote side that needs fixing.
I will be more than happy to make issues, fill in contact forms, and help technically where possible!

[B0r1sD]

The initial priority list for us seems to be quite simple:

Priority

• EBI SRA

For this tool, I will create an issue on the main Galaxy repo (as discussed here: galaxyproject/galaxy#18771)

Not priority

• Biomart
• Biomart test server
• Gramene
• CBI Rice Mart
• EuPathDB
• Flymine
• HbVar
• GDCWebApp
• Get Microbial Data
• MetabolicMine
• modENCODE fly
• modENCODE modMine
• modENCODE worm
• NCBI Datasets Genome
• Ratmine
• Wormbase
• YeastMine
• ZebrafishMine

[mvdbeek]

The EBI SRA wool works, it's just not super-obvious how to use it the first time, instructions here: galaxyproject/galaxy#18771 (comment).

I would discuss this with EBI, but it's not broken on the galaxy side.

[B0r1sD]

I commented on the older EBI SRA update PR to keep the information about that tool together, see galaxyproject/galaxy#18771 (comment) for my latest findings on that tool. Grateful for your input Marius!

[B0r1sD]

Added gdcwebapp as a data source tool not working. Cfr. galaxyproject/tools-iuc#7377.