Biaftp update by pavanvidem · Pull Request #1582 · bgruening/galaxytools

pavanvidem · 2025-02-26T19:47:56Z

make mode param as select
put output images into a collection

bgruening · 2025-02-26T22:15:33Z

.. ERROR (XSD): Invalid XML: Element 'discovered_dataset': This element is not expected. Expected is ( element ).
.. ERROR (XSD): Invalid XML: Element 'discovered_dataset': This element is not expected. Expected is ( element ).
.. CHECK (TestsNoValid): 2 test(s) found.
.. ERROR (TestsOutputCollectionCheckDiscovered): Test 1: test collection 'images' must have a 'count' attribute or 'element' children
.. ERROR (TestsOutputCollectionCheckDiscovered): Test 2: test collection 'images' must have a 'count' attribute or 'element' children
.. INFO (OutputsNumber): 2 outputs found.
.. INFO (InputsNum): Found 3 input parameters.
.. CHECK (HelpPresent): Tool contains help section.
.. CHECK (HelpValidRST): Help contains valid reStructuredText.
.. CHECK (ToolIDValid): Tool defines an id [bia_download].
.. CHECK (ToolNameValid): Tool defines a name [FTP Link for Bioimage Archive].
.. CHECK (ToolProfileValid): Tool specifies profile version [22.05].
.. CHECK (ToolVersionValid): Tool defines a version [0.1.0+galaxy1].
.. INFO (CommandInfo): Tool contains a command.
.. CHECK (CitationsFound): Found 1 citations.
.. WARNING: Missing shed metadata field [long_description] for repository
.. WARNING: Missing shed metadata field [homepage_url] for repository

pavanvidem · 2025-02-27T10:36:33Z

Currently, this tool can download from BioImages-core. Let me check if we can download from other collections.

pavanvidem · 2025-02-28T17:15:43Z

I don't find what the error is :(

bgruening · 2025-02-28T17:36:13Z

error code of one ...

Try to not use the pipe and see where it is crashing. In the curl, or somewhere else...

bgruening · 2025-03-05T22:24:57Z

tools/image_processing/bia-ftplinks/biaftplink.xml

-            curl https://www.ebi.ac.uk/biostudies/api/v1/studies/$study/info -s |jq -r .ftpLink >>ftpLink.txt
-        #end if
+    curl -s https://www.ebi.ac.uk/biostudies/api/v1/studies/$accession/info | jq -r .ftpLink >> ftpLink.txt &&
+    wget -q -r -l 0 -i ftpLink.txt &&


How do we make sure that we only get tif files?

We get the whole ftp directory of that accession.

There is no particular directory structure. So I download everything and filter for tif later.

Can you not filter this file first, instead of downloading everything and throwing that away later?

bgruening · 2025-03-05T22:25:59Z

tools/image_processing/bia-ftplinks/biaftplink.xml

-        </data>
-        <data format="txt" name="ftplinks" from_work_dir="ftpLink.txt" label="FTP Links">
+        <collection name="images" type="list" label="${tool.name} on ${on_string}: TIFF Images">
+            <discover_datasets pattern="(?P&lt;designation&gt;.*)\.tif" format="tif" directory="ftp.ebi.ac.uk" recurse="true" />


Do you know if the format is tiff or ome.tiff ?

In the UI filtering options, it shows only tif as an option. I kept as it is.

Btw, there are also several other image formats like lif, lsm, mrc and so on. I skipped them because there is no Galaxy datatype.

We can change the UI. The question is if the downloaded files are proper OME.tiff files. https://github.com/galaxyproject/galaxy/blob/5c686c62ecb5c23fa31ebae30db4f5a56c63630a/lib/galaxy/datatypes/images.py#L309

how to check that?

https://github.com/galaxyproject/galaxy/blob/5c686c62ecb5c23fa31ebae30db4f5a56c63630a/lib/galaxy/datatypes/images.py#L313

bgruening · 2025-03-07T10:50:23Z

tools/image_processing/bia-ftplinks/biaftplink.xml

-        <collection name="images" type="list" label="${tool.name} on ${on_string}: TIFF Images">
+        <collection name="images" type="list" label="${tool.name} on ${on_string}: Images">
            <discover_datasets pattern="(?P&lt;designation&gt;.*)\.tif" format="tif" directory="ftp.ebi.ac.uk" recurse="true" />
+            <discover_datasets pattern="(?P&lt;designation&gt;.*)\.png" format="png" directory="ftp.ebi.ac.uk" recurse="true" />


should this 3 different collections? what is the usecase? Or should the users then filter the collection afterwards according to filetypes?

in most cases, there is only one type of images. But it is better to create different collections.

tools/image_processing/bia-ftplinks/biaftplink.xml

Co-authored-by: Björn Grüning <bjoern@gruenings.eu>

pavanvidem added 3 commits February 26, 2025 19:52

update BIA download tool

8a0dedd

hide collection files

3d4ad0c

add label to collection

bba9624

pavanvidem added 2 commits February 27, 2025 10:07

fix test

bdb1fda

fix .shed.yml

596e9ae

bgruening approved these changes Feb 27, 2025

View reviewed changes

pavanvidem marked this pull request as draft February 28, 2025 08:41

pavanvidem added 2 commits February 28, 2025 16:50

Remove mode param. Extract ftplink from the API

c0d496d

replace ext with format

d49e029

fix test by overwriting additional license/readme files

5a6c318

pavanvidem marked this pull request as ready for review March 3, 2025 15:57

use only .tif format

a0ea238

bgruening reviewed Mar 5, 2025

View reviewed changes

add png and jpg support

2997d09

bgruening reviewed Mar 7, 2025

View reviewed changes

tools/image_processing/bia-ftplinks/biaftplink.xml Outdated Show resolved Hide resolved

pavanvidem and others added 4 commits March 7, 2025 11:58

Update tools/image_processing/bia-ftplinks/biaftplink.xml

66d4a74

Co-authored-by: Björn Grüning <bjoern@gruenings.eu>

download only .tif,.png,.jpg and zip files

e1641e1

separate collection for each format

39337d3

fix expect_num_outputs

7fb18de

pavanvidem merged commit 2723029 into bgruening:master Mar 7, 2025
11 checks passed

pavanvidem deleted the biaftp-update branch March 7, 2025 14:39

pavanvidem mentioned this pull request Oct 20, 2025

[BH2024] Add EDAM annotations for Bioimaging Archive retrieval tool #1541

Merged

kostrykin mentioned this pull request Oct 29, 2025

Refactor the BIA retrieval tool FAIR-imaging/galaxy-image-community#11

Closed

7 tasks

Conversation

pavanvidem commented Feb 26, 2025

Uh oh!

bgruening commented Feb 26, 2025

Uh oh!

pavanvidem commented Feb 27, 2025

Uh oh!

pavanvidem commented Feb 28, 2025

Uh oh!

bgruening commented Feb 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants