Skip to content

Commit 3bfe3bf

Browse files
Marie59bgruening
Marie59
and
bgruening
authored
Bgc ocean new odv collection manager (#155)
* Add ingestor tool * Add test data files * fix lint rm directory * fix test * fix test * Rm useless test data file * Improve id, name, description help * Add new tool on bgc ocean * Delete tools/ocean/bgc_harmonizer.xml * Delete tools/ocean/test-data/D6901758_001.nc * Delete tools/ocean/test-data/D6901758_002.nc * Delete tools/ocean/test-data/D6901758_003.nc * Delete tools/ocean/test-data/D6901758_004.nc * Delete tools/ocean/test-data/D6901758_005.nc * Create .shed.yml * rm useless if * Reopen PR with tool ocean * Create .shed.yml * Update tool_odv.xml * Update tool_odv.xml * change max file for test * version with configfile not working when no need of the config file * Update tools/ocean_data_view_manager/tool_odv.xml Co-authored-by: Björn Grüning <bjoern@gruenings.eu> * final update before test * Update tools/ocean_data_view_manager/tool_odv.xml Co-authored-by: Björn Grüning <bjoern@gruenings.eu> * Update tests * Update tool_odv.xml * Update .github/workflows/pr.yaml Co-authored-by: Björn Grüning <bjoern@gruenings.eu> * imrpove test * Delete tools/ocean_data_view_manager/test-data/data_raw.nc * Delete tools/ocean_data_view_manager/test-data/data_ref.nc * Update tools/ocean_data_view_manager/tool_odv.xml Co-authored-by: Björn Grüning <bjoern@gruenings.eu> * Update tools/ocean_data_view_manager/tool_odv.xml Co-authored-by: Björn Grüning <bjoern@gruenings.eu> * Update tools/ocean_data_view_manager/tool_odv.xml Co-authored-by: Björn Grüning <bjoern@gruenings.eu> * Update tools/ocean_data_view_manager/tool_odv.xml Co-authored-by: Björn Grüning <bjoern@gruenings.eu> --------- Co-authored-by: Björn Grüning <bjoern@gruenings.eu>
1 parent 22bb5ef commit 3bfe3bf

2 files changed

Lines changed: 200 additions & 0 deletions

File tree

‎tools/ocean_data_view_manager/.shed.yml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
categories:
2+
- Ecology
3+
owner: ecology
4+
remote_repository_url: https://github.com/galaxyecology/tools-ecology/tree/master/tools/ocean_data_view_manager
5+
homepage_url: https://github.com/Marie59/FE-ft-ESG/tree/main/ocean
6+
long_description: |
7+
The tool merges various datasets with a common vocabulary and creates a single generic ODV spreadsheet in an automatic way.
8+
type: unrestricted
9+
auto_tool_repositories:
10+
name_template: "{{ tool_id }}"
11+
description_template: "Wrapper for ocean biogechemical data tool: {{ tool_name }}."

‎tools/ocean_data_view_manager/tool_odv.xml‎

Lines changed: 189 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,189 @@
1+
<tool id="tool_odv" name="ODV collection manager" version="@VERSION@+galaxy@VERSION_SUFFIX@" profile="22.05" license="MIT">
2+
<description>tool for qualificiation and validation</description>
3+
<macros>
4+
<token name="@VERSION@">0.2</token>
5+
<token name="@VERSION_SUFFIX@">0</token>
6+
</macros>
7+
<requirements>
8+
<container type="docker">pokapok/qcv-odv-tool:@VERSION@</container>
9+
</requirements>
10+
<command detect_errors="exit_code"><![CDATA[
11+
export HOME=\$PWD &&
12+
mkdir -p /runtime/config &&
13+
mkdir -p '/runtime/data-in-raw/' &&
14+
mkdir -p '/runtime/data-in-ref/' &&
15+
mkdir -p '/runtime/data-out/' &&
16+
mkdir -p ./outputs &&
17+
18+
cp '$input_raw' '/runtime/data-in-raw/${input_raw.element_identifier}' &&
19+
#if $input_ref:
20+
cp '$input_ref' '/runtime/data-in-ref/${input_ref.element_identifier}' &&
21+
#end if
22+
23+
24+
#if $config.type == 'yes':
25+
## Write YAML content
26+
cp '${config_file}' '/runtime/config/config_user_odv_tool.yml' &&
27+
#end if
28+
29+
/app/launchers/start-app.sh '$mode' &&
30+
31+
#if $mode == 'R_APP_QV':
32+
cp '/runtime/data-out/odv_collection.txt' '$output' &&
33+
#else:
34+
cp '/runtime/data-out/demo_odv_collection.txt' '$output' &&
35+
#end if
36+
cp -r '/runtime/data-out/' './outputs/'
37+
]]></command>
38+
<configfiles>
39+
<configfile name="config_file">
40+
#if $config.type == 'yes':
41+
default:
42+
file_name_list_raw: '${input_raw.element_identifier}'
43+
operator: '${config.operator}'
44+
QV:
45+
file_name_list_ref: '${input_ref.element_identifier}'
46+
odv_collection:
47+
name: "odv_collection.txt"
48+
qc_convention: '${config.qc_convention}'
49+
tool_option:
50+
subsetting: '${config.subsetting}'
51+
plt: '${config.plt}'
52+
#end if
53+
</configfile>
54+
</configfiles>
55+
<inputs>
56+
<param name="input_raw" type="data" format="netcdf" label="Input raw data"/>
57+
<param name="input_ref" type="data" format="netcdf" optional="true" label="Input reference data"/>
58+
<param name="mode" type="select" label="Select in which mode you want to run the tool">
59+
<option value="R_APP_QV">QV that creates a odv collection with raw and ref files to qualify</option>
60+
<option value="DEMO">Valide with ODV demo that performes the demo</option>
61+
</param>
62+
<conditional name="config">
63+
<param name="type" type="select" label="Select if you want to write your own configuration file or not.">
64+
<option value="no">No, I don't want to, I'll use the default one</option>
65+
<option value="yes">Yes, I to write my own configuration file</option>
66+
</param>
67+
<when value="yes">
68+
<param name="operator" type="text" value="Anonymous" label="Enter operator name"/>
69+
<param name="qc_convention" type="select" label="Enter QC convention for the ODV collection output (default: ARGO)">
70+
<option value="ARGO">ARGO</option>
71+
<option value="ODV">ODV</option>
72+
<option value="GTSPP">GTSPP</option>
73+
<option value="SEADATANET">SEADATANET</option>
74+
<option value="ESEAS">ESEAS</option>
75+
<option value="WOD">WOD</option>
76+
<option value="WODSTATION">WODSTATION</option>
77+
<option value="WOCEBOTTLE">WOCEBOTTLE</option>
78+
<option value="WOCECTD">WOCECTD</option>
79+
<option value="QARTOD">QARTOD</option>
80+
<option value="BODC">BODC</option>
81+
<option value="PANGAEA">PANGAEA</option>
82+
<option value="SMHI">SMHI</option>
83+
<option value="OceanSITES">OceanSITES</option>
84+
<option value="IODE">IODE</option>
85+
</param>
86+
<param name="subsetting" type="float" min="-1" max="1" value="1" label="Enter subsetting (default: 1)"/>
87+
<param name="plt" type="float" min="0" max="1" value="0" label="Enter plt (default: 0 for QV, 1 for demo)"/>
88+
</when>
89+
<when value="no">
90+
</when>
91+
</conditional>
92+
</inputs>
93+
<outputs>
94+
<data name="output" format="txt" label="ODV tool output"/>
95+
<collection type="list" name="files" label="ODV tool collection">
96+
<discover_datasets pattern="(?P&lt;designation&gt;.+)" directory="./outputs" recurse="true" format="txt"/>
97+
</collection>
98+
</outputs>
99+
<tests>
100+
<test expect_num_outputs="2">
101+
<param name="input_raw" location="https://gitlab.com/pokapok-projects/easy-qcv-bgc/qcv-odv-tool/-/raw/7b3557c89438b7347c601d6959af354103b167a1/galaxy_tool/test-data/data_raw.nc"/>
102+
<param name="input_ref" location="https://gitlab.com/pokapok-projects/easy-qcv-bgc/qcv-odv-tool/-/raw/7b3557c89438b7347c601d6959af354103b167a1/galaxy_tool/test-data/data_ref.nc"/>
103+
<param name="mode" value="R_APP_QV"/>
104+
<conditional name="config">
105+
<param name="type" value="no"/>
106+
</conditional>
107+
<output name="output">
108+
<assert_contents>
109+
<has_n_lines n="26377"/>
110+
</assert_contents>
111+
</output>
112+
<output_collection name="files" type="list" count="4"/>
113+
</test>
114+
<test expect_num_outputs="2">
115+
<param name="input_raw" location="https://gitlab.com/pokapok-projects/easy-qcv-bgc/qcv-odv-tool/-/raw/7b3557c89438b7347c601d6959af354103b167a1/galaxy_tool/test-data/data_raw.nc"/>
116+
<param name="mode" value="R_APP_QV"/>
117+
<conditional name="config">
118+
<param name="type" value="yes"/>
119+
<param name="operator" value="anonymous"/>
120+
<param name="qc_convention" value="ARGO"/>
121+
<param name="subsetting" value="1"/>
122+
<param name="plt" value="0"/>
123+
</conditional>
124+
<output name="output">
125+
<assert_contents>
126+
<has_n_lines n="17966"/>
127+
</assert_contents>
128+
</output>
129+
<output_collection name="files" type="list" count="4"/>
130+
</test>
131+
<test expect_num_outputs="2">
132+
<param name="input_raw" location="https://gitlab.com/pokapok-projects/easy-qcv-bgc/qcv-odv-tool/-/raw/dev_virginie/galaxy_tool/test-data/data_raw.nc?ref_type=heads"/>
133+
<param name="mode" value="DEMO"/>
134+
<conditional name="config">
135+
<param name="type" value="no"/>
136+
</conditional>
137+
<output name="output">
138+
<assert_contents>
139+
<has_n_lines n="17966"/>
140+
</assert_contents>
141+
</output>
142+
<output_collection name="files" type="list" count="4"/>
143+
</test>
144+
</tests>
145+
<help><![CDATA[
146+
======================
147+
ODV collection manager
148+
======================
149+
150+
**What it does**
151+
152+
This tool present two type of actions :
153+
154+
- qualificiation & validation : the tool merges various datasets with a common vocabulary and creates a single generic ODV spreadsheet following ODV user’s guide (v5.7.0) in an automatic way.
155+
- history : the tool report in the input file the ODV history including the change of QC flag (not yet operational)
156+
157+
**Input description**
158+
159+
- file_name_raw : nc file to be qualified
160+
- file_name_ref : nc file used for validation / comparison. noted NULL if no files
161+
- odv_collection$qc_convention : tool makes mapping when its necessary and when global attribute qc_convention is available and filled in file_name_raw
162+
- tool_option$subsetting : 1 apply a subbsetting to ref data searching the smallest box (default) / 0 no subsetting / -1 apply an inverse subseting
163+
- tool_option$plt : figure option to see the subseted box (0 or 1) - default name subset_box.png
164+
165+
with default values different from NULL :
166+
167+
- odv_collection$name : odv_collection.txt
168+
- odv_collection$qc_convention : ARGO
169+
- tool_action :
170+
- tool_option$subsetting :
171+
- tool_option$plt :
172+
173+
If the list of raw file name is empty, tool stops If the user make an error in the file extension, tool changes it by itself. If the user miss the yaml configuration, tool uses its default config and upload file in data-in-raw and data-in-ref
174+
175+
**Output**
176+
177+
This tool creates a single generic ODV spreadsheet following ODV user’s guide
178+
179+
]]></help>
180+
<citations>
181+
<citation type="bibtex">
182+
@Manual{,
183+
title = {ODV tool},
184+
author = {Pokapok},
185+
year = {2024},
186+
note = {https://gitlab.com/pokapok-projects/easy-qcv-bgc/qcv-odv-tool}
187+
</citation>
188+
</citations>
189+
</tool>

0 commit comments

Comments
 (0)