Skip to content

Conversation

@SantaMcCloud
Copy link
Contributor

I check some mem between .ORG, .FR and .EU for some tools which are used in a WF.

After comapre them i got ask to change some mem or add some tools/rules to the shared db.

I copy metaSPAdes and Semibin from .EU : https://github.com/usegalaxy-eu/infrastructure-playbook/blob/59ae3e284aa1ea32ebb1d9f25f0119f5b3821879/files/galaxy/tpv/tools.yml

Binette, CheckM2 and CheckM lineage_wf where copied from .FR: https://gitlab.com/ifb-elixirfr/usegalaxy-fr/infrastructure/-/blob/preprod/production/files/total_perspective_vortex/tools.yml?ref_type=heads

@SantaMcCloud
Copy link
Contributor Author

hm strange error

@SantaMcCloud
Copy link
Contributor Author

SantaMcCloud commented Apr 30, 2025

Add a tool which failed in .ORG for a GTA. The job had the ID: bbd44e69cb8906b5db17c96263a18ec0

This are the metrics:

image

@SantaMcCloud
Copy link
Contributor Author

SantaMcCloud commented Apr 30, 2025

Another tool which need more memory in .ORG. ID: bbd44e69cb8906b58921b72541fe2018
Also failed in .AU. ID: a6e389a98c2d16785e9a4afe167c45ed
Both are for GTA

Metrics .ORG:

image

Metrics .AU:

image

tools.yml Outdated
mem: 250
cores: 2
rules:
- if: 0.05 <= input_size < 1
Copy link
Collaborator

@cat-bro cat-bro May 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These rules could be given ids, e.g.:

- id: metaspades_medium_input_rule
  if: 0.05 <= input_size < 1

That makes it easier to understand where a rule has come from downstream, and also to override a rule on a downstream galaxy

Copy link
Member

@nuwang nuwang May 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also prefix the rule id with tpvdb_?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be fine like this? 4026dbb

tools.yml Outdated
toolshed.g2.bx.psu.edu/repos/iuc/binette/binette/.*:
cores: 6
mem: 32
toolshed.g2.bx.psu.edu/repos/iuc/fastp/fastq/.*:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not look like it matches a tool ID. The shared db already has an entry for toolshed.g2.bx.psu.edu/repos/iuc/fastp/fastp/.*

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i had a typo there but yes you are right there was a rule strange i didnt found it at the beginning. 45309ff removed it here and change the entry to have at least 8 GB at the beginning instead of only 3.7

@nuwang
Copy link
Member

nuwang commented May 9, 2025

@SantaMcCloud Thanks for this. Could you also please run tpv format to fix the linting error?

@SantaMcCloud
Copy link
Contributor Author

@nuwang should be done!

@nuwang nuwang merged commit d9e3478 into galaxyproject:main May 15, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants