Description of feature
A common question a newcomer to a pipeline may ask is how much computing resources (memory/CPUs) a pipeline run will need (at least on default).
Currently this information is buried across multiple places: conf/base.config for values, modules/**/*.nf for labels etc, making it hard for a user to find this information.
I propose that we add a markdown table to pipeline docs that summarises this information:
- Name of module
- How much CPU/memory/time it is requested
- Where is the source of this definition (
withName? withLabel? defaults?)
This would make it easier for people to get an idea of what they would normally expect to need.
Technical considerations
This would need to be included in e.g. a linting action to be auto-updated, as I suspect it would be regularly forgotten to be updated when a user adds/tweaks a module.
The rules are quite straight forward to assess this based on Nextflow configuration order: withName > withLabel > default, so it is better to do this rather than have an AI LLM (which was slow when I did it once, and made a lot of mistakes - so wouldn't be worth the environmental impact).
The code that sweeps the code bases will need to be careful to also check for module aliases and report these in the table, not the 'installed' module itself.
Additional context
Description of feature
A common question a newcomer to a pipeline may ask is how much computing resources (memory/CPUs) a pipeline run will need (at least on default).
Currently this information is buried across multiple places:
conf/base.configfor values,modules/**/*.nffor labels etc, making it hard for a user to find this information.I propose that we add a markdown table to pipeline docs that summarises this information:
withName?withLabel? defaults?)This would make it easier for people to get an idea of what they would normally expect to need.
Technical considerations
This would need to be included in e.g. a linting action to be auto-updated, as I suspect it would be regularly forgotten to be updated when a user adds/tweaks a module.
The rules are quite straight forward to assess this based on Nextflow configuration order:
withName>withLabel>default, so it is better to do this rather than have an AI LLM (which was slow when I did it once, and made a lot of mistakes - so wouldn't be worth the environmental impact).The code that sweeps the code bases will need to be careful to also check for module aliases and report these in the table, not the 'installed' module itself.
Additional context