3 develop testing guidelines - #5
Closed
emi728 wants to merge 9 commits into
Closed
Conversation
- Generated with nf-core tools v4.0.2 - Includes CI workflows, linting config, base/test profiles - Adds community boilerplate (LICENSE, CODE_OF_CONDUCT, CITATIONS) - Includes nf-core shared subworkflows and modules
- column_standardise: map CSV headers to ABCD schema terms (WP6) - biodiv_annotate: wrap BiodivPortal Annotator REST API (WP2) - taxonomy_classify: resolve taxon names to GBIF/WFO IDs (WP3) - outlier_detect: statistical + LLM quality detection (WP4) - provisional_concepts: register unresolved terms (WP5) - rdf_transform: tabular data to RDF triples (WP7) - collect_reports: aggregate per-step quality reports All modules are stub implementations ready for WP teams to replace.
- main.nf: entry point accepting --input CSV and --mapping_schema - workflows/biodivpipeline.nf: orchestrates all 7 modules - Step 1: Column standardisation (WP6) - Step 2: Parallel annotation (WP2) + taxonomy (WP3) + outlier detection (WP4) - Step 3: Provisional concept registration (WP5, after WP2) - Step 4: RDF transformation (WP7, combines all upstream outputs) - Step 5: Report collection - nextflow.config: add input/mapping_schema params - nextflow_schema.json: update input parameter validation for CSV data
- Remove FASTQC/MultiQC module references from workflow - Replace samplesheet parsing with direct CSV file channel - Remove genome/igenomes/fasta parameter references - Update PIPELINE_INITIALISATION to validate CSV input - Replace validateInputSamplesheet with validateInputParameters - Update modules.config: per-module publishDir for our 7 modules - Update test.config: point to test_data/sample.csv
- test_data/sample.csv: 109 records sampled from BELEGE_aus_D (every 1000th row) - modules/stubs/: FastAPI mock service for end-to-end testing - Dockerfile + stub_service.py (POST /process, GET /health) - assets/default_mapping.jsonld: stub DwC RDF mapping schema - Maps BGBM CSV columns to Darwin Core terms Pipeline runs end-to-end with: nextflow run main.nf -profile test
Replace nf-core genomics boilerplate with project-specific content: - README.md: project summary, Mermaid diagrams, quick start, output tree - docs/usage.md: CSV input format, parameters, sequence diagram - docs/output.md: all 18 output files across 7 directories - docs/CONTRIBUTING.md: branch model, conventional commits, module interface - docs/README.md: documentation index with tech stack table Remove unused template files: - modules/nf-core/fastqc, modules/nf-core/multiqc (genomics QC) - assets: samplesheet.csv, schema_input.json, multiqc_config.yml, logos - conf: igenomes configs, container lock files, test_full.config - tower.yml, methods_description_template.yml Clean code references: - nextflow.config: remove igenomes include, test_full profile - modules.json: remove fastqc/multiqc entries - main.nf, subworkflow: remove multiqc_report channel - subworkflow: remove samplesheetToList import, FastQC/MultiQC citations
WP1 owns the integration backbone, not module internals. Strip stub scripts to minimal valid-output emitters and remove pre-claimed domain content: - Delete modules/stubs/ (FastAPI HTTP service contradicts Pattern A) - Replace 6 module main.nf stubs with minimal emitters - Strip default_mapping.jsonld to empty placeholder - Replace biodiv/wpX-*:latest container refs with placeholder image - Remove WP-to-module assignments from workflow comments - Drop BGBM example from input description Module teams (WP2-WP7) own real implementations, container images, and domain content. WP1 ships interface contracts.
- README.md: remove module diagram, dataset claims, project-structure module descriptions, credits section - docs/output.md: reduce to directory layout + Nextflow pipeline_info - docs/README.md: shrink to TOC + tech stack - docs/usage.md: remove dataset specifics, column listing, unzip step, sequence diagram - docs/CONTRIBUTING.md: remove project contacts, dataset record count - nextflow.config: empty manifest.contributors placeholder - .nf-core.yml: clear author placeholder - CITATIONS.md: drop tools not used in this pipeline
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.