fix: check out correct Loculus commit to deploy pinned version (#214)

* deploy locally

* Add helm role

* feat(loculus): automate Loculus repo checkout and improve deployment

- Add git checkout task to clone Loculus at specific commit SHA
- Move from shell-based helm to kubernetes.core.helm module
- Pin Loculus version in group_vars (loculus_commit_sha: 4948577ed)
- Add loculus_repo_path and loculus_helm_chart_path to role defaults
- Remove hardcoded helm_chart_path from host_vars
- Enable proper Helm values file merging (base + custom)

This makes Loculus deployment fully automated and version-controlled,
following the pattern documented in LOCULUS_DEPLOYMENT_GUIDE.md.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* use root to checkout loculus

* do not print the helm output, it's not useful

* feat: add kubernetes.core collection to requirements

Add kubernetes.core collection (>=2.3.0) to requirements.yml to support
the kubernetes.core.helm module used in the Loculus deployment role.

This fixes the CI linting error:
  Error loading plugin 'kubernetes.core.helm': No module named
  'ansible_collections.kubernetes'

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat(loculus): add Helm installation to deployment role

Install Helm using gantsign.helm role before deploying Loculus.
This ensures Helm is available for the kubernetes.core.helm module.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(ci): install Ansible collections and roles before syntax check

Add steps to install collections and roles from requirements.yml before
running syntax checks. This fixes the linting error:
  Error loading plugin 'kubernetes.core.helm': No module named
  'ansible_collections.kubernetes'

The CI was only installing ansible-core but not the required collections
(kubernetes.core, community.general, etc.) or roles (gantsign.helm).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

Author: fhennig

.github/workflows/ansible-quality.yml

@@ -15,7 +15,13 @@ jobs:
 
       - name: Install Ansible
         run: pip install ansible-core
-      
+
+      - name: Install Ansible collections
+        run: ansible-galaxy collection install -r requirements.yml
+
+      - name: Install Ansible roles
+        run: ansible-galaxy role install -r requirements.yml
+
       - name: Setup CI environment
         run: |
           # Temporarily remove vault files to avoid decryption attempts

docs/configuration/reference.md

@@ -57,15 +57,6 @@ srsilo_viruses:
 
 ## Loculus Configuration
 
-### Required Variables
-
-Set in `host_vars/` or `group_vars/`:
-
-| Variable | Description |
-|----------|-------------|
-| `kubectl_context` | kubectl context name |
-| `helm_chart_path` | Path to Loculus Helm chart |
-
 ### Application Settings
 
 In `group_vars/loculus/main.yml`:

docs/deployment/loculus.md

@@ -20,13 +20,6 @@ Ansible role for deploying Loculus (pathogen sequence sharing platform) to Kuber
 | `loculus_cleanup_temp_files` | Clean up temp files after deployment (default: `true`) |
 | `loculus_kubeconfig_path` | Path to kubeconfig file |
 
-**Required** (set in `host_vars/` or `group_vars/`):
-
-| Variable | Description |
-|----------|-------------|
-| `kubectl_context` | kubectl context name |
-| `helm_chart_path` | Path to Loculus Helm chart |
-
 **Configuration** (`group_vars/loculus/main.yml`):
 
 - Application settings (name, host, organisms, URLs, etc.)

group_vars/loculus/main.yml

@@ -1,6 +1,9 @@
 name: 'W-ASAP'
 host: 'db.wasap.genspectrum.org'
 
+# Pin the Loculus version
+loculus_commit_sha: 4948577ed8a054341e155b5c6f593996290a3320
+
 logo:
   url: 'https://db.wasap.genspectrum.org/static/images/wasap-logo.png'
   width: 200
@@ -111,10 +114,7 @@ organisms:
         args:
           - "--watch"
           - "--disableConsensusSequences"
-    referenceGenomes:
-      singleReference:
-        nucleotideSequences: []
-        genes: []
+    referenceGenomes: []
    rsva:
     schema:
       organismName: 'RSV-A'
@@ -187,10 +187,7 @@ organisms:
         args:
           - "--watch"
           - "--disableConsensusSequences"
-    referenceGenomes:
-      singleReference:
-        nucleotideSequences: []
-        genes: []
+    referenceGenomes: []
 
 dataUseTerms:
   enabled: false

host_vars/localhost/main.yml

@@ -5,8 +5,6 @@
 # Loculus Deployment Configuration
 # ============================================================================
 host: 'db.wasap.genspectrum.org'
-kubectl_context: 'k3d-loculusCluster'
-helm_chart_path: '/opt/loculus/loculus/kubernetes/loculus'
 cleanup_temp_files: true
 
 # ============================================================================

requirements.yml

@@ -11,3 +11,9 @@ collections:
     version: ">=8.0.0"
   - name: ansible.posix
     version: ">=1.5.0"
+  - name: kubernetes.core
+    version: ">=2.3.0"
+
+roles:
+  - src: gantsign.helm
+    version: "2.13.0"

roles/loculus/defaults/main.yml

@@ -13,6 +13,13 @@ loculus_cleanup_temp_files: true
 # Note: Run playbook without sudo (use --ask-become-pass instead)
 loculus_kubeconfig_path: "{{ lookup('env', 'HOME') }}/.kube/config"
 
+# Loculus repository configuration
+loculus_repo_path: "/opt/loculus/loculus"
+loculus_commit_sha: "main"  # Default, should be overridden in group_vars
+
+# Derived path to Helm chart (subdirectory within the repo)
+loculus_helm_chart_path: "{{ loculus_repo_path }}/kubernetes/loculus"
+
 # Static images to deploy
 # Add new virus images here as needed
 virus_images:

roles/loculus/tasks/main.yml

@@ -1,10 +1,4 @@
 ---
-- name: Ensure temp directory exists
-  file:
-    path: "{{ loculus_temp_dir }}"
-    state: directory
-    mode: '0750'
-
 # TODO: Add nginx configuration automation for static file serving in future
 # This should include adding location /static/ block to nginx config
 
@@ -37,29 +31,53 @@
   become: yes
   loop: "{{ static_images }}"
 
+- name: Install Helm
+  ansible.builtin.include_role:
+    name: gantsign.helm
+
+# Loculus deployment
+- name: Ensure temp directory exists
+  file:
+    path: "{{ loculus_temp_dir }}"
+    state: directory
+    mode: '0750'
+
+- name: Ensure loculus repository parent directory exists
+  file:
+    path: "{{ loculus_repo_path | dirname }}"
+    state: directory
+    mode: '0755'
+
+- name: Checkout Loculus Repository at specific commit
+  ansible.builtin.git:
+    repo: 'https://github.com/loculus-project/loculus.git'
+    dest: "{{ loculus_repo_path }}"
+    version: "{{ loculus_commit_sha }}"
+    update: true
+    force: true
+  register: loculus_checkout
+  become: yes
+
+- name: Display checked out commit
+  debug:
+    msg: "Loculus checked out at commit {{ loculus_commit_sha }}"
+
 - name: Generate values file from template
   template:
     src: values.yaml.j2
     dest: "{{ loculus_temp_dir }}/values.yaml"
     mode: '0640'
 
-- name: Verify kubectl context is available
-  shell: kubectl config get-contexts {{ kubectl_context }}
-  register: kubectl_context_check
-  environment:
-    KUBECONFIG: "{{ loculus_kubeconfig_path }}"
-
-- name: Apply Helm chart with values to loculus cluster
-  shell: |
-    kubectl config use-context {{ kubectl_context }}
-    helm upgrade --install my-loculus {{ helm_chart_path }} -f {{ loculus_temp_dir }}/values.yaml
-  register: helm_output
-  environment:
-    KUBECONFIG: "{{ loculus_kubeconfig_path }}"
-
-- name: Display Helm output
-  debug:
-    var: helm_output.stdout_lines
+- name: Deploy Loculus via Helm
+  kubernetes.core.helm:
+    release_name: my-loculus
+    release_namespace: loculus
+    create_namespace: true
+    chart_ref: "{{ loculus_helm_chart_path }}"
+    values_files:
+      - "{{ loculus_helm_chart_path }}/values.yaml"
+      - "{{ loculus_temp_dir }}/values.yaml"
+    kubeconfig: "{{ loculus_kubeconfig_path }}"
 
 - name: Clean up temporary files
   file:

roles/loculus/templates/values.yaml.j2

@@ -1,5 +1,8 @@
 name: '{{ name }}'
 host: '{{ host }}'
+{% if loculus_commit_sha is defined %}
+sha: '{{ loculus_commit_sha[:7] }}'
+{% endif %}
 environment: '{{ loculus_environment }}' # Do not change this: this setup uses a "local" environment as the public traffic is proxied through nginx.
 
 runDevelopmentMainDatabase: {{ runDevelopmentMainDatabase | lower }}
@@ -138,13 +141,11 @@ organisms:
 {% endfor %}
 {% endfor %}
     referenceGenomes:
-      singleReference:
-        nucleotideSequences: {{ organism_config.referenceGenomes.singleReference.nucleotideSequences }}
-        genes: {{ organism_config.referenceGenomes.singleReference.genes }}
+      {{ organism_config.referenceGenomes }}
 {% endfor %}
 
 dataUseTerms:
   enabled: {{ dataUseTerms.enabled | lower }}
 disableIngest: {{ disableIngest | lower }}
 disableEnaSubmission: {{ disableEnaSubmission | lower }}
-accessionPrefix: "{{ accessionPrefix }}"
+accessionPrefix: "{{ accessionPrefix }}"