Skip to content

Commit dd77ed8

Browse files
authored
Merge pull request #3 from 33Fraise33/fix/noninteractive-ci-validation
feat: Improve github CI and linting
2 parents d45fd3e + 26f8e0d commit dd77ed8

10 files changed

Lines changed: 233 additions & 34 deletions

File tree

.github/workflows/ansible.yml

Lines changed: 55 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,79 @@
11
---
2+
# yaml-language-server: $schema=https://json.schemastore.org/github-workflow.json
23
name: Ansible validation
34

45
on:
5-
pull_request: true
6+
pull_request: {}
67
push:
78
branches:
89
- main
910

11+
permissions:
12+
contents: read
13+
1014
jobs:
1115
validate:
1216
runs-on: ubuntu-latest
1317
steps:
14-
- uses: actions/checkout@v4
15-
- uses: actions/setup-python@v5
18+
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
19+
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
1620
with:
17-
python-version: '3.x'
21+
python-version: '3.13'
1822
- name: Install Ansible tooling
19-
run: python -m pip install --requirement requirements.txt molecule molecule-docker
23+
run: python -m pip install --requirement requirements.txt
2024
- name: Install Ansible collections
2125
run: ansible-galaxy collection install --requirements-file requirements.yml
2226
- name: Lint YAML
2327
run: yamllint .
2428
- name: Lint Ansible
2529
run: ansible-lint
26-
- name: Syntax check Dawarich playbook
27-
run: ansible-playbook --syntax-check playbooks/dawarich.yml
30+
- name: Syntax check playbooks
31+
shell: bash
32+
run: |
33+
set -euo pipefail
34+
shopt -s nullglob
35+
playbooks=(playbooks/*.yml playbooks/*.yaml)
36+
if (( ${#playbooks[@]} == 0 )); then
37+
echo "No playbooks found." >&2
38+
exit 1
39+
fi
40+
for playbook in "${playbooks[@]}"; do
41+
ansible-playbook --syntax-check --inventory 'localhost,' "$playbook"
42+
done
2843
- name: Test Dawarich role
2944
env:
3045
ANSIBLE_ALLOW_BROKEN_CONDITIONALS: 'true'
3146
run: molecule test --scenario-name dawarich
47+
48+
validate-vault:
49+
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
50+
needs: validate
51+
runs-on: ubuntu-latest
52+
environment: ansible-validation
53+
steps:
54+
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
55+
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
56+
with:
57+
python-version: '3.13'
58+
- name: Install Ansible tooling
59+
run: python -m pip install --requirement requirements.txt
60+
- name: Create temporary Vault password file
61+
shell: bash
62+
env:
63+
ANSIBLE_VAULT_PASSWORD: ${{ secrets.ANSIBLE_VAULT_PASSWORD }}
64+
run: |
65+
set -euo pipefail
66+
if [[ -z "$ANSIBLE_VAULT_PASSWORD" ]]; then
67+
echo "ANSIBLE_VAULT_PASSWORD is not configured for the ansible-validation environment." >&2
68+
exit 1
69+
fi
70+
umask 077
71+
printf '%s' "$ANSIBLE_VAULT_PASSWORD" > "$RUNNER_TEMP/ansible-vault-password"
72+
- name: Validate Vault decryption
73+
env:
74+
ANSIBLE_VAULT_PASSWORD_FILE: ${{ runner.temp }}/ansible-vault-password
75+
run: python scripts/validate_vault.py
76+
- name: Remove temporary Vault password file
77+
if: always()
78+
shell: bash
79+
run: rm -f "$RUNNER_TEMP/ansible-vault-password"
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
---
2+
# yaml-language-server: $schema=https://json.schemastore.org/github-workflow.json
3+
name: GitHub workflow validation
4+
5+
on:
6+
pull_request:
7+
paths:
8+
- .github/workflows/**
9+
push:
10+
branches:
11+
- main
12+
paths:
13+
- .github/workflows/**
14+
15+
permissions:
16+
contents: read
17+
18+
jobs:
19+
actionlint:
20+
runs-on: ubuntu-latest
21+
steps:
22+
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
23+
- name: Validate GitHub Actions workflows
24+
uses: docker://rhysd/actionlint@sha256:ef8299f97635c4c30e2298f48f30763ab782a4ad2c95b744649439a039421e36
25+
with:
26+
args: -color

.pre-commit-config.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
repos:
3+
- repo: https://github.com/rhysd/actionlint
4+
rev: v1.7.10
5+
hooks:
6+
- id: actionlint

AGENTS.md

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,23 +3,23 @@
33
## Persona And Workflow
44

55
- Act as an expert DevOps engineer focused on Ansible roles that deploy and manage Docker containers. Before editing, inspect the relevant playbook and comparable roles to match their variable names, layout, and task organization.
6-
- Never commit directly to `main` or `master`. Branch from `master` using descriptive names such as `feature/add-nginx-role` or `fix/db-container-restart`; use Conventional Commits and submit all changes through a pull or merge request with a concise review summary.
6+
- Never commit directly to `main`. Branch from `main` using descriptive names such as `feature/add-nginx-role` or `fix/db-container-restart`; use Conventional Commits and submit all changes through a pull or merge request with a concise review summary.
77

88
## Automation Boundary
99

1010
- Never execute code against the live environment. Do not run `ansible-playbook` with the external inventory, including `--check`, and do not use credentials or SSH to access managed hosts.
1111
- Write and modify roles, playbooks, CI pipelines, and tests only. Live execution and deployment are the operator's responsibility.
12-
- Run Ansible tooling from the repository root when performing local-only validation: `ansible.cfg` resolves `roles_path = ./roles` and `vault_password_file = ./vault_pass.sh` relatively.
12+
- Run Ansible tooling from the repository root when performing local-only validation: `ansible.cfg` resolves `roles_path = ./roles` relatively. Lint, syntax checks, and Molecule must not require Vault decryption.
1313
- Inventory is external: `../netbox-ansible/netbox.yml`; this repository does not contain a local inventory.
14-
- The vault password script requires the 1Password CLI and the `Ansible Vault Password` item. Do not retrieve it. Keep existing `!vault` values encrypted; create new values with `ansible-vault encrypt_string` only when the operator provides the required vault access.
14+
- The vault password script requires the 1Password CLI and the `Ansible Vault Password` item. Operators must explicitly pass `--vault-password-file ./vault_pass.sh` when decrypting or modifying Vault values. Do not retrieve it. Keep existing `!vault` values encrypted; create new values with `ansible-vault encrypt_string` only when the operator provides the required vault access.
1515

1616
## Layout And Safety
1717

1818
- `playbooks/` contains runnable entrypoints; `roles/` contains reusable service configuration. Inventory-derived variables are in `playbooks/group_vars/` and `playbooks/host_vars/`.
1919
- Tags are the intended way to select roles and role components. `common.yml` requires `--ask-become-pass --ask-pass` for an initial host run.
2020
- `patch.yml` deliberately rolls Debian hosts out at 30%, 60%, then 100%, and Proxmox one host at a time; preserve that rollout behavior.
2121
- Firewall changes are high impact: the role validates `/etc/nftables.conf` with `nft --check` before applying it only when templates changed.
22-
- No repository-defined CI or automated test suite exists. `ansible-lint` and `yamllint` are available through `requirements.txt`, but no project-specific invocation is configured.
22+
- GitHub Actions runs secret-free lint, syntax, and Molecule validation for pull requests. The `ansible-validation` environment must restrict deployments to `main` and supply `ANSIBLE_VAULT_PASSWORD` only to trusted pushes for decryption validation. It must not require approval because CI is non-interactive.
2323

2424
## Roles And Containers
2525

@@ -35,6 +35,5 @@
3535
- Molecule scenarios must pause after convergence so the agent can inspect the deployed containers' logs for runtime errors and warnings. A container being running or reporting healthy is not sufficient; investigate and address warnings as well as failures before considering the scenario successful.
3636
- Where possible, Molecule scenarios must include basic application-level tests in addition to idempotency and container configuration checks.
3737
- During local Molecule debugging, temporarily override container `state: healthy` with `state: started` so convergence returns quickly and container logs can be inspected before adapting the role. Restore health-state assertions before merging.
38-
- Run `yamllint` and `ansible-lint` for changed Ansible and YAML files; use two-space YAML indentation and no trailing whitespace.
39-
- CI changes must lint modified roles, run `ansible-playbook --syntax-check` for playbooks using the modified role, and run `molecule test` in an ephemeral environment. Require a green pipeline before merging to `master`.
40-
- Since no CI configuration exists, add the required pipeline configuration as part of feature work that introduces or changes a role.
38+
- Run `pre-commit run --all-files`, `yamllint`, and `ansible-lint` for changed files; use two-space YAML indentation and no trailing whitespace.
39+
- CI changes must lint modified roles, run `ansible-playbook --syntax-check` for playbooks using the modified role, and run `molecule test` in an ephemeral environment. Require a green pipeline before merging to `main`.

ansible.cfg

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
1-
[defaults]
2-
vault_password_file = ./vault_pass.sh
3-
roles_path = ./roles
1+
[defaults]
2+
roles_path = ./roles
43

54
interpreter_python = auto_silent
65

molecule/dawarich/molecule.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,11 @@
22
driver:
33
name: docker
44

5+
ansible:
6+
cfg:
7+
defaults:
8+
roles_path: ${MOLECULE_PROJECT_DIRECTORY}/roles
9+
510
scenario:
611
test_sequence:
712
- destroy

readme.md

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,12 +22,18 @@ As a network engineer I prefer to segregate my network a bit more than normally
2222
### Running a playbook
2323
``eval `ssh-agent` ``
2424
`ssh-add <ssh-key here>`
25-
`ansible-playbook -i ../netbox-ansible/netbox.yml playbooks/<playbook>.yml --tags <tags> --diff <--check>` (vault pass is handled by 1password cli)
25+
`ansible-playbook -i ../netbox-ansible/netbox.yml playbooks/<playbook>.yml --tags <tags> --diff --vault-password-file ./vault_pass.sh [--check]` (vault pass is handled by the 1Password CLI)
2626

2727
### Encrypting passwords
28-
`ansible-vault encrypt_string`
28+
`ansible-vault encrypt_string --vault-password-file ./vault_pass.sh`
2929
Next enter your variable to encrypt and press ctrl+d twice.
3030

31+
### Validating GitHub Actions workflows
32+
GitHub Actions workflow files use the GitHub workflow schema in VS Code and are checked by a dedicated actionlint workflow. Install the repository hook once with `pre-commit install`, then run all local checks with `pre-commit run --all-files`. Actionlint can also run directly with `docker run --rm --volume "$PWD:/repo" --workdir /repo rhysd/actionlint:1.7.10`.
33+
34+
### GitHub Actions Vault validation
35+
Create a GitHub environment named `ansible-validation`, restrict its deployment branches to `main`, and add an environment secret named `ANSIBLE_VAULT_PASSWORD` containing only the raw Ansible Vault password. Do not add an approval rule if validation must remain non-interactive. Trusted pushes to `main` run `python scripts/validate_vault.py` after lint, syntax, and Molecule checks succeed. Pull requests never receive the Vault password.
36+
3137
## Servicer Specific Info
3238

3339
### DNS
@@ -41,5 +47,3 @@ DNS is running in multiple stages.
4147
### Matrix Setup
4248
- nginx serves frai.se => /.well-known/matrix/server redirects to https://matrix.frai.se:443
4349
- matrix.frai.se directs directly towards the matrix server
44-
45-

requirements.txt

Lines changed: 14 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
1-
ansible
2-
ansible-lint
3-
yamllint
4-
passlib # needed for macos hashing and encryption
5-
pymongo # needed to create databases
6-
psycopg2-binary # needed to create postgres database
7-
pywinrm # needed to manage windows hosts
8-
netaddr # needed for ansible.utils.ipaddr
9-
bcrypt # needed for adguard password hash
10-
proxmoxer # needed for proxmox modules
11-
jmespath # json_query
12-
molecule
13-
molecule-docker
1+
ansible==13.6.0
2+
ansible-lint==26.4.0
3+
yamllint==1.38.0
4+
passlib==1.7.4 # needed for macos hashing and encryption
5+
pymongo==4.17.0 # needed to create databases
6+
psycopg2-binary==2.9.12 # needed to create postgres database
7+
pywinrm==0.5.0 # needed to manage windows hosts
8+
netaddr==1.3.0 # needed for ansible.utils.ipaddr
9+
bcrypt==4.0.1 # needed for adguard password hash
10+
proxmoxer==2.3.0 # needed for proxmox modules
11+
jmespath==1.1.0 # json_query
12+
molecule==26.6.0
13+
molecule-docker==2.1.0
14+
pre-commit==4.6.0

requirements.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
11
---
22
collections:
33
- name: community.docker
4+
version: 5.2.0
45
- name: community.general
6+
version: 12.6.0
57
- name: ansible.posix
8+
version: 2.1.0
69
- name: ansible.utils
10+
version: 6.0.2

scripts/validate_vault.py

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
#!/usr/bin/env python3
2+
"""Validate every Ansible Vault payload without templating variable values."""
3+
4+
from __future__ import annotations
5+
6+
import argparse
7+
import os
8+
import sys
9+
from collections.abc import Mapping, Sequence
10+
from pathlib import Path
11+
12+
from ansible.parsing.dataloader import DataLoader
13+
from ansible.parsing.vault import VaultLib, VaultSecret, is_encrypted
14+
15+
16+
VAULT_HEADER = b"$ANSIBLE_VAULT;"
17+
YAML_SUFFIXES = {".yaml", ".yml"}
18+
EXCLUDED_PARTS = {".ansible", ".git", ".venv"}
19+
20+
21+
def encrypted_payloads(value: object):
22+
"""Yield ciphertext from parsed Vault-tagged scalars without rendering Jinja."""
23+
ciphertext = getattr(value, "_ciphertext", None)
24+
if ciphertext is not None and is_encrypted(ciphertext):
25+
yield ciphertext
26+
return
27+
28+
if isinstance(value, Mapping):
29+
for key, item in value.items():
30+
yield from encrypted_payloads(key)
31+
yield from encrypted_payloads(item)
32+
elif isinstance(value, Sequence) and not isinstance(value, (str, bytes, bytearray)):
33+
for item in value:
34+
yield from encrypted_payloads(item)
35+
36+
37+
def candidate_files(root: Path):
38+
for path in root.rglob("*"):
39+
if not path.is_file() or EXCLUDED_PARTS.intersection(path.parts):
40+
continue
41+
42+
contents = path.read_bytes()
43+
if contents.lstrip().startswith(VAULT_HEADER) or (
44+
path.suffix.lower() in YAML_SUFFIXES and VAULT_HEADER in contents
45+
):
46+
yield path, contents
47+
48+
49+
def validate_file(path: Path, contents: bytes, vault: VaultLib) -> int:
50+
expected = contents.count(VAULT_HEADER)
51+
if contents.lstrip().startswith(VAULT_HEADER):
52+
vault.decrypt(contents.strip())
53+
return 1
54+
55+
parsed = DataLoader().load_from_file(str(path))
56+
payloads = list(encrypted_payloads(parsed))
57+
if len(payloads) != expected:
58+
raise ValueError(
59+
f"found {expected} Vault header(s), but parsed {len(payloads)} encrypted value(s)"
60+
)
61+
62+
for payload in payloads:
63+
vault.decrypt(payload)
64+
return len(payloads)
65+
66+
67+
def main() -> int:
68+
parser = argparse.ArgumentParser(description=__doc__)
69+
parser.add_argument("--root", type=Path, default=Path.cwd())
70+
parser.add_argument(
71+
"--password-file",
72+
type=Path,
73+
default=os.environ.get("ANSIBLE_VAULT_PASSWORD_FILE"),
74+
)
75+
args = parser.parse_args()
76+
77+
if args.password_file is None:
78+
parser.error("--password-file or ANSIBLE_VAULT_PASSWORD_FILE is required")
79+
80+
password = args.password_file.read_bytes().rstrip(b"\r\n")
81+
if not password:
82+
parser.error("the Vault password file is empty")
83+
84+
vault = VaultLib([("default", VaultSecret(password))])
85+
files = list(candidate_files(args.root.resolve()))
86+
if not files:
87+
print("No Ansible Vault payloads found.", file=sys.stderr)
88+
return 1
89+
90+
decrypted = 0
91+
failures = 0
92+
for path, contents in files:
93+
try:
94+
decrypted += validate_file(path, contents, vault)
95+
except Exception as error: # Ansible exposes several version-specific Vault errors.
96+
failures += 1
97+
print(f"{path.relative_to(args.root.resolve())}: {error}", file=sys.stderr)
98+
99+
if failures:
100+
return 1
101+
102+
print(f"Validated {decrypted} Vault value(s) in {len(files)} file(s).")
103+
return 0
104+
105+
106+
if __name__ == "__main__":
107+
raise SystemExit(main())

0 commit comments

Comments
 (0)