Improved molecule support (#147)

* Improved molecule support

Present molecule scenarios using a fixture.

Author: cidrblock

README.md

@@ -19,7 +19,41 @@ Install this plugin using `pip`
 pip install pytest-ansible
 ```
 
-## Usage
+## Running molecule scenarios using pytest
+
+Molecule scenarios can be tested using 2 different methods if molecule is installed.
+
+**Recommended:**
+
+Add a `test_integration.py` file to the `tests/integration` directory of the ansible collection:
+
+```
+"""Tests for molecule scenarios."""
+from __future__ import absolute_import, division, print_function
+
+from pytest_ansible.molecule import MoleculeScenario
+
+
+def test_integration(molecule_scenario: MoleculeScenario) -> None:
+    """Run molecule for each scenario.
+
+    :param molecule_scenario: The molecule scenario object
+    """
+    proc = molecule_scenario.test()
+    assert proc.returncode == 0
+```
+
+The `molecule_scenario` fixture provides parameterized molecule scenarios discovered in the collection's `extensions/molecule` directory, as well as other directories within the collection.
+
+`molecule test -s <scenario>` will be run for each scenario and a completed subprocess returned from the `test()` call.
+
+**Legacy:**
+
+Run molecule with the `--molecule` command line parameter to inject each molecule directory found in the current working directory. Each scenario will be injected as an external test in the the tests available for pytest. Due to the nature of this approach, the molecule scenarios are not represented as python tests and may not show in the IDE's pytest test tree.
+
+## Fixtures and helpers for use in tests
+
+### Usage
 
 Once installed, the following `pytest` command-line parameters are available:
 
@@ -43,7 +77,7 @@ pytest \
     [--check]
 ```
 
-## Inventory
+### Inventory
 
 Using ansible first starts with defining your inventory. This can be done in
 several ways, but to start, we'll use the `ansible_adhoc` fixture.
@@ -85,7 +119,7 @@ tests. To accomplish this, the fixture `ansible_adhoc` allows you to customize
 the inventory parameters. Read on for more detail on using the `ansible_adhoc`
 fixture.
 
-## Extra Inventory
+### Extra Inventory
 
 Using ansible first starts with defining your extra inventory. This feature was added in version 2.3.0, and is intended
 to allow the user to work with two different inventories. This can be done in several ways, but to start,
@@ -97,7 +131,7 @@ For example,
 pytest --inventory my_inventory.ini --extra-inventory my_second_inventory.ini --host-pattern host_in_second_inventory
 ```
 
-### Fixture `ansible_adhoc`
+#### Fixture `ansible_adhoc`
 
 The `ansible_adhoc` fixture returns a function used to initialize
 a `HostManager` object. The `ansible_adhoc` fixture will default to parameters
@@ -151,7 +185,7 @@ def test_host_manager(ansible_adhoc):
         a_host.ping()
 ```
 
-### Fixture `localhost`
+#### Fixture `localhost`
 
 The `localhost` fixture is a convenience fixture that surfaces
 a `ModuleDispatcher` instance for ansible host running `pytest`. This is
@@ -179,7 +213,7 @@ def test_do_something_cloudy(localhost, ansible_adhoc):
     localhost.ec2(**params)
 ```
 
-### Fixture `ansible_module`
+#### Fixture `ansible_module`
 
 The `ansible_module` fixture allows tests and fixtures to call [ansible
 modules](http://docs.ansible.com/modules.html). Unlike the `ansible_adhoc`
@@ -224,7 +258,7 @@ def test_sshd_config(ansible_module):
     # do other stuff ...
 ```
 
-### Fixture `ansible_facts`
+#### Fixture `ansible_facts`
 
 The `ansible_facts` fixture returns a JSON structure representing the system
 facts for the associated inventory. Sample fact data is available in the
@@ -256,7 +290,7 @@ def test_terminate_us_east_1_instances(ansible_adhoc):
             '''do some testing'''
 ```
 
-### Parameterizing with `pytest.mark.ansible`
+#### Parameterizing with `pytest.mark.ansible`
 
 Perhaps the `--ansible-inventory=<inventory>` includes many systems, but you
 only wish to interact with a subset. The `pytest.mark.ansible` marker can be
@@ -306,7 +340,7 @@ class Test_Local(object):
         '''do some testing'''
 ```
 
-### Inspecting results
+#### Inspecting results
 
 When using the `ansible_adhoc`, `localhost` or `ansible_module` fixtures, the
 object returned will be an instance of class `AdHocResult`. The
@@ -371,7 +405,7 @@ The contents of the JSON returned by an ansible module differs from module to
 module. For guidance, consult the documentation and examples for the specific
 [ansible module](http://docs.ansible.com/modules_by_category.html).
 
-### Exception handling
+#### Exception handling
 
 If `ansible` is unable to connect to any inventory, an exception will be raised.
 

src/pytest_ansible/molecule.py

@@ -8,6 +8,7 @@
 import subprocess
 import sys
 import warnings
+from pathlib import Path
 from shlex import quote
 
 import pkg_resources
@@ -234,3 +235,34 @@ def __str__(self):
 
 class MoleculeExceptionError(Exception):
     """Custom exception for error reporting."""
+
+
+class MoleculeScenario:
+    """Molecule subprocess wrapper."""
+
+    # pylint: disable=too-few-public-methods
+
+    def __init__(self, name: str, parent_directory: Path, test_id: str):
+        """Initialize the MoleculeScenario class.
+
+        :param molecule_parent: The parent directory of 'molecule'
+        :param scenario_name: The name of the molecule scenario
+        :param test_id: The test id
+        """
+        self.parent_directory = parent_directory
+        self.name = name
+        self.test_id = test_id
+
+    def test(self) -> subprocess.CompletedProcess:
+        """Run molecule test for the scenario.
+
+        :returns: The completed process
+        """
+        return subprocess.run(
+            args=[sys.executable, "-m", "molecule", "test", "-s", self.name],
+            capture_output=False,
+            check=False,
+            cwd=self.parent_directory,
+            shell=False,
+            text=True,
+        )

src/pytest_ansible/plugin.py

@@ -1,7 +1,9 @@
 """PyTest Ansible Plugin."""
 from __future__ import annotations
 
+import contextlib
 import logging
+import subprocess
 from pathlib import Path
 from typing import TYPE_CHECKING
 
@@ -21,12 +23,13 @@
 from pytest_ansible.host_manager import get_host_manager
 
 try:
-    from .molecule import MoleculeFile
+    from .molecule import MoleculeFile, MoleculeScenario
 
     HAS_MOLECULE = True
 except ImportError:
     HAS_MOLECULE = False
 
+
 from .units import inject, inject_only
 
 if TYPE_CHECKING:
@@ -217,7 +220,6 @@ def pytest_collect_file(
     parent: pytest.Collector,
 ) -> Node | None:
     """Transform each found molecule.yml into a pytest test."""
-
     if not parent.config.option.molecule:
         return None
     if not HAS_MOLECULE:
@@ -268,6 +270,53 @@ def pytest_generate_tests(metafunc):
         metafunc.parametrize("ansible_group", iter(hosts[g] for g in groups))
         metafunc.parametrize("ansible_group", iter(hosts[g] for g in extra_groups))
 
+    if "molecule_scenario" in metafunc.fixturenames:
+        if not HAS_MOLECULE:
+            pytest.exit("molecule not installed or found.")
+
+        # Find all molecule scenarios not gitignored
+        # Replace this with molecule --list in the future if json output is available
+        rootpath = metafunc.config.rootpath
+
+        scenarios = []
+
+        candidates = list(rootpath.glob("**/molecule/*/molecule.yml"))
+        command = ["git", "check-ignore"] + candidates
+        with contextlib.suppress(subprocess.CalledProcessError, FileNotFoundError):
+            proc = subprocess.run(
+                args=command,
+                capture_output=True,
+                check=True,
+                text=True,
+                shell=False,
+            )
+
+        try:
+            ignored = proc.stdout.splitlines()
+            scenario_paths = [
+                candidate for candidate in candidates if str(candidate) not in ignored
+            ]
+        except NameError:
+            scenario_paths = candidates
+
+        for fs_entry in scenario_paths:
+            scenario = fs_entry.parent
+            molecule_parent = scenario.parent.parent
+            scenarios.append(
+                MoleculeScenario(
+                    parent_directory=molecule_parent,
+                    name=scenario.name,
+                    test_id=f"{molecule_parent.name}-{scenario.name}",
+                ),
+            )
+        if not scenarios:
+            pytest.exit(f"No molecule scenarions found in: {rootpath}")
+        metafunc.parametrize(
+            "molecule_scenario",
+            scenarios,
+            ids=[scenario.test_id for scenario in scenarios],
+        )
+
 
 class PyTestAnsiblePlugin:
     """Ansible PyTest Plugin Class."""

tests/fixtures/collections/ansible_collections/company/product/extensions/molecule/default/converge.yml

@@ -0,0 +1,8 @@
+---
+- name: Converge
+  hosts: localhost
+  gather_facts: false
+  tasks:
+    - name: Print out information
+      ansible.builtin.debug:
+        msg: "converge ran successfully"

tests/fixtures/collections/ansible_collections/company/product/extensions/molecule/default/molecule.yml

@@ -0,0 +1,31 @@
+---
+# dependency:
+#   name: galaxy
+#   state: present
+
+# driver:
+#   name: delegated
+
+platforms:
+  - name: instance
+
+# verifier:
+#   name: ansible
+
+provisioner:
+  name: ansible
+  # log: true
+  # config_options:
+  #   defaults:
+  #     interpreter_python: auto
+  #     verbosity: 1
+
+scenario:
+  name: default
+  test_sequence:
+    # - dependency
+    # - syntax
+    # - create
+    # - prepare
+    - converge
+    # - verify

tests/integration/test_molecule.py

@@ -5,9 +5,13 @@
 import os
 import subprocess
 import sys
+from typing import TYPE_CHECKING
 
 import pytest
 
+if TYPE_CHECKING:
+    from pytest_ansible.molecule import MoleculeScenario
+
 
 def test_molecule_collect() -> None:
     """Test pytest collection of molecule scenarios."""
@@ -56,3 +60,13 @@ def test_molecule_runtest() -> None:
     assert "collected 1 item" in proc.stdout
     assert "tests/fixtures/molecule/default/molecule.yml::test " in proc.stdout
     assert "1 passed" in proc.stdout
+
+
+def test_molecule_fixture(molecule_scenario: MoleculeScenario) -> None:
+    """Test the scenario fixture.
+
+    :param molecule_scenario: One scenario
+    """
+    assert molecule_scenario.test_id in ["fixtures-default", "extensions-default"]
+    assert molecule_scenario.name == "default"
+    molecule_scenario.test()