Skip to content

feat: Support for using Ansible playbooks for installing model components #113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 128 commits into
base: main
Choose a base branch
from

Conversation

sdelliot
Copy link
Collaborator

@sdelliot sdelliot commented Apr 18, 2025

Current State

Currently, some Model Components may require additional data to be downloaded (e.g., VM resources, etc.) that cannot or should not be stored within the MC repo package. In this case, the Model Components can have an INSTALL file, which can be any executable script (as denoted by a shebang line). See: Model Component INSTALL file.

This PR enables using Ansible playbooks in lieu of using a bash-based script.

Primary Benefits

There are several primary motivators for this change:

  1. Idempotence - By design, it is significantly easier to design Ansible playbooks that can be run multiple times without changing the final state. Specifically, this is necessary when installation fails and needs to be re-run. While other scripts can have this property, Ansible makes it significantly easier to do so.
  2. Readability - INSTALL files that are written with Shell scripts are significantly harder to read/understand than Ansible-based YAML. This not only impacts maintainability, but also the security as reviewing shell code for issues is more difficult.
  3. Pulling Cached Files - Downloading files and creating VM Resources or images each time FIREWHEEL is installed is time consuming and, in the case of offline installations, impractical. The common solution for this is to perform this action once and cache the resulting files. By using Ansible, FIREWHEEL can provide a few common ways to retrieve these cached files. When combined with the Idempotence of Ansible scripts, this will further reduce installation errors and speed new installs.

Key Changes

  1. A new top-level FIREWHEEL config option called ansible where users can add other values (URLs, API Keys, etc.) for getting the various cached files.
  2. New Ansible playbooks that support caching from Amazon S3 buckets, Git repositories, and file servers.
  3. Modifications to the model_component_install.py to check and see if the INSTALL file has a shebang, and if not, we check if it is a directory. At this point, we check for a tasks.yml and vars.yml to indicate that it is a proper Ansible structure. If it is, then we will use ansible-runner to execute the the tasks, which get included into main.yml.
  4. Updated documentation which explains, in detail, the new structure.

NOTE: Despite this large feature, this is not a breaking change as the current functionality still works and will be deprecated/removed in a future release.

@github-actions github-actions bot added the feature New feature or request label Apr 18, 2025
@sdelliot sdelliot marked this pull request as draft April 18, 2025 14:56
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Apr 22, 2025
@sdelliot sdelliot requested a review from gregjacobus May 22, 2025 16:14
@sdelliot sdelliot marked this pull request as ready for review May 22, 2025 16:17
Copy link
Collaborator

@gregjacobus gregjacobus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a few spelling things for now. Still getting through some more architecture review, but so far, looking really good

sdelliot and others added 3 commits June 4, 2025 10:21
Co-authored-by: gregjacobus <[email protected]>
Signed-off-by: Steven Elliott <[email protected]>
Co-authored-by: gregjacobus <[email protected]>
Signed-off-by: Steven Elliott <[email protected]>
Co-authored-by: gregjacobus <[email protected]>
Signed-off-by: Steven Elliott <[email protected]>
@sdelliot sdelliot requested a review from gregjacobus June 17, 2025 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants