Skip to content

feat: Support for using Ansible playbooks for installing model components #113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 26 commits into
base: main
Choose a base branch
from

Conversation

sdelliot
Copy link
Collaborator

Current State

Currently, some Model Components may require additional data to be downloaded (e.g., VM resources, etc.) that cannot or should not be stored within the MC repo package. In this case, the Model Components can have an INSTALL file, which can be any executable script (as denoted by a shebang line). See: Model Component INSTALL file.

This PR enables using Ansible playbooks in lieu of using a bash-based script.

Primary Benefits

There are several primary motivators for this change:

  1. Idempotence - By design, it is significantly easier to design Ansible playbooks that can be run multiple times without changing the final state. Specifically, this is necessary when installation fails and needs to be re-run. While other scripts can have this property, Ansible makes it significantly easier to do so.
  2. Readability - INSTALL files that are written with Shell scripts are significantly harder to read/understand than Ansible-based YAML. This not only impacts maintainability, but also the security as reviewing shell code for issues is more difficult.
  3. Pulling Cached Files - Downloading files and creating VM Resources or images each time FIREWHEEL is installed is time consuming and, in the case of offline installations, impractical. The common solution for this is to perform this action once and cache the resulting files. By using Ansible, FIREWHEEL can provide a few common ways to retrieve these cached files. When combined with the Idempotence of Ansible scripts, this will further reduce installation errors and speed new installs.

Key Changes

  1. A new top-level FIREWHEEL config option called ansible where there is a subkey cache_type="online". That is, by default assume that the system is online and will not pull from any kind of cached system. This top-level option will enable users to add other values (URLs, API Keys, etc.) for getting the various cached files.
  2. New Ansible playbooks that support caching from Amazon S3 buckets and from Git repositories.
  3. Modifications to the model_component_install.py to check and see if the INSTALL script is a valid Ansible playbook. If it is, then we will use ansible-runner to execute the script. Otherwise, the existing behavior is maintained.

Pending Tasks

  • The relevant documentation needs to be updated to describe these options.
  • Test cases for the new install code would be valuable, though it may be difficult to construct.
  • Additional cache options should be supported. Options include: JFrog Artifactory, a standard get_url, MS Onedrive, etc. These can also be added in future PRs as need arises.

@github-actions github-actions bot added the feature New feature or request label Apr 18, 2025
@sdelliot sdelliot marked this pull request as draft April 18, 2025 14:56
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Apr 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants