Skip to content

Latest commit

 

History

History
242 lines (179 loc) · 15 KB

README.md

File metadata and controls

242 lines (179 loc) · 15 KB

Formulas - Wick

Formulas are sets of instructions that result in a specific state. You use them to install software, configure services and perform other actions in a system. Formulas may contain dependencies, explorers, files, functions, a run script and templates.

Formulas should be idempotent, which means that running them once or running them a hundred times will result in the exact same state. Let's take a formula that adds a line to /etc/rc.local. When executed once, the line should be added. When executed again, the formula should check to see if the line already exists and will add the line only if it is missing.

All formulas run in a strict mode. This means you won't be able to reference undefined variables nor commit other minor infractions. Further implications and code examples are documented with strict mode.

Specific Formulas

See the formula-specific documentation that explain what each of these do. Formulas can reference each other in the chain of parents in the invocation of wick.

  • Add Repository - Exposes functions for adding a repository to a package management system.
  • Apache2 - Installs the Apache 2.x web server and enables named virtual hosts. Provides functions to add virtual hosts and config files.
  • At - Installs and sets up the at package.
  • AWS Cli - Installs Amazon's command-line tools.
  • CA-Certificates - Adds a function that will install a CA certificate in the system's certificate store.
  • Consul-Locker - Adds Consul-Locker to /usr/local/bin/ to enable process locking via Consul.
  • Cron - Adds useful cron manipulation functions to the Wick environment. Performs no actions.
  • Curl - Ensures curl is installed
  • Dig - Installs dig to query domain name system name servers.
  • Dnsmasq - Installs dnsmasq and adds it as a nameserver.
  • Encrypt Folder - Creates a script that encrypts a folder on a machine.
  • Erlang - Determines the erlang package based on the OS distribution.
  • fsfreeze - Utility for freezing writes to a device.
  • gcc - Installs the Gnu C compiler (gcc).
  • GitHub - Adds functions for interacting with GitHub repositories via GitHub's API.
  • graphviz - Installs Graphviz graph visualization software.
  • g++ - Installs the Gnu C++ compiler (g++).
  • Hostname - Sets the hostname and domain. Supports templates. Can automatically update the hostname when network connections are made.
  • htop - Installs the htop interactive process viewer.
  • IPTables - Installs iptables if it is not already installed.
  • Java-Util - Adds useful functions into the Wick environment that are related to Java.
  • Jq - Installs jq, a command-line tool to process JSON.
  • Kermit - This installs the Kermit (from the ckermit package), allowing file transfer to and from a remote machine.
  • Lsof - This installs the lsof command, which may be needed by other programs.
  • Md5deep - Sets up md5deep and its related utilities.
  • Mo - Installs mo mustache template parser into /usr/local/bin/ and adds the ability to use "mo" templates with wickMakeFile (in the wick-base formula).
  • MySQL Client - Installs the mysql client for mysql-server
  • MySQL Server - Installs the MySQL relational database server.
  • Nano - Installs nano, a tiny clone of Pico.
  • Netcat - Installs the netcat (nc) package.
  • network-layer - Provides a library that allows you to include services into the network layer. Your service would be added after the rest of the network is up but before the rest of the services that depend on the network.
  • Node.js - Installs Node.js from their website, not a distribution's packaged version.
  • NSCD - This installs a name server caching daemon. The unscd package is preferred over nscd, but the two are essentially interchangeable.
  • NTP - Installs ntp Network Time Protocol package and ntpdate set time utility.
  • OpenJDK 1.7 JDK - Installs OpenJDK 1.7 JDK.
  • OpenJDK 1.7 - Installs OpenJDK 1.7 JRE.
  • OpenJDK 1.8 JDK Formula - Installs OpenJDK 1.8 JDK.
  • OpenJDK 1.8 - Installs OpenJDK 1.8 JRE.
  • Openssh Server - Ensures openssh-server is installed which will provide sshd/ssh.
  • Python-Pip - Installs the Python package manager, pip.
  • Python-virtualenv - Installs the Python virtual environment manager, virtualenv.
  • Redhat LSB Core - Installs the requirements that most init scripts depend upon without including many unnecessary dependencies.
  • Redis - Installs redis onto the box.
  • Rvm - This installs RVM from http://get.rvm.io/. Passes all extra parameters to the installer.
  • Node.js - Alternate version of s3cmd, parallelizes operations.
  • S3cmd - This installs a tool to work with Amazon S3 from the command-line.
  • Screen - Installs GNU screen package.
  • SELinux - Configures SELinux into a given state.
  • SSHD Tweak - Alters the SSH server's settings.
  • Stunnel - Installs stunnel and starts it at boot. Provides functions to install service specific configurations.
  • Sysctl - This adds a function to Wick to set a system configuration setting.
  • Telnet - Installs telnet if it is not already installed. Consider netcat instead.
  • Timezone - Sets the time zone to a specified zone, defaulting to UTC.
  • Tomcat 7 - Installs Tomcat 7, which depends on Java 1.7 or Java 1.8.
  • Unzip - Installs unzip. This does not install zip automatically.
  • Update Packages - Updates all packages on the system except kernel updates.
  • Vim - Makes sure vim (enhanced vi)
  • Wick-Base - This formula only adds useful functions and explorers to Wick. Nearly everything depends on this formula.
  • Wick-Infect - Installs a Bash configuration at /usr/local/lib/wick-infect that shell scripts can source into their environment in order to get a copy of all of the library functions.
  • Wick-Init-D-Lib - Not all systems are created equally. In that regard, not all /etc/init.d/ environments are the same either. Instead of rewriting a shell that's extremely similar for multiple services, this formula will install a library that can be used to make init-style scripts quickly.
  • Yum-EPEL - Adds the EPEL repository to yum. Does not do anything for other package managers.
  • Yum-Remi - Adds the Remi repository for yum. Will not make it enabled by default. Does nothing for other platforms that do not use yum.
  • Zip - Installs zip. This does not install unzip automatically.
  • ZModem - This installs the lrzsz package, providing the rz and sz commands to send/receive files using ZModem.

Dependencies

The depends file exists so you can list the formulas that need to be installed before this formula executes. For instance, if you have a formula that requires apache2, then your depends file would look like this:

#!/usr/bin/env bash

wickFormula apache2

The dependencies can also use explorers. If you need to use rvm on centos and redhat, your depends file would contain lines similar to this example.

#!/usr/bin/env bash

wickExplorer os wick-base os

case "$os" in
    centos|redhat)
        wickFormula rvm
        ;;
esac

While executing, the depends script will have WICK_FORMULA_DIR set to the path of the currently executing formula and WICK_FORMULA_NAME will be the name of the formula.

Explorers

The shell scripts in explorers/ are executed by wickExplorer, one of the main binaries provided by Wick. Their mission is to identify some information about the target system. This example will detect the Linux kernel version.

#!/usr/bin/env bash

uname -r

Please remember that all explorers run in strict mode and all errors in running will cause the programs to abort.

When executed, the output of this will be something like "3.16.0-30-generic". You can use explorers to determine the currently running operating system, versions of software, if files exist in specific locations or see if some packages are installed (among many other things). Take a look at the ones provided in wick-base to see some that already exist.

Explorer scripts have the full Wick environment, so functions like wickDebug, wickInfo (both from wick-base) and any other functions defined by earlier formulas are available during script execution.

On success, information should be written to stdout. Failure would cause Wick to stop and display all of the messages written to stderr. (See Bash concepts for stdout, stderr, success and failure.)

Files

Anything under files/ is intended to be copied directly to the target system with wickMakeFile (a function provided by wick-base). They can also be used indirectly, such as with the apache2-add-vhost function (from apache2).

Functions

The files in functions/ are sourced into Wick's environment before a formula runs, allowing the run script and any formula afterwards to use them. (See Bash concepts for sourcing.)

It is a good idea to include functions to eliminate repeated tasks such as adding users, installing configuration files and managing processes. Having copious amounts of logging in the functions helps troubleshoot difficulties as well. You should use wickInfo, wickDebug, wickWarn and wickError (all from wick-base) to write output.

Run Script

The run file is the meat of your formula. It performs whatever actions are necessary in order to complete the desired action. Typically you will leverage existing functions as much as possible in order to keep the size of your code down and to reduce duplication.

All run scripts automatically start in strict mode, so any error that is encountered will cause the script to immediately fail.

These scripts can also be passed arguments. For instance, the hostname formula accepts the desired name for the target machine. Arguments are set in another function's dependencies or in roles. They use the command wickFormula, like this:

# This is how a role would pass arguments to the run script
wickFormula hostname server1.example.com

# The run script would act like it was called this way:
# ./formulas/hostname/run server1.example.com

In the above example, the run script for the hostname formula will get "server1.example.com" as $1 in the script.

Here is a sample run script that will download a copy of the application from an internal server. It uses wickGetOption and wickGetArgument to simplify command-line arguments (available in Libraries and detailed in [argument processing]).

#!/usr/bin/env bash
# Download and install the application code from a remote server
#
# $1 - Codebase name to download, such as myApp.
# --version - Version number to download, default is "latest".
#
# Examples
#
#   # Downloads theCodebase, version "latest"
#   wickFormula mySampleFormula theCodebase
#
#   # Downloads anotherExample, version 1.2.3
#   wickFormula mySampleFormula --version=1.2.3
#
# Returns nothing.

# If `--version=VALUE` was used then `$version` is set to VALUE.
# If `--version` was used, then `$version` will be set to "true".
# If `--version` is not used, `$version` will be set to "".
wickGetOption version version "$@"

# Get the first non-option argument.  Use this syntax instead of `$1`
# because `$1` might be the `--version` option.
wickGetArgument name 0 "$@"

# Check to make sure the codebase was passed.
if [[ -z "$name" ]]; then
    # Write a message with wickError
    wickError "You must specify a codebase to download"

    # Return a non-zero code to indicate failure
    exit 1
fi

: ${version:=latest}
URL="http://10.0.0.1/installer-files/${name}-${version}.tar.gz"
wickInfo "Installing $name ($version)"

# Remove a previous installation if one exists
rm -rf /opt/application

# Create destination directory
wickMakeDir --owner=root:root --mode=0755 /opt/application

(
    # Use a subshell when changing directories.  This way we don't need to
    # reset the environment back to "normal" and change back to where we
    # started.  It will happen automatically.  Changing back to the
    # original directory is not a requirement, but it is simpler to debug
    # scripts when you use a consistent method like this.
    cd /opt/application

    wickDebug "Downloading $URL"
    wickGetUrl "$URL" application.tar.gz

    wickDebug "Extracting"
    tar xfz application.tar.gz --strip=1

    rm application.tar.gz
)

There you have it, a complete run script. It's calling several other functions defined in [formulas] and libraries. It even cleans up after itself. When anything fails, the script will automatically terminate and Wick will stop processing. It's fairly simple Bash, with the complexities hidden away in functions. For example, wickGetUrl will use curl or wget if available. You don't need to worry, force the installation nor do the check yourself. Instead, let that function do the hard work.

Templates

The files contained within the templates/ folder are extremely similar to the files that are copied verbatim to the target system, but they follow a naming convention and are processed by a template system. This is explained more in the templates page.