dataplatform-pipeline-alerts-email

Sends alerts about failing pipelines as an email.

Usage

Copy repo content to your new repository, or create a new branch where you quickly can test a custom lambda function

Setup

In these examples, we use the default python3 distribution on your platform. If you need a specific version of python you need to run the command for that specific version. Ie. for 3.8 run python3.8 -m venv .venv instead to get a virtualenv for that version.

Installing global python dependencies

You can either install globally. This might require you to run as root (use sudo).

python3 -m pip install tox black pip-tools

Or, you can install for just your user. This is recommended as it does not require root/sudo, but it does require ~/.local/bin to be added to PATH in your .bashrc or similar file for your shell. Eg: PATH=${HOME}/.local/bin:${PATH}.

python3 -m pip install --user tox black pip-tools

Installing local python dependencies in a virtualenv

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

make init

Tests

Tests are run using tox: make test

For tests and linting we use pytest, flake8 and black.

Deploy

Deploy to both dev and prod is automatic via GitHub Actions on push to main. You can alternatively deploy from local machine (requires saml2aws) with: make deploy or make deploy-prod.