Yupana Upload Processor II

How it Works

See more details in this Yuptoo Documentation.

What is different in yuptoo compared to legacy yupana

• No database.
• No explicit state-machine.
• No django service for readiness and liveness probes.
• More robust and decoupled code.

Usage

pipenv install --dev
pipenv run python main.py

Modifiers

Yuptoo has the concept of modifiers which are used for manipulating host data before sending it to host inventory.

Read more about modifiers - yuptoo/modifiers

Testing and Linting

To run pytest hit the below command in yuptoo root directory

pipenv run python -m pytest

For linting run the below command.

pipenv run flake8

Konflux Hermetic Build

Konflux Hermetic Build had been enabled for Yuptoo repo.

A hermetic build is a secure, self-contained build process that doesn’t depend on anything outside of the build environment. Konflux can prefetch dependencies for your hermetic builds using Cachi2 by generating a software bill of materials (SBOM) where all dependencies are properly declared and pinned to specific versions.

For any dependencies update introduced in Pipfile and Pipfile.lock, please update the following required files referring to this Hermetic Build Process:

• Enabling prefetch builds for rpm

  • rpms.in.yaml
  • rpms.lock.yaml

• Enabling prefetch builds for pip

  • requirements.txt
  • requirements-build.in
  • requirements-build.txt
  • requirements-extras.txt

• Enabling prefetch builds for generic fetcher

  • artifacts.lock.yaml

More Konflux Hermetic Build resources:

• https://konflux-ci.dev/docs/building/hermetic-builds/
• https://konflux-ci.dev/docs/building/prefetching-dependencies/

Running with Docker Compose

Two docker-compose files are made available in this repo for standing up a local dev environment. The full-stack.yml file stands up ingress, kafka, yuptoo, minio, and inventory components so that the entire first bits of the platform pipeline can be tested. The docker-compose.yml file stands up services without yuptoo, the yuptoo can be run manaully in local for development tests.

Stand Up Full Stack

🐧 For Linux Users

cd scripts && source .env && sudo docker-compose -f full-stack.yml up

🍎 For macOS Users

cd scripts && source .env && sudo docker-compose -f full-stack-mac.yml up

Stand Up Yuptoo Manually outside the docker-compose

cd scripts && source .env && sudo docker-compose up

Read more about local development tests env - docs/local_environment.md

NOTE: The full stack expects you to have an ingress and inventory image available, by default, those will be pulled from quay.io. See those projects for steps for building the images needed.