Splitting up deployment play»book« into roles and smaller playbooks

[mfocko]

Outcome of the arch discussion from today.

Separate points of discussion

• Fedora-source-git & stream deployments are no longer used, since they are not deployed and we currently don't even plan on deploying them (we would need to request another CMDB entry, ESS, MP+ namespaces, etc.) let's remove them
  • the files will be kept in the history anyways
  • we can also simplify the structure of our variables
• «Wait for …» — currently somewhat broken, expects the CLI to be switched to the correct project + does not really make sense to be present in the playbook?
  • Testing Farm deployment tests depend on this, could be just moved around
  • Usefulness during manual deployments is questionable…
  • We could probably keep it as an optional thing via tag / variable, such as wait/blocking
• Variable mess — they should be, ideally, tied to the related k8s objects, but they’re global…
  • global variables (API, project, etc.) should be global…
  • deployment-specific variables (resources for workers, scaling of workers, etc.) should be deployment-specific; related to the next point
• Tight coupling of k8s definitions — right now everything per deployment is in one file (deployment, route, volume, etc.); splitting up could result in less frequent redeployment on the OpenShift side when deploying manually (also allows for better “monitoring” ok/changed), but at the same time implodes the amount of k8s definitions in the repository (better directory structure would be definitely needed)

TODO

• Remove the fedora-source-git and stream deployments #617
• Make blocking wait on running workers/deployments optional #618
• Factor out one of the OpenShift deployments to a role to have a better idea of the results of the complete refactor #619

Notes

• Keep in mind testing on the TF when refactoring.
• Create PRs against deployment's upstream (so that we can test changes on the Testing Farm; for security reasons, it is required to have the branch in a repo where team has write access)