Homework 3: CI CD, Kubernetes, and other components
- https://www.xsede.org/documents/10165/2374841/kubernetes.pdf/3e1f0d0e-9bb4-422b-a0de-d72770931d28
- Refer to the document [here] (https://github.com/airavata-courses/CloudElves/blob/kubernetes/README.md) to deploy a kubernetes cluster and connect to it
sudo apt update
sudo apt install openjdk-11-jdk
wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt update
sudo apt install jenkins
- Check Jenkins status
systemctl status jenkins
- Open port
sudo ufw allow 8080
- The user jenkins needs to be added to the group docker to successfully run docker commands:
sudo usermod -a -G docker jenkins
- Check if user is added to the group
grep docker /etc/group
- Restart Jenkins
sudo systemctl restart jenkins
- NodeJS 14 or above:
- https://computingforgeeks.com/install-node-js-14-on-ubuntu-debian-linux/
sudo apt update
curl -sL https://deb.nodesource.com/setup_14.x | sudo bash -
cat /etc/apt/sources.list.d/nodesource.list
sudo apt -y install nodejs
node -v
-
NPM:
sudo apt install npm -
Python: already present on jetstream
- Use Jenkins Kubernetes
- Use Ansible playbook
- Use Helm charts
- Handle shell scripts manually
2,3,4 requires ssh ing into the K8s master
- https://faun.pub/ci-cd-pipeline-using-jenkins-to-deploy-on-kubernetes-cf2fd5e185b8
- https://www.magalix.com/blog/create-a-ci/cd-pipeline-with-kubernetes-and-jenkins
- https://www.jenkins.io/doc/book/pipeline/docker/
-
Listed are the branch links :
-
Created Jenkinsfile for each microservice with respective configuration and stages :
-
As soon as the developer makes changes, the hook will trigger the Jenkins pipeline and the respective stages will start. Firstly, for some services, local build will take place, once the dependencies are correctly installed, and the build is successful, for some services, unit testing will take place, new docker image will be built and published to docker hub. The pipeline will then checkout with the Ansible Deployment GitHub branch and deployed to kubernetes clusters.
-
All Deployment yml files in the branch Ansible-K8s-deployments
-
Created playbook yml files for each service
We are using [Zenko Cloudserver] (https://www.zenko.io/cloudserver/), an S3 compatible object store service.
To run with a file backend, run the following command:
mkdir -m 700 $(pwd)/dataPath
mkdir -m 700 $(pwd)/metadataPath
docker run -d --name s3server -e S3DATAPATH="$(pwd)/dataPath" -e S3METADATAPATH="$(pwd)/metadataPath" -p 8000:8000 scality/cloudserver
To run with an in-memory backend, run the following command:
docker run -d --name s3server -p 8000:8000 scality/cloudserver-latest
Install AWS CLI from here
add the following into ~/.aws/creditials file
[default]
aws_access_key_id = accessKey1
aws_secret_access_key = verySecretKey1
region = us-east-1
After this, you can use the AWS CLI to access the local deployment of s3 using --endpoint=http://localhost:8000 option as follows:
aws s3 ls --endpoint-url http://localhost:8000