CI/CD Pipeline

[TOC]

CI/CD stands for continuous integration and continuous deployment (or continuous delivery) and is an essential practice in modern software development. It focuses on automating and streamlining the process of integrating code changes, testing, and deploying software.

CI/CD

Continuous Integration(CI)

Continuous Integration(CI) means developers frequently merge small code changes into a shared repository instead of doing large merges later.

Types Of CI Tests:

Unit Tests
Integration Tests

Each push automatically triggers:

Application build
Automated tests(unit + integration)

Main Goal of CI:

Massive conflicts
Hidden bugs
Long debugging cycle

Continuous Delivery(CD)

Continuous Delivery(CD) extends continuous integration by automating everything needed to prepare code for production release. Where CI stops after building & testing, CD starts and moves the validated artifact through preproduction environments.

After CI passes, the pipeline automatically deploys the build to:

Testing environment
Staging environment

Main Goal of CD:

Ensure there is always a production-ready build that has passed all automated checks.
Enable releases at any time with confidence and stability.

Continuous Deployment(CD)

Continuous Deployment(CD) is the highest level of automation in the CI/CD pipeline. It takes Continuous Delivery one step further by removing the final manual approval and automatically deploying every validated change straight to production.

A code change passes all automated checks in:

CI(build + unit test + integration tests)
CD(E2E tests, performance tests, security tests)

CI/CD Components

Source

The Source stage is the entry point and trigger for the entire CI/CD pipeline. It is intrinsically linked to the version control system, which acts as the single source of truth for the codebase.
Build

Once the source code passes the initial checks, the Build stage transforms it from human-readable code into a runnable, packaged application.
Test

The Test stage is where the build artifact is subjected to a rigorous and multi-layered validation process to ensure its quality, correctness, and stability. This stage is often the most time-consuming part of the pipeline and is composed of several sub-stages of automated testing.
Deploy

After the build artifact has successfully passed all automated tests, the Deploy stage is responsible for releasing it to various environments and, ultimately, to end-users.

CI/CD Tools

These tools help automate the entire software delivery process, from code integration to deployment, making development faster and more reliable.

Jenkins

Open-source automation server; highly extensible with plugins; ideal for complex pipelines.
GitLab CI/CD

Built into GitLab; easy setup; supports pipelines as code and Kubernetes integration.
CircleCI

Cloud-based; fast builds with parallelism and caching; works well with Docker.
Travis CI

Cloud-based; simple setup; popular for open-source projects and multiple languages.
GitHub Actions

Integrated into GitHub, event-driven workflows, and easy automation using prebuilt actions.

Ship Code To Production

Deploy Services

Multi-Service Deployment

In this model, we deploy new changes to multiple services simultaneously. This approach is easy to implement. But since all the services are upgraded at the same time, it is hard to manage and test dependencies. It’s also hard to rollback safely.
Blue-Green Deployment

With blue-green deployment, we have two identical environments: one is staging (blue), and the other is production (green). The staging environment is one version ahead of production. Once testing is done in the staging environment, user traffic is switched to the staging environment, and the staging environment becomes the production. This deployment strategy is simple to perform a rollback, but having two identical production quality environments could be expensive.
Canary Deployment

A canary deployment upgrades services gradually, each time to a subset of users. It is cheaper than blue-green deployment and easy to perform a rollback. However, since there is no staging environment, we have to test on production. This process is more complicated because we need to monitor the canary while gradually migrating more and more users away from the old version.
A/B Test

In the A/B test, different versions of services run in production simultaneously. Each version runs an “experiment” for a subset of users. An A/B test is a cheap method to test new features in production. We need to control the deployment process in case some features are pushed to users by accident.

Git

Git Storage

Git has two layers:

Porcelain (user-facing commands): add, commit, checkout, rebase, etc.
Plumbing (low-level building blocks): hash-object, cat-file, read-tree, update-index, and more.

When you trigger a Git command:

Your porcelain command is translated by Git
It calls lower-level plumbing operations
Plumbing writes directly into the .git directory (Git’s entire internal database)

Inside the .git directory: Git stores everything it needs to reconstruct your repo.

objects/: all file content and metadata stored by hash
refs/: branches and tags
index : staging area
config: repo configuration
HEAD: current branch pointer

The .git folder is your repository. If you delete it, the project loses its entire history.

Everything in Git is built from just four objects:

blob: file contents
tree: directories
commit: metadata + parents
tag: annotated reference

Git Reset

Git Reset moves your current git branch (HEAD) to a different commit and can make the index and working directory match it.

Git Fetch vs Git Pull

Git Fetch	Git Pull
Used to fetch all changes from the remote repository to the local repository without merging into the current working directory	Brings the copy of all the changes from a remote repository and merges them into the current working directory
Repository data is updated in the `.git` directory	The working directory is updated directly
Review of commits and changes can be done	Updates the changes to the local repository immediately

Git Merge vs Git Rebase

Git Merge	Git Rebase
Git Merge merges two branches to create a "feature" branch.	Git Rebase rebases the feature branch to add the feature branch to the main branch.
Git Merge is comparatively easy.	Git Rebase is comparatively harder.
Git Merge safeguards history.	Git Rebase doesn't safeguard history.
Git Merge is more suitable for projects with a less active main branch.	Git Rebase is suitable for projects with frequently active main branches.