In Dissemination we build, deploy and manage applications which run as services. The twelve-factor methodology helps us follow best practices and build applications which are portable and resilient. This module will introduce you to the Twelve-factor principles and provide you with examples of how we use them in our team.
Note: The examples refer primarily to Go apps. Please add more details or alternative examples which might be useful.
- Introduction and Background to the twelve-factor app
- About version control
- What is Continuous Integration (CI) and Continuous Delivery (CD)?
See Prerequisites in each section.
In this section you can find out how we apply each principle in Dissemination:
Prerequisite: Codebase
One codebase tracked in revision control, many deploys
The app should be tracked using version control. In dissemination, We create one GitHub repository per application, for example the dp-image-api repository contains our codebase for that app.
Each deploy of the dp-image-api will use a version of the codebase. The version is defined by an annotated tag which references a specific commit in the dp-image-api repository. The production environment will contain a version deployed from the master/main branch of the repository, which may be different from the develop branch.
Find out more:
Prerequisite: Dependencies
Explicitly declare and isolate dependencies
A 12-factor app will need to have ownership of its dependencies by defining the version/release of a library (what's the difference between an app and a library?). This allows for better control of how the application functions instead of always inheriting the latest version of a library which might not be backward compatible.
We use dependency managers to be able to maintain them all: Maven for Java apps and modules for Go apps.
Prerequisite: Config
Store config in the environment
The configuration we want for a given app may vary for each environment (develop and production) and local setup. In order to change the configuration without modifying code, we set it using environment variables which overwrite default values. In this example, the config/config.go contains a default value for BindAddr of ":24700"; to overwrite it, you can set the environment variable BIND_ADDR to a different value.
We aim to provide open source software; if we were to hard code our configuration in the service, we might disclose information which could raise security issues. In order to protect information, it is good practice to make sure any secrets, passwords are set using config and never hard coded into an app.
The configuration for each app exists in one place so that we can collaborate and manage them easily, for example: config/config.go in our go applications.
In our Go apps we use the envconfig package to help us implement this principle.
Prerequisite: Backing services
Treat backing services as attached resources
Our apps rely on the use of other resources, such as APIs (e.g. other apps of ours; and third-party resources, like vault) and databases (e.g. MongoDB). A 12-factor app should be able to switch out any of its resources by modifying the configuration, e.g. with no code changes.
Prerequisite: Build, release, run
Strictly separate build and run stages
We have created pipelines that build, release and run our apps.
- Build: based on a commit, the code is converted into an executable bundle i.e. a build.
- Release: the dp-configs configuration is applied to the build from the previous stage.
- Run: the app is deployed to the environment (
developorproduction) using the selected release.
Strict separation means that any change has to go through the stages, rather than being applied directly at runtime.
We use Semantic Versioning to give each release a unique id. This allows us to easily rollback to earlier releases.
Find out more:
- Concourse documentation
- Head to concourse.dp-ci.aws.onsdigital.uk/ to see our pipelines.
Prerequisite: Processes
Execute the app as one or more stateless processes
A 12-factor app is stateless, i.e. it does not save client data from a session to be used in the next session with that client. This means that restarting the app does not result in different outcomes when processing information or requests.
In dissemination, data that should persist is stored in a stateful backing service (typically a database like Neptune or MongoDB, but also a message queue, such as kafka).
Prerequisite: Port binding
Export services via port binding
A 12-factor app is self-contained and services are exported by port binding. In dissemination, we follow this principle, exporting HTTP and TCP as services to listen to requests and events, respectively.
Often, apps in dissemination become backing services for each other by providing their URL in the configuration. You can see examples of this by decrypting some secrets in dp-configs. We keep track of port allocations.
Prerequisite: Concurrency
Scale out via the process model
The applications must be able to scale horizontally. This means creating multiple instances of a single application, allowing them to share the work load.
To be able to achieve this, the app must share-nothing (following the process model in section 6).
In Dissemination nearly all of our apps are compliant with this. A notable exception is Zebedee (CMS), as the data processed and served by it exists as files on disk.
Prerequisite: Disposability
Maximize robustness with fast startup and graceful shutdown
A 12 factor app should maximise robustness with fast start up and gracefully shutting down:
Fast start up is important to enable rapid deployment of new code or config and being able to scale the app horizontally or vertically quickly (this can be based on varying demand). We use Nomad to manage our running services. We can easily stop, move and restart apps. If needed, we can also re-deploy apps in Concourse.
A Graceful Shutdown is important to enable an app to finish processing data before shutting down, to protect data integrity as well as responding to the requesting service or user. To achieve this the application must be able to stop handling new requests when receiving a SIGTERM and finish existing processes before shutting down.
Find out more:
Prerequisite: Dev/prod parity
Keep development, staging, and production as similar as possible
A 12-factor app is continuously-deployed, keeping differences between the development and production environments to a minimum. In order to do this, we need minimise the:
- Time gap: when we write code, we can deploy within hours.
- Personnel gap: developers who write the code are involved in the deployment and monitoring.
- Tools gap:
developandproductionenvironments are similar. However, we limit the resources indevelopto minimise costs.
Prerequisite: Logs
Treat logs as event streams
Logs should be written to STDOUT and the environment will decide how to stream these events using other services like fluentd.
- We have logging libraries that follow our logging standards. Using structured logs means we can aggregate the log events.
- The logs are streamed, centralised and aggregated in Kibana
Prerequisite: Admin processes
Run admin/management tasks as one-off processes
One-off administrative tasks should be one-off processes shipped with application code. Then the process (admin task) should be run remotely, for example by SSHing to the environment.
In dissemination, we sometimes write these scripts in a separate git repo dp-data-tools and hence it is not deployable; instead, we use flags to allow us to inject the correct configuration for each environment.