Skip to content

Commit

Permalink
New Continuous Integration section (#378)
Browse files Browse the repository at this point in the history
* New Continuous Integration section

Co-authored-by: Brian J. Cardiff <[email protected]>
Co-authored-by: Johannes Müller <[email protected]>
  • Loading branch information
3 people authored Feb 4, 2020
1 parent 7b069de commit 321a6b5
Show file tree
Hide file tree
Showing 3 changed files with 363 additions and 0 deletions.
2 changes: 2 additions & 0 deletions SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,3 +130,5 @@
* [Writing Shards](guides/writing_shards.md)
* [Hosting on GitHub](guides/hosting/github.md)
* [Hosting on GitLab](guides/hosting/gitlab.md)
* [Continuous Integration](guides/continuous_integration.md)
* [Using TravisCI](guides/ci/travis.md)
264 changes: 264 additions & 0 deletions guides/ci/travis.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,264 @@
# Travis CI

In this section we are going to use [Travis CI](https://travis-ci.org/) as our continuous-integration service. Travis CI is [mostly used](https://docs.travis-ci.com/user/tutorial/#more-than-running-tests) for building and running tests for projects hosted at GitHub. It supports [different programming laguages](https://docs.travis-ci.com/user/tutorial/#selecting-a-different-programming-language) and for our particular case, it supports the [Crystal language](https://docs.travis-ci.com/user/languages/crystal/).

>**Note:**If you are new to continuous integration (or you want to refresh the basic concepts) we may start reading the [core concepts guide](https://docs.travis-ci.com/user/for-beginners/).
Now let's see some examples!

## Build and run specs

### Using `latest` and `nightly`

A first (and very basic) Travis CI config file could be:

```yaml
# .travis.yml
language: crystal
```
That's it! With this config file, Travis CI by default will run `crystal spec`.
Now, we just need to go to Travis CI dashboard to [add the GitHub repository](https://docs.travis-ci.com/user/tutorial/#to-get-started-with-travis-ci).

Let's see another example:

```yaml
# .travis.yml
language: crystal
crystal:
- latest
- nightly
script:
- crystal spec
- crystal tool format --check
```

With this configuration, Travis CI will run the tests using both Crystal `latest` and `nightly` releases on every push to a branch on your Github repository.

**Note:** When [creating a Crystal project](../../using_the_compiler/#creating-a-crystal-project) using `crystal init`, Crystal creates a `.travis.yml` file for us.

### Using a specific Crystal release

Let's suppose we want to pin a specific Crystal release (maybe we want to make sure the shard compiles and works with that version) for example [Crystal 0.31.1](https://github.com/crystal-lang/crystal/releases/tag/0.31.1).

Travis CI only provides _runners_ to `latest` and `nightly` releases directly and so, we need to install the requested Crystal release manually. For this we are going to use [Docker](https://www.docker.com/).

First we need to add Docker as a service in `.travis.yml`, and then we can use `docker` commands in our build steps, like this:

```yml
# .travis.yml
language: minimal
services:
- docker
script:
- docker run -v $PWD:/src -w /src crystallang/crystal:0.31.1 crystal spec
```

**Note:** We may read about different (languages)[https://docs.travis-ci.com/user/languages/] supported by Travis CI, included [minimal](https://docs.travis-ci.com/user/languages/minimal-and-generic/).

**Note:** A list with the different official [Crystal docker images](https://hub.docker.com/r/crystallang/crystal/tags) is available at [DockerHub](https://hub.docker.com/r/crystallang/crystal).

### Using `latest`, `nightly` and a specific Crystal release all together!

Supported _runners_ can be combined with Docker-based _runners_ using a [Build Matrix](https://docs.travis-ci.com/user/customizing-the-build#build-matrix). This will allow us to run tests against `latest` and `nightly` and pinned releases.

Here is the example:

```yaml
# .travis.yml
matrix:
include:
- language: crystal
crystal:
- latest
script:
- crystal spec
- language: crystal
crystal:
- nightly
script:
- crystal spec
- language: bash
services:
- docker
script:
- docker run -v $PWD:/src -w /src crystallang/crystal:0.31.1 crystal spec
```

## Installing shards packages

In native _runners_ (`language: crystal`), Travis CI already automatically installs shards dependencies using `shards install`. To improve build performance we may add [caching](#caching) on top of that.

#### Using Docker

In a Docker-based _runner_ we need to run `shards install` explicitly, like this:

```yml
# .travis.yml
language: bash
services:
- docker
script:
- docker run -v $PWD:/src -w /src crystallang/crystal:0.31.1 shards install
- docker run -v $PWD:/src -w /src crystallang/crystal:0.31.1 crystal spec
```

**Note:** Since the shards will be installed in `./lib/` folder, it will be preserved for the second docker run command.

## Installing binary dependencies

Our application or maybe some shards may required libraries and packages. This binary dependencies may be installed using different methods. Here we are going to show an example using the [Apt](https://help.ubuntu.com/lts/serverguide/apt.html) command (since the Docker image we are using is based on Ubuntu)

Here is a first example installing the `libsqlite3` development package using the [APT addon](https://docs.travis-ci.com/user/installing-dependencies/#installing-packages-with-the-apt-addon):

```yaml
# .travis.yml
language: crystal
crystal:
- latest
before_install:
- sudo apt-get -y install libsqlite3-dev
addons:
apt:
update: true
script:
- crystal spec
```

#### Using Docker

We are going to build a new docker image based on [crystallang/crystal](https://hub.docker.com/r/crystallang/crystal/), and in this new image we will be installing the binary dependencies.

To accomplish this we are going to use a [Dockerfile](https://docs.docker.com/engine/reference/builder/):

```dockerfile
# Dockerfile
FROM crystallang/crystal:latest
# install binary dependencies:
RUN apt-get update && apt-get install -y libsqlite3-dev
```

And here is the Travis CI configuration file:

```yml
# .travis.yml
language: bash
services:
- docker
before_install:
# build image using Dockerfile:
- docker build -t testing .
script:
# run specs in the container
- docker run -v $PWD:/src -w /src testing crystal spec
```

**Note:** Dockerfile arguments can be used to use the same Dockerfile for latest, nightly or a specific version.

## Using services

Travis CI may start [services](https://docs.travis-ci.com/user/database-setup/) as requested.

For example, we can start a [MySQL](https://docs.travis-ci.com/user/database-setup/#mysql) database service by adding a `services:` section to our `.travis.yml`:

```yaml
# .travis.yml
language: crystal
crystal:
- latest
services:
- mysql
script:
- crystal spec
```

Here is the new test file for testing against the database:

```crystal
# spec/simple_db_spec.cr
require "./spec_helper"
require "mysql"
it "connects to the database" do
DB.connect ENV["DATABASE_URL"] do |cnn|
cnn.query_one("SELECT 'foo'", as: String).should eq "foo"
end
end
```

When pushing this changes Travis CI will report the following error: `Unknown database 'test' (Exception)`, showing that we need to configure the MySQL service **and also setup the database**:

```yaml
# .travis.yml
language: crystal
crystal:
- latest
env:
global:
- DATABASE_NAME=test
- DATABASE_URL=mysql://root@localhost/$DATABASE_NAME
services:
- mysql
before_install:
- mysql -e "CREATE DATABASE IF NOT EXISTS $DATABASE_NAME;"
- mysql -u root --password="" $DATABASE_NAME < db/schema.sql
script:
- crystal spec
```

We are [using a `schema.sql` script](https://andidittrich.de/2017/06/travisci-setup-mysql-tablesdata-before-running-tests.html) to create a more readable `.travis.yml`. The file `./db/schema.sql` looks like this:

```sql
-- schema.sql
CREATE TABLE ... etc ...
```

Pushing these changes will trigger Travis CI and the build should be successful!

## Caching

If we read Travis CI job log, we will find that every time the job runs, Travis CI needs to fetch the libraries needed to run the application:

```log
Fetching https://github.com/crystal-lang/crystal-mysql.git
Fetching https://github.com/crystal-lang/crystal-db.git
```

This takes time and, on the other hand, these libraries might not change as often as our application, so it looks like we may cache them and save time.

Travis CI [uses caching](https://docs.travis-ci.com/user/caching/) to improve some parts of the building path. Here is the new configuration file **with cache enabled**:

```yml
# .travis.yml
language: crystal
crystal:
- latest
cache: shards
script:
- crystal spec
```

Let's push these changes. Travis CI will run, and it will install dependencies, but then it will cache the shards cache folder which, usually, is `~/.cache/shards`. The following runs will use the cached dependencies.
97 changes: 97 additions & 0 deletions guides/continuous_integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Continuous Integration

The ability of having immediate feedback on what we are working should be one of the most important characteristics in software development. Imagine making one change to our source code and having to wait 2 weeks to see if it broke something? oh! That would be a nightmare! For this, Continuous Integration will help a team to have immediate and frequent feedback about the status of what they are building.

Martin Fowler [defines Continuous Integration](https://www.martinfowler.com/articles/continuousIntegration.html) as
_a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly._

In the next subsections, we are going to present 2 continuous integration tools: [Travis CI](https://travis-ci.org/) and [Circle CI](https://circleci.com/) and use them with a Crystal example application.

These tools not only will let us build and test our code each time the source has changed but also deploy the result (if the build was successful) or use automatic builds, and maybe test against different platforms, to mention a few.

## The example application

We are going to use Conway's Game of Life as the example application. More precisely, we are going to use only the first iterations in [Conway's Game of Life Kata](http://codingdojo.org/kata/GameOfLife/) solution using [TDD](https://martinfowler.com/bliki/TestDrivenDevelopment.html).

Note that we won't be using TDD in the example itself, but we will mimic as if the example code is the result of the first iterations.

Another important thing to mention is that we are using `crystal init` to [create the application](../using_the_compiler/#creating-a-crystal-project).

And here's the implementation:

```crystal
# src/game_of_life.cr
class Location
getter x : Int32
getter y : Int32
def self.random
Location.new(Random.rand(10), Random.rand(10))
end
def initialize(@x, @y)
end
end
class World
@living_cells : Array(Location)
def self.empty
new
end
def initialize(living_cells = [] of Location)
@living_cells = living_cells
end
def set_living_at(a_location)
@living_cells << a_location
end
def is_empty?
@living_cells.size == 0
end
end
```

And the specs:

```crystal
# spec/game_of_life_spec.cr
require "./spec_helper"
describe "a new world" do
it "should be empty" do
world = World.new
world.is_empty?.should be_true
end
end
describe "an empty world" do
it "should not be empty after adding a cell" do
world = World.empty
world.set_living_at(Location.random)
world.is_empty?.should be_false
end
end
```

And this is all we need for our continuous integration examples! Let's start!

## Continuous Integration step by step

Here's the list of items we want to achieve:

1. Build and run specs using 3 different Crystal's versions:
* latest
* nightly
* 0.31.1 (using a Docker image)
2. Install shards packages
3. Install binary dependencies
4. Use a database (for example MySQL)
5. Cache dependencies to make the build run faster

From here choose your next steps:

* I want to use [Travis CI](./ci/travis.html)
* I want to use Circle CI (coming soon...)

0 comments on commit 321a6b5

Please sign in to comment.