Skip to content

Docker Workflow for Development

Jump to bottom Edit New page
Max Patiiuk edited this page Oct 25, 2022 · 13 revisions

Instructions for setting up the development environment

NOTE: Development Docker composition is not suitable for Production use.

This is a Development Docker composition of Specify 7

It has nice features like hot reloading and debugging support

However, it is not suited for production use due to memory leaks and security issues

Though, you can use it for evaluation of Specify 7

Members of Specify Collections Consortium get access to production-ready Docker compositions. Please contact us at [email protected] if you would like to get access

This tutorial assumes basic knowledge of Docker and Docker Compose.

You can read these tutorials on Docker and Docker Compose to get started.

Installation

  1. Install Docker and Docker compose:

    On macOS (assuming brew is installed):

    brew install --cask docker

    On Linux:

    sudo apt-get install \
  apt-transport-https \
  ca-certificates \
  curl \
  gnupg \
  lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo \
  "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io
sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker
sudo systemctl enable docker.service
sudo systemctl enable containerd.service
curl -L https://raw.githubusercontent.com/docker/compose-cli/main/scripts/install/install_linux.sh | sh
docker plugin install --grant-all-permissions vieux/sshfs

  2. Checkout the repository (assuming git is installed)

    git clone https://github.com/specify/specify7/
cd specify7

  3. Put a MySQL dump of the database you want to use into the ./seed-database folder (name of the file does not matter)

  4. Build the images and run the containers:

    docker compose up --build
# or, if you are on a legacy version:
docker-compose up --build

    On subsequent runs, you don't have to provide the "--build" argument, unless the list of front-end or back-end dependencies has changed.

  5. Make the code changes you want. Both the backend and the frontend would be updated in real-time.

    If you want to receive audible notifications when the rebuild process is finished, here are the instructions for setting that up

Afterward, you can stop the containers by pressing Ctrl+C or typing docker compose down in a separate terminal window.

Changing the schema version

  1. Open the ./docker-compose.yml file

  2. Find the following lines:

    specify6:                                                             
  image: specifyconsortium/specify6-service:6.8.01

  3. Replace "6.8.01" with the desired version. Full list of available versions

  4. Rebuild the containers:

    docker compose up --build

Changing the branch

  1. Checkout the branch you want:

    git switch production

    Full list of branches

  2. Get the latest changes from GitHub:

    git pull

  3. If the branch you are switcing from and the new one have the same dependencies, you don't have to stop the containers and rebuild them. Simply run docker compose up if containers are not yet running.

    If there were some dependency changes, or you are not sure, run docker compose up --build to rebuild the containers.

Changing the database

  1. Put the database dump into ./seed_database

  2. SSH into the MariaDB container. You can do so by pressing the >_ button next to the container name in Docker Dashboard, or by executing the following command in the terminal:

    # Assumming your container is named "specify7-mariadb-1":
docker exec -it specify7-mariadb-1 env TERM=xterm-256color /bin/sh

  3. Run the following commands in order:

    # Navigate into the mounted volume in the container that holds the database dump
cd doc*
# List the files in that directory. If everything is correct, you should see the name of your database dump file
ls
# Connect to the database
mysql -uroot -proot
    # Delete current database from the MariaDB server. This will delete all data. If you need to preserver data, make a database dump using `mysqldump` first
DROP DATABASE `specify`;
# Create a blank database
CREATE DATABASE `specify`;
# Select the database
USE `specify`;
# Import the database from the dump. Assuming your dump file is called "dump.sql":
SOURCE "./dump.sql";
# Disconnect from MySQL
EXIT;
    # Disconnect from the container
exit;

  4. Run database migrations. This step is needed only once for a database before it is used with Specify 7 for the first time or after a Specify 7 update.

    docker exec -it specify7-specify7-1 ve/bin/python manage.py migrate

After this, refresh the "Specify 7" window in your browser and log in again if prompted.

Connecting to database from outside the container

  1. Open ./docker-compose.yml

  2. Find the section that begins with mariadb:

  3. Add a new line:

    mysql:
   # Add these two lines line:
   ports:
   - '127.0.0.1:3306:3306'

  4. Rebuild the containers:

    docker compose up --build

This would make database accessible at localhost with port 3306.

Running the server on a remote machine

A common use case for using Docker is having the containers run on a dedicated machine. This frees up resources on your laptop for other tasks.

This can be atchived in Docker by using code syncing and Docker contexts.

Configuring Docker Contexts

Docker Contexts allows to manage remote Docker servers.

Read more about them here

And here is how to use them

For example:

# Create a context
docker context create maxxxxxdlp-pc --default-stack-orchestrator=swarm --docker "host=tcp://192.168.50.165:2375" --description "Home Ubuntu PC"

Then, you can define default context like docker context use maxxxxxdlp-pc or define an alias and prepend it to your commands:

alias dd="DOCKER_CONTEXT=maxxxxxdlp-pc "
dd docker container ls

Configuring Code Sync

We need to update the remove server in real time as soon as any code changes are made. mirror is designed just for that task.

Instructions for setting up Code Sync with Docker

The commands I used:

# Add this to your .zshrc file:
alias sync_client='docker run --rm --init -it -u $(id -u):$(id -g) -v $(pwd):/data \
  quay.io/stephenh/mirror client \
  --local-root /data \
  --remote-root /data \
  --host 192.168.50.165 \
  --include "./seed-database" --exclude "./idea" --exclude "nohup.out"'
alias sync_server='docker run \
  --rm --init -it -u $(id -u):$(id -g) -v $(pwd):/data -p 49172:49172 \
  quay.io/stephenh/mirror server'

Then just run sync_server on the server and sync_client on the client machine.

Overwriting default preferences

The easiest way currently to override Specify settings that aren't connected to an environment variable in a Docker container is:

  1. Copy the local_specify_settings.py out of the container docker-compose exec specify7 cat /opt/specify7/settings/local_specify_settings.py > override_specify_settings.py
  2. Make the desired changes in override_specify_settings.py
  3. Mount the modified settings file on top of the file in the container by adding - "./override_specify_settings.py:/opt/specify7/settings/local_specify_settings.py" to the volumes section of specify7 service.
Add a custom footer

Clone this wiki locally