Merge pull request #1 from starters-dev/v0.1.0

v0.1.0

Author: kanzitelli

.env.example

@@ -1,3 +1,5 @@
-# [SYSTEM]
-DOMAIN_NAME=your-domain.com
-DO_AUTH_TOKEN=XXXXXXXXXXXXXXXXX
+# [GENERAL]
+DOMAIN_NAME=your-website.com
+DO_AUTH_TOKEN=xxxxxxxxxxxxxxxxxxxxx # for DNS challenge
+ACME_EMAIL=[email protected]   # for let's encrypt
+GITHUB_TOKEN=xxxxxxxxxxxxxxxxxxxxxx # (optional) for private repos

.gitignore

@@ -1,3 +1,4 @@
+# General
 .DS_Store
 .env
 acme/

README.md

@@ -1,61 +1,195 @@
-# https-backend
+# 🔐 Universal HTTPS Backend
 
-This repository is a minimal setup for running your services with Docker (Compose) behind Traefik proxy and with autogenerated HTTPS certificates. That means if you have the `API` service (e.g. NodeJS app) and would like to make it available through `https://api.your-domain.com`, then you are at the right place.
+![Traefik](https://img.shields.io/badge/Proxy-Traefik-skyblue)
+![HTTPS](https://img.shields.io/badge/HTTPS-Configured-green)
+![PostgreSQL](https://img.shields.io/badge/Database-PostgreSQL-blue)
+![Redis](https://img.shields.io/badge/Cache-Redis-red)
+![Plausible](https://img.shields.io/badge/Analytics-Plausible-purple)
+![Website](https://img.shields.io/badge/Frontend-React%20App-blue)
 
-## Quickstart
+This repository is a minimalistic setup for running services with Docker Compose behind Traefik and autogenerated `https` certificates.
 
-1. Connect to the remote server
+Let's say, you have the `API` service (e.g. NodeJS app), React app for frontend and would like to make it available via `https://api.your-domain.com` and `https://dashboard.your-domain.com`, then you are at the right place!
+
+It includes the collection of `docker-compose` files and bash scripts that automate and simplify startup of your project, by taking care of some DevOps routine. List with available services can be found at [Services section](#services).
+
+> _HTTPS Backend_ is a part of [starters.dev](https://github.com/starters-dev) collection.
+
+## Getting Started
+
+### Quickstart
+
+1. Connect to your remote server
 
 ```bash
 > ssh root@IP_ADDRESS
 ```
 
+> Hint: you can use a cloud computing providers such as [DigitalOcean](https://digitalocean.com).
+
 2. Clone this repo
 
 ```bash
 > git clone https://github.com/starters-dev/https-backend backend
 > cd backend
 ```
 
-3. Fill in `.env` file with your information
+3. Run `setup` script and fill in `env` variables
 
 ```bash
-> mv .env.example .env
+> bash setup.sh
 > nano .env
 ```
 
-4. Get the service repo (e.g. `nextjs-tailwindcss`)
+Check [Environment variables](#environment-variables) section for more information.
+
+4. Get a service repo
+
+> Our example is based on [backend.starters.dev-website](https://github.com/starters-dev/backend.starters.dev-website) repo.
 
 ```bash
-> bash run/get-repo.sh nextjs-tailwindcss
+> git clone https://github.com/starters-dev/backend.starters.dev-website services/frontend
 ```
 
-It will fetch the following repo `https://github.com/starters-dev/nextjs-tailwindcss`
-
 5. Build and run
 
 ```bash
-> bash run/build.sh
+> bash build.sh
 ```
 
-It will setup everything, including ACME (https certificates), and will run docker.
+It will setup everything, including https certificates, and will start Docker Compose services in the background.
 
-## Tips
+### Service creation
 
----
+If you'd like to add your own service, then you should follow the steps below to keep the same structure:
+
+1. Add service's Docker Compose file `services/docker-compose.YOUR_SERVICE.yml`.
+
+2. `(optional)` Add service's folder `services/YOUR_SERVICE`. For example, if you are going to store some data or configuration files only related to that service.
+
+3. `(optional)` Add service's script file `services/YOUR_SERVICE/index.sh`. For example, if you need to craete some folders in advance on `setup` step.
+
+### Service deletion
 
-If you'd like to add your service or from [starters-dev](https://github.com/starters-dev), create a dedicated `docker-compose.your-service.yml` file and add it to `run/build.sh` and then run:
+If you don't need a service, you can easily delete it by running:
 
 ```bash
-> bash run/get-repo.sh <service-repo>
+> bash run/sdel.sh SERVICE_NAME
 ```
 
+For example, if you don't need `postgresql` service, you will run `bash run/sdel.sh postgresql` and it will delete related files.
+
+## Structure
+
+The folder structure is pretty simple but lets you reuse and write less boilerplate code.
+
+- `docker-compose.main.yml` - main Docker Compose file with DNS challence, ACME and ports configuration.
+- `build.sh` - `build` script that runs `setup` script with all scripts under `run/services` folder and builds and runs Docker images in background.
+- `.env` - list of environment variables.
+- `run/` - folder with general bash scripts.
+- `services/` - folder with Docker services files. For example, if you have `redis` service, then you would have `services/docker-compose.redis.yml` with specific configuration for `redis` service. Or if your service is a frontend app, then you'll need to clone it to `services/frontend/` and create `services/docker-compose.frontend.yml`.
+- `services/your-service/` - folder with your service, for ex. `frontend`.
+
+## What's inside
+
+The setup uses [Traefik](https://github.com/traefik/traefik), the cloud native application proxy, to simplify the process of getting `https` certificates and the way to describe services.
+
+You can always delete existing (if you don't need them) or add your own services, just make sure to apply necessary changes.
+
+### Services
+
+---
+
+[PostgreSQL](https://www.postgresql.org) - open source object-relational database known for reliability and data integrity.
+
+It uses [Bitnami Docker image](https://hub.docker.com/r/bitnami/postgresql).
+
+<details>
+<summary>Env variables</summary>
+
+- `POSTGRESQL_USERNAME` - username for PostgreSQL.
+- `POSTGRESQL_PASSWORD` - password for PostgreSQL.
+- `POSTGRESQL_DATABASE` - name of the database in PostgreSQL.
+
+</details>
+
 ---
 
+[Redis](https://redis.io) - open source, in-memory data store used by millions of developers as a database, cache, streaming engine, and message broker.
+
+It uses [Bitnami Docker image](https://hub.docker.com/r/bitnami/redis).
+
+<details>
+<summary>Env variables</summary>
+
+- `REDIS_PASSWORD` - password for Redis.
+- `REDIS_DISABLE_COMMANDS` - commands disabled from execution.
+
+</details>
+
 ---
 
-You can find example `env` file in the root folder.
+[Plausible](https://plausible.io) - simple and privacy-friendly Google Analytics alternative.
+
+[Official documentation](https://plausible.io/docs/self-hosting).
+
+<details>
+<summary>Env variables</summary>
+
+`services/plausible/conf.env` file:
+
+- `SECRET_KEY_BASE` - secret key of your app. Can be generated by running `openssl rand -base64 64 | tr -d '\n' ; echo`.
+- `BASE_URL` - base url where this instance is accessible, including the scheme (eg. `http://` or `https://`), the domain name, and optionally a port.
+- `ADMIN_USER_EMAIL` - admin email.
+- `ADMIN_USER_NAME` - admin usenamename.
+- `ADMIN_USER_PWD` - admin password.
+
+</details>
+
+<details>
+<summary>Troubleshooting</summary>
+
+- [ClickHouse] _Database plausible_events_db doesn't exist_. If you encounter this issue, check out [this comment](https://github.com/plausible/hosting/issues/49#issuecomment-1229183219) for the solution.
+
+</details>
 
-`DO_AUTH_TOKEN` is used to generate https certificates against [DigitalOcean](https://digitalocean.com) challenge. You can generate one in the DO Networking dashboard or choose one of the [available providers](https://doc.traefik.io/traefik/https/acme/#providers).
+---
+
+[React App](https://backend.starters.dev) - simple [frontend app](https://github.com/starters-dev/backend.starters.dev-website).
 
 ---
+
+> more services will be added...
+
+### Env variables
+
+Current setup requires you to fill in `.env` file with variables that are used in services.
+
+- `DOMAIN_NAME` - your registered domain.
+- `DO_AUTH_TOKEN` - Digital Ocean token that is going to be used for DNS challenge and generating https certificates. It's required by Traefik and they provide other options, you can find them [here](https://doc.traefik.io/traefik/https/acme/#providers). If you'd like to continue with Digital Ocean, then you can create a token in `Dashboard` -> `API` -> `Tokens/Keys`.
+- `ACME_EMAIL` - email that is used for [Let's Encrypt](https://letsencrypt.org) and `https` certificates.
+- `GITHUB_TOKEN` - `(optional)` github token for private repos.
+
+## Enhancements
+
+There are still some things I would like to add to the backend setup:
+
+- [x] [PostgreSQL](https://www.postgresql.org) - open source object-relational database known for reliability and data integrity.
+- [x] [Redis](https://redis.io) - open source, in-memory data store used by millions of developers as a database, cache, streaming engine, and message broker.
+- [x] [Plausible](https://plausible.io) - simple and privacy-friendly Google Analytics alternative.
+- [ ] [GlitchTip](https://glitchtip.com) - open source reimplementation of Sentry error tracking platform.
+- [ ] [Cal.com](https://cal.com) - scheduling infrastructure for absolutely everyone. [Github repo](https://github.com/calcom/docker).
+- [ ] [Mattermost](https://mattermost.com) - open source platform for developer collaboration. [Github repo](https://github.com/starters-dev/mattermost).
+- [ ] [Taiga](https://www.taiga.io) - open source, self-hosted project management tool. [Github repo](https://github.com/starters-dev/taiga).
+- [ ] [Focalboard](https://focalboard.com) - open source, self-hosted alternative to Trello, Notion, and Asana. [Github repo](https://github.com/starters-dev/focalboard).
+- [ ] Github actions or similar technique
+
+## Why?
+
+While developing `API` and similar services for mobile apps, you can not really access `localhost` (on mobile device) if, let's say, you have running Docker image on you local machine. Also you have to make only `https` requests from mobile app.
+
+Another reason was to have `PostgreSQL` and `Redis` always running in the cloud but for cheap cost. You can run both of them easily on the $6 server.
+
+#### Note
+
+This backend setup is a great fit if you just want to start writing business logic without messing with DevOps and spending a few days on that what's already has been done for you. It can be used for development and early production stages (of course, depends on your project), however, it's strongly **recommended** to have services like `PostgreSQL`, `Redis`, and similar to be seperated and independent on production stage.

build.sh

@@ -0,0 +1,9 @@
+# -- getting docker-compose files
+DC_FILES="-f docker-compose.yml"
+for F in `find ./services -maxdepth 1 -type f -name "docker-compose*.yml"`
+do
+  DC_FILES="$DC_FILES -f $F "
+done
+
+# -- building containers
+docker-compose $DC_FILES build && docker-compose $DC_FILES up -d

docker-compose.nextjs-tailwindcss.yml

@@ -9,16 +9,20 @@ services:
       - "--providers.docker.exposedbydefault=false"
       - "--entrypoints.http.address=:80"
       - "--entrypoints.https.address=:443"
+      - "--entrypoints.postgresql.address=:5432" # PostgreSQL entry point
+      - "--entrypoints.redis.address=:6379" # Redis entry point
       - "--certificatesresolvers.mydnschallenge.acme.dnschallenge=true"
       - "--certificatesresolvers.mydnschallenge.acme.dnschallenge.provider=digitalocean"
-      - "--certificatesresolvers.mydnschallenge.acme.email=[email protected]"
+      - "--certificatesresolvers.mydnschallenge.acme.email=${ACME_EMAIL}"
       - "--certificatesresolvers.mydnschallenge.acme.storage=/acme.json"
     environment:
       - DO_AUTH_TOKEN=${DO_AUTH_TOKEN}
     ports:
       - "80:80"
       - "443:443"
-      - "6969:8080"
+      - "6969:8080" # Traefik dashboard
+      - "5432:5432" # PostgreSQL port
+      - "6379:6379" # Redis port
     volumes:
       - /var/run/docker.sock:/var/run/docker.sock
       - ./acme/acme.json:/acme.json

docker-compose.yml

@@ -1,3 +1,4 @@
+# `acme/acme.json` file for storing https certificates
 DIR=acme
 FILE=acme/acme.json
 
@@ -7,5 +8,6 @@ fi
 
 if [ ! -f "$FILE" ]; then
   touch "$FILE"
-  chmod 600 "$FILE"
-fi
+fi
+
+chmod 600 "$FILE"

run/acme.sh

@@ -0,0 +1,19 @@
+SNAME=$1
+
+# -- adding docker compose file
+SDC=services/docker-compose.$SNAME.yml
+if [ ! -f "$SDC" ]; then
+  touch $SDC
+fi
+
+# -- adding service folder
+SDIR=services/$SNAME
+if [ ! -d "$SDIR" ]; then
+  mkdir $SDIR
+fi
+
+# -- adding script
+SSH=services/$SNAME/index.sh
+if [ ! -f "$SSH" ]; then
+  touch $SSH > SNAME=$SNAME
+fi 

run/build.sh

@@ -0,0 +1,13 @@
+SNAME=$1
+
+# -- deleting service folder
+SDIR=services/$SNAME
+if [ -d "$SDIR" ]; then
+  rm -rf $SDIR
+fi
+
+# -- deleting docker compose file
+SDC=services/docker-compose.$SNAME.yml
+if [ -f "$SDC" ]; then
+  rm -rf $SDC
+fi

run/get-repo.sh

@@ -0,0 +1,15 @@
+version: "3.7"
+
+services:
+  frontend:
+    build: ./services/frontend
+    restart: always
+    labels:
+      - "traefik.enable=true"
+      - "traefik.http.routers.frontend.rule=Host(`backend.${DOMAIN_NAME}`)"
+      - "traefik.http.routers.frontend.entryPoints=http"
+      - "traefik.http.routers.frontend.middlewares=https_redirect"
+      - "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https"
+      - "traefik.http.routers.frontend_tls.rule=Host(`backend.${DOMAIN_NAME}`)"
+      - "traefik.http.routers.frontend_tls.entryPoints=https"
+      - "traefik.http.routers.frontend_tls.tls.certresolver=mydnschallenge"