Skip to content

Commit b77a326

Browse files
RA341RA341
authored
Merge pull request #39 from makeopensource/dev
added new docs
2 parents 75bbd87 + 13a14ee commit b77a326

10 files changed

Lines changed: 304 additions & 2 deletions

File tree

‎docs/docs/users/configuration/_category_.json‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"label": "Configuration",
3+
"position": 3,
4+
"link": {
5+
"type": "generated-index",
6+
"description": "Configure leviathan"
7+
},
8+
"collapsed": false
9+
}

‎docs/docs/users/configuration/config_yaml.md‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
---
2+
sidebar_position: 2
3+
title: config.yaml
4+
---
5+
6+
# Configuration file
7+
8+
Leviathan uses a `config.yaml` file for configration, this file is automatically created on first run with
9+
some default values, everything about leviathan can be configured here ([except logging](#logging))
10+
11+
You can also modify some of the options using environment variables.
12+
13+
Below is the default configuration file with explanations for each option.
14+
15+
```yaml
16+
clients:
17+
# Enable local Docker execution for jobs
18+
enable_local_docker: true
19+
# SSH client configurations for remote execution
20+
ssh:
21+
example:
22+
enable: false # Enable/disable this SSH client
23+
host: 192.168.1.69 # Remote host address
24+
port: 22 # SSH port
25+
user: test # SSH username
26+
password: "" # SSH password (if not using key auth)
27+
remotepublickey: "" # Remote public key path
28+
usepublickeyauth: false # Whether to use public key authentication
29+
db:
30+
postgres:
31+
enable_postgres: false # Enable PostgreSQL instead of SQLite
32+
postgres_db: leviathan # Database name
33+
postgres_host: localhost # PostgreSQL host
34+
postgres_pass: postgres # Database password
35+
postgres_port: "5432" # Database port
36+
postgres_ssl: disable # SSL mode (disable, require, verify-ca, verify-full)
37+
postgres_user: postgres # Database user
38+
sqlite:
39+
db_path: ./appdata/config/leviathan.db # SQLite database file path
40+
folder:
41+
# All folder paths should remain within ./appdata to ensure proper hardlinking
42+
job_output_dir: ./appdata/output # Directory for job outputs
43+
labs: ./appdata/labs # Directory for lab files
44+
log_dir: ./appdata/config/logs/leviathan.log # Log file location
45+
ssh_config: ./appdata/config/ssh_config # SSH configuration directory
46+
tmp_submission_dir: ./appdata/submissions # Temporary submission directory
47+
tmp_uploads: ./appdata/tmp_uploads # Temporary upload directory
48+
jobs:
49+
concurrent_jobs: 50 # Maximum number of concurrent jobs
50+
server:
51+
log_level: info # Logging level (debug, info, warn, error)
52+
port: "9221" # Server port
53+
apikey: "" # API key for authentication
54+
```
55+
56+
# Environment Variables
57+
58+
Some configuration options can be overridden using environment variables. Here are the available environment variables:
59+
60+
## Database Configuration
61+
62+
Read more [here](database.md)
63+
64+
## Directory Configuration
65+
66+
Read more [here](folders.md)
67+
68+
## Logging
69+
70+
You can set the following logger options these are available using only env vars
71+
72+
| Environment Variable | Description | Default Value |
73+
|----------------------------------|----------------------------------------------------|---------------|
74+
| `LEVIATHAN_LOG_LEVEL` | Set log level (debug, info, warn, error, disabled) | `info` |
75+
| `LEVIATHAN_LOG_SHOW_CALLER_FILE` | Shows the file where the log was generated from | `false` |
76+
77+
The logging system has the following behaviour:
78+
79+
- Log rotation: Maximum file size of 10MB
80+
- Backup retention: Keeps up to 5 backup files
81+
- Age limit: Logs are kept for up to 30 days
82+
- Compression: Old log files are automatically compressed

‎docs/docs/users/configuration/database.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
---
2+
sidebar_position: 4
3+
title: Database
4+
---
5+
6+
# Database options
7+
8+
Leviathan uses sqlite by default this makes development and testing easier, but it also supports postgres.
9+
10+
## PostgresSQL
11+
12+
To enable either modify the config.yaml or pass in the following environment variables
13+
14+
| Environment Variable | Description | Default Value |
15+
|----------------------|--------------------------|---------------|
16+
| `POSTGRES_ENABLE` | Enable PostgreSQL | `false` |
17+
| `POSTGRES_HOST` | PostgreSQL host | `"localhost"` |
18+
| `POSTGRES_PORT` | PostgreSQL port | `"5432"` |
19+
| `POSTGRES_USER` | PostgreSQL user | `"postgres"` |
20+
| `POSTGRES_PASSWORD` | PostgreSQL password | `"postgres"` |
21+
| `POSTGRES_DB` | PostgreSQL database name | `"postgres"` |
22+
| `POSTGRES_SSL` | PostgreSQL SSL mode | `"disable"` |
23+
24+
## Sqlite
25+
26+
No additional setup is required for sqlite.
27+
28+
If you wish to keep using sqlite, set `postgres_enable: false` and leave the postgres option as is.

‎docs/docs/users/configuration/docker_hosts.md‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
---
2+
sidebar_position: 3
3+
title: Docker Hosts
4+
---
5+
6+
# Connecting to Docker Hosts
7+
8+
Leviathan uses SSH to connect to remote docker hosts and supports multiple ways to connect to Docker hosts.
9+
10+
This can be configured in the `clients` section in `config.yaml`
11+
Here is an example ssh section
12+
13+
```yaml
14+
clients:
15+
enable_local_docker: true
16+
ssh:
17+
example: # key is a nickname for this machine
18+
enable: false
19+
host: 192.168.1.69
20+
port: 22
21+
user: test
22+
password: ""
23+
remotepublickey: ""
24+
usepublickeyauth: false
25+
```
26+
27+
## SSH user considerations
28+
29+
You must ensure that the ssh user that the leviathan uses can access the docker demon properly,
30+
31+
You can create a user with limited permissions that can only control the Docker daemon remotely. This approach follows
32+
the principle of the least privilege, providing only the specific access needed.
33+
34+
## Connection hierarchy
35+
36+
When leviathan connects to a docker host it follows the following order
37+
38+
1. Checks if `use_public_key_auth` is enabled then uses [public key auth](#ssh-public-key-authentication-recommended)
39+
2. If that is disabled, it checks the `passoword` section and uses [password auth](#ssh-password-authentication)
40+
3. If password is empty, then it will fallback to [host public key auth](#ssh-public-key-authentication-host-dependent)
41+
42+
Regardless of the auth method, if the host public key is automatically stored on first
43+
connection and verified on subsequent connections.
44+
45+
If the `remotepublickey` is set before first connect then it will be used to verify.
46+
47+
**Leviathan requires at least 1 docker host connected at anytime,
48+
if it is unable to connect to any docker host then it will exit with a fatal error.**
49+
50+
## Authentication Methods
51+
52+
### SSH Public Key Authentication (Recommended)
53+
54+
```yaml
55+
use_public_key_auth: true
56+
```
57+
58+
- Uses auto-generated key pair stored in Leviathan's config directory, found in `appdata/config/ssh_config`
59+
- Requires adding Leviathan's public key to remote host's `~/.ssh/authorized_keys`
60+
- No password required
61+
62+
### SSH Password Authentication
63+
64+
```yaml
65+
password: "some password"
66+
```
67+
68+
- Basic authentication method
69+
- Recommended for initial setup/testing only
70+
71+
### SSH Public Key Authentication (Host dependent)
72+
73+
- Uses key pair stored on the machine running leviathan typically at `~/.ssh`
74+
- Requires mounting the `.ssh` dir into the docker container or
75+
- transferring the keys manually and setting up `.ssh` inside the container file system
76+
77+
### Local Docker Connection
78+
79+
- Enabled by default on first run.
80+
- Requires Docker daemon running on local machine
81+
- Requires mounting `/var/run/docker.sock:/var/run/docker.sock` into the container
82+
- Disable with `enable_local_docker: false` in config.yaml
83+
84+
# Configuration Parameters
85+
86+
| Parameter | Required | Default | Description | Notes |
87+
|-----------------------|:--------:|:-------:|----------------------------------|----------------------------------------------------|
88+
| `enable` | No | `true` | Host availability | Set to `false` to disable host |
89+
| `host` | Yes | - | Host machine IP | `http://192.168.1.1` or `https:remote.docker.com` |
90+
| `port` | No | `22` | SSH port number | Standard SSH port used if not specified |
91+
| `user` | Yes | - | SSH login username | |
92+
| `password` | No | - | SSH password | Used for password-based authentication |
93+
| `remote_public_key` | No | - | Remote host's public key | Automatically set by Leviathan on first connection |
94+
| `use_public_key_auth` | No | `false` | Enable public key authentication | Set to `true` to use leviathan's SSH key pairs |
95+
96+

‎docs/docs/users/configuration/folders.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
---
2+
sidebar_position: 5
3+
title: Folders
4+
---
5+
6+
# Folder config
7+
8+
Leviathan uses the following folders for its operations.
9+
10+
| Environment Variable | Description | Purpose/Usage | Default Value |
11+
|-------------------------|------------------------------|--------------------------------------------------------------------------------|-------------------------------|
12+
| `TMP_SUBMISSION_DIR` | Temporary submission storage | Stores job submissions temporarily; contents are deleted after job completion | `./appdata/submissions` |
13+
| `SUBMISSION_OUTPUT_DIR` | Job output storage | Preserves stdout and logs from containers where jobs executed | `./appdata/output` |
14+
| `SSH_CONFIG_DIR` | SSH configuration storage | Contains Leviathan-managed SSH keys for remote Docker host connections | `./appdata/config/ssh_config` |
15+
| `LABS_DIR` | Laboratory files storage | Stores lab files | `./appdata/labs` |
16+
| `TMP_UPLOAD_DIR` | Temporary upload storage | Facilitates file transfers to Leviathan; contents are deleted after processing | `./appdata/tmp_uploads` |
17+
18+
# Folder Configuration
19+
20+
Leviathan uses a structured directory system for its operations.
21+
22+
## Hardlinks
23+
24+
Leviathan uses hardlinks for efficient job processing. Changing folder locations requires special consideration:
25+
26+
- 🚫 **Avoid network mounts** - NFS/SMB volumes often lack proper hardlink support
27+
- 📂 **Maintain same filesystem** - All paths must reside on the same physical storage device
28+
29+
### Docker Deployment Considerations
30+
31+
When deploying via Docker, volume mounting requires special attention. For example:
32+
33+
`/some_host_mount:/appdata/folder1`
34+
35+
`/some_host_mount:/appdata/folder2`
36+
37+
Docker treats these paths as separate filesystems, preventing hardlinks between them.
38+
39+
❗ Changing these paths incorrectly may lead to:
40+
41+
- Failed job submissions
42+
- Corrupted submission artifacts
43+
- Incomplete output generation

‎docs/docs/users/configuration/intro.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
---
2+
sidebar_position: 1
3+
title: Intro
4+
---
5+
6+
# Leviathan Configuration
7+
8+
Leviathan creates an `appdata` directory in its working directory upon startup. This directory stores all configuration and runtime data.
9+
10+
For Docker deployments, the `appdata` directory is located at `/app/appdata` inside the container. To persist data between container restarts, you should mount this directory as a volume.
11+
12+
## Docker Compose Example
13+
14+
Below is a recommended `docker-compose.yml` configuration:
15+
16+
```yaml
17+
services:
18+
leviathan:
19+
container_name: leviathan
20+
image: ghcr.io/makeopensource/leviathan:master
21+
volumes:
22+
- ./appdata:/app/appdata
23+
ports:
24+
- "9221:9221"
25+
restart: unless-stopped
26+
```

‎docs/docs/users/how-it-works.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
sidebar_position: 2
3+
title: How it works
4+
---

‎docs/docs/users/installation/_category_.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
"position": 2,
44
"link": {
55
"type": "generated-index",
6-
"description": "Guide to installing leviathan on various platforms."
6+
"description": "Guide to installing leviathan on various platforms."
77
},
88
"collapsed": false
99
}

‎docs/docs/users/intro.md‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,20 @@ title: About
66

77
# Leviathan
88

9-
Leviathan is a container orchestrator and job runner intended to be code runner component of the [DevU](https://github.com/makeopensource/devU).
9+
A container orchestrator and job runner for executing code in docker containers.
10+
11+
## Overview
12+
13+
Leviathan is the code execution engine for the [DevU project](https://github.com/makeopensource/devU).
14+
It securely runs submitted code in isolated containers, providing evaluation results and feedback.
15+
Designed as a modern replacement for [Autolab Tango](https://github.com/autolab/Tango).
16+
17+
Leviathan maintains full compatibility with existing Autolab/Tango graders. Any graders written for Tango will run on Leviathan without modification.
18+
19+
## Features
20+
- Secure container-based execution
21+
- Resource limiting and management
22+
- gRPC API for type-safe, efficient communication
1023

1124
## How it Works
1225

‎docs/package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
"serve": "docusaurus serve",
1313
"write-translations": "docusaurus write-translations",
1414
"write-heading-ids": "docusaurus write-heading-ids",
15+
"md": "echo '# New Markdown File' >",
1516
"typecheck": "tsc"
1617
},
1718
"dependencies": {

0 commit comments

Comments
 (0)