Deprecate config.json, add config.toml support

Author: retrixe

.gitignore

@@ -1,4 +1,5 @@
 # configuration files
+*config.toml
 *config.json
 users.json
 logs/

API.md

@@ -107,19 +107,25 @@ Get Octyne's configuration file contents. Added in v1.1.0.
 
 **Response:**
 
-HTTP 200 response with the configuration file contents in the body. The configuration file format is documented in the [README](https://github.com/retrixe/octyne/blob/main/README.md).
+HTTP 200 response with the configuration file contents in the body. The configuration file format is documented in the [README](https://github.com/retrixe/octyne/blob/main/README.md) and is formatted with either [TOML (supported since v1.4.0)](https://toml.io) or [HuJSON (⚠️ deprecated)](https://github.com/tailscale/hujson).
 
-⚠️ *Warning:* The configuration file uses [HuJSON](https://github.com/tailscale/hujson) instead of JSON! This means that comments and trailing commas are allowed. Don't parse the body assuming that it's JSON.
+The `Content-Type` header will be set to `application/toml` if the config is using [TOML](https://toml.io) or `application/json` if the config is using [HuJSON](https://github.com/tailscale/hujson).
+
+⚠️ *Note:* `Content-Type` was added to the response headers in v1.4.0. If absent, HuJSON can be assumed, since prior versions only support HuJSON.
 
 ---
 
 ### PATCH /config
 
 Modify Octyne's configuration. Added in v1.1.0.
 
+**Request Headers:**
+
+- `Content-Type` - The content type of the request body (either `application/toml` or `application/json`). If not provided, it will be inferred from the request body. Added in v1.4.0.
+
 **Request Body:**
 
-New configuration file contents in the body. The configuration file format is documented in the [README](https://github.com/retrixe/octyne/blob/main/README.md) and is formatted with [HuJSON](https://github.com/tailscale/hujson).
+New configuration file contents in the body. The configuration file format is documented in the [README](https://github.com/retrixe/octyne/blob/main/README.md) and is formatted with either [TOML (supported since v1.4.0)](https://toml.io) or [HuJSON (⚠️ deprecated)](https://github.com/tailscale/hujson).
 
 **Response:**
 

README.md

@@ -4,15 +4,15 @@ A process manager with an HTTP API for remote console and file access.
 
 Octyne allows running multiple apps on a remote server and provides an HTTP API to manage them. This allows for hosting web servers, game servers, bots and so on on remote servers without having to mess with SSH, using `screen` and `systemd` whenever you want to make any change, in a highly manageable and secure way.
 
-It incorporates the ability to manage files and access the terminal output and input over HTTP remotely. For further security, it is recommended to use HTTPS (see [config.json](#configjson)) to ensure end-to-end secure transmission.
+It incorporates the ability to manage files and access the terminal output and input over HTTP remotely. For further security, it is recommended to use HTTPS (see [config.toml](#configtoml)) to ensure end-to-end secure transmission.
 
 [retrixe/ecthelion](https://github.com/retrixe/ecthelion) complements octyne by providing a web interface to control apps on octyne remotely.
 
 ## Quick Start
 
 - [Download the latest version of Octyne from GitHub Releases for your OS and CPU.](https://github.com/retrixe/octyne/releases/latest) Alternatively, you can get the latest bleeding edge version of Octyne from [GitHub Actions](https://github.com/retrixe/octyne/actions?query=branch%3Amain), or by compiling it yourself.
 - Place octyne in a folder (on Linux/macOS/\*nix, mark as executable with `chmod +x <octyne file name>`).
-- Create a `config.json` next to Octyne (see [here](https://github.com/retrixe/octyne#configuration) for details).
+- Create a `config.toml` next to Octyne (see [here](https://github.com/retrixe/octyne#configuration) for details).
 - Run `./<octyne file name>` in a terminal in the folder to start Octyne. An `admin` user will be generated for you.
 - You may want to get [Ecthelion](https://github.com/retrixe/ecthelion) to manage Octyne over the internet, and [octynectl](https://github.com/retrixe/octynectl) as a CLI tool to manage Octyne locally on your machine. [Additionally, make sure to follow the security practices here to prevent attacks against your setup!](https://github.com/retrixe/octyne#security-practices-and-reverse-proxying)
 - You might want to manage Octyne using systemd on Linux systems, which can start/stop Octyne, start it on boot, store its logs and restart it on crash. [This article should help you out.](https://medium.com/@benmorel/creating-a-linux-service-with-systemd-611b5c8b91d6)
@@ -27,10 +27,61 @@ Octyne depends on two files in the current working directory to get configuratio
 
 The path to these files can be customised using the `--config=/path/to/config.json` and `--users=/path/to/users.json` CLI flags (if relative, resolved relative to the working directory).
 
-### config.json
+### config.toml
 
 Used to configure the apps Octyne should start, Redis-based authentication for allowing more than a single node, Unix socket API, and HTTPS support. A reverse proxy can also be used for HTTPS if it supports WSS.
 
+```toml
+port = 42069 # optional, default is 42069
+
+[unixSocket]
+# enables Unix socket API for auth-less actions by locally running apps e.g. octynectl
+enabled = true
+# optional, if absent, default is TMP/octyne.sock.PORT (see API.md for details)
+# location = ""
+# optional, sets the socket's group owner, if absent, default is current user's primary group
+# group = ""
+
+[redis]
+# whether the authentication tokens should sync to Redis for more than 1 node
+enabled = false
+# link to Redis server
+url = "redis://localhost"
+
+[https]
+# whether Octyne should listen using HTTP or HTTPS
+enabled = false
+# path to HTTPS certificate
+cert = "/path/to/cert.pem"
+# path to HTTPS private key
+key = "/path/to/key.pem"
+
+[logging]
+# whether Octyne should log actions
+enabled = true
+# path to log files, can be relative or absolute
+path = "logs"
+
+[logging.actions]
+# optional, disable logging for specific actions, more info below
+
+# each key has the name of the server
+[servers.test1]
+# optional, default true, Octyne won't auto-start when false
+# enabled = true
+# the directory in which the server is located
+directory = "/home/test/server"
+# the command to run to start the server
+command = "java -jar spigot-1.12.2.jar"
+```
+
+### config.json
+
+In Octyne v1.4.0, JSON was replaced with TOML for configuration. However, JSON still works for backwards compatibility, so the documentation below has been retained. Note that JSON is deprecated and will be removed with v2.0. Migrate to v1.4.0 or later to use TOML.
+
+<details>
+<summary>config.json example</summary>
+
 *NOTE: Octyne supports comments and trailing commas in the config.json file, they don't need to be removed.*
 
 ```jsonc
@@ -65,6 +116,8 @@ Used to configure the apps Octyne should start, Redis-based authentication for a
 }
 ```
 
+</details>
+
 ### users.json
 
 Contains users who can log into Octyne. This file is automatically generated on first start with an `admin` user and a generated secure password which is logged to terminal. You can perform account management via Ecthelion, octynectl or other such tools.
@@ -81,7 +134,15 @@ Note: actions performed locally using the Unix socket API by apps like [octynect
 
 **Note: Fine-grained control over logging is currently *experimental*. Therefore, action names may change in any version, not just major versions. However, we will generally try to avoid this in the interest of stability.**
 
-By default, Octyne will log all actions performed by users. You can enable/disable logging for specific actions by setting the action to `true` or `false` in the `logging.actions` object in `config.json`. For example, to disable logging for `auth.login` and `auth.logout`, your `actions` object would be:
+By default, Octyne will log all actions performed by users. You can enable/disable logging for specific actions by setting the action to `true` or `false` in the `logging.actions` section in `config.toml`. For example, to disable logging for `auth.login` and `auth.logout`:
+
+```toml
+[logging.actions]
+"auth.login" = false
+"auth.logout" = false
+```
+
+Or, with the older `config.json` format:
 
 ```json
 "actions": {

config.go

@@ -3,7 +3,9 @@ package main
 import (
 	"encoding/json"
 	"os"
+	"strings"
 
+	"github.com/pelletier/go-toml/v2"
 	"github.com/tailscale/hujson"
 )
 
@@ -19,58 +21,69 @@ var defaultConfig = Config{
 	Servers: map[string]ServerConfig{},
 }
 
+func isJSON(data []byte) bool {
+	return strings.HasPrefix(strings.TrimSpace(string(data)), "{")
+}
+
 func ReadConfig() (Config, error) {
 	config := defaultConfig
-	contents, err := os.ReadFile(ConfigJsonPath)
+	contents, err := os.ReadFile(ConfigFilePath)
 	if err != nil {
 		return config, err
 	}
-	contents, err = hujson.Standardize(contents)
-	if err != nil {
-		return config, err
-	}
-	err = json.Unmarshal(contents, &config)
-	if err != nil {
-		return config, err
+	if isJSON(contents) {
+		contents, err = hujson.Standardize(contents)
+		if err != nil {
+			return config, err
+		}
+		err = json.Unmarshal(contents, &config)
+		if err != nil {
+			return config, err
+		}
+	} else {
+		err = toml.Unmarshal(contents, &config)
+		if err != nil {
+			return config, err
+		}
 	}
 	return config, nil
 }
 
 // Config is the main config for Octyne.
 type Config struct {
-	Port       uint16                  `json:"port"`
-	UnixSocket UnixSocketConfig        `json:"unixSocket"`
-	HTTPS      HTTPSConfig             `json:"https"`
-	Redis      RedisConfig             `json:"redis"`
-	Logging    LoggingConfig           `json:"logging"`
-	Servers    map[string]ServerConfig `json:"servers"`
+	Port       uint16                  `json:"port" toml:"port"`
+	UnixSocket UnixSocketConfig        `json:"unixSocket" toml:"unixSocket"`
+	HTTPS      HTTPSConfig             `json:"https" toml:"https"`
+	Redis      RedisConfig             `json:"redis" toml:"redis"`
+	Logging    LoggingConfig           `json:"logging" toml:"logging"`
+	Servers    map[string]ServerConfig `json:"servers" toml:"servers"`
 }
 
 // RedisConfig contains whether or not Redis is enabled, and if so, how to connect.
 type RedisConfig struct {
-	Enabled bool   `json:"enabled"`
-	URL     string `json:"url"`
+	Enabled bool   `json:"enabled" toml:"enabled"`
+	URL     string `json:"url" toml:"url"`
 }
 
 // HTTPSConfig contains whether or not HTTPS is enabled, and if so, path to cert and key.
 type HTTPSConfig struct {
-	Enabled bool   `json:"enabled"`
-	Cert    string `json:"cert"`
-	Key     string `json:"key"`
+	Enabled bool   `json:"enabled" toml:"enabled"`
+	Cert    string `json:"cert" toml:"cert"`
+	Key     string `json:"key" toml:"key"`
 }
 
 // UnixSocketConfig contains whether or not Unix socket is enabled, and if so, path to socket.
 type UnixSocketConfig struct {
-	Enabled  bool   `json:"enabled"`
-	Location string `json:"location"`
-	Group    string `json:"group"`
+	Enabled  bool   `json:"enabled" toml:"enabled"`
+	Location string `json:"location" toml:"location"`
+	Group    string `json:"group" toml:"group"`
 }
 
 // ServerConfig is the config for individual servers.
 type ServerConfig struct {
-	Enabled   bool   `json:"enabled"`
-	Directory string `json:"directory"`
-	Command   string `json:"command"`
+	Enabled   bool   `json:"enabled" toml:"enabled"`
+	Directory string `json:"directory" toml:"directory"`
+	Command   string `json:"command" toml:"command"`
 }
 
 // UnmarshalJSON unmarshals ServerConfig and sets default values.
@@ -84,9 +97,9 @@ func (c *ServerConfig) UnmarshalJSON(data []byte) error {
 
 // LoggingConfig is the config for action logging.
 type LoggingConfig struct {
-	Enabled bool            `json:"enabled"`
-	Path    string          `json:"path"`
-	Actions map[string]bool `json:"actions"`
+	Enabled bool            `json:"enabled" toml:"enabled"`
+	Path    string          `json:"path" toml:"path"`
+	Actions map[string]bool `json:"actions" toml:"actions"`
 }
 
 // ShouldLog returns whether or not a particular action should be logged.

endpoints_misc.go

@@ -11,6 +11,7 @@ import (
 	"time"
 
 	"github.com/gorilla/websocket"
+	"github.com/pelletier/go-toml/v2"
 	"github.com/retrixe/octyne/auth"
 	"github.com/retrixe/octyne/system"
 	"github.com/tailscale/hujson"
@@ -32,12 +33,17 @@ func configEndpoint(connector *Connector, w http.ResponseWriter, r *http.Request
 		return
 	}
 	if r.Method == "GET" {
-		contents, err := os.ReadFile(ConfigJsonPath)
+		contents, err := os.ReadFile(ConfigFilePath)
 		if err != nil {
-			log.Println("Error reading "+ConfigJsonPath+" when user accessed /config!", err)
+			log.Println("Error reading "+ConfigFilePath+" when user accessed /config!", err)
 			httpError(w, "Internal Server Error!", http.StatusInternalServerError)
 			return
 		}
+		if isJSON(contents) {
+			w.Header().Set("content-type", "application/json")
+		} else {
+			w.Header().Set("content-type", "application/toml")
+		}
 		connector.Info("config.view", "ip", GetIP(r), "user", user)
 		writeJsonStringRes(w, string(contents))
 	} else if r.Method == "PATCH" {
@@ -49,25 +55,34 @@ func configEndpoint(connector *Connector, w http.ResponseWriter, r *http.Request
 		}
 		var origJson = buffer.String()
 		var config Config
-		contents, err := hujson.Standardize(buffer.Bytes())
-		if err != nil {
-			httpError(w, "Invalid JSON body!", http.StatusBadRequest)
-			return
-		}
-		err = json.Unmarshal(contents, &config)
-		if err != nil {
-			httpError(w, "Invalid JSON body!", http.StatusBadRequest)
-			return
+		contentType := r.Header.Get("content-type")
+		if contentType == "application/json" || (contentType == "" && isJSON(buffer.Bytes())) {
+			contents, err := hujson.Standardize(buffer.Bytes())
+			if err != nil {
+				httpError(w, "Invalid JSON body!", http.StatusBadRequest)
+				return
+			}
+			err = json.Unmarshal(contents, &config)
+			if err != nil {
+				httpError(w, "Invalid JSON body!", http.StatusBadRequest)
+				return
+			}
+		} else {
+			err = toml.Unmarshal(buffer.Bytes(), &config)
+			if err != nil {
+				httpError(w, "Invalid TOML body!", http.StatusBadRequest)
+				return
+			}
 		}
-		err = os.WriteFile(ConfigJsonPath+"~", []byte(strings.TrimSpace(origJson)+"\n"), 0666)
+		err = os.WriteFile(ConfigFilePath+"~", []byte(strings.TrimSpace(origJson)+"\n"), 0666)
 		if err != nil {
-			log.Println("Error writing to " + ConfigJsonPath + " when user modified config!")
+			log.Println("Error writing to " + ConfigFilePath + " when user modified config!")
 			httpError(w, "Internal Server Error!", http.StatusInternalServerError)
 			return
 		}
-		err = os.Rename(ConfigJsonPath+"~", ConfigJsonPath)
+		err = os.Rename(ConfigFilePath+"~", ConfigFilePath)
 		if err != nil {
-			log.Println("Error writing to " + ConfigJsonPath + " when user modified config!")
+			log.Println("Error writing to " + ConfigFilePath + " when user modified config!")
 			httpError(w, "Internal Server Error!", http.StatusInternalServerError)
 			return
 		}

go.mod

@@ -6,6 +6,7 @@ require (
 	github.com/gomodule/redigo v2.0.0+incompatible
 	github.com/gorilla/handlers v1.4.2
 	github.com/gorilla/websocket v1.4.2
+	github.com/pelletier/go-toml/v2 v2.2.3
 	github.com/puzpuzpuz/xsync/v3 v3.0.2
 	github.com/tailscale/hujson v0.0.0-20221223112325-20486734a56a
 	go.uber.org/zap v1.24.0

go.sum

@@ -11,6 +11,8 @@ github.com/gorilla/handlers v1.4.2 h1:0QniY0USkHQ1RGCLfKxeNHK9bkDHGRYGNDFBCS+YAR
 github.com/gorilla/handlers v1.4.2/go.mod h1:Qkdc/uu4tH4g6mTK6auzZ766c4CA0Ng8+o/OAirnOIQ=
 github.com/gorilla/websocket v1.4.2 h1:+/TMaTYc4QFitKJxsQ7Yye35DkWvkdLcvGKqM+x0Ufc=
 github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
+github.com/pelletier/go-toml/v2 v2.2.3 h1:YmeHyLY8mFWbdkNWwpr+qIL2bEqT0o95WSdkNHvL12M=
+github.com/pelletier/go-toml/v2 v2.2.3/go.mod h1:MfCQTFTvCcUyyvvwm1+G6H/jORL20Xlb6rzQu9GuUkc=
 github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
 github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
@@ -19,8 +21,8 @@ github.com/puzpuzpuz/xsync/v3 v3.0.2 h1:3yESHrRFYr6xzkz61LLkvNiPFXxJEAABanTQpKbA
 github.com/puzpuzpuz/xsync/v3 v3.0.2/go.mod h1:VjzYrABPabuM4KyBh1Ftq6u8nhwY5tBPKP9jpmh0nnA=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
-github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
-github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
+github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
+github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
 github.com/tailscale/hujson v0.0.0-20221223112325-20486734a56a h1:SJy1Pu0eH1C29XwJucQo73FrleVK6t4kYz4NVhp34Yw=
 github.com/tailscale/hujson v0.0.0-20221223112325-20486734a56a/go.mod h1:DFSS3NAGHthKo1gTlmEcSBiZrRJXi28rLNd/1udP1c8=
 go.uber.org/atomic v1.7.0 h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw=

main.go

@@ -28,18 +28,22 @@ func getPort(config *Config) string {
 }
 
 var info *log.Logger
-var ConfigJsonPath = "config.json"
+var ConfigFilePath = "config.toml"
 var UsersJsonPath = "users.json"
 
 func main() {
+	if _, err := os.Stat(ConfigFilePath); os.IsNotExist(err) {
+		ConfigFilePath = "config.json" // Fallback to JSON if TOML not available.
+	}
+
 	for _, arg := range os.Args {
 		if arg == "--help" || arg == "-h" {
 			println("Usage: " + os.Args[0] + " [--version] [--config=<path>] [--users=<path>]")
 			return
 		} else if strings.HasPrefix(arg, "--users=") {
 			UsersJsonPath = arg[8:]
 		} else if strings.HasPrefix(arg, "--config=") {
-			ConfigJsonPath = arg[9:]
+			ConfigFilePath = arg[9:]
 		} else if arg == "--version" || arg == "-v" {
 			println("octyne version " + OctyneVersion)
 			return