Switch from YAML to JSON for configuration, remove Telegram integrati…

…on (#46)

* remove telegram & switch from YAML to JSON for configuration

* fix .gitignore

* allow reading config from an envar

* fix readme

* polish readme

.editorconfig

@@ -14,6 +14,9 @@ indent_style = space
 [*.md]
 trim_trailing_whitespace = false
 
+[*.json]
+indent_size = 2
+
 [*.yml]
 indent_size = 2
 

.github/workflows/ci.yml

@@ -11,7 +11,7 @@ jobs:
     - name: Setup Go
       uses: actions/setup-go@v1
       with:
-        go-version: 1.14
+        go-version: 1.16
 
     - name: Check gofmt
       run: test -z "$(gofmt -s -d .)"
@@ -22,6 +22,9 @@ jobs:
     - name: Make sure that go.mod has already been tidied
       run: go mod tidy && git diff --no-patch --exit-code
 
+    - name: Build
+      run: go build ./cmd/entrello
+
     - name: Run tests
       run: go test -covermode=count -coverprofile=profile.cov ./...
 

.github/workflows/release.yml

@@ -18,7 +18,7 @@ jobs:
       - name: Setup Go
         uses: actions/setup-go@v1
         with:
-          go-version: 1.14.x
+          go-version: 1.16.x
 
       - name: GoReleaser
         uses: goreleaser/goreleaser-action@v1

.gitignore

@@ -1,2 +1,2 @@
-*config.yml
 *.json
+!config.example.json

README.md

@@ -4,84 +4,76 @@
 [![Go Report Card](https://goreportcard.com/badge/github.com/utkuufuk/entrello)](https://goreportcard.com/report/github.com/utkuufuk/entrello)
 [![Coverage Status](https://coveralls.io/repos/github/utkuufuk/entrello/badge.svg)](https://coveralls.io/github/utkuufuk/entrello)
 
-Run this as a cron job to periodically poll your data sources via HTTP and automatically create/update/delete Trello cards based on fresh data.
+Polls compatible data sources and keeps your Trello cards synchronized with fresh data. Meant to be run as a scheduled job.
 
-An example use case could be to create a Trello card for each GitHub issue that's assigned to you.
+Let's say you have an HTTP endpoint that returns GitHub issues assigned to you upon a `GET` request.
+You can point `entrello` to that endpoint to keep your GitHub issues synchronized in your Trello board.
 
-Your data sources must return a JSON array of Trello card objects upon a `GET` request. You can import and use the `NewCard` function from `pkg/trello/trello.go` in order to construct Trello card objects.
+Each data source must return a JSON array of Trello card objects upon a `GET` request. You can import and use the `NewCard` function from `pkg/trello/trello.go` in order to construct Trello card objects.
 
 ## Configuration
-Copy and rename `config.example.yml` as `config.yml` (default), then set your own values in `config.yml`.
+Copy and rename `config.example.json` as `config.json` (default), then set your own values in `config.json`.
 
 You can also use a custom config file path using the `-c` flag:
 ```sh
 go run ./cmd/entrello -c /path/to/config/file
 ```
 
+Alternatively, you can store the configuration inside the `ENTRELLO_CONFIG` environment variable as a JSON string.
+
 ### Trello
 You need to set your [Trello API key & token](https://trello.com/app-key) in the configuraiton file, as well as the Trello board ID.
 
-### Telegram
-You need a Telegram token & a chat ID in order to receive alerts in case of errors.
-
 ### Data Sources
-Each data source must have the following configuration parameters. Refer to `config.example.yml` for examples.
+For each data source, the following parameters have to be specified. (See `config.example.json`)
 
-#### **`name`**
-Data source name.
+- `name` — Data source name.
 
-#### **`endpoint`**
-Data source endpoint. `entrello` will make a `GET` request to this endpoint to fetch fresh cards from the data source. 
+- `endpoint` — Data source endpoint. `entrello` will make a `GET` request to this endpoint to fetch fresh cards from the data source.
 
-#### **`strict`**
-When strict mode is enabled, previously auto-generated cards that are no longer present in the fresh data will be deleted.
+- `strict` — When strict mode is enabled, previously auto-generated cards that are no longer present in the fresh data will be deleted. For instance, with a GitHub data source, strict mode can be useful for automatically removing previously auto-generated cards for issues/PRs from the board when the corresponding issues/PRs are closed/merged.
 
-For instance, with a GitHub data source, strict mode can be useful for automatically removing previously auto-generated cards for issues/PRs from the board when the corresponding issues/PRs are closed/merged.
+- `label_id` — **Distinct** Trello label ID associated with the data source.
 
-#### **`label_id`**
-**Distinct** Trello label ID associated with the data source.
+- `list_id` — Trello list ID for the data source to determine where to insert new cards. The selected list must be in the same board as configured by the `board_id` parameter.
 
-#### **`list_id`**
-Trello list ID for the data source to determine where to insert new cards. The selected list must be in the same board as configured by the `board_id` parameter.
+- `period` — Polling period for the data source. Some examples:
+    ```json
+    // query at 3rd, 6th, 9th, ... of each month
+    "period": {
+      "type": "day",
+      "interval": 3
+    }
 
-#### **`period`**
-Polling period for the data source.
+    // query at 00:00, 02:00, 04:00, ... every day
+    "period": {
+      "type": "hour",
+      "interval": 2
+    }
 
-Example configuration:
-```yml
-# query at 3rd, 6th, 9th, ... of each month
-period:
-  type: day
-  interval: 3
+    // query at XX:00, XX:15, XX:30 and XX:45 every hour
+    "period": {
+      "type": "minute",
+      "interval": 15
+    }
 
-# query at 00:00, 02:00, 04:00, ... every day
-period:
-  type: hour
-  interval: 2
-
-# query at XX:00, XX:15, XX:30 and XX:45 every hour
-period:
-  type: minute
-  interval: 15
-
-# query on each execution
-period:
-  type: default
-  interval:
-```
+    // query on each execution
+    "period": {
+      "type": "default",
+      "interval": 0
+    }
+    ```
 
 ## Example Cron Job
-It's important to make sure that the cron job runs frequently enough to accomodate the most frequent custom interval for a source. It wouldn't make sense to define a custom period of 15 minutes while the cron job only runs every hour.
+Make sure that the cron job runs frequently enough to keep up with the most frequent custom interval in your configuration.
+
+For instance, it wouldn't make sense to define a custom period of 15 minutes while the cron job only runs every hour.
 
-Both of the following jobs run every hour and both assume that `config.yml` is located in the current working directory.
+Both of the following jobs run every hour and both assume that `config.json` is located in the current working directory, or its contents are stored within the `ENTRELLO_CONFIG` environment variable.
 ``` sh
-# use "go run"
-# 'config.yml' should be located in '/home/utku/git/entrello'
-# your go executable may or may not be located in the same location (i.e. /usr/local/go/bin/)
-0 * * * * cd /home/utku/git/entrello && /usr/local/go/bin/go run ./cmd/entrello
-
-# use binary executable
-# see releases: https://github.com/utkuufuk/entrello/releases
-# 'config.yml' should be located in '/path/to/binary'
+# using "go run"
+0 * * * * cd /home/you/git/entrello && /usr/local/go/bin/go run ./cmd/entrello
+
+# use binary executable (see releases: https://github.com/you/entrello/releases)
 0 * * * * cd /path/to/binary && ./entrello
 ```

cmd/entrello/main.go

@@ -3,33 +3,29 @@ package main
 import (
 	"flag"
 	"log"
+	"os"
 	"sync"
 	"time"
 
 	"github.com/utkuufuk/entrello/internal/config"
-	"github.com/utkuufuk/entrello/internal/syslog"
+	"github.com/utkuufuk/entrello/internal/logger"
 	"github.com/utkuufuk/entrello/pkg/trello"
 )
 
-var (
-	logger syslog.Logger
-)
-
 func main() {
 	var configFile string
-	flag.StringVar(&configFile, "c", "config.yml", "config file path")
+	flag.StringVar(&configFile, "c", "config.json", "config file path")
 	flag.Parse()
 
 	cfg, err := config.ReadConfig(configFile)
 	if err != nil {
 		log.Fatalf("Could not read configuration: %v", err)
 	}
 
-	logger = syslog.NewLogger(cfg.Telegram)
-
 	loc, err := time.LoadLocation(cfg.TimezoneLocation)
 	if err != nil {
-		logger.Fatalf("Invalid timezone location: %v", loc)
+		logger.Error("Invalid timezone location: %v", loc)
+		os.Exit(1)
 	}
 
 	sources, labels := getSources(cfg.Sources, time.Now().In(loc))
@@ -39,11 +35,13 @@ func main() {
 
 	client, err := trello.NewClient(cfg.Trello)
 	if err != nil {
-		logger.Fatalf("Could not create trello client: %v", err)
+		logger.Error("Could not create trello client: %v", err)
+		os.Exit(1)
 	}
 
 	if err := client.LoadBoard(labels); err != nil {
-		logger.Fatalf("Could not load existing cards from the board: %v", err)
+		logger.Error("Could not load existing cards from the board: %v", err)
+		os.Exit(1)
 	}
 
 	var wg sync.WaitGroup

cmd/entrello/source.go

@@ -9,6 +9,7 @@ import (
 	"time"
 
 	"github.com/utkuufuk/entrello/internal/config"
+	"github.com/utkuufuk/entrello/internal/logger"
 	"github.com/utkuufuk/entrello/pkg/trello"
 )
 
@@ -17,7 +18,7 @@ func getSources(srcArr []config.Source, now time.Time) (sources []config.Source,
 	for _, src := range srcArr {
 		if ok, err := shouldQuery(src, now); !ok {
 			if err != nil {
-				logger.Errorf("could not check if '%s' should be queried or not, skipping", src.Name)
+				logger.Error("could not check if '%s' should be queried or not, skipping", src.Name)
 			}
 			continue
 		}
@@ -64,7 +65,7 @@ func process(src config.Source, client trello.Client, wg *sync.WaitGroup) {
 
 	resp, err := http.Get(src.Endpoint)
 	if err != nil {
-		logger.Errorf("could not make GET request to source '%s' endpoint: %v", src.Name, err)
+		logger.Error("could not make GET request to source '%s' endpoint: %v", src.Name, err)
 		return
 	}
 	defer resp.Body.Close()
@@ -75,23 +76,23 @@ func process(src config.Source, client trello.Client, wg *sync.WaitGroup) {
 		if err != nil {
 			msg = err.Error()
 		}
-		logger.Errorf("could not retrieve cards from source '%s': %v", src.Name, msg)
+		logger.Error("could not retrieve cards from source '%s': %v", src.Name, msg)
 		return
 	}
 
 	var cards []trello.Card
 	if err = json.NewDecoder(resp.Body).Decode(&cards); err != nil {
-		logger.Errorf("could not decode cards received from source '%s': %v", src.Name, err)
+		logger.Error("could not decode cards received from source '%s': %v", src.Name, err)
 		return
 	}
 
 	new, stale := client.FilterNewAndStale(cards, src.Label)
 	for _, c := range new {
 		if err := client.CreateCard(c, src.Label, src.List); err != nil {
-			logger.Errorf("could not create Trello card: %v", err)
+			logger.Error("could not create Trello card: %v", err)
 			continue
 		}
-		logger.Debugf("created new card: %s", c.Name)
+		logger.Info("created new card: %s", c.Name)
 	}
 
 	if !src.Strict {
@@ -100,9 +101,9 @@ func process(src config.Source, client trello.Client, wg *sync.WaitGroup) {
 
 	for _, c := range stale {
 		if err := client.DeleteCard(c); err != nil {
-			logger.Errorf("could not delete Trello card: %v", err)
+			logger.Error("could not delete Trello card: %v", err)
 			continue
 		}
-		logger.Debugf("deleted stale card: %s", c.Name)
+		logger.Info("deleted stale card: %s", c.Name)
 	}
 }

config.example.json

@@ -0,0 +1,43 @@
+{
+  "timezone_location": "Europe/Istanbul",
+  "trello": {
+    "api_key": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
+    "api_token": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
+    "board_id": "xxxxxxxxxxxxxxxxxxxxxxxx"
+  },
+  "sources": [
+    {
+      "name": "Github Issues",
+      "endpoint": "http://<github-issues-endpoint>",
+      "strict": true,
+      "label_id": "xxxxxxxxxxxxxxxxxxxxxxxx",
+      "list_id": "xxxxxxxxxxxxxxxxxxxxxxxx",
+      "period": {
+        "type": "minute",
+        "interval": 15
+      }
+    },
+    {
+      "name": "Google Spreadsheet Habits",
+      "endpoint": "http://<habits-endpoint>",
+      "strict": true,
+      "label_id": "xxxxxxxxxxxxxxxxxxxxxxxx",
+      "list_id": "xxxxxxxxxxxxxxxxxxxxxxxx",
+      "period": {
+        "type": "hour",
+        "interval": 1
+      }
+    },
+    {
+      "name": "TodoDock",
+      "endpoint": "http://<tododock-endpoint>",
+      "strict": true,
+      "label_id": "xxxxxxxxxxxxxxxxxxxxxxxx",
+      "list_id": "xxxxxxxxxxxxxxxxxxxxxxxx",
+      "period": {
+        "type": "default",
+        "interval": 0
+      }
+    }
+  ]
+}

go.mod

@@ -4,12 +4,7 @@ go 1.13
 
 require (
 	github.com/adlio/trello v1.8.0
-	github.com/go-telegram-bot-api/telegram-bot-api v4.6.4+incompatible
 	github.com/google/go-cmp v0.5.4
-	github.com/kr/text v0.2.0 // indirect
 	github.com/pkg/errors v0.9.1 // indirect
-	github.com/technoweenie/multipartstreamer v1.0.1 // indirect
 	golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
-	gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
-	gopkg.in/yaml.v2 v2.4.0
 )