GitHub - sven-seyfert/apiprobe: The project "APIProbe 📡" is a Go-based lightweight CLI tool designed for automated API monitoring, structured request testing and response change detection. It loads JSON-defined API requests, applies test cases, handles secrets securely, diffs responses and sends webhook notifications when changes or errors occur. See the "why this" section.

Welcome to `APIProbe` 📡

Description

🥇 What

The project APIProbe 📡 is a Go-based lightweight CLI tool designed for automated API monitoring, structured request testing and response change detection. It loads JSON-defined API requests, applies test cases, handles secrets securely, diffs responses and sends webhook notifications when changes or errors occur.

🥈 Why this

Unlike GUI-based tools such as Postman, APIProbe 📡 is built with developers in mind and optimized for fully automated, data-driven workflows. You can invoke it interactively for quick ad-hoc checks on your local machine or integrate it seamlessly into your CI/CD pipelines for continuous monitoring on remote machines.

🥉 Stability notice

Currently in a stable initial state — core features implemented; more advanced capabilities planned.

Features

Structured API definitions:
Define and load multiple API requests based on JSON files.
Test case support:
Define multiple test cases per request to cover various scenarios (data driven approach).
Secrets management:
Securely store secrets in an encrypted database instead of plain credentials/secrets in the JSON definition files (SQLite).
Authentication token handling:
Send auth requests, store returned tokens and automatically inject them into dependent requests via <auth-token> placeholder.
Response diffing:
Detect changes through a before and after comparison.
Webhook notifications:
Send summary reports or error alerts to collaboration tools (like WebEx, MS Teams).
Custom Logging:
Log to console and log file with multiple log levels.
Flexible filtering:
Filter by ID or tags, generate new IDs, insert new secrets into database.

Getting started

🏃‍♂️ Preconditions | Installation | Usage

Preconditions

Go 1.20+ installed (download) or simply run the executable instead.
Ensure $GOPATH/bin is in your PATH.
Dependency (binary) curl in ./lib/.
SQLite available (preinstalled on the most OS and systems).

Installation

Clone the repository:

git clone https://github.com/sven-seyfert/apiprobe.git
cd apiprobe

Ensure dependencies are available:

curl in the ./lib/ folder or adjust the path in ./internal/exec/curl.go.

Run or build the program:

# load
go mod tidy
go mod download

# run program
go run main.go

# or build and run executable
go build
./apiprobe.exe

Or see Makefile commands.

Usage

🏃‍♂️ Global Flags | Examples | Remote execution

Global Flags

Flags	Description
`--help`	Show all flags (switches) and their explanations. Shows also the program version.
`--name "Environment: PROD"`	Set custom name for the test run (for the execution). Shown in final notification.
`--id "<hex hash>"`	Run only the request matching this ID.
`--tags "animals, cars"`	Run all requests containing any of the comma-separated tags.
`--exclude-ids "bb5599abcd, ff00fceb61"`	Do not run any request that contains ANY of the IDs in the comma-separated ID list.
`--exclude-tags "daily-based-execution"`	Do not run any request that contains ANY of the tags in the comma-separated tag list.
`--new-id`	Generates and returns a new random hex ID for use in JSON definitions.
`--new-file`	Generates a new JSON definition template file. Then enter the request values/data and done.
`--add-secret "<value>"`	Securely stores secrets in SQLite database. Returns a placeholder like "<secret-b29ff12b50>" for use in JSON definitions.
`--notify-channel "<channel>"`	Specify the WebEx or MS Teams channel where notifications should be sent. The name must match a key in the 'webEx.webhooks' or 'msTeams.webhooks' map in the config file apiprobe.json. Default is "default".

Examples

Run all API requests:

go run main.go
# or by executable (faster)
./apiprobe.exe

Filter and run API requests by ID:

go run main.go --id "ff00fceb61"
# or by executable (faster)
./apiprobe.exe --id "ff00fceb61"

Filter and run API requests by tags:

go run main.go --tags "reqres, booker, env-prod"
# or by executable (faster)
./apiprobe.exe --tags "reqres, booker, env-prod"

Exclude API requests from run by ID:

go run main.go --exclude-ids "ff00fceb61, bb11abc987"
# or by executable (faster)
./apiprobe.exe --exclude-ids "ff00fceb61, bb11abc987"

# combination example:
# run every request with tag "<tag-name>" except request with specific <ID> and name the test run "Environment: PROD"
go run main.go --tags "env-prod" --exclude-ids "bb11abc987" --name "Environment: PROD"

Generate new ID:

go run main.go --new-id
# or by executable (faster)
./apiprobe.exe --new-id

Generate new JSON definition template file:

go run main.go --new-file
# or by executable (faster)
./apiprobe.exe --new-file

Add new secret:

go run main.go --add-secret "myApiKey123"
# or by executable (faster)
./apiprobe.exe --add-secret "myApiKey123"

For more instructions, see section secret management below.

Send notifications to a specific channel:

go run main.go --notify-channel "prod"
# or by executable (faster)
./apiprobe.exe --notify-channel "prod"

# combination example:
# run requests with tag "env-test" and send notifications to the "test" channel and name the test run "Environment: TEST"
go run main.go --tags "env-test" --notify-channel "test" --name "Test Run"

Remote execution

You can run the CLI regularly via various schedulers or task runners.

Windows Task Scheduler

A sample XML definition is provided under ./remote/windows-tasks-scheduler.xml.
Use it to register a scheduled task that invokes apiprobe.exe at your desired interval.
For example, to schedule a daily run at 2 AM, import the XML and adjust the <Triggers> section accordingly.

Configuration

🏃‍♂️ apiprobe.json | JSON definitions | Secret management

apiprobe.json

Setup your webhook URL for WebEx or MS Teams. The MS Teams webhook usage is currently in development and cannot be used yet, but coming very soon.

debugMode

Activate or deactivate debug mode. This will print the cURL format representation of the request to the console. You then can simply test your request via cURL directly.

heartbeat

Define the interval (in hours) how often a heartbeat message should be sent. This is useful when you don't receive many failures or changes with you API requests and still want to know is the program running and healthy.

notification

Configure webhook notifications for collaboration tools like WebEx and MS Teams. Notifications are sent automatically when errors occur, responses change, or on heartbeat intervals.

webEx / msTeams: Set active to true to enable notifications for the respective tool. Define multiple webhook URLs under webhooks as a map (e.g., "default", "prod", "test"). Use the --notify-channel flag to specify which channel to use (defaults to "default" if not set).
You can use placeholders like <secret-f0f0f0f0f0> to avoid plaintext secrets in webhook URLs. Add the secret using --add-secret "<value>" and replace it in the config. For more instructions, see section secret management below.

Example config snippet:

{
    ...
    ...
    "notification": {
        "webEx": {
            "active": true,
        "webhooks": {
            "default": "https://webexapis.com/v1/webhooks/incoming/<secret-abcde12345>",
            "prod": "https://webexapis.com/v1/webhooks/incoming/<secret-54321edcba>",
            "test": "https://webexapis.com/v1/webhooks/incoming/<secret-edcba54321>"
        }
        },
        "msTeams": {
            "active": false,
            "webhooks": {
                "default": "<webhook-url-with-secret>",
                "prod": "<webhook-url-with-secret>",
                "test": "<webhook-url-with-secret>"
            }
        }
    }
}

### JSON definitions

Define your APIs in JSON files under `./data/input/`. Each file contains an array of objects following the schema:

#### *Minimal definition*

``` json
[
    {
        "id": "0f1e2d3c4b",
        "isActive": true,
        "isAuthRequest": false,
        "preRequestId": "",
        "request": {
            "description": "Short description of the request (purpose)",
            "method": "GET|POST|PUT",
            "url": "https://api.example.com",
            "endpoint": "/api/path",
            "basicAuth": "",
            "headers": [],
            "params": [],
            "postBody": {},
            "name": ""
        },
        "testCases": [
            {
                "name": "",
                "paramsData": "",
                "postBodyData": {}
            }
        ],
        "tags": [
            "env-prod"
        ],
        "jq": ""
    }
]

Advanced definition

[
    {
        "id": "0f1e2d3c4b",
        "isActive": true,
        "isAuthRequest": false,
        "preRequestId": "",
        "request": {
            "description": "Short description of the request (purpose)",
            "method": "GET|POST|PUT",
            "url": "https://api.example.com",
            "endpoint": "/api/path",
            "basicAuth": "<secret-b4c3d2e1f0>",
            "headers": [
                "Content-Type: application/json"
            ],
            "params": [
                "animalId=1337",
                "pageSize=25",
                "page=3"
            ],
            "postBody": {
                "Username": "John Doe",
                "Password": "<secret-b4c3d2e1f0>"
            },
            "name": ""
        },
        "testCases": [
            {
                "name": "Test with Marry Doe",
                "paramsData": "",
                "postBodyData": {
                    "Username": "Marry Doe",
                    "Password": "<secret-ff00ee11cc>"
                }
            },
            {
                "name": "Test with Julia Ismo",
                "paramsData": "",
                "postBodyData": {
                    "Username": "Julia Ismo",
                    "Password": "<secret-cc11ee00ff>"
                }
            },
            {
                "name": "Test with John Doe and different animalId",
                "paramsData": "animalId=4567",
                "postBodyData": {}
            }
        ],
        "tags": [
            "animals",
            "cars",
            "env-prod"
        ],
        "jq": ".data = (.data | sort_by(.type))"
    }
]

Explanation

Mandatory = (M)
Mandatory for POST request = (P)

JSON key	JSON value description	Default value
id (M)	Unique 10 character hex hash. Use `--new-id` to generate.
isActive (M)	Toggle the request execution by this boolean flag. In case the endpoint still exists but is temporary inactive, simply set 'false' and this requests will not be processed.	true
isAuthRequest (M)	Marks this as an authentication request (e.g. login). When true, the tool will make the resulting token available to subsequent requests.	false
preRequestId	ID of the preconditional request to run before this one. The response payload (e.g. token) of that pre-request will automatically be made available to this request’s headers or body if referenced.	"" (empty string)
request	JSON node for all request related values.
request.description	Endpoint description (purpose).	"" (empty string)
request.method (M)	HTTP Method; currently only GET, POST and PUT requests are supported.
request.url (M)	Interface (API) URL
request.endpoint (M)	Request endpoint.
request.basicAuth	User and password for a basic authentification; format <user>:<password>.	"" (empty string)
request.headers	Request header list (one or n headers).	[] (empty string array)
request.params	URL query parameter list (one or n params); no ? or & needed, only the raw query parameter(s).	[] (empty string array)
request.postBody (P)	JSON message body (payload) for POST requests. Custom JSON object.	{} (empty JSON object)
request.name	Define the name of the first test case.	"" (empty string)
testCases	Data driven test data list (one or n test data entries); these variations apply to query params or post body. See minimal definition.
testCases.name	Define the name of your test case.	"" (empty string)
testCases.paramsData	Define which query parameter should be applied (replaced) in request.params for the test cases. See advanced definition.	"" (empty string)
testCases.postBodyData	Define post body data that will be applied (replaced) in request.postBody for the test cases. See advanced definition.	{} (empty JSON object)
tags	Representation of the topic, of a application, environment etc.	[] (empty string array)
jq	JSON query syntax; prettify JSON response (default ".").	"." (dot is the fallback if "" is provided)

Secret management

Insert a new secret:

go run main.go --add-secret "superSecretValue"
# or by executable (faster)
./apiprobe.exe --add-secret "superSecretValue"

You'll receive a message with a placeholder like:

Use this placeholder "<secret-ab12cd34ef>" in your JSON file instead of the actual secret value.

In your JSON, replace the real value with this placeholder:
```
"headers": ["Authorization: Bearer <secret-ab12cd34ef>"]
```
Secrets are securely stored in the SQLite database ./db/store.db.

Authentication

This section details how authentication token requests are handled.

Define Auth Requests: In your JSON definitions set "isAuthRequest": true and include the endpoint to obtain your token.
Token Extraction: Auth responses are parsed (via jq) and added to the Token Store under the auth request ID.
Injecting Tokens: For any request that depends on an auth request, set "preRequestId": "<auth-request-id>" and include the header placeholder:
```
"headers": [
  "Authorization: Bearer <auth-token>"
]
```
The tool replaces <auth-token> with the actual token from the Token Store before executing the request.
Usage: Ensure your JSON definitions reference <auth-token> exactly, so that the CLI can locate and replace it.

Behind the scenes

🏃‍♂️ Project layout | How it works | Logging, Reporting

Project layout

Most important parts (directories and files):

apiprobe/
├── assets/
│   └── images/         # Images, screenshots
├── config/
│   └── apiprobe.json   # User defined config entries (like notification settings)
├── data/
│   ├── input/          # JSON request definitions organized by service and environment
│   └── output/         # Auto-generated responses (snapshots)
├── db/
│   ├── seed.csv        # Initial secrets data
│   └── store.db        # SQLite database
├── internal/           # Go packages
├── lib/                # Dependency binary (curl)
├── logs/               # Execution logs (auto-generated)
├── remote/             # Windows Task Scheduler templates
├── CHANGELOG.md        # Version history
├── LICENSE.md          # MPL-2.0 License
├── main.go             # CLI entrypoint
└── Makefile            # Build & run helpers

How it works

Initialization: Logger setup, DB connection, CLI flags setup and config load. Also seed default data insertion.
Loading: Recursively parse JSON files (API request definitions) into APIRequest objects.
Filtering: Apply --exclude-ids, --exclude-tags, --id and --tags CLI flag filters.
Prepending: Dependent pre-requests will be merged (prepended) to the list of requests.
Secrets: Replace <secret-...> placeholders with actual secrets from the database.
Authentication: If an API definition has isAuthRequest: true, the response token is stored in an in-memory Token Store keyed by the request ID. For any subsequent requests with preRequestId, the <auth-token> placeholder in headers is replaced with the stored token before execution.
Execution:
- Build cURL arguments and run HTTP request.
- Capture status code and response body.
- Filter response body through jq.
Diffing:
- Compute SHA256 of formatted response.
- Compare with existing snapshot file in ./data/output.
- Update file and record change if different.
Reporting:
- Increment counters for errors and changes.
- Depending on counter results write ./logs/report.json
  or with suffix ./logs/report-test.json, ./logs/report-prod.json depending on --name flag content.
- Send WebEx webhook summary.

Logging, Reporting

Console & file logging: All logs to console and to file, like ./logs/2025-06/18/2025-06-18-12-58-54.938.log.
Report file: JSON report at ./logs/report.json or ./logs/report-test.json (see above) when errors/changes occur.
Webhook: Automatic notifications to WebEx (WebEx only at the moment). Later also to MS Teams etc.

Contributing

Fork repository.
Create feature branch: git checkout -b feature/my-new-feature.
Commit changes: git commit -m "Added: My new feature.".
Push to branch: git push origin feature/my-new-feature.
Open a pull request (PR).

Please ensure:

You added function comments for new functions (*.go).
Code passes golangci-lint (golangci-lint run ./...).
You added documentation for new features (README.md).

License

Copyright (c) 2025 Sven Seyfert (SOLVE-SMART)
Distributed under the MPL-2.0 License. See LICENSE for more information.

Acknowledgements

Opportunity by GitHub
Badges by Shields and SimpleIcons
Thanks to the authors, maintainers and contributors of the various projects and products
- golang by the Go team at Google; License: BSD-3-Clause
- cURL by Daniel Stenberg; License: MIT
- jq by Stephen Dolan; License: MIT
- gojq by itchyny; License: MIT
- SQLite by Richard Hipp; License: Public Domain
- go-sqlite by Roxy Light; License: ISC

To the top

Navigation Menu

Repository files navigation

Welcome to `APIProbe` 📡

Description

🥇 What

🥈 Why this

🥉 Stability notice

Features

Getting started

Preconditions

Installation

Usage

Global Flags

Examples

Remote execution

Configuration

apiprobe.json

debugMode

heartbeat

notification

Advanced definition

Explanation

Secret management

Authentication

Behind the scenes

Project layout

How it works

Logging, Reporting

Contributing

License

Acknowledgements

About

Releases

Packages

Used by

Contributors

Languages

Name		Name	Last commit message	Last commit date
.vscode		.vscode
assets/images		assets/images
config		config
coverage		coverage
data		data
db		db
internal		internal
lib		lib
logs		logs
remote		remote
reports		reports
.gitattributes		.gitattributes
.gitignore		.gitignore
.golangci.yml		.golangci.yml
CHANGELOG.md		CHANGELOG.md
LICENSE.md		LICENSE.md
Makefile		Makefile
README.md		README.md
go.mod		go.mod
go.sum		go.sum
main.go		main.go

Latest commit

History

Folders and files

Repository files navigation

Welcome to APIProbe 📡

Description

🥇 What

🥈 Why this

🥉 Stability notice

Features

Getting started

Preconditions

Installation

Usage

Global Flags

Examples

Remote execution

Configuration

apiprobe.json

debugMode

heartbeat

notification

Advanced definition

Explanation

Secret management

Authentication

Behind the scenes

Project layout

How it works

Logging, Reporting

Contributing

License

Acknowledgements

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages

Welcome to `APIProbe` 📡