feat(minithor): Improved deploy script automation

This improves deployment automation for the minithor development by making
the deploy script self contained and inlining many of the resources previously
created by kustomization files in infrastructure. This allows the singular
script to be pulled and then applied to a development environment without
needing to pull multiple files from the repo.

Author: gabaker

Cargo.lock

@@ -32,7 +32,7 @@ default-members = [
 ]
 
 [workspace.package]
-version = "1.6.1"
+version = "1.7.0"
 authors = ["mcarson <mcarson@sandia.gov>", "gmbaker <gmbaker@sandia.gov>", "jehamza <jehamza@sandia.gov>"]
 edition = "2024"
 
@@ -41,8 +41,8 @@ inherits = "release"
 debug = true
 
 [workspace.dependencies]
-thorium-api = { version= "1.6.1", path="api", default-features = false }
-thorium-derive = { version = "1.6.1", path = "thorium-derive" }
+thorium-api = { version= "1.7.0", path="api", default-features = false }
+thorium-derive = { version = "1.7.0", path = "thorium-derive" }
 reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls", "multipart", "stream", "blocking"] }
 tokio = { version = "1.45", features = ["full"] }
 kube = { git = "https://github.com/stackabletech/kube-rs.git", rev="cfdfcd1", features = ["rustls-tls", "derive", "config", "client", "runtime", "http-proxy"] }

Cargo.toml

@@ -1,4 +1,7 @@
 *.secret
 .dockerconfigjson
+toolbox.json
+thorctl
 .certs/*
 .s3cfg
+*.crt

minithor/.gitignore

@@ -3,135 +3,103 @@
 
 Minithor utilizes Minikube to provide a local Kubernetes instance with minimal custom configuration required. Such an instance is useful for development and testing of Thorium as well as small stand alone analyst fly-away kits where external network access may not be available. Minithor deployments are not highly available distributed systems like our production instances and provides minimal redundancy. The Thorium deployment produced by following these instructions should be considered Beta. We will work to improve its stability over time. While a Minithor deployment is accessible only from your localhost, it has not been configured to be secure. Please change DB passwords if working with sensitive data on a multi-user system using a Minithor deployment.
 
-### Requirements and "Disclosures"
+### Requirements
 
-To deploy Minithor, you will need a container runtime such as that provided by the docker engine. Minithor also requires a relatively beefy machine, with > 12 GiB of memory, 8+ CPUs, and 100GiB of local storage. You will also need to install s3cmd if it is not already installed on the host system, this enables deployment of a s3 bucket for storing traces pushed to Quickwit.
+To deploy Minithor, you will need a container runtime such as that provided by the docker engine or podman. Minithor also requires a relatively beefy machine, with > 12 GiB of memory, 8+ CPUs, and 100GiB of local storage.
 
-On a Mac using brew:
-
-```bash
-brew install s3cmd
-```
-
-On Linux:
-
-```bash
-apt install s3cmd
-```
-
-### Deploy Minikube
+### Install Minikube
 
 Install and start minikube and any necessary plugins.
 
 ```bash
-./install-linux
-# or ./install-mac-m1
+./install
 ```
 
-Add this to your environment settings after installation:
-
-```bash
-alias kubectl="minikube kubectl --"
-```
+### Create registry auth file (optional)
 
-### Create registry auth file
-
-In the project directory you will need to create a file called `.dockerconfigjson` containing the authentication credentials for the user account/registry containing the thorium container image.
-
-Create the `.dockerconfigjson` via the `docker login` command. The registry url must match that used by the images Thorium will run:
+If the Thorium container image is hosted in a private registry, create a `.dockerconfigjson` file in this directory containing the registry credentials. The deploy script will detect this file and create a Kubernetes image pull secret automatically.
 
 ```bash
 docker login registry.domain:port
+cp ~/.docker/config.json .dockerconfigjson
 ```
 
-The registry auth information will be structured like this:
+If omitted, the operator will pull images without authentication (works for public registries like `ghcr.io`).
 
-```bash
-cat .dockerconfigjson
-{
-	"auths": {
-		"registry.domain:port": {
-			"auth": "<base64 of username:token/password>"
-		}
-    }
-}
-```
-
-Once this registry auth file has been created, copy the file (default path is `~/.docker/config.json` for most linux systems, must be manually created on mac) to the project directory and rename it to `.dockerconfigjson`.
+### Proxy configuration (optional)
 
-### Deploy Dependencies
+If your organization maintains a proxy for all traffic going to the internet, export proxy settings before running the deploy script:
 
-Thorium requires persistent storage interfaces a tracing API and an operator. Lets deploy these dependencies.
-
-If your organization maintains a proxy for all traffic going to the internet, you will need to export proxy settings such as the following:
 ```bash
-cat proxy
-
-#!/bin/bash
 export HTTP_PROXY=<HTTP_PROXY_URL:PORT>
 export HTTPS_PROXY=<HTTPS_PROXY_URL:PORT>
 export NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,192.168.0.0/16
 ```
 
-Once you have built that proxy file it can be reused in different terminal windows with:
-
-```bash
-source proxy
-```
+Or use the provided proxy file: `source proxy`
 
-Alternatively, those proxy settings can be added into your shell's settings file.
+### Deploy
 
-Now deploy the dependencies:
+The `deploy` script handles the full deployment in a single step: all backing services (Redis, Elasticsearch, ScyllaDB, MinIO, Quickwit, Jaeger), the Thorium operator, the ThoriumCluster resource, and a default test user.
 
 ```bash
 ./deploy
 ```
 
-### Deploy Thorium
+This will:
 
-```bash
-kubectl create -n thorium -f thorium-cluster.yml
-```
-
-### Set Password For Node's Docker User
+1. Wait for the minikube cluster to be healthy
+2. Install Helm and add required chart repos
+3. Deploy Redis, Elasticsearch (ECK), cert-manager, ScyllaDB, MinIO, Jaeger, Kubegres, and Quickwit
+4. Configure databases (Scylla roles/keyspace, Elasticsearch index/user, MinIO buckets)
+5. Deploy the Thorium operator and create the ThoriumCluster CRD
+6. Wait for all Thorium components (API, scaler, event-handler, search-streamer) to be running
+7. Create a test admin user (`test` / `INSECURE_DEV_PASSWORD`)
+8. Install `thorctl` from the API and import the default toolbox (analysis tools and pipelines)
 
-You only have to do this once and only when using priveleged ports for your local host port mapping. Kkeep track of the docker-in-docker password you set so you can tunnel to the Thorium UI/API later.
+To customize the ThoriumCluster configuration, copy the example and edit it before running the deploy script:
 
 ```bash
-minikube ssh
-sudo su -
-passwd docker
-# New password: 
-# Retype new password: 
-# passwd: password updated successfully
-exit
-exit
+cp thorium-cluster.yml.example thorium-cluster.yml
+# edit thorium-cluster.yml as needed
+./deploy.sh
 ```
 
-### Setup Tunnel (when using Thorium)
+If no `thorium-cluster.yml` exists, the script falls back to `thorium-cluster.yml.example`.
 
-This is a blocking command that can must be run in a dedicated terminal window or put in the background.
+### Access Thorium
+
+Start the minikube tunnel in a separate terminal (this is a blocking command):
 
 ```bash
-minikube tunnel
-# or ./expose
+./expose --help
+Usage: ./expose [--dev] [--stop] [--status]
+  --dev     Also forward database ports (Elastic, Kibana, Redis, MinIO, Scylla)
+  --stop    Stop all running port-forwards
+  --status  Show which port-forwards are running
 ```
 
-### Setup Dev Tunnels (Elastic/Kibana, Scylla, Redis)
+Then open http://localhost in your browser and log in:
+- **Username:** `test`
+- **Password:** `INSECURE_DEV_PASSWORD`
+
+### Dev tunnels (Elastic/Kibana, Scylla, Redis)
 
-This is a blocking command that can must be run in a dedicated terminal window or put in the background.
+To access backing services directly from your host (useful for debugging):
 
 ```bash
-./expose-dev
+./expose --dev
 ```
 
-### Get Thorium admin password
+### Cleanup
+
+Remove all deployed resources (without deleting the minikube cluster itself):
 
 ```bash
-kubectl get secret -n thorium thorium-pass --template={{.data.thorium}} | base64 --decode; echo
+./cleanup
 ```
 
-### Cleanup of Minithor
+To fully remove minikube:
 
 ```bash
 ./stop