Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .agents/skills/debug-openshell-cluster/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -435,11 +435,14 @@ kubectl -n openshell get endpoints openshell
For local port-forward testing:

```bash
kubectl -n openshell port-forward svc/openshell 8080:8080
openshell gateway add http://127.0.0.1:8080 --local --name local
mise run helm:k3s:forward
openshell gateway list
openshell status
```

A successful plaintext `helm:skaffold:run` registers and selects the local
gateway; the forwarding task does not modify gateway metadata.

If the gateway is healthy but sandbox creation fails:

```bash
Expand Down
23 changes: 17 additions & 6 deletions .agents/skills/helm-dev-environment/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,17 @@ sidecar-mTLS profile reuses `ci/values-sidecar.yaml` and restores
Job that runs `openshell-gateway generate-certs`) generates mTLS secrets on first
install. Envoy Gateway opt-in; see the Optional Add-ons section below.

The gateway Service uses ClusterIP. Access is via Envoy Gateway (port `8080`) or `kubectl port-forward`.
The gateway Service uses ClusterIP. Access is via Envoy Gateway (port `8080`) or
the local forwarding task:

```bash
mise run helm:k3s:forward
```

The task forwards the Kubernetes gateway to `http://127.0.0.1:8090`. A
successful plaintext `helm:skaffold:run` or `helm:skaffold:run:sidecar`
registers and selects the gateway under the worktree-specific k3d cluster name.
Keep the forwarding task running while using the endpoint.

**HA test deploy** (two gateway replicas + external PostgreSQL Secret): uncomment
`#- ci/values-high-availability.yaml` in `deploy/helm/openshell/skaffold.yaml`,
Expand All @@ -99,19 +109,20 @@ plaintext by default. To test sidecar topology with TLS enabled, use
| Skaffold dev (default) | `true` | `http://` |
| TLS enabled | `false` (or omitted) | `https://` |

### Connecting via port-forward
### Connecting through the forwarding task

Port `8080` is already bound by the k3d load balancer when Envoy Gateway is active, so
the port-forward uses local port `8090` to avoid a collision:
Port `8080` is already bound by the k3d load balancer when Envoy Gateway is
active, so the forwarding task uses local port `8090` for the gateway. In a
second terminal, confirm that the gateway is registered and active:

```bash
KUBECONFIG=kubeconfig kubectl port-forward -n openshell svc/openshell 8090:8080
openshell gateway list
```

**Plaintext (default Skaffold deploy):**

```bash
openshell sandbox list --gateway-endpoint http://localhost:8090
openshell sandbox list
```

**With mTLS enabled** — extract the client cert the PKI hook wrote to the cluster,
Expand Down
12 changes: 8 additions & 4 deletions tasks/helm.toml
Original file line number Diff line number Diff line change
Expand Up @@ -69,14 +69,14 @@ dir = "deploy/helm/openshell"
run = "skaffold dev -p sidecar-mtls"

["helm:skaffold:run"]
description = "Run skaffold run for deploy/helm/openshell (one-shot deploy)"
description = "Run a one-shot Skaffold deploy and register its local plaintext gateway"
dir = "deploy/helm/openshell"
run = "skaffold run"
run = "skaffold run && ../../../tasks/scripts/helm-k3s-local.sh register"

["helm:skaffold:run:sidecar"]
description = "Run skaffold run with the Kubernetes supervisor sidecar topology"
description = "Run a sidecar Skaffold deploy and register its local plaintext gateway"
dir = "deploy/helm/openshell"
run = "skaffold run -p sidecar"
run = "skaffold run -p sidecar && ../../../tasks/scripts/helm-k3s-local.sh register"

["helm:skaffold:run:sidecar-mtls"]
description = "Run skaffold run with the Kubernetes supervisor sidecar topology and TLS/mTLS enabled"
Expand Down Expand Up @@ -130,6 +130,10 @@ description = "List local k3d clusters"
run = "tasks/scripts/helm-k3s-local.sh status"
hide = true

["helm:k3s:forward"]
description = "Forward the local Kubernetes gateway to port 8090"
run = "tasks/scripts/helm-k3s-local.sh forward"

# Install Envoy Gateway's GatewayClass into the cluster

["helm:gateway:apply"]
Expand Down
58 changes: 57 additions & 1 deletion tasks/scripts/helm-k3s-local.sh
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,13 @@ PRELOAD_SANDBOX_IMAGE="${HELM_K3S_PRELOAD_SANDBOX_IMAGE-${DEFAULT_SANDBOX_PRELOA
# exercise the v1alpha1 controller release.
AGENT_SANDBOX_VERSION="${AGENT_SANDBOX_VERSION:-v0.5.0}"

# Host endpoint registered for the Skaffold-deployed gateway. Derive the
# gateway name from the worktree-specific cluster name so concurrent local
# clusters do not overwrite each other's CLI metadata.
GATEWAY_NAMESPACE="openshell"
GATEWAY_HOST_PORT="${HELM_K3S_GATEWAY_HOST_PORT:-8090}"
GATEWAY_NAME="${HELM_K3S_GATEWAY_NAME:-${CLUSTER_NAME}}"

default_kubeconfig="${ROOT}/kubeconfig"
if [[ -n "${HELM_K3S_KUBECONFIG:-}" ]]; then
KUBECONFIG_TARGET="${HELM_K3S_KUBECONFIG}"
Expand All @@ -54,7 +61,7 @@ fi

usage() {
cat >&2 <<EOF
usage: $(basename "$0") <create|delete|start|stop|status>
usage: $(basename "$0") <create|delete|start|stop|status|register|forward>

Environment:
HELM_K3S_CLUSTER_NAME k3d cluster name (default: openshell-dev-<branch-suffix>)
Expand All @@ -65,6 +72,9 @@ Environment:
HELM_K3S_PRELOAD_SANDBOX_IMAGE
Sandbox image to docker pull and import into k3d
(default: ${DEFAULT_SANDBOX_PRELOAD_IMAGE}; set empty to skip)
HELM_K3S_GATEWAY_HOST_PORT Host port forwarded to the gateway (default: 8090)
HELM_K3S_GATEWAY_NAME CLI gateway registration name
(default: worktree-specific k3d cluster name)

macOS uses k3d from mise (Docker required). Linux can use this flow only when
k3d is installed explicitly; otherwise use kind or an existing cluster context.
Expand Down Expand Up @@ -312,6 +322,50 @@ cmd_status() {
k3d cluster list
}

register_local_gateway() {
local config_home openshell_dir gateway_dir endpoint

if [[ ! "${GATEWAY_NAME}" =~ ^[A-Za-z0-9._-]+$ ]]; then
echo "error: HELM_K3S_GATEWAY_NAME must contain only letters, numbers, dots, underscores, or dashes" >&2
return 2
fi

config_home="${XDG_CONFIG_HOME:-${HOME}/.config}"
openshell_dir="${config_home}/openshell"
gateway_dir="${openshell_dir}/gateways/${GATEWAY_NAME}"
endpoint="http://127.0.0.1:${GATEWAY_HOST_PORT}"

mkdir -p "${gateway_dir}"
chmod 700 "${gateway_dir}" 2>/dev/null || true
cat >"${gateway_dir}/metadata.json" <<EOF
{
"name": "${GATEWAY_NAME}",
"gateway_endpoint": "${endpoint}",
"is_remote": false,
"gateway_port": ${GATEWAY_HOST_PORT},
"auth_mode": "plaintext"
}
EOF
chmod 600 "${gateway_dir}/metadata.json" 2>/dev/null || true
printf '%s' "${GATEWAY_NAME}" >"${openshell_dir}/active_gateway"
chmod 600 "${openshell_dir}/active_gateway" 2>/dev/null || true

echo "Registered and selected local gateway '${GATEWAY_NAME}' at ${endpoint}."
}

cmd_forward() {
require_supported_os
require_kubectl

echo "Forwarding gateway to http://127.0.0.1:${GATEWAY_HOST_PORT}"
echo "Press Ctrl-C to stop."
kubectl \
--kubeconfig="${KUBECONFIG_TARGET}" \
--context="$(k3d_context_name)" \
--namespace="${GATEWAY_NAMESPACE}" \
port-forward service/openshell "${GATEWAY_HOST_PORT}:8080"
}

main() {
local sub="${1:-}"
case "${sub}" in
Expand All @@ -320,6 +374,8 @@ main() {
start) cmd_start ;;
stop) cmd_stop ;;
status) cmd_status ;;
register) register_local_gateway ;;
forward) cmd_forward ;;
-h | --help | help | "") usage ; [[ -n "${sub}" ]] || exit 1 ;;
*)
echo "error: unknown command '${sub}'" >&2
Expand Down
Loading