Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.idea/
50 changes: 50 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,14 @@ docker compose

Higress Console 在服务器本地监听的端口。默认值为 8080。

* --use-plugin-server

使用内置的 Plugin Server 通过 HTTP 方式分发 WASM 插件。此为默认行为。

* --no-plugin-server

禁用内置的 Plugin Server,WASM 插件将通过 OCI 镜像方式加载。

* -r, --rerun

在 Higress 已配置完成后重新执行配置流程。
Expand Down Expand Up @@ -140,6 +148,48 @@ higress-precheck-1 "/bin/bash ./prechec…" precheck exited (0)

查看 Higress 各组件的运行日志。

## Docker All-in-One 模式

除了 Docker Compose 多容器部署外,Higress 还提供了 All-in-One 单容器部署模式,将所有组件打包在一个 Docker 镜像中运行。

### 构建镜像

```bash
docker build -t higress-all-in-one ./all-in-one
```

### 启动容器

```bash
docker run -d --name higress \
-p 8080:8080 -p 8443:8443 -p 8001:8001 \
higress-all-in-one
```

### 环境变量

All-in-One 模式通过环境变量控制各组件的行为,可在 `docker run` 时通过 `-e` 参数传入:

* `MODE`

运行模式。可选值为 `full`(默认)、`gateway`、`console`。

* `O11Y`

是否启用可观测性组件(Prometheus、Grafana 等)。可选值为 `on`、`off`(默认)。需镜像中已包含相关组件。

* `USE_PLUGIN_SERVER`

是否启用内置的 Plugin Server 通过 HTTP 方式分发 WASM 插件。默认启用。设置为 `false`、`off`、`no` 或 `N` 可禁用,禁用后 WASM 插件将通过 OCI 镜像方式加载。

```bash
# 禁用 Plugin Server
docker run -d --name higress \
-e USE_PLUGIN_SERVER=off \
-p 8080:8080 -p 8443:8443 -p 8001:8001 \
higress-all-in-one
```

## 设计文档

- [方案整体设计](./docs/design.md)
Expand Down
3 changes: 2 additions & 1 deletion all-in-one/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@
/*.cmd
/*.bat
/docker-compose.yml
/higress
/higress
.idea
16 changes: 11 additions & 5 deletions all-in-one/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
ARG HUB=higress-registry.cn-hangzhou.cr.aliyuncs.com/higress
ARG BASE_VERSION=2022-10-27T19-02-22
ARG CORE_VERSION=2.1.11
ARG CONSOLE_VERSION=2.1.11
ARG CORE_VERSION=2.2.1
ARG CONSOLE_VERSION=2.2.1
ARG APISERVER_VERSION=0.0.29
ARG ENVOY_VERSION=2.1.11
ARG ENVOY_VERSION=2.2.1
ARG PLUGIN_SERVER_VERSION=2.2.1

FROM ${HUB}/api-server:${APISERVER_VERSION} AS apiserver
FROM ${HUB}/higress:${CORE_VERSION} AS controller
FROM ${HUB}/pilot:${CORE_VERSION} AS pilot
FROM ${HUB}/gateway:${CORE_VERSION} AS gateway
FROM ${HUB}/console:${CONSOLE_VERSION} AS console
FROM ${HUB}/eclipse-temurin:21-jre AS jdk
FROM ${HUB}/plugin-server:${PLUGIN_SERVER_VERSION} AS plugin-server

FROM ${HUB}/base:${BASE_VERSION}

Expand All @@ -37,6 +39,9 @@ COPY --from=gateway /usr/local/bin/supercronic* /usr/local/bin/
# Install console
COPY --from=console /app /app

# Install plugin-server (nginx with wasm plugins)
COPY --from=plugin-server /usr/share/nginx/html/plugins /usr/share/nginx/html/plugins

# Install JDK required by console
ENV JAVA_HOME=/opt/java/openjdk
COPY --from=jdk $JAVA_HOME $JAVA_HOME
Expand All @@ -53,7 +58,7 @@ RUN arch="$(dpkg --print-architecture)"; \
esac; \
apt-get update --allow-unauthenticated; \
apt-get install --no-install-recommends -y --allow-unauthenticated \
wget supervisor logrotate cron; \
wget supervisor logrotate cron nginx; \
apt-get upgrade -y --allow-unauthenticated; \
apt-get clean; \
rm -rf /var/log/*log /var/lib/apt/lists/* /var/log/apt/* /var/lib/dpkg/*-old /var/cache/debconf/*-old; \
Expand All @@ -74,7 +79,8 @@ COPY ./gateway/podinfo /etc/istio/pod
COPY ./scripts /usr/local/bin
COPY ./apiserver/config /app/kubeconfig
COPY ./config /opt/data/defaultConfig
COPY ./plugin-server/nginx.conf /etc/nginx/plugin-server/nginx.conf

EXPOSE 8080 8443 8001
EXPOSE 8080 8443 8001 8002

ENTRYPOINT ["supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]
46 changes: 46 additions & 0 deletions all-in-one/plugin-server/nginx.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
daemon off;
user www-data;
worker_processes auto;
pid /run/nginx.pid;
error_log /var/log/nginx/error.log notice;

events {
worker_connections 1024;
}

http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent"';

access_log /var/log/nginx/access.log main;

sendfile on;
keepalive_timeout 65;

server {
listen 8002;
server_name localhost;

# Static files root directory
root /usr/share/nginx/html;

# Hide Nginx version
server_tokens off;

# Health check endpoint
location = /healthz {
return 200 'ok';
add_header Content-Type text/plain;
}

# Error pages
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
}
11 changes: 11 additions & 0 deletions all-in-one/scripts/base.sh
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,17 @@ case $O11Y in
esac
echo "O11Y=$O11Y"

case $USE_PLUGIN_SERVER in
false|FALSE|off|OFF|no|NO|N|n)
USE_PLUGIN_SERVER=off
;;
*)
# Default to on
USE_PLUGIN_SERVER=on
;;
esac
echo "USE_PLUGIN_SERVER=$USE_PLUGIN_SERVER"

CONSOLE_USED_MARKER='/data/.console-used'
CONSOLE_USED='false'
if [ -f "$CONSOLE_USED_MARKER" ]; then
Expand Down
4 changes: 4 additions & 0 deletions all-in-one/scripts/start-console.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ if [ "$O11Y" == "on" ]; then
export HIGRESS_CONSOLE_DASHBOARD_DATASOURCE_LOKI_URL="http://localhost:3100"
fi

if [ "$USE_PLUGIN_SERVER" == "on" ]; then
export HIGRESS_ADMIN_WASM_PLUGIN_CUSTOM_IMAGE_URL_PATTERN="http://localhost:8002/plugins/\${name}/\${version}/plugin.wasm"
fi

HIGRESS_CONSOLE_KUBE_CONFIG="/app/kubeconfig" \
HIGRESS_CONSOLE_SERVICE_HOST="higress-console.static" \
HIGRESS_CONSOLE_SERVICE_PORT=80 \
Expand Down
5 changes: 5 additions & 0 deletions all-in-one/scripts/start-controller.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ waitForApiServer

set -e

# Use HTTP for loading wasm plugins from plugin-server
if [ "$USE_PLUGIN_SERVER" == "on" ]; then
export MCP_SERVER_WASM_IMAGE_URL="http://localhost:8002/plugins/mcp-server/1.0.0/plugin.wasm"
fi

/usr/local/bin/higress \
serve \
--kubeconfig=/app/kubeconfig \
Expand Down
15 changes: 15 additions & 0 deletions all-in-one/scripts/start-plugin-server.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/bash

cd "$(dirname -- "$0")"
ROOT=$(pwd)
cd - >/dev/null
source $ROOT/base.sh

if [ "$USE_PLUGIN_SERVER" != "on" ]; then
echo "Plugin-server won't run when USE_PLUGIN_SERVER is not turned on."
sleep 2
exit 0
fi

# Start nginx for plugin-server (foreground mode via daemon off in nginx.conf)
exec nginx -c /etc/nginx/plugin-server/nginx.conf
12 changes: 12 additions & 0 deletions all-in-one/supervisord/supervisord.conf
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,18 @@ stdout_logfile_maxbytes=10MB
redirect_stderr=true
environment=

[program:plugin-server]
directory=/
command=bash /usr/local/bin/start-plugin-server.sh
priority=50
autostart=true
autorestart=unexpected
startsecs=1
stdout_logfile=/var/log/higress/plugin-server.log
stdout_logfile_maxbytes=10MB
redirect_stderr=true
environment=

[program:controller]
directory=/
command=bash /usr/local/bin/start-controller.sh
Expand Down
57 changes: 55 additions & 2 deletions bin/configure.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ DEFAULT_GATEWAY_HTTP_PORT=80
DEFAULT_GATEWAY_HTTPS_PORT=443
DEFAULT_GATEWAY_METRICS_PORT=15020
DEFAULT_CONSOLE_PORT=8080
DEFAULT_USE_PLUGIN_SERVER="Y"

COMMAND_PREPARE="prepare"
COMMAND_INIT="init"
Expand All @@ -36,6 +37,8 @@ source "$ROOT/bin/base.sh"

source "$COMPOSE_ROOT/.env"

HIGRESS_PLUGIN_SERVER_TAG=${HIGRESS_PLUGIN_SERVER_TAG:-2.2.1}

CONFIGURED_MARK="$COMPOSE_ROOT/.configured"

initArch() {
Expand Down Expand Up @@ -171,6 +174,16 @@ parseArgs() {
MODE="params"
shift
;;
--use-plugin-server)
USE_PLUGIN_SERVER="Y"
MODE="params"
shift
;;
--no-plugin-server)
USE_PLUGIN_SERVER="N"
MODE="params"
shift
;;
-h | --help)
outputUsage
exit 0
Expand Down Expand Up @@ -212,6 +225,7 @@ configure() {
configureByArgs
else
configureStorage
configurePluginServer
configureConsole
configurePorts
fi
Expand Down Expand Up @@ -243,6 +257,7 @@ resetEnv() {
GATEWAY_HTTPS_PORT=$DEFAULT_GATEWAY_HTTPS_PORT
GATEWAY_METRICS_PORT=$DEFAULT_GATEWAY_METRICS_PORT
CONSOLE_PORT=$DEFAULT_CONSOLE_PORT
USE_PLUGIN_SERVER=""
}

configureByArgs() {
Expand All @@ -255,6 +270,9 @@ configureByArgs() {
exit -1
fi
configureConsoleByArgs
if [ -z "$USE_PLUGIN_SERVER" ]; then
USE_PLUGIN_SERVER="$DEFAULT_USE_PLUGIN_SERVER"
fi
configurePortsByArgs
}

Expand Down Expand Up @@ -499,6 +517,16 @@ configureFileStorage() {
done
}

configurePluginServer() {
echo "==== Configure Plugin Server ===="
readWithDefault "Use built-in plugin server for WASM plugin distribution? (Y/N) [${DEFAULT_USE_PLUGIN_SERVER}]: " "$DEFAULT_USE_PLUGIN_SERVER"
if [ "$input" == "Y" ] || [ "$input" == "y" ]; then
USE_PLUGIN_SERVER="Y"
else
USE_PLUGIN_SERVER="N"
fi
}

configureConsole() {
# echo "==== Configure Higress Console ===="
:
Expand Down Expand Up @@ -569,6 +597,10 @@ outputUsage() {
--console-port=CONSOLE-PORT
the port used to visit Higress Console
default to 8080 if unspecified
--use-plugin-server use the built-in plugin server for WASM plugin
distribution via HTTP (default)
--no-plugin-server disable the built-in plugin server, WASM plugins
will be loaded from OCI images instead
-r, --rerun re-run the configuration workflow even if
Higress is already configured
-h, --help give this help list'
Expand Down Expand Up @@ -612,6 +644,24 @@ writeConfiguration() {
NACOS_SERVER_HTTP_URL=${NACOS_SERVER_HTTP_URL%/}/nacos
fi

# Build COMPOSE_PROFILES with plugin-server support
if [ "$USE_PLUGIN_SERVER" == "Y" ]; then
if [ -n "$COMPOSE_PROFILES" ]; then
COMPOSE_PROFILES="${COMPOSE_PROFILES},plugin-server"
else
COMPOSE_PROFILES="plugin-server"
fi
fi

# Generate wasm plugin URL variables
if [ "$USE_PLUGIN_SERVER" == "Y" ]; then
WASM_PLUGIN_URL_PATTERN="http://plugin-server:8002/plugins/\${name}/\${version}/plugin.wasm"
MCP_SERVER_WASM_URL="http://plugin-server:8002/plugins/mcp-server/1.0.0/plugin.wasm"
else
WASM_PLUGIN_URL_PATTERN=''
MCP_SERVER_WASM_URL=''
fi

cat <<EOF >$COMPOSE_ROOT/.env
COMPOSE_PROFILES='${COMPOSE_PROFILES}'
CONFIG_STORAGE='${CONFIG_STORAGE}'
Expand All @@ -629,6 +679,9 @@ HIGRESS_CONTROLLER_TAG='${HIGRESS_CONTROLLER_TAG}'
HIGRESS_PILOT_TAG='${HIGRESS_PILOT_TAG}'
HIGRESS_GATEWAY_TAG='${HIGRESS_GATEWAY_TAG}'
HIGRESS_CONSOLE_TAG='${HIGRESS_CONSOLE_TAG}'
HIGRESS_PLUGIN_SERVER_TAG='${HIGRESS_PLUGIN_SERVER_TAG}'
WASM_PLUGIN_URL_PATTERN='${WASM_PLUGIN_URL_PATTERN}'
MCP_SERVER_WASM_URL='${MCP_SERVER_WASM_URL}'
PROMETHEUS_TAG='${PROMETHEUS_TAG}'
PROMTAIL_TAG='${PROMTAIL_TAG}'
LOKI_TAG='${LOKI_TAG}'
Expand All @@ -649,7 +702,7 @@ runInitializer() {

echo "==== Build Configurations ==== "

if [ "$COMPOSE_PROFILES" == "nacos" ]; then
if [[ "$COMPOSE_PROFILES" == *"nacos"* ]]; then
echo "Starting built-in Nacos service..."
cd "$COMPOSE_ROOT" && runDockerCompose -p higress up -d nacos
retVal=$?
Expand All @@ -666,7 +719,7 @@ runInitializer() {
exit -1
fi

if [ "$COMPOSE_PROFILES" == "nacos" ] && [ "${AUTO_START}" != "Y" ]; then
if [[ "$COMPOSE_PROFILES" == *"nacos"* ]] && [ "${AUTO_START}" != "Y" ]; then
echo "Stopping built-in Nacos service..."
cd "$COMPOSE_ROOT" && runDockerCompose -p higress down --remove-orphans
local retVal=$?
Expand Down
Loading
Loading