Skip to content

feat(openresty): activate brotli when the module is enabled, and fix gzip defaults - #13639

Open
Snrat wants to merge 8 commits into
1Panel-dev:dev-v2from
Snrat:feat/gzip-brotli-defaults
Open

feat(openresty): activate brotli when the module is enabled, and fix gzip defaults#13639
Snrat wants to merge 8 commits into
1Panel-dev:dev-v2from
Snrat:feat/gzip-brotli-defaults

Conversation

@Snrat

@Snrat Snrat commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Enabling ngx_brotli only ever emitted load_module. The module was loaded
but inert: no response was brotli-encoded until the user added brotli on and
brotli_types to nginx.conf by hand. The module is prebuilt into the OpenResty
image and listed in the catalog, so the only missing step was the runtime
configuration.

Depends on 1Panel-dev/appstore#9230, which adds the conf/http.d include.
Merge that first. This PR probes for the directory and keeps the previous
behaviour when it is absent, so merging out of order degrades silently rather
than breaking.

Why a separate directory

load_module is main-context; brotli on is http-context. They cannot share a
file, and conf/modules-enabled is included at the top level of nginx.conf.
Runtime directives therefore go to conf/http.d, managed with the same
snapshot/rollback discipline as the existing module files.

Files carry a 1panel-http- prefix; anything else in the directory is left
alone. Writes are atomic (temp file + rename).

Runtime defaults are declared per module in a table, so another module needing
http-context configuration can be added without touching the reconcile logic.

Ordering guarantee

Both directory sets are written before nginx -t runs, so nginx only ever
observes a consistent state, and a failed check rolls back load_module files
and runtime directives together.

This matters: leaving brotli on behind after the .so is unloaded makes
nginx fail to start with unknown directive "brotli". The reverse order is
equally fatal.

gzip

The embedded gzip.conf template was dead code — nothing referenced it, so its
values never reached an installation. It is now embedded and used.

Upgrades deliberately preserve the user's nginx.conf, so corrected defaults
shipped with a new OpenResty version would never reach existing installations.
The upgrade now rewrites the gzip values in place, but only when the block
is provably untouched: every directive must match the factory values byte for
byte, with none missing, added or duplicated. Any deviation means the user
tuned compression and their configuration is left alone.

gzip stays in nginx.conf rather than moving to http.d: nginx rejects a
duplicate gzip directive across contexts, and the compression settings page
reads and writes these same keys in nginx.conf, so a relocated block would be
reintroduced on the next save and break nginx -t.

The config parser is not used for this. Its dumper regenerates the whole file,
drops standalone comments and reorders proxy includes, which would be
destructive on a user's main config. Lines are edited individually so
everything outside the gzip block stays byte-identical.

Module builds on older installs

Two defects surfaced while testing the above against pre-dynamic versions.

Module state written before build modes existed carries no buildMode.
validateNginxModuleBuildMode rejects the empty value, which fails
loadNginxModules and with it GetModules, UpdateModule, Build and the
upgrade itself — the whole module subsystem, not just the static feature. The
missing value is now inferred from what the install can actually do: dynamic
when the builder and catalog are present, static when the compose file still
has a build section and build/Dockerfile to recompile the image.

Asking such an install to build a module used to return "the installed
OpenResty version does not support dynamic module builds", which is a dead
end: these versions produce modules by compiling them into the image, and they
still can. The build is now retargeted to the static path, with
--add-dynamic-module rewritten back to --add-module and =dynamic
switches reduced to their plain form. The retarget applies to a copy that
drives one build and is never persisted, so the catalog stays authoritative
and modules return to dynamic once the install gains a builder. The error is
kept only for installs that reference a prebuilt image and genuinely cannot
compile anything, and it now says so and points at the upgrade.

Separately, manual builds and upgrades disagreed on when a full image rebuild
is required: executeNginxModuleBuild also treated a non-empty
RESTY_CONFIG_OPTIONS_MORE in .env as a reason to rebuild. That value is
derived state — configureStaticNginxModules rewrites it from the current
module list before every build, writing an empty string when no static module
is enabled. The rebuild it triggered therefore ran with an empty option list
and could only reproduce the image it started from, up to 120 minutes to
arrive back where it began. Both paths now decide on the module list alone.

Also fixed: size units in the performance page

The form stripped the unit suffix when reading a directive and then always
appended a fixed one when saving, silently reinterpreting the unit.

A config carrying gzip_min_length 512; — 512 bytes — was read as 512 and
written back as 512k, inflating the threshold by 1024 and effectively
disabling compression for every response under 512 KB. The same applied to
client_header_buffer_size and client_max_body_size, where the value grew
by 1024× in the riskier direction.

Additionally Number(value.match(/\d+/g)) coerces a multi-number match to
NaN, so a directive such as gzip_buffers 4 16k would blank the field.

UI

The brotli section appears only once the module is enabled and built, since
nginx rejects the directives while the module is not loaded. Values are read
from and written to the managed file rather than nginx.conf, so they are
removed together with the module.

brotli_types is deliberately not exposed: it is kept aligned with
gzip_types so both encoders cover the same content, and exposing it would
invite the two lists to drift apart.

Commits

Commit
f61fcb9 manage http-context directives via conf/http.d
0db2a9c correct gzip defaults and add missing compressible types
23484fe activate brotli directives when the module is enabled
77bb04d refresh stock gzip defaults on upgrade
eef4438 preserve size units in nginx performance settings
9ba509b expose brotli settings in the compression page
ed75c0e stop a stale build option from forcing a full rebuild
98e17f3 build modules on versions without a dynamic builder

Testing

Verified against the real 1panel/openresty:1.31.1.1-2-4-noble image with a
real ngx_brotli .so, compiled using the appstore's own
Dockerfile.modules. 73 integration assertions plus 15 Go tests, all
passing.

Static checks: go build, go vet, gofmt and go test on linux/amd64 with
go1.26.1; frontend type-check (no new errors against a 267-error baseline),
prettier --check, and vite build.

brotli enableContent-Encoding: br confirmed; .so verified mapped
into the worker via /proc/*/maps; nginx -t and live reload both pass.

brotli disablenginx -t passes, live reload passes, and a full
container restart succeeds. The restart is the real risk and it is covered.

Negative tests — directives without the module are rejected
(unknown directive "brotli"), and unloading the module before removing its
directives breaks nginx. Both confirm why the ordering above is required.

gzip rewrite — driven by a harness compiled from the shipped source file,
so tested logic cannot drift from shipped logic. A stock config is rewritten
and nginx -t passes; configs with a tuned comp level, gzip switched off, a
byte-valued min_length, or an extra directive are all left byte-identical
(md5 unchanged); the rewrite is idempotent; non-gzip lines diff to zero.

Older installs — verified end to end against 1.27.1.2-5-1-focal, which
ships no Dockerfile.modules and no module.catalog.json: its module.json
has no buildMode, ngx_brotli compiles into the image, nginx -t accepts
the brotli directives with no load_module present, and the server responds
with Content-Encoding: br (5226 B → 48 B).

Full journey — old install → upgrade → enable brotli → disable → restart.

Measured on a 16 KB JSON response:

Size
before 16791 B uncompressed — gzip_types had no application/json
gzip 1093 B −93.5%
brotli 529 B −51.6% vs gzip

HTML page: raw 9042 B → gzip 146 B → brotli 83 B.

Not covered

  • reconcileDynamicNginxModuleConfig was not driven end to end; it needs a
    full agent plus database plus install record. The managed files were
    constructed to match its output format exactly and validated against real
    nginx behaviour.
  • The UI was type-checked and built but not exercised in a browser.
  • zstd is out of scope. Unlike brotli it is not prebuilt into the image and
    not in the catalog, so adding it means introducing a new third-party source
    rather than finishing an existing integration. The runtime-defaults table
    makes it a small change once that decision is made.

Snrat added 6 commits August 26, 2026 04:17
Add a managed-file mechanism for http-context nginx directives, mirroring
the existing one for conf/modules-enabled.

A separate directory is required because load_module is a main-context
directive, so modules-enabled is included at the top level of nginx.conf and
cannot host http-context directives.

Files carry a 1panel-http- prefix; anything else in the directory is left
untouched. Writes are atomic via a temporary file plus rename, and the
directory is snapshotted so a failed nginx -t can be rolled back.

The mechanism is inert when conf/http.d does not exist, which is the case
for OpenResty installations predating the directory.
Bring the embedded gzip template in line with how sites are actually served.
It was previously dead code: nothing referenced gzip.conf, so the values
never reached an installation. It is now embedded and used by the migration
that follows.

gzip_types was missing application/json, so JSON API responses were served
uncompressed. Also add ld+json, text/xml, xhtml+xml, rss+xml, atom+xml,
wasm, svg+xml and ttf/otf. Already compressed formats (images, woff2,
archives) stay out on purpose.

gzip_comp_level 6 -> 5, at the cost/ratio knee for gzip.

gzip_proxied any, so that proxied responses are compressed regardless of
their Cache-Control semantics.

gzip_static is intentionally not enabled: nginx does not verify that a .gz
file is newer than its source, so a stale artifact would be served
indefinitely with no error.
Enabling ngx_brotli only emitted load_module, leaving the module loaded but
inert: no response was ever brotli-encoded until the user added
`brotli on` and `brotli_types` to nginx.conf by hand. The module is
prebuilt into the OpenResty image and listed in the catalog, so the only
missing step was the runtime configuration.

Enabling the module now also writes its http-context directives to
conf/http.d, and disabling or deleting it removes them. Removal matters:
leaving `brotli on` behind after the .so is unloaded makes nginx fail to
start on an unknown directive.

Both directory sets are written before nginx -t runs, so nginx only ever
observes a consistent state, and a failed check rolls back load_module
files and runtime directives together.

Runtime defaults are declared per module in a table, so other modules
needing http-context configuration can be added without touching the
reconcile logic.

brotli_types matches gzip_types so both encoders cover the same content.
brotli_comp_level is 5 rather than the nginx default of 6: level 5 reaches
roughly gzip level 9 ratio at a fraction of the cost, while 6 is tuned for
static assets and is too expensive for dynamic responses.

brotli_static is deliberately omitted, for the same reason gzip_static is:
nginx does not verify that a precompressed artifact is newer than its
source, so a stale file would be served indefinitely with no error.

Installations without conf/http.d keep the previous behaviour instead of
failing.
Upgrades deliberately preserve the user's nginx.conf, so corrected gzip
defaults shipped with a new OpenResty version never reach existing
installations. Rewrite the values in place during upgrade, but only when the
block is provably untouched.

The rewrite requires every gzip directive to match the factory values byte
for byte, with none missing, none added and none duplicated. Any deviation
means the user tuned compression, and their configuration is left alone.

gzip stays in the http block of nginx.conf rather than moving to an included
file: nginx rejects a duplicate gzip directive across contexts, and the
compression settings page reads and writes these same keys in nginx.conf, so
a relocated block would be reintroduced on the next save and break nginx -t.

The config parser is not used either. Its dumper regenerates the whole file,
drops standalone comments and reorders proxy includes, which would be
destructive on a user's main config. Lines are edited individually so
everything outside the gzip block stays byte-identical.

The rewrite is idempotent, and a failed nginx -t restores the previous file.
A failure is logged as a warning instead of failing the upgrade.
The form stripped the unit suffix when reading a directive and then always
appended a fixed one when saving, so the unit was silently reinterpreted.

A config carrying `gzip_min_length 512;`, meaning 512 bytes, was read as 512
and written back as `512k`, inflating the threshold by 1024 and effectively
disabling compression for every response under 512 KB. The same applied to
client_header_buffer_size and client_max_body_size, where the value grew by
a factor of 1024 in the opposite, riskier direction.

Remember the unit that was read and write it back unchanged, defaulting to
the previous suffix only when the directive carries no unit information. The
input suffix now shows the unit actually in use instead of a hardcoded
label.

Also fix the value parsing itself: `Number(value.match(/\d+/g))` coerces a
multi-number match to NaN, so a directive such as `gzip_buffers 4 16k` would
blank the field. Take the first captured number instead.
Brotli could be enabled as a module but never configured from the panel, so
its behaviour was invisible and unchangeable without editing nginx.conf by
hand.

The section appears only once the module is enabled and built, since the
directives are rejected by nginx while the module is not loaded. Values are
read from and written to the panel-managed http.d file rather than
nginx.conf, so they are removed together with the module.

brotli_types stays out of the form on purpose: it is kept aligned with
gzip_types so both encoders cover the same content, and exposing it would
invite the two lists to drift apart.

Saving reuses the existing scope endpoint with a dedicated brotli scope,
which keeps the managed file as the single source of truth instead of
duplicating the values into nginx.conf.
Snrat added 2 commits August 26, 2026 06:25
Manual builds and upgrades disagreed on when a full OpenResty image rebuild
is required. `executeNginxModuleBuild` used `staticNginxBuildRequired`, which
also treated a non-empty `RESTY_CONFIG_OPTIONS_MORE` in .env as a reason to
rebuild, while `buildNginx` looked only at the module list.

The env value is derived state, not an input: `configureStaticNginxModules`
rewrites it from the current module list, and every build path calls that
function before building. With no static module enabled it writes an empty
string, so the rebuild the latch triggered ran with an empty option list and
could only reproduce the image it started from — up to 120 minutes of build
time to arrive back where it began.

Decide on the module list alone, which is what the upgrade path already did.

An install that genuinely has an enabled static module is unaffected: both
predicates already agreed in that case. Leftover values are still cleared, by
`configureStaticNginxModules` on the next build or upgrade.
Module state written before build modes existed carries no buildMode.
validateNginxModuleBuildMode rejects the empty value, which fails
loadNginxModules and with it every module operation and the upgrade itself —
the whole module subsystem, not just the static feature.

Infer the missing value from what the install can actually do instead:
dynamic when the builder and catalog are present, static when the compose
file still has a build section and build/Dockerfile to recompile the image.

Builds follow the same principle. Asking a pre-dynamic install to build a
module used to return "the installed OpenResty version does not support
dynamic module builds", which is a dead end: these versions produce modules
by compiling them into the image, and they still can. Such a build is now
retargeted to the static path, with --add-dynamic-module rewritten back to
--add-module and =dynamic switches reduced to their plain form. The error is
kept only for installs that reference a prebuilt image and genuinely cannot
compile anything, and it now says so and points at the upgrade.

The retarget applies to a copy that drives one build and is never persisted,
so the catalog stays authoritative and modules return to dynamic once the
install gains a builder.

Verified end to end against 1.27.1.2-5-1-focal, which ships no
Dockerfile.modules and no module.catalog.json: ngx_brotli compiles into the
image, nginx -t accepts the brotli directives with no load_module present,
and the server responds with Content-Encoding: br.
@wanghe-fit2cloud
wanghe-fit2cloud marked this pull request as draft August 26, 2026 06:31
@Snrat
Snrat marked this pull request as ready for review August 26, 2026 08:32
@zhengkunwang223

Copy link
Copy Markdown
Member

如果用户已经有
brotli 配置 怎么处理呢

@Snrat

Snrat commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

如果用户已经有 brotli 配置 怎么处理呢

感谢指出,目前这个场景确实没有进行覆盖。

appstore 的 upgrade.sh 会在 nginx.conf 中幂等插入 http.d 的 include 并创建空目录,但升级过程不产生任何托管文件,且 ngx_brotli 会针对新版本重新编译并自测——用户的手写配置在升级后照常生效。

如果用户先前手动执行过相关配置,说明用户已经成功启用了 br 模块,否则 nginx 会直接抛出报错。在这种情况下用户执行 Openrestry 升级时,用户不会遇到报错信息。

  1. 如果用户升级后在面板压缩配置页中修改相关配置,则会在 conf/http.d/ 下写入 brotli 相关配置。此时由于用户先前已经手动进行配置,会触发 duplicate directive 错误,面板通过 nginx -t 检测到该错误后会执行回滚并报错。并且页面此时显示的相关配置信息将失真。
  2. 用户模块页面中移除 br 模块时,会出现 unknown directive "brotli" 报错,属于合理范围,非本 PR 引入。

目前考虑针对此场景在压缩配置页面内增加配置文件扫描,识别用户是否已经手动启用 brotli 功能,并且要求用户执行相关操作以规避潜在的错误。

或者请问是否有其他的思路来解决此问题?此场景的影响面较小,因此不会影响到绝大部分用户。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants