refactor(shader): migrate GLSL shaders to ShaderLab and clean up shader infrastructure #2961
Open
zhuxudong wants to merge 126 commits intogalacean:dev/2.0from
Open
refactor(shader): migrate GLSL shaders to ShaderLab and clean up shader infrastructure #2961zhuxudong wants to merge 126 commits intogalacean:dev/2.0from
zhuxudong wants to merge 126 commits intogalacean:dev/2.0from
Conversation
… and clean up old files All GLSL chunks and complete shaders have been migrated from packages/core/src/shaderlib/ to packages/shader/src/shaders/ with a unified directory structure. ShaderLib.ts is now an empty runtime registry populated by the shader package's registerIncludes(). Old VS/FS shader pairs in extra/ are replaced by ShaderLab .shader files. Post-process and AO GLSL includes updated to use new .glsl-suffixed keys. Blit.vs.glsl relocated from extra/ to shaderlib root.
… assertions Move camera_ProjectionParams declaration from Transform.glsl to Common.glsl where it is actually used by remapDepthBufferEyeDepth(). Update ShaderLab test to match new PBR.shader structure with inlined ShadowCaster/DepthOnly passes. Fix PrecompileABTest macro expansion test to find Forward Pass by name instead of hardcoded index.
…te PostProcess/AO Split shader package into two layers, create .shader files for PostProcess and AO, simplify FXAA3_11.glsl for ShaderLab compatibility, remove Shader.create() from core passes.
…ng with pre-migration source FXAA3_11.glsl was incorrectly stripped of conditional branches (FXAA_DISCARD, FXAA_FAST_PIXEL_OFFSET, FXAA_GATHER4_ALPHA) and type alias macros during the initial migration. Restore the full original 1028-line version to preserve all code paths. FinalAntiAliasing.glsl now includes the FXAA_GLSL_130/120 conditional defines and uses FxaaFloat type aliases, matching the pre-migration FinalAntiAliasing.fs.glsl source exactly.
…ly across all material shaders PBR/BlinnPhong/Unlit/PBRSpecular now reference Utility/ShadowMap and Utility/DepthOnly via UsePass instead of inlining or referencing PBR. Fix _resolveUsePass to handle shader names containing "/" by parsing from the end. Reorder registerShaders() so Utility shaders are created before material shaders.
- Remove `path` parameter from `Shader.create()` ShaderLab overload - Remove `_shaderRootPath` from ShaderPass - Remove `basePathForIncludeKey` from IShaderLab, ShaderLab, and Preprocessor - Remove relative path resolution in Preprocessor._replace() - Delete ShaderChunkLoader and simplify ShaderLoader - Clean up basePath references in tests and devtools example
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughReplaces the ShaderLab pipeline with a new ShaderCompiler and precompiled .gsp shader assets, renames and reorganizes shader include paths, updates engine/material/render pipeline to use shader-driven render states and constant-property masks, adjusts examples/E2E/tests to use ShaderCompiler, and adds bundler/CLI precompile tooling. Changes
Sequence Diagram(s)sequenceDiagram
participant App as App / Examples
participant Engine as WebGLEngine
participant Compiler as ShaderCompiler
participant Pool as ShaderPool
participant GPU as GPU/ShaderProgram
App->>Engine: create({ shaderCompiler: new ShaderCompiler() })
Engine->>Compiler: set as global compiler
Compiler->>Pool: provide precompiled .gsp assets (register via precompile or runtime)
Pool->>Engine: ShaderPool.registerShaders() (creates Shader/ShaderPass entries)
App->>Engine: load material / Shader.find("PBR")
Engine->>Pool: lookup Shader by name
Engine->>GPU: request program via ShaderPass._getShaderProgram(macroCollection)
GPU-->>Engine: compiled/linked ShaderProgram
Engine->>GPU: render draw calls (using shaderData + constant-property masks)
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related issues
Possibly related PRs
Suggested reviewers
✨ Finishing Touches🧪 Generate unit tests (beta)
|
…alacean#2960 - Delete unused noise files (NoiseCellular, NoisePerlin, NoisePsrd, NoiseSimplex) - Add NoiseSimplexGrad.glsl with simplexGrad() returning vec3 gradient - Add algorithm attribution comments to NoiseCommon.glsl - Add Particle/Module/NoiseModule.glsl with curl noise sampling - Integrate noise velocity into ParticleFeedback.glsl - Update Shaders/index.ts registrations for new includes
- Remove _createShader wrapper that handled ShaderPool name conflicts - ShaderPool no longer registers VS/FS fallbacks, so no conflict exists - Fix Shaders/index.ts noise registrations reverted by case issue
…m core - Strip shader package to zero dependencies (pure data exports only) - Move include registration to ShaderPool.init() and shader registration to ShaderPool.registerShaders() in core - Split exports: ShaderLibrary/index.ts for fragmentList, Shaders/index.ts for complete shader sources - Auto-register built-in shaders in Engine._initialize() after ShaderLab is set, so external callers no longer need manual register calls - Remove registerIncludes/registerShaders from all tests, examples, e2e - Update docs to reflect automatic registration
… DepthOnly passes - ShadowMap.shader: bind RenderQueueType = material_ShadowCasterRenderQueue - DepthOnly.shader: bind RenderQueueType = material_DepthOnlyRenderQueue - Remove duplicate lowercase shaders/utility/ entries from git index
github-actions
Bot
added
the
documentation
- rollup-plugin-shaderlab transforms .shader to IPrecompiledShader JSON when PRECOMPILE=true - ShaderPool.registerShaders() detects string vs precompiled object and calls appropriate Shader factory method - Move ShaderLab setting and registerShaders() from Engine._initialize() to constructor, before BasicResources - Route .shader through shaderlabPlugin in rollup.config.js - Tests read .shader source via readFile() instead of package import - Remove redundant Shader.create(PBRSource) calls from e2e cases
…ransform - Remove double shadow intensity mixing in sampleShadowMap, move fade calc back inside the if block - Restore inverse-transpose normal transform for BlinnPhong skinning
… add sources subpath - Add libs/ directory with 21 precompiled .gsp files (checked into git) - libs/index.ts aggregates all .gsp imports as IPrecompiledShader objects - src/index.ts re-exports from libs/ (FinalAntiAliasing kept as string fallback) - Add src/sources.ts for editor use: raw .shader strings via /sources subpath - Add package.json exports field with "." and "./sources" subpaths - Add *.gsp type declaration in global.d.ts - rollup-plugin-glsl: handle .gsp files (JSON → object export) - rollup-plugin-shaderlab: revert to .gs only, add buildStart/watchChange hooks - rollup.config.js: add sources build entry for shader package - scripts/precompile-shaders.mjs: full/incremental precompile script - ShaderPool: add missing UIDefaultSource registration
- Revert lazy getter in BasicResources, Background, PostProcessUberPass back to eager constructor init - Migrate render states from TS to .shader files: * Blit/BlitScreen: DepthState disabled * BackgroundTexture: DepthState CompareFunction LessEqual * Sprite/Text: BlendState, DepthState, RasterState - Remove unused imports
macOS case-insensitive FS caused both Shaders/ and shaders/ paths to coexist in the git index after directory rename.
…ile compatibility - Sprite/Text shaders: use PBR-consistent variable names for render states (renderQueueType, sourceColorBlendFactor, blendEnabled, etc.) - Skybox/SkyProcedural shaders: add hardcoded DepthState and RasterState - BasicResources._create2DMaterial: use shaderData.setInt() instead of renderState API - BaseMaterial: migrate setIsTransparent/setBlendMode/setRenderFace/_setAlphaCutoff from renderState API to shaderData.setInt() - PBRMaterial: migrate renderQueueType to shaderData.setInt() - SkyBoxMaterial/SkyProceduralMaterial: remove TS render state code (now in shader) - Fix precompile script: handle undefined throw, non-fatal exit on failure - Add precompile dedup flag to avoid repeated runs in b:all - Regenerate all .gsp files to reflect shader source changes
- Move shaderLab configuration back to _initialize() where it was on dev/2.0 - registerShaders() stays in constructor (precompiled path doesn't need ShaderLab)
- PBR: material_OcclusionTextureCoord Float → Enum(UV0/UV1), material_AttenuationDistance range max 1 → 5 - PBRSpecular: same OcclusionTextureCoord fix, add Clear Coat group (5 properties matching PBR) - BlinnPhong: material_EmissiveColor Color → HDRColor, material_Shininess range (1,1024) → (0,100) - Particle: add full Editor block (Base, Emissive, Common) - 2D/Trail: add full Editor block (same as Particle) - Sky/Skybox: add Editor block (TintColor, Exposure, Rotation, CubeTexture) - Sky/SkyProcedural: add Editor block (Exposure, SunMode, SunSize, etc.) - Fix precompile script error logging to print full stack trace
- Add libs/ to tsconfig include so tsc generates types for .gsp imports - Update package.json types paths to types/src/ (tsc output structure)
…omment handling - Add ShaderLabUtils.skipComment() as shared primitive for comment detection - Add ShaderLabUtils.removeComments() to strip all comments preserving newlines - Preprocessor.parse() now strips comments before include expansion and macro collection, fixing false matches on #include inside block comments (e.g. FXAA3_11.glsl) - BaseLexer.skipCommentsAndSpace() reuses ShaderLabUtils.skipComment() - Lexer overrides skipCommentsAndSpace() to only skip whitespace since comments are already stripped by Preprocessor - IShaderLab._parseShaderPass return type: IShaderProgramSource | undefined
…d unify export names
- precompile-shaders.mjs: auto-generate libs/index.ts from actual .gsp files
on disk (variable names derived from src/Shaders/index.ts); add --index-only
mode; add cleanOrphanedGsp to remove stale .gsp files; detect stale
shader-lab dist and rebuild automatically
- rollup-plugin-shaderlab: remove buildStart for non-watch builds (precompile
runs as npm script before rollup); in watch mode: .shader changes precompile
immediately in watchChange, shader-lab/glsl changes defer to writeBundle;
syncGsp deletes gsp and updates index when .shader is deleted
- package.json: add precompile script; b:module/b:umd/b:bundled/b:all run
precompile before rollup; watch/dev enable PRECOMPILE=true
- src/index.ts: simplify to export * from ../libs
- Unify export names to {FileName}Source convention:
* UberShaderSource → UberSource
* FinalSRGBShaderSource → FinalSRGBSource
* FinalAntiAliasingShaderSource → FinalAntiAliasingSource
* BloomShaderSource → BloomSource
* SAOShaderSource → ScalableAmbientOcclusionSource
- Fix BlinnPhong shadow: SCENE_IS_CALCULATE_SHADOWS → NEED_CALCULATE_SHADOWS in MobileBlinnPhong.glsl - Fix BlinnPhong tangent: use renderer_NormalMat instead of renderer_ModelMat - Restore MATERIAL_OMIT_NORMAL guard in VertexPBR and ForwardPassBlinnPhong - Fix project-loader.ts: restore ShaderLab instance - Add const RenderState to UIDefault.shader and remove TS render state code from ui/index.ts - ShaderPool: throw on registration failure instead of swallowing with Logger.warn
This reverts commit 33f0318.
Add `scripts/generate-shader-library-index.js` that scans `src/ShaderLibrary/` for every `.glsl` file and emits the import + fragmentList block to `src/ShaderLibrary/index.ts`. The `includeKey` is the path relative to that directory (forward slashes), matching the URL-resolved key the Preprocessor produces for `#include "X/Y.glsl"`. Hook the generator into `precompile` so it runs before the bundler each build, keeping the index always in sync with the directory contents. This also picked up `Particle/ParticleFeedback.glsl`, which was missing from the hand-written list.
…runtime split - Rename `compiled/` to `compiledShaders/` so the precompiled-output dir parallels the `Shaders/` source dir at the package level. - Auto-generate `Shaders/index.ts` alongside `ShaderLibrary/index.ts` via a unified `scripts/generate-source-indexes.js`. Each entry carries a `path` (or `includeKey`) relative to `packages/shader/src/`, so identities match what users type in `#include` and what the editor displays in its asset tree. - Drop legacy `<Name>Source` exports from the editor-side surface; the `/sources` subpath now exposes only the `shaderLibrary` and `shaders` arrays, removing the same-name-different-shape collision with the runtime main entry's `.gsp` exports. - Rename `fragmentList` / `IShaderFragment` → `shaderLibrary` / `IShaderLibraryItem` to match the directory name and align with how users see "the shader library". - Engine-side index/sources cleanup. Editor still imports the old names — it needs a follow-up adapt pass against the new arrays + the `/sources` path.
Add `// prettier-ignore` ahead of the shaderLibrary / shaders arrays so the build does not produce diff churn each time the post-build prettier pass reflows entries past the print width.
ShaderLibrary chunks now register with their full path prefix (e.g. "ShaderLibrary/Common/Common.glsl"), so e2e cases that wrote `#include "Common/Common.glsl"` could no longer resolve their includes — the runtime Preprocessor logged "shader slice not founded" and dropped the chunk, producing broken GLSL and 14–42% visual regressions. Update the three offenders (camera-opaque-texture, material-shaderReplacement, spriteMask-customStencil) to match the registered keys.
Roll back the source-level changes from the four shader-compiler bug-fix commits so the parser/lexer/preprocessor logic matches the dev/2.0 baseline (the state right after the macro-AST refactor was merged in 1f17f50). The package rename, bundler restructuring, include path resolution, and other PR-specific work are kept intact. Reverted source changes: - Lexer.ts: restore the `skipCommentsAndSpace` override that fed the Preprocessor's pre-stripped output (6312a6c). - Preprocessor.ts: drop the `removeComments` pre-pass and its import (bc91ce5). - lalr/CFG.ts + parser/TargetParser.y: drop the `MACRO_CALL` declarator variants (2a7e6b4) and re-introduce the per-declaration-site `[macro_call_symbol, MACRO_CALL]` productions removed by the `type_specifier_nonarray` unification (25c6fe2). - parser/AST.ts: revert the macro-as-type-alias unification. Tests will regress until the bug fixes are reconstructed from first principles in follow-up commits.
The first revert pass only covered the four bug-fix commits I had on my list. PR-author commits earlier in the branch (a69041d "support macro-defined type aliases" and 6629e9e "strip comments before preprocessing") had layered additional `macro_call_symbol` productions across declaration sites and a Lexer comment-skip override on top of the parser/lexer pipeline, which the first pass missed. Restore CFG.ts / AST.ts / TargetParser.y / lexer/Lexer.ts to their dev/2.0 baseline (just after the macro AST refactor merged), then re-apply only the package-rename touch-ups (ShaderLab → ShaderCompiler, ShaderLabUtils → ShaderCompilerUtils, comment header). The Preprocessor keeps the PR-specific switch to `ShaderFactory.getInclude`; everything else in these files now matches dev/2.0 byte-for-byte modulo the rename.
…er tweaks
Two more lines from `a69041d6f` ("support macro-defined type aliases")
slipped past the previous revert pass:
- ShaderTargetParser.parse: `traceBackStack.length = 0;` reset injected
before the initial `push(0)` ("prevent cross-shader contamination").
- ShaderCompiler._parseShaderPass: extra `clearAllShaderCompilerObjectPool()`
call at the top ("clear object pool at start of _parseShaderPass").
Both lines are gone; the two files now match dev/2.0 modulo the package
rename.
…pers The two helpers were added for the strip-comments-before-include-regex pre-pass in 6629e9e / bc91ce5. Both consumer call-sites (Preprocessor and the Lexer override) were already reverted; the helpers are now dead code. Drop them so ShaderCompilerUtils matches dev/2.0's ShaderLabUtils modulo the package rename.
When this chunk was migrated from `core/src/lighting/ambientOcclusion/shaders/`
to `shader/src/ShaderLibrary/AO/`, the rewrite that dropped the `uniform` /
`varying` keywords (no longer needed under ShaderLab) also stripped a number
of explanatory comments that survive on dev/2.0:
- Per-uniform docs ("Inverse of the squared radius", etc.).
- Algorithm references and URLs above `computeViewSpaceNormal`.
- Direction labels ("left"/"right"/"down"/"up", "left2"/"right2", ...) on
the four-tap depth fetches and the edge-weight steps.
- Step-by-step commentary inside `computeAmbientOcclusionSAO` /
`scalableAmbientObscurance` and the `frag` entry.
Restore them. The functional structure stays as-is (ShaderLab-style:
attribute-block-driven `Varyings v`, `v.v_uv` accessor, `frag` entry).
…recompile failures
ShaderLab is a deliberately *lazy* preprocessor — `#define`s aren't expanded
at parse time so the GLSL driver can do it later, and `MacroCallSymbol`
nodes ride into the AST as first-class citizens that variant codegen can
expand at runtime against the active macro state. Two implementation gaps
made the lexer over-aggressively tag identifiers as `MACRO_CALL` and the
grammar reject the resulting tokens in legal GLSL positions, breaking
FXAA's compilation. The bundler then masked the breakage by exiting 0
even when individual shaders failed.
Lexer: branch precision
-----------------------
- `#if expr` opens used to share a single `{name: "", defined: true}`
sentinel, so the `#else` polarity flip was indistinguishable from `#if`
in `isVisibleFrom`. A `#define X` registered inside `#if` was visible
from `#else`. Replace with a per-`#if` synthetic name (`__if_<n>`) so
the matching `#else` is genuinely mutually exclusive. `#elif` likewise
gets a fresh tag per chain link.
- The MACRO_CALL classifier checked only `macroDefineList[word]` non-empty,
not whether any of those defs is reachable from the current branch. Add
`_isVisibleMacro` so identifiers shadowed in a sibling `#if` arm parse
as plain `ID`. Fixes FXAA's `#define lumaS luma4A.x` (in `#if FXAA_GATHER4_ALPHA == 1`)
shadowing `FxaaFloat lumaS = …` (in `#else`).
Grammar: residual coverage
--------------------------
The lexer is conservative about `#if expr` (no expression evaluator), so
two independent `#if` blocks tagged with different synthetic names look
non-mutually-exclusive even when they aren't (`#if X == 1` vs `#if X == 0`).
Two grammar variants pick up that residual:
- `type_specifier_nonarray → macro_call_symbol` so any declaration site
(variable, struct member, function return / parameter, local) accepts
a macro-as-type alias (`#define FxaaFloat float; FxaaFloat x;`). Goes
through the existing non-terminal so the macro stays a first-class AST
node and the LALR table disambiguates from expression-position calls.
- `single_declaration → fully_specified_type MACRO_CALL [ = initializer ]`
for the cross-`#if` declarator-name collision (FXAA's `lumaNW` with two
independent `#if` arms).
AST glue
--------
`TypeSpecifierNonArray.init` recognises the new `MacroCallSymbol` child
and emits `{ type: TypeAny, lexeme: macroName }` — same shape as a
user-typed struct, so downstream codegen needs no further changes.
Bundler: surface failures
-------------------------
The bundler used to log per-shader `FAILED:` lines and still exit 0, so
a broken built-in shader would slip through CI as long as a stale `.gsp`
sat in the working tree. Make `runFull` return the failure count and let
the one-shot CLI path forward it as a non-zero exit so `pnpm run b:all`
breaks the build. Watch mode keeps streaming — failures don't tear the
watcher down.
The `./sources` entry exposed two structurally identical but field-named
differently arrays:
- `shaders[]` items used `path` (relative to `packages/shader/src/`)
- `shaderLibrary[]` items used `includeKey` (same semantic, same value
shape — relative path matching the string consumers write in
`#include "..."`)
Unify on `path` so:
- One shared `IShaderSource` interface covers both arrays — consumers
can write a single helper that works against either.
- The interface is `export`ed (was internal in the auto-generated index),
so consumers can type their helpers explicitly instead of
`(typeof shaders)[number]`.
- Field name matches the `#include` string, the registered key in
`ShaderFactory.registerInclude`, and the relative path on disk —
closing the equivalence chain instead of having two parallel names.
Also switches the precompile benchmark off the cross-package
`readFile("../../../packages/shader/src/...")` magic path onto the
public `./sources` entry — same content, no fs cwd assumption, no
hard-coded directory layout.
Engine-side consumer updated:
- ShaderPool.init: `item.includeKey` -> `item.path` (one call site).
Editor-side consumers in the editor repo
(`fragment.includeKey` in `packages/constants/src/builtin-shaders.ts`
and `packages/model/src/modules/assets/builtinShaders.ts`) need to be
updated to `fragment.path` when they upgrade to this version.
…tion rules `TargetParser.y` is the bison sanity-check resource for the runtime grammar in `lalr/CFG.ts` (header: `// For cfg conflict test, used by bison`). The two are hand-maintained in parallel — the recent fix (87cb2b5) added macro-as-type-alias and cross-`#if` declarator-name collision rules to CFG.ts but missed the .y mirror, so anyone running `bison TargetParser.y` to vet a future grammar change would now be vetting a stale snapshot. Mirror the two CFG.ts additions: - `type_specifier_nonarray → macro_call_symbol` - `single_declaration → fully_specified_type MACRO_CALL [= initializer]` bison reports 2 shift/reduce conflicts (was 1): - State 117 (new, expected): `macro_call_symbol .` looking at `(` — shift wins, producing `macro_call_function` instead of reducing to `type_specifier_nonarray`. Pseudo-ambiguity: GLSL has no C-style cast, so the reduce path leads to no valid parse — only shift produces a complete derivation. Same disambiguation the LALR table at runtime resolves; matches expression-position macro call semantics. - State 439 (was 424 on dev/2.0, dangling-else): pre-existing GLSL/C grammar ambiguity, shift attaches `else` to nearest `if`. State number shifted because the new rules grew the state table. No `%expect` declaration added — repo convention already accepts bison reporting a known conflict count without a declaration.
GuoLei1990
force-pushed
the
refactor/glsl-to-shaderlab
branch
from
aadf8e9 to
0a269a7
Compare
…if shadowing
Under ShaderLab's lazy preprocessor the same name can be a macro in
one preprocessor arm and a variable in the mutually-exclusive arm
(FXAA-style cross-arm shadowing):
#if FXAA_GATHER4_ALPHA == 1
#define lumaS 1.0
#else
float lumaS = 0.5;
#endif
...
use(lumaS);
Grammar `single_declaration -> fully_specified_type MACRO_CALL [= initializer]`
(added in 87cb2b5) accepts the parse, but `MacroCallSymbol.referenceSymbolNames`
collected only the macro value's referenced names, not the macro name itself.
Use sites of `lumaS` are also tagged MACRO_CALL, so `referenceGlobal` was never
called for `lumaS` -- the global-decl emitter then dropped the sibling-arm
`float lumaS = 0.5;` and the variant where the sibling preprocessor condition
was false had `lumaS` undeclared.
Fix `VariableIdentifier.semanticAnalyze` to also probe the macro name itself
when the use site is MACRO_CALL, with a one-name `macroDefineList`
short-circuit bypass so the symbol-table lookup actually reaches the
sibling-arm declaration.
Tests:
- New fixture `cross-if-declarator-collision.shader` reproduces the FXAA
pattern; test asserts the emitted vertex retains `float lumaS = ...` to
lock both halves of the fix (grammar + codegen).
- Drive-by: clear stale "long-standing limitation" comment in
`type-alias-repro.shader` (the limitation was lifted in 87cb2b5;
`macro-type-alias.shader` now covers macro-as-type usage).
Verified: 35/36 test fixtures pass (the 1 failure is `render-state.shader`,
already failing on baseline, unrelated), 22/22 built-in shaders pass.
Replace the catch-all `Utility/` and `AO/` buckets with semantic directories that mirror the runtime's role: Shaders/AO/ -> Shaders/Lighting/ Shaders/Utility/ShadowMap.shader -> Shaders/Pipeline/ShadowCaster.shader Shaders/Utility/DepthOnly.shader -> Shaders/Pipeline/DepthOnly.shader Shaders/Utility/Blit.shader -> Shaders/Blit/Blit.shader Shaders/Utility/BlitScreen.shader -> Shaders/Blit/BlitScreen.shader Shader names follow the directory: AO/ScalableAmbientOcclusion -> Lighting/ScalableAmbientOcclusion Utility/ShadowMap -> Pipeline/ShadowCaster Utility/DepthOnly -> Pipeline/DepthOnly Utility/Blit -> Blit/Blit Utility/BlitScreen -> Blit/BlitScreen Rationale: - `Lighting/` mirrors core/src/lighting/ -- SSAO is a screen-space lighting effect, not a post-process effect (matches Unity URP/HDRP and Unreal classification). - `Pipeline/` holds the shared base passes that material shaders reference via UsePass -- distinct from `Blit/` which holds the Blitter utility's internal shaders. Both were lumped into `Utility/` despite serving different roles. - `AO/` was using a non-spell-out abbreviation while core uses the full `ambientOcclusion`; with only one AO shader today, flattening to `Lighting/ScalableAmbientOcclusion` avoids redundant `AmbientOcclusion/ScalableAmbientOcclusion` until a second AO algorithm appears. ShaderLibrary chunks unscatter from Common/ to their semantic homes: ShaderLibrary/AO/ -> ShaderLibrary/Lighting/ ShaderLibrary/Common/MobileBlinnPhong -> ShaderLibrary/BlinnPhong/ ShaderLibrary/Common/BlitVertex -> ShaderLibrary/Blit/ `Common/` now only holds genuinely shared utilities (Common, Color, Fog, Light, Normal, Transform, Attributes, UV, Position*, ViewDirection, WorldPosition). MobileBlinnPhong was only consumed by BlinnPhong; BlitVertex is the vertex implementation for Blit-family shaders. Drive-by: remove stale packages/shader/types/compiled/ and packages/shader/types/shaders/ directories left over from a pre-rename build. Updated: PBR/BlinnPhong/Unlit UsePass references, ShaderPool source imports & registration order comment, BasicResources.ts Shader.find calls, ScalableAmbientObscurancePass.SHADER_NAME, Shader.test.ts UsePass, PrecompileBenchmark.test.ts builtinSource paths and Shader.find. Verified: 35/36 test fixtures pass (the 1 failure is render-state.shader, already failing on baseline, unrelated), 22/22 built-in shaders compile.
…ia DI
The bundler used to obtain its include map indirectly: importing
`@galacean/engine` triggered `ShaderPool.init()` (an Engine.ts top-level
side effect) which registered engine-shader's dist snapshot into
`ShaderFactory._includeMap`. Preprocessor then read that map via a hard
import. This created a build-pipeline cycle:
precompile -> consumes engine-shader's dist -> rebuilt by precompile
Renaming a chunk path (e.g. `ShaderLibrary/AO/X.glsl` ->
`ShaderLibrary/Lighting/X.glsl`) required rebuilding engine-shader before
precompile could see the new path, otherwise precompile failed against the
stale snapshot.
Replace the implicit global with explicit DI:
- `Preprocessor.parse(source, basePath, includeMap)` — pure function over
the injected map; no `ShaderFactory` import.
- `ShaderCompiler` carries a `_includeMap` instance field (defaults to `{}`)
and forwards it to `Preprocessor`. Public `new ShaderCompiler()` signature
unchanged.
- Runtime side: `Engine._initialize` binds
`shaderCompiler._includeMap = ShaderFactory._includeMap` once when the
user-supplied compiler is registered, so `Shader.create` keeps querying
the live engine registry — runtime behaviour identical.
- Build-time side: bundler builds its own map by scanning src
(`<inputDir>/*.glsl` + sibling `<ShaderLibrary>/*.glsl` by convention)
and injects it into `_includeMap`. Bundler no longer imports
`@galacean/engine`; it inlines `_shaderRootPath` as a literal.
Net effect: build artifacts no longer participate in their own build, the
chunk-rename scenario works on the very first precompile pass, and the
preprocessor is a pure function. No public-API changes.
Drive-by: trim verbose JSDoc in bundler/precompile.ts (310 -> 245 lines).
…aderCompiler After ea2d000 decoupled the preprocessor's include map via DI (ShaderCompiler holds a `_includeMap` field defaulting to `{}`), tests that bypass `WebGLEngine.create({ shaderCompiler })` and assign `Shader._shaderCompiler` directly leave the map empty -- every `#include` lookup then fails. Inject the runtime map (`ShaderFactory._includeMap`) right before assigning `Shader._shaderCompiler` in each affected test setup. Production code stays unchanged: binding is the consumer's responsibility, not shader-compiler's nor engine's. Verified: 1330/1330 tests pass.
GuoLei1990
force-pushed
the
refactor/glsl-to-shaderlab
branch
from
632de77 to
bd50c31
Compare
…methods ShaderFactory was the only meaningful resident of `shaderlib/`. Move it under `shader/` next to its consumers (ShaderPool/ShaderPass/Shader), delete the now-empty `shaderlib/` directory, and tag the class `@internal` (`stripInternal: true` removes it from the public d.ts). Also drop dead and over-wrapped surface: - delete unused `getInclude` / `unRegisterInclude` / `parseIncludes` (Preprocessor switched to dependency-injected map, no callers left) - drop the unused Logger import that came with `parseIncludes` - inline `_has300Output` (1-call wrapper around a regex test) - rename internal-only `_includeMap` / `_shaderExtension` -> `includeMap` / `shaderExtension` (class is fully `@internal`; underscore prefix added no info) Updated callers: Engine.ts, ShaderPool.ts, ShaderPass.ts plus four test files; shader-compiler doc comment refreshed. Verified: npm run b:module clean, 1330/1330 tests pass.
…used work Replace the over-permissive `RENDERER_HAS_TANGENT`-only gating with two narrower macros that match what the surface actually consumes: - `NEED_VERTEX_TANGENT` — vertex stage reads mesh tangent and writes the tangentWS / bitangentWS varying. Requires both a mesh tangent attribute AND a tangent-space normal map (base or clear coat). Anisotropy alone falls back to dFdx/dFdy in fragment, so it does not pull tangent through the vertex pipeline. - `NEED_TANGENT_SPACE` — fragment stage builds a tangent space (T/B vectors on SurfaceData + a temporary `mat3 tbn`). Triggered by any tangent-space material feature: normal map, clear coat normal map, or anisotropy. With NEED_VERTEX_TANGENT the basis comes from the interpolated varying; otherwise it is derived from screen-space derivatives. Previously a model carrying tangent attributes always paid for tangent skin/blend-shape transform, world-space projection, and two extra vec3 varyings — even when no normal map / anisotropy was bound. With this change those costs are skipped in the common "GLTF mesh + plain PBR" path. Matches the gating philosophy of dev/2.0's pbr_helper.glsl, Unity URP's REQUIRES_WORLD_SPACE_TANGENT_INTERPOLATOR, and Filament's HAS_TANGENT_SPACE. The two macros are defined at the top of ForwardPassPBR.glsl / ForwardPassBlinnPhong.glsl (the entry chunks for each shader family) so all downstream sub-chunks see the resolved values without needing a separate defines file.
- shader-mrt.ts: update the inline custom shader's `UsePass` from the obsolete `Utility/ShadowMap/Default/ShadowCaster` to the current `Pipeline/ShadowCaster/Default/ShadowCaster`. The old path stopped resolving after the shader directory reorganization, leaving the scene with no valid material so the canvas never rendered and Playwright timed out waiting for the screenshot download event. - config.ts: drop the residual diff tolerance for particleRenderer-shape-transform now that the regenerated baseline matches output exactly. - fixtures/originImage/Particle_particleRenderer-shape-transform.jpg: refresh the baseline image to match current renderer output.
…tity activation `rootEntity.createChild()` activates the entity immediately, so `addComponent(ParticleRenderer)` triggers _onEnable → generator.play() before the next-line `useAutoRandomSeed = false` assignment runs. play() then takes the default-true branch and seeds the generator with `Math.random()`, producing a fresh screenshot every run. Construct the entity detached, configure useAutoRandomSeed and the emission shape, then `addChild` to activate. play() now sees useAutoRandomSeed=false and skips the random seed roll, making the particle emission stream deterministic across runs. Refresh the baseline image to match the now-stable output.
- Remove unused legacy Common chunks (Color/Position/PositionClipSpace/ UV/WorldPosition). They were carried over from the old chunk-style shader pipeline and lost all consumers after the ShaderLab function rewrite — every defined function had zero call sites. - Inline the 4-line `getViewDirection` into Common.glsl and remove ViewDirection.glsl. Single consumer (BlinnPhong) and trivial body did not justify a standalone chunk file. - Move Light.glsl from Common/ into Lighting/. Its semantic class (light source struct + scene uniforms + cull helpers) belongs with the lighting effects rather than with low-level data scaffolding. - Group AO chunks into Lighting/AmbientOcclusion/ subdirectory now that Lighting/ contains both basic light declarations and SAO-specific helpers; the longer subdir name reads better than the bare "AO" abbreviation. - Update all `#include` paths in PBR / BlinnPhong / SAO shaders and regenerate the auto-generated ShaderLibrary/index.ts (58 chunks, down from 64).
The previous `.gsp` extension carried Galacean-internal branding
without conveying intent. Rename to `.shaderc` for two reasons:
1. Mirror Python's `.py → .pyc` convention (source extension + `c`
marker for "compiled"): visually adjacent to the source, suffix
blood-relationship is obvious, and the `c` is recognized in DX
land too (`.cso` = Compiled Shader Object — same `c=compiled`
semantics).
2. Drop the `g` prefix that only made sense inside Galacean. The
artifact is a runtime-ready shader bundle compiled from .shader
sources; "shaderc" is descriptive without project-specific noise.
Also fix a long-standing bug in the loader registration: the
@resourceLoader decorator listed only `["shader"]`, so the
ResourceManager could never route precompile artifacts to
ShaderLoader. Add `"shaderc"` so URLs ending in `.shaderc` reach
the precompiled-loading branch.
Renamed:
- 22 precompile artifacts under packages/shader/compiledShaders/
- `gspPathToVarName` → `shadercPathToVarName`
- `cleanOrphanedGsp` → `cleanOrphanedBundles`
- `removeGspFor` → `removeBundleFor`
- Local var `gspRelative` / `gspPath` → `bundleRelative` / `bundlePath`
Updated extension references:
- packages/shader/src/global.d.ts (module declaration)
- packages/shader-compiler/src/bundler/{transform,precompile,index,utils}.ts
- packages/loader/src/ShaderLoader.ts
- packages/core/src/shader/ShaderPool.ts (comment)
- tests/src/shader-compiler/Precompile{ABTest,Benchmark}.test.ts
- packages/shader/compiledShaders/index.ts (auto-regenerated)
The shader-compiler used to import Logger / Color / ObjectPool / enums from
@galacean/engine, which made it depend on the entire engine umbrella at
runtime. That created a build-time cycle: precompile loads shader-compiler/
dist, which require()s engine-core, but engine-core's dist depends on the
.shaderc bundles produced by precompile. The PR worked around it with
SKIP_GALACEAN, fs.existsSync guards, a graceful tryLoadShaderCompiler
fallback, and effectively required two b:all runs on a cold checkout.
Cut the cycle at the source: shader-compiler no longer imports anything
from @galacean/engine. Only @galacean/engine-math (for Color) remains, as
a single runtime dep — math is a leaf with no further engine dependencies,
so the workspace dependency graph becomes a clean DAG.
What moves into shader-compiler:
- src/enums/ — local copies of the 9 .shaderc wire-format enums
(BlendFactor, BlendOperation, ColorWriteMask, CompareFunction,
CullMode, RenderQueueType, RenderStateElementKey,
StencilOperation, ShaderLanguage), kept in lockstep with
the engine-core copies via README convention. Industry-
standard for offline shader compilers (Unity, Unreal,
glslang all do this).
- src/common/ObjectPool.ts — local ClearableObjectPool / ReturnableObjectPool /
IPoolElement implementations.
- Inlined SHADER_ROOT_PATH constant in Preprocessor.ts (was
ShaderPass._shaderRootPath).
- console.error/warn at error/warning sites (was a noop-by-default Logger
that silently swallowed errors).
Build-side changes:
- shader-compiler/package.json: drop @galacean/engine deps and peerDependencies,
add @galacean/engine-math as the only runtime dep, add
umd.globals mapping math to Galacean (so UMD doesn't inline
math, ~33KB minified savings on browser.min.js).
- shader-compiler/rollup.config.js: runtimeExternal=[] in b:compiler so math
gets inlined into the self-contained dist used by precompile;
resolve mainFields=["debug","module","main"] lets the math
source resolve via its `debug` field on a fresh checkout
where math/dist doesn't exist yet.
- bundler/precompile.ts: drop the graceful skip + window/document polyfill
— neither is needed once shader-compiler is engine-free.
Build pipeline collapses back to a single rollup pass, structurally
identical to dev/2.0 plus a precompile step in front. SKIP_GALACEAN /
fs.existsSync / tryLoadShaderCompiler / two-pass orchestration are all
gone. Cold-boot `pnpm b:all` succeeds in one run.
Bundle size impact vs the previous PR head: zero on every consuming package
(core, math, loader, rhi-webgl, shader, galacean, physics-*, ui, xr*).
shader-compiler itself grows ~16KB (the inlined enums + ObjectPool).
GuoLei1990
approved these changes
May 2, 2026
…nd clarify intent - mainFields=["debug"] (was ["debug", "module", "main"]): the runtime entry now resolves workspace deps strictly to source via the `debug` field, removing the dist-fallback path that could never legitimately fire (math always has a `debug` field) but invited stale-dist surprises if it ever did. - Reword the header / runtimeExternal / swcPluginRuntime comments to spell out exactly which build this is, what it produces, and why nothing is externalized at the runtime entry. No behavior change beyond the mainFields tightening.
…Info) to galacean umbrella
The umbrella `@galacean/engine` package now owns the three top-level side
effects that previously lived in `engine-core`:
- `Polyfill.registerPolyfill()` (matchAll / AudioContext / TextMetrics
/ Promise.finally — touches `window`)
- `SystemInfo._initialize()` (browser platform detection)
- `ShaderPool.init()` + `registerShaders()` (built-in shader assets)
Why
---
`engine-core` is supposed to be a generic engine runtime — neutral about
which shader set ships in the box and which environment it runs in. With
those three top-level effects sitting in `core/src/index.ts` (and
`core/src/Engine.ts`), any consumer that only imports an enum or utility
from core was forced to drag the entire flavor closure (engine-shader,
window touches) along with it. That ruled out two things:
- shader-compiler couldn't import `engine-core` enums / pools — even via
rollup's tree-shake — because `Engine.ts`'s top-level `ShaderPool.init()`
pulled in engine-shader, whose `export * from "../compiledShaders"` is a
physical file that doesn't exist yet at precompile time. We had been
working around this by maintaining local copies of nine enums plus
`ObjectPool` inside shader-compiler.
- core could never advertise `"sideEffects": false`, so user-app bundles
couldn't tree-shake unused core modules.
What moves
----------
- `core/src/shader/ShaderPool.ts` → `galacean/src/ShaderPool.ts`
Plus drop the cached `particleFeedbackPass` static field — particle code
now does `Shader.find("Effect/ParticleFeedback")` directly.
- `core/src/Engine.ts`: drop the top-level `ShaderPool.init()` and the
in-constructor `ShaderPool.registerShaders()` call.
- `core/src/index.ts`: drop the top-level `Polyfill.registerPolyfill()`,
re-export `Polyfill` instead so the umbrella can call it.
- `core/src/SystemInfo.ts`: drop the trailing `SystemInfo._initialize()`.
- `core/src/particle/ParticleTransformFeedbackSimulator.ts`: look up the
feedback shader via `Shader.find` with an explicit error when the
umbrella hasn't registered it (which would only happen if the consumer
built a custom flavor without registering built-in shaders).
- `core/package.json`: drop the `@galacean/engine-shader` dependency.
- `galacean/src/index.ts`: at module load, call `Polyfill.registerPolyfill()`
→ `SystemInfo._initialize()` → `ShaderPool.init()` →
`ShaderPool.registerShaders()` (the order matters — polyfills first,
platform detection next, then `#include` map and shader registration
before any `Material` constructor's `Shader.find` runs).
- `galacean/package.json`: pick up the new `@galacean/engine-shader`
dependency that core dropped.
- `galacean/src/ShaderPool.ts`: same shape as the old core copy minus the
`particleFeedbackPass` cache.
Drop the local enum / pool copies in shader-compiler
----------------------------------------------------
With core no longer flavor-bound, shader-compiler can finally import the
shared enums and pools from `@galacean/engine-core` without the closure
expansion that previously hit the missing `compiledShaders/index.ts`. So:
- Delete `shader-compiler/src/enums/` (nine wire-format enum copies + a
README describing the sync convention) and `shader-compiler/src/common/
ObjectPool.ts`.
- Rewrite the seven import sites to pull from `@galacean/engine-core`.
- Add `@galacean/engine-core` to `shader-compiler/package.json` deps and
to the UMD `globals` map so `browser.min.js` doesn't inline core.
Verification
------------
- Cold-boot `pnpm b:all` succeeds in a single pass; no `.shaderc` files
needed up front, no skipped precompile, no fallback paths.
- shader-compiler dist contains zero inline ShaderPool / Engine / Polyfill
/ SystemInfo / MathUtil / GLSL chunk markers; the only `require` calls
on the `@galacean/*` namespace are math and core.
- Total dist bytes across all packages: 56,119,531 → 55,907,479
(-212,052 bytes, ≈ -207 KB). The shader-compiler subtree shrinks ~263 KB
thanks to nine enum + pool copies no longer being inlined into 16 output
artifacts (8 .js + 8 .map across release / verbose × CJS / ESM / UMD /
minified). core shrinks ~4 KB; galacean grows ~21 KB to host the
bootstrap. Eight unrelated packages are byte-identical.
- vitest suite passes (manually verified).
Note: this still doesn't set `"sideEffects": false` on core — four
animation curve assemblers still rely on bare `import "./..."` to
self-register. Switching them to explicit `registerAssembler(...)` calls
would unlock the flag and let user bundles tree-shake unused core modules.
That's a follow-up; the present change keeps the assemblers as-is so the
behavior surface stays identical.
…r code
Until now the test suite mostly imported `WebGLEngine` from
`@galacean/engine-rhi-webgl` directly. That worked only because of an
accident: `@galacean/engine-core` had three top-level side effects
(`ShaderPool.init()`, `Polyfill.registerPolyfill()`,
`SystemInfo._initialize()`) that fired on any core import, no matter how
indirect — so the tests got built-in shaders, polyfills, and platform
detection wired up "for free" even though they never touched the umbrella
package the way real consumers do.
The previous commit moved those three bootstraps from `engine-core` into
the `@galacean/engine` umbrella so core can be flavor-agnostic. That
exposes the test path mismatch: 75 test files import `WebGLEngine` from
`engine-rhi-webgl`, never load the umbrella, and now have no built-in
shaders → `BasicResources`'s `Shader.find("Blit/Blit")` returns null →
`new WebGLEngine` fails with `Cannot read properties of undefined`.
Fix by routing every test's `WebGLEngine` (and the three Polyfill tests'
dynamic `import("@galacean/engine-core")`) through the umbrella, which is
the same import users write in real applications. No test logic changes —
77 files, only the import sources move:
-import { WebGLEngine } from "@galacean/engine-rhi-webgl"
+import { WebGLEngine } from "@galacean/engine"
`WebGLEngine` is the same class either way (the umbrella re-exports
rhi-webgl's), so the tests behave identically — they just exercise the
real consumer entry point and pick up the umbrella's bootstrap as a
side effect of the import they were already doing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
core/shaderlib(raw GLSL + rollup-plugin-glsl) toshaderpackage as ShaderLab.shaderfilesShaders/(ShaderLab entry points) andShaderLibrary/(GLSL include fragments)UsePassto reference themShaderChunkLoaderand shader path/chunk loading infrastructure (_shaderRootPath,basePathForIncludeKey)_resolveUsePassto support shader names containing "/" (e.g.Utility/ShadowMap)Motivation
Raw GLSL shaders scattered across
core/shaderlibrelied onrollup-plugin-glslfor bundling and lacked the render state / pass structure that ShaderLab provides. This migration:shaderpackage with a clear Shaders + ShaderLibrary layoutregisterIncludes()for include registration and ShaderLab for shader creationKey Changes
Shader Migration (
packages/shader/).glslinclude fragments organized by category (Common, Lighting, PBR, Skin, Shadow, Fog, PostProcess, Particle)UsePass Architecture
Utility/ShadowMapandUtility/DepthOnlyare canonical shared passesUsePass "Utility/ShadowMap/Default/ShadowCaster"registerShaders(): Utility first, then material shaders_resolveUsePassnow throws on malformed names or missing referenced shaders (was silently returning undefined)Render State Per-Property Priority
_constantPropertyMask(per-property bitmask) on ShaderPass_mergeUnmanagedFrom,_managedGroupMask,RenderStateGroupFlag,_copyFromRenderState._resolveValuehelper handles bool / numeric / enum priority resolutionInfrastructure Cleanup
ShaderChunkLoader, remove_shaderRootPath,basePathForIncludeKeyfrom Preprocessor/ShaderLab/IShaderLabTransformFeedbackShader(replaced by ShaderPass with_feedbackVaryings)Shader.create()only accepts ShaderLab source (vertex/fragment string overload kept as signature for tests)engine-toolkitdependency from e2e (WireframeManager, OrbitControl)Breaking Changes
Built-in shader names renamed (no aliases — alpha stage)
pbrPBRpbr-specularPBRSpecularblinn-phongBlinnPhongunlitUnlitSprite2D/SpriteSpriteMask2D/SpriteMaskText2D/TextTrail2D/TrailUIDefault2D/UIDefaultskyboxSky/SkyboxSkyProceduralSky/SkyProceduralbackground-textureSky/BackgroundTextureparticle-shaderParticleblitUtility/Blitblit-screenUtility/BlitScreenshadow-mapUtility/ShadowMapdepth-onlyUtility/DepthOnlyShader include paths reorganized
<common>Common/Common.glsl<transform>Common/Transform.glsl<fog>Common/Fog.glsl<skin>Skin/Skin.glsl<blendShape>Skin/BlendShape.glslAPI removals
setIsTransparent(value, passIndex?)/setBlendMode(value, passIndex?)/setRenderFace(value, passIndex?)—passIndexparameter removed. For per-pass render state, declare it directly in ShaderLabRenderStateblock.ShaderLibnamed exports (ShaderLib.common,ShaderLib.pbr_helper, etc.) — useShader.find()for shader sources instead.TransformFeedbackShaderclass removed (useShaderPasswith_feedbackVaryings).ShaderChunkLoader,_shaderRootPath,basePathForIncludeKeyremoved.Test Plan
npm run buildpasses (CI green)npx vitest run— all unit tests pass, including:tests/src/shader-lab/Precompile.test.ts— 92/92 passtests/src/core/shader/state/RenderState.test.ts— 18/18 pass (new, covers 3-tier priority)tests/src/core/material/BaseMaterial.test.ts— 9/9 pass (extended)_createFromPrecompiledround-trip works for PBRSummary by CodeRabbit
New Features
Documentation