Skip to content

Commit a409641

Browse files
committed
feat: complete automatic dual-runtime rigging
1 parent 1878881 commit a409641

104 files changed

Lines changed: 12494 additions & 2608 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,6 +179,7 @@ Lance versions and media/caption export are documented in the [Import / Export g
179179
| PSD layer export | [PSD Export](docs/tools/psd_export.en.md) | `2.4.psdexport.ps1` |
180180
| Audio separation | [Audio Separation](docs/tools/audio_separation.en.md) | `2.5.audio_separator.ps1` |
181181
| Image2PSD | [See-through](docs/tools/image2psd.en.md) | `2.6.image2psd.ps1` |
182+
| PSD auto-rig export | [Auto-rig](docs/tools/auto_rig.en.md) | `qinglong-auto-rig <item-or-final.psd>` |
182183
| Audio to MIDI | [MuScriptor](docs/tools/muscriptor.en.md) | `2.7.music_transcription.ps1` |
183184
| Sheet-music OMR | [MuSViT](docs/tools/sheet_music.en.md) | GUI Tools |
184185
| Text and document translation | [Translation](docs/tools/text_translation.en.md) | `5.translate.ps1` |

docs/superpowers/plans/2026-08-01-auto-rig-stage-b-geometry-implementation.md

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,12 @@
1010

1111
## Global Constraints
1212

13-
- Follow design spec Revision 24.
13+
- Follow design spec Revision 30.
1414
- B consumes authenticated `MaskComponentPlan v1`/QCL and `StageAJointPlan v1`; it never thresholds PNG/PSD, labels connected components, or runs a pose model.
1515
- `bone/root` is the only zero-length/synthetic bone. Every other emitted bone requires two resolved joints and a positive finite length; a missing parent is promoted to the closest emitted ancestor.
16-
- Meshes are built per frozen component ID. No triangle may cross another component or accepted transparent/hole samples.
16+
- Meshes are built per frozen component ID. No triangle may cross another component. Within one component, mesh geometry
17+
is a carrier envelope and may cover transparent/hole samples because the canonical straight-alpha texture remains the
18+
visibility source; every cleaned nonzero alpha pixel center must be covered by at least one triangle.
1719
- Persisted vertices are quantized to `1/256` canvas px. Qhull `QJ` is forbidden; symbolic perturbation exists only in the topology copy and is `<1/4096 px`.
1820
- Mesh rest positions remain LayerDiff canvas coordinates; UVs remain part-local top-left `u-right/v-down` values in `[0,1]`.
1921
- Each vertex has 1-4 finite non-negative influences summing to one. Limb transitions scale with local mask/joint radius and chain arc length, never a fixed pixel constant.
@@ -80,7 +82,10 @@
8082

8183
- [x] **Step 3: Implement RED/green contour and topology fixtures**
8284

83-
Cover rectangle, concave C, hole, two components, duplicate/near-collinear/cocircular points, and a degenerate component. Assert positive signed area, in-range flat indices, no accepted centroid or edge quarter sample in alpha zero, no cross-component edge, canonical vertex/triangle order, and `degenerate_mesh` instead of a random joggle.
85+
Cover rectangle, concave C, hole, two components, duplicate/near-collinear/cocircular points, and a thin diagonal.
86+
Assert positive signed area, in-range flat indices, no cross-component edge, canonical vertex/triangle order, and complete
87+
coverage of every cleaned nonzero alpha pixel center. Transparent samples inside a component envelope are legal and are
88+
clipped by the texture; do not reintroduce the disproven all-samples-inside-alpha rule.
8489

8590
- [x] **Step 4: Prove deterministic topology under input/library ordering changes**
8691

docs/superpowers/plans/2026-08-01-auto-rig-stage-e-live2d-implementation.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66

77
**Architecture:** Correct the Stage C primitive identity so per-mesh sampled deformation targets ArtMesh keyforms rather than inventing an unverified WarpDeformer. Project the canonical Rig through isolated Live2D symbol, parameter, binding/liveness, coordinate, ArtMesh, animation, and artifact plans into a disposable `Moc3V400Document`. Encode through the attested V4.00 section codec, reload through a strict pure-Python validator, then run Core consistency/model-state and SDK D3D11 WARP render/playback gates. Publish the exact E inventory marker-last. The packaged E0 attestation remains the startup proof for coordinate/section semantics; each item still receives its own runtime proof.
88

9-
**Frozen scope:** Revision 27 A-D outputs remain immutable. Stage E targets MOC3 V4.00 header version 3 and model3 Version 3, fixed basename `model`, at most four 2048x2048 straight-alpha pages copied byte-for-byte from C, one Part per Rig Part, one ArtMesh per Rig mesh component, single-parameter ArtMesh keyforms for sampled deform/opacity, and nested RotationDeformers keyed by `(bone_id, parameter_id)`. Multi-parameter ArtMesh grids, Glue, physics, pose files, CMO3, and formal Live2D limb bends remain unsupported.
9+
**Frozen scope:** Follow the Revision 41 public contract. Stage E targets MOC3 V4.00 header version 3 and model3 Version 3, fixed basename `model`, at most four 2048x2048 straight-alpha pages copied byte-for-byte from C, one Part per Rig Part, one ArtMesh per Rig mesh component, single-parameter ArtMesh keyforms for sampled deform/opacity, and nested RotationDeformers keyed by `(bone_id, parameter_id)`. Multi-parameter ArtMesh grids, Glue, physics, pose files, CMO3, and formal Live2D limb bends remain unsupported.
1010

1111
---
1212

@@ -118,19 +118,19 @@
118118

119119
- [x] **Step 1: Write failing model3/cdi3 tests**
120120

121-
Freeze basename `model`, Version 3, contiguous `textures/page_0.png...`, exact supported motion/expression references, no empty sections, no physics/pose/PMA extension, parameter/Part IDs from symbols, and EyeBlink/LipSync groups only when their used parameters exist. Missing/stale references or JCS/minified runtime JSON must fail.
121+
Freeze basename `model`, Version 3, contiguous `textures/page_0.png...`, exact supported motion/expression references, no empty sections, no physics/pose/PMA extension, parameter/Part IDs from symbols, and a LipSync group only when its used parameter exists. Do not emit EyeBlink: blinking is an explicit motion, and a global group lets compatible runtimes inject it into unrelated motions or overwrite wink expressions. Missing/stale references or JCS/minified runtime JSON must fail.
122122

123123
- [x] **Step 2: Implement deterministic runtime JSON encoders**
124124

125125
Reuse `cubism_runtime_json_bytes()` (ASCII, sorted keys, indent 2, terminal newline) so Framework 5-r.5 sees numeric terminators. Keep reports/JCS separate from runtime JSON.
126126

127127
- [x] **Step 3: Write failing motion/expression tests**
128128

129-
Convert each C-supported MotionClip curve exactly at `frame/30`, encode only linear segment type 0, one curve per parameter, zero fade, exact duration/loop/count metadata, and no resampling. Encode each supported ExpressionPreset as full-weight `Overwrite`, zero fade, with no pseudo-time axis. Reject duplicate parameters, unsupported blend, stale artifacts, or missing parameter keyforms.
129+
Convert each C-supported MotionClip curve exactly at `frame/30`, encode only linear segment type 0, one curve per parameter, and write `Meta.FadeInTime=0.0` plus `Meta.FadeOutTime=0.0` in the motion asset itself. Freeze Blink at `(0,1),(6,0),(9,0),(16,1),(24,1)` so it has both a closed hold and an eight-frame pre-completion rest hold. Encode each supported ExpressionPreset with zero fade and no pseudo-time axis: the two eye-open controls use full-weight `Multiply` so a persistent expression cannot overwrite explicit blink motion; every other target uses full-weight `Overwrite`. Reject duplicate parameters, missing/nonzero motion fade, missing Blink recovery hold, unsupported/per-control blend, stale artifacts, or missing parameter keyforms.
130130

131131
- [x] **Step 4: Implement motion3/exp3 compilation**
132132

133-
Preserve C artifact names and runtime application ordering. Blink/talk remain motions; happy/sad/surprised remain expressions. Optional format asymmetry stays explicit in the C motion manifest rather than being guessed from the output directory.
133+
Preserve C artifact names and runtime application ordering. Blink/talk remain motions; static facial presets remain expressions. Keep `.model3.json` free of `EyeBlink`; verify blink alone and blink while a wink/expression remains active. The SDK harness must consume serialized motion fade and must not call `SetFadeInTime`/`SetFadeOutTime` to repair missing asset fields. Sample Blink at its effect point, exact end, and the frame immediately before completion; the latter two must be rest. Optional format asymmetry stays explicit in the C motion manifest rather than being guessed from the output directory.
134134

135135
## Task 5: Official Core/SDK per-item release validation
136136

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# Auto-Rig Generic Expression Implementation Plan
2+
3+
**Goal:** Replace the false-positive procedural facial path with deterministic, exporter-neutral eye/mouth variants, expose a useful expression set in Spine 4.2 and Live2D, and validate declared effects in their anatomical ROI.
4+
5+
**Design source:** `2026-07-31-auto-rig-from-see-through-layers-design.md`, Revision 31.
6+
7+
## Constraints
8+
9+
- Authored `rig_inputs/variants` entries win over generated variants for the same semantic role.
10+
- Generated pixels come only from the validated item snapshot; no Anime2.5DRig sample artwork is copied.
11+
- Stage A owns generated PNGs and `GenericVariantSynthesisPlan`; Stage C remains the only public Rig writer.
12+
- Spine and Live2D consume the same admitted parts, controls, transfers, presets, and symbols.
13+
- A whole-frame SHA change is evidence only. It is never sufficient proof of a facial capability.
14+
- Every behavior change starts with a failing focused test and ends with the real Lucy SDK render.
15+
16+
## Task 1: Freeze Generic Variant Facts
17+
18+
1. Add failing unit tests for deterministic generated eye-closed, mouth-closed, smile, and frown records.
19+
2. Add `generic_variants.py` with a versioned synthesis plan and canonical PNG encoder.
20+
3. Derive eye geometry from sided eyewhite/irides/eyelash union and mouth geometry from the ordinary mouth layer; derive colors from alpha-weighted item pixels.
21+
4. Persist generated PNGs below `rig/cache/A/generated_variants/`, merge them with authored candidates, and include the plan/output inventory in Stage A.
22+
5. Verify byte determinism, authored override precedence, missing-feature degradation, and no path escape.
23+
24+
## Task 2: Add Crossfade Composite Semantics
25+
26+
1. Add failing quality/admission tests for transparent `crossfade_overlay_v1` candidates.
27+
2. Extend candidate/composite schemas without weakening `occluding_overlay_v1` replacement gates.
28+
3. Require complete open/closed target pairs and side-correct anchors; admit the atomic blink and mouth-crossfade bundles before atlas planning.
29+
4. Add open-target opacity bindings and closed-target inverse bindings to the same control.
30+
5. Verify default rest opacity and endpoints in the common Rig validator.
31+
32+
## Task 3: Fix Preset Selection and Expression Coverage
33+
34+
1. Add failing planner tests showing `talk`, `happy`, and `sad` can coexist when their typed targets do not overlap.
35+
2. Delete the hard-coded incompatibility map; keep only primitive-union conflict analysis.
36+
3. Add `unimpressed`, `wink_screen_left`, and `wink_screen_right` to the versioned preset/capability registries.
37+
4. Ensure happy/sad smile/frown targets are independent of talk's open/closed targets.
38+
5. Verify both format plans expose every supported preset with matching IDs and manifest records.
39+
40+
## Task 4: Project to Spine and Live2D
41+
42+
1. Add exporter tests for crossfade opacity timelines/keyforms and all expression files.
43+
2. Update Spine animations and Live2D bindings/model3 groups without exporter-local semantic inference.
44+
3. Validate default setup, parameter endpoints, expression application, exact artifact inventory, and deterministic bytes.
45+
46+
## Task 5: Replace SHA-Only Runtime Validation
47+
48+
1. Add failing tests where a one-pixel mutation changes SHA but must fail semantic validation.
49+
2. Record rest/effect RGBA and canonical face-part ROIs from the public Rig.
50+
3. Implement eye, iris, mouth, and brow metrics with the fixed Revision 31 thresholds.
51+
4. Make required facial effects fail with `runtime_effect_semantics_failed` when the target ROI contract is not met.
52+
53+
## Task 6: Add Minimum Depth-Aware Head Motion
54+
55+
1. Add tests showing equal motion for every head layer is rejected.
56+
2. Bind bounded translation/shear gains from normalized `depth_median`, with face as the reference plane.
57+
3. Verify near/far relative displacement in common Rig, Spine, Live2D parser, and SDK render evidence.
58+
59+
## Task 7: Real Lucy Release Verification
60+
61+
1. Remove stale generated Stage A-E/G owner artifacts through the normal inventory transaction.
62+
2. Run Lucy with `dual_runtime_avatar_v1`, official SDK/Core 5-r.5, and the D3D11 renderer.
63+
3. Render rest, every facial motion/expression, and head controls at 768x768.
64+
4. Assert the Revision 31 ROI metrics, inspect the contact sheet, run focused/full tests, Ruff, compileall, and `git diff --check`.
65+
5. Report exact supported/omitted presets and any residual visual limitation; do not call a hash-only effect fixed.

0 commit comments

Comments
 (0)