Skip to content

Commit 583a5d9

Browse files
Nodes to Code (#47)
* initial codegen * stage 2 of codegen complete with simple inlining * initial stage 3 generation initial stage 3 generation initial stage 3 generation * ruff cleanup * bump version and changelog * refactor and update of plan * more iteration on codegen * initial refactor of dynamic inputs mixin * step 2 complete * step 3 complete * step 4 complete * finalise zone emitting * string emitters, compare lifting, float32 literals - FormatString emits fmt.format({...}) when the format string is linked, else g.FormatString("...", items={...}); StringJoin emits delim.join((...)) / g.JoinStrings((...)) with multi-input tuple in creation order. - Compare lifts to Python comparison operators (CompareOp IR, operands always parenthesised) when node state matches the operator overloads: ELEMENT mode, data_type matching lhs socket dispatch, default epsilon for ==/!=. Otherwise falls back to the factory spelling. - Float literals emit the shortest decimal round-tripping through float32, removing repr noise like 0.10000000149011612. - ItemsMixin items= dicts accept plain default values ("label": "hello") via the new _add_unlinked_input hook. * suppress test warning * emit shader/compositor TreeBuilder constructors to_python now maps the tree's bl_idname to TreeBuilder.shader(...) / TreeBuilder.compositor(...) instead of always emitting the geometry default; the tree name renders through _fmt so quotes escape correctly. socket-method specs: Mix, tuple-result methods, grid info - Mix lifts to factor.mix.float/vector/color/rotation when its state matches what those methods produce; new always_args spec field makes required params render positionally even at default values (skipping them would emit calls with missing arguments). - TupleMethodSpec dissolves NamedTuple-returning socket methods — s.find(x), mat.svd(), rot.to_quaternion(), rot.to_axis_angle() — into attribute accesses on the call; the call binds to a variable when more than one link consumes it so re-rendering cannot duplicate nodes. - GridInfo dissolves into grid.transform / grid.background_value; DissolveSpec gains consumed_props, "{data_type}" receiver resolution and an inferred-structure check. GridSocketMixin._info now reuses one GridInfo node per grid socket (it cannot use _find_or_create_linked because data_type must match the grid type). - _output_expr falls back to identifier-based accessor attributes when a node duplicates output names (Mix's four "Result" sockets), keeping name-based attrs for custom groups. skip disabled sockets in constructor literal kwargs Sockets disabled by the active mode/data_type (length= on an EVALUATED CurveToPoints, Mix's inactive typed inputs) hold stale values that evaluation ignores — emitting them as kwargs is noise. The enabled flag on the real node is the direct signal; no probe-node prop copying needed. interface fidelity: socket props and panels - Interface emission now compares every keyword-only parameter of the builder's SocketContext factory methods against a probed fresh interface socket and emits the ones that differ — subtype, min/max, hide_value, hide_in_modifier, default_input, structure_type, attribute_domain, default_attribute, menu expanded. description renders as a kwarg when non-empty. - Top-level single-direction panels re-emit as `with tree.inputs.panel("Name", default_closed=True):` blocks in items_tree order; nested or mixed input/output panels fall back to flat emission (the builder cannot author them). - Builder fix: _set_props now writes default_attribute to the bpy property default_attribute_name — previously any non-None default_attribute raised AttributeError. re-emit frames as with-blocks - _frame_order clusters each top-level frame's members contiguously via a cluster-level topological sort (each `with g.Frame():` block creates exactly one frame); a frame whose cluster sits on a cycle through outside nodes falls back to flat emission, as do nested frames. - Body lines are tagged with their node's frame and wrapped/indented into `with g.Frame("Label"):` blocks after the emission loop; the Frame alias follows the tree type (g/s/c). - Inlining stops at frame boundaries — an inlined node is created at its consumer's statement, so a single-consumer node in a different frame now binds to a variable instead. inline-width budget for deep graphs to_python gains max_inline_width (default 88): a node whose rendered expression exceeds the budget binds to a variable instead of inlining into its consumer, so deep operator graphs split into readable steps. Chain (>>) continuations are exempt — pipelines stay one statement and wrap fine under a formatter. None disables the budget. * expose and on TreeBuilder * fix tests * test coverage * move codegen and diagram behind export/ * make export output independent of link insertion order The arrange pass inserts reroute links in set-iteration order, so structurally identical trees can carry their links in different order between runs (PYTHONHASHSEED) — and both to_python and to_mermaid followed that order, breaking snapshots. - _effective_links now returns canonical (structural) order: sorted by node creation order and socket position via _canonical_links; _ordering_edges feeds the topological sort from it, so emission and statement order are a pure function of tree structure. - to_mermaid emits edges from _effective_links instead of raw node_tree.links (also replacing its own reroute tracing). - test_codegen/test_diagram imports updated for the export/ move; snapshots re-recorded in canonical order and verified identical across PYTHONHASHSEED 1/2/5/42/1337. * cleanup a test * test coverage * docs and better splitting across lines * parametrised usecase round-trip tests + valid identifier var names Every tree in test_usecases.py is now a module-level build_*() function (collected in ROUNDTRIP_BUILDERS) and round-tripped through codegen by a parametrised test. This caught _make_var emitting invalid Python identifiers (keywords like `with`, punctuation, leading digits) — fixed. Two strict xfails document the open gaps (MenuSwitch/group-node emission; unreproducible item identifiers on hand-built zones). PLAN.md updated to reflect completed Stage 9 polish and remaining To Do items. * codegen: MenuSwitch/IndexSwitch emitters MenuSwitch emits the factory dict form so enum item names round-trip (g.MenuSwitch.geometry(menu, {"Name": value, ...})); IndexSwitch emits the factory tuple form (g.IndexSwitch.float(index, (a, b, ...))) so item count and order survive. Three library fixes surfaced along the way: - items= values of None now declare an unlinked item (type comes from the node data_type) instead of crashing - an explicit string menu= selection is no longer clobbered by the first-item default - the ambiguous *NodeGroup bl_idnames are dropped from the codegen registry, so group nodes report unsupported instead of emitting an arbitrary CustomGeometryGroup subclass build_mask_grid's MenuSwitch now round-trips; its xfail narrows to the remaining custom-group-node gap. * codegen: recursive node-group emission A group node now round-trips as a Custom{Geometry,Shader,Compositor}Group subclass whose _build_group recreates the inner tree, instantiated as GeneratedClass(**{"Socket Name": value, ...}). to_python's body is split into a per-tree _emit_tree (returning a _TreeEmission of interface/body/output lines) plus module assembly. A _GroupCollector threads through EmitContext, dedupes inner trees by name, renders each class once, and orders them innermost-first (a group is appended only after the groups it nests). Only linked inputs and unlinked inputs differing from the group's own interface default are passed; the GroupCall IR renders the **{...} form since socket names need not be valid identifiers. Tests force a fresh _build_group rebuild (renaming existing groups) so the inner structure is verified, not just reuse-by-name. build_mask_grid's group now round-trips; its xfail narrows to the separate variable-items constructor gap (FieldToGrid emits an invalid field_0= kwarg). * Better code highlighting in documentation (#108) * better code highlighting for docs * more docs * codegen: variable-items node emitter (CaptureAttribute / FieldToGrid) These nodes take items={name: field}; the generic constructor path emitted each item input as its own field_0=/value= kwarg, which the constructors reject. A small _ItemsNodeSpec table names the fixed inputs and the factory method chosen by the node's data_type/domain, and item sockets are found generically as the trailing N input/output sockets (N = items-collection length). Emits the canonical factory form (g.CaptureAttribute.point(geometry=…, items={…}), g.FieldToGrid.boolean(topology=…, items={…})), reading captured outputs by item name. Bails to the generic path when a fixed input it can't author (e.g. a linked CaptureAttribute Selection) is in use. build_mask_grid now round-trips; the only remaining xfail is the out-of-reach hand-built-zone item identifier. * update snapshot * round-trip test for geometry PCA asset * codegen: extend variable-items emitter to Bake and FieldToList Both nodes have no node-level type/domain property, so they emit the plain constructor with the items dict: g.Bake(items={...}) (no fixed inputs) and g.FieldToList(count=..., items={...}). _ItemsNodeSpec's factory_prop/ factory_map are now optional, and a fixed input is emitted only when linked or differing from a fresh node's socket default, so an unlinked non-default FieldToList count survives. Item sockets keep the generic trailing-N rule. * codegen: round-trip variable-items sockets by role, not counter A hand-built node whose item collection was clear()ed and rebuilt keeps a higher creation-order counter in its socket identifiers (Generation_1) than a fresh node authored by codegen (Generation_0). The counter is an implementation detail — codegen does create a corresponding socket — so the round-trip _structure comparison now keys variable-items sockets on their role prefix + name (Generation:Geometry) instead of the raw identifier. Clears the last xfail (build_import_microscopy_meshes_api); all 19 usecase builders now round-trip. * tests: round-trip Blender's bundled geometry node-group assets Parametrise test_roundtrip_bundled_asset over every geometry node-group asset bpy ships (essentials, dynamics, hair, principal-components) — the broadest available codegen coverage, since these are real trees not authored through nodebpy. The 14 that round-trip cleanly are asserted hard for regression protection; the ~49 that hit codegen gaps are non-strict xfail so a future fix surfaces as XPASS rather than breaking the build. PLAN.md records the categorised backlog of gaps they exercise (menu/enum defaults, Bundle/Closure items nodes, string escaping, vector defaults, socket-method faithfulness, structural mismatches). * codegen: escape control characters in string literals _fmt used a naive backslash/quote replace, so a string default containing a newline emitted an unterminated literal that broke exec (SyntaxError). Use json.dumps(..., ensure_ascii=False): it escapes all control characters, stays double-quoted, leaves printable non-ASCII as-is (so existing snapshots are unchanged), and is always valid Python. Clears the unterminated-literal failures for the Cloth Dynamics and Hair Dynamics bundled assets (they now hit the separate menu/enum-default gap). * bundle: items= API and codegen emitter for CombineBundle/SeparateBundle CombineBundle and SeparateBundle had no way to author their named bundle items, so codegen emitted raw item_N kwargs the constructors reject. Both now take an items= dict: - CombineBundle(items={name: source}) links each source via the __extend__ virtual socket (Blender infers the item's type from the source) then renames the new item; a socket-type string declares an unlinked item. - SeparateBundle(bundle, items={name: "TYPE"}) declares each output by name and socket-type string, read back via .o[name]. Codegen emitters produce these forms (item inputs/outputs identified from the bundle_items collection). The bundle parts of the dynamics/hair assets now round-trip; those assets stay xfailed on unrelated gaps (menu defaults, multiple Group Input nodes). Closures (EvaluateClosure/ClosureZone) are more involved and remain in the backlog. * closure: input_items/output_items API and emitter for EvaluateClosure EvaluateClosure feeds values into a closure and reads results, but had no way to author its two item collections, so codegen emitted raw item_N kwargs the constructor rejects. It now takes input_items= and output_items=: - input_items={name: source} links each source via the input __extend__ socket (Blender infers the item type), then renames it — like CombineBundle. A type string declares an unlinked input item. - output_items={name: "TYPE"} declares each result by name and socket type, read back via .o[name] — like SeparateBundle. The codegen emitter produces this form. All three EvaluateClosure assets (Custom Effector, Custom Force, Displace Geometry) now round-trip the closure correctly; Displace Geometry is left blocked only on the separate multiple-Group-Input gap. Inline closure definition (ClosureZone via ClosureInput/Output) remains in the backlog. * tests: collapse multiple Group Input/Output nodes in round-trip comparison A tree may hold several Group Input nodes (editor convenience to shorten wires) that are functionally one interface. nodebpy authors a single logical interface, so codegen collapses them to one node — the correct, idiomatic behaviour, exactly like reroute collapsing. The round-trip _structure comparison now excludes NodeGroupInput/NodeGroupOutput from the node multiset; links still reference group sockets by name, so a genuinely missing interface socket is still caught. This, together with the bundle/closure emitters, moves 16 bundled assets to passing (_ASSET_ROUNDTRIP_OK 14 -> 30) and resolves most of the old structural-mismatch category (Box/Normal/Sphere Selection, Randomize Transforms, Smooth by Angle, Collider, Displace Geometry, ...). * codegen: defer menu interface defaults until after the body A menu interface input's valid values come from the MenuSwitch linked to it, so its default can only be set once the body has created that node. Setting it at interface-creation time raised 'enum "X" not found in ()'. Codegen now emits the menu input without its default and appends a deferred '<var>.default_value = "X"' statement after the body, via a new EmitContext.iface_deferred list threaded into _TreeEmission.deferred_lines and _assemble_tree_body. Removes the enum error from ~9 bundled assets (Array, Curve to Tube, Scatter on Surface, hair generators); those now reach the separate vector/scalar-default gap. * builder: vector interface dimensions (2D/4D) default handling The interface vector socket's default_value RNA is a fixed 3-float array regardless of dimensions, so a 2-element default (what a dimensions=2 socket reads back) could not be assigned, and the (0,0,0) param default failed the len==dimensions assert for dimensions != 3. tree.inputs/outputs.vector now defaults default_value=None -> (0.0,)*dimensions and pads/truncates to exactly 3 floats when assigning to the RNA. The dimensions=3 path is unchanged. Flips 5 bundled assets to passing (_ASSET_ROUNDTRIP_OK 30 -> 35): 3D to Screen Space, Screen to 3D Space, Project with Depth, Set Attachment Surface, Transform and Project. The separate operator-lifting issue (vector_const * scalar) still blocks Array/Combine Cylindrical. * codegen: refuse VectorMath lift when no operand is a linked vector Codegen lifted a VectorMath multiply to a Python operator like '(1.0, 0.0, 0.0) * r', but tuple * float_socket re-creates a scalar Math node whose float input rejects the tuple. A VectorMath only round-trips through the operator when at least one operand is a linked vector socket (so the operator dispatches to VectorMath); otherwise the lift now refuses and falls back to the g.VectorMath.* constructor. _lift_plan gains the incoming sockets' source types (_linked_src_types) to make this check. Flips Combine Cylindrical and Combine Spherical to passing (_ASSET_ROUNDTRIP_OK 35 -> 37). * docs: mark operator-lifting backlog item done * codegen: emit GeometryToInstance multi-input as positional args GeometryToInstance takes its multi-input geometry as *args (unlike JoinGeometry, whose geometry= iterable param absorbs the tuple), so the generic constructor's geometry=(...) kwarg was rejected with 'unexpected keyword argument'. A custom emitter now renders the multi-input geometry links as positional arguments in creation order: g.GeometryToInstance(a, b, c). Removes the error from Scatter on Surface / Curve to Tube / Instance on Elements (they now reach the socket-method/output-accessor gap). Locked in with a synthetic round-trip test. * codegen: refuse Math lift when both operands are integer sockets A ShaderNodeMath whose operands are both integer sockets was lifted to e.g. 'g.Index() / integer_math', but int_socket / int_socket re-creates a FunctionNodeIntegerMath (INT output), not a ShaderNodeMath. Downstream this produced an integer receiver where a float was expected (Array: 'IntegerSocket' has no attribute 'mix'). Generalised the operator-dispatch faithfulness check (was VectorMath-only) into _operator_dispatch_ok: a Math lift needs at least one float operand — a linked VALUE socket or an unlinked input (whose default renders as a float literal, forcing float dispatch). Otherwise it falls back to the g.Math.* constructor, which faithfully builds a ShaderNodeMath. * builder: resolve .o.<name> accessor for multi-word socket names The socket accessor normalised identifiers but not names, and denormalize_name('flip_and_cyclic') title-cases the connector -> 'Flip And Cyclic', which doesn't match the real socket name 'Flip and Cyclic'. So .o.flip_and_cyclic (which codegen derives from the socket name) failed to resolve a CaptureAttribute output named 'Flip and Cyclic'. SocketAccessor._index now also matches the key against normalised socket names (the inverse of how _output_expr builds the attribute), with the same ambiguity guard. Flips 5 hair assets to passing (_ASSET_ROUNDTRIP_OK 37 -> 42): Attachment Info, Curl/Duplicate/Roll/Rotate Hair Curves. * test: multi-word output accessor round-trip regression * codegen: force lifted operator's left operand to a socket A Python operator dispatches on its left operand, and nodebpy's operators only return a socket when the left operand is already a socket: e.g. g.Compare.float.equal(...) & g.Compare.float.equal(...) operates on two Compare *nodes* and returns a BooleanMath node, so a lifted result used as a socket method (boolean_math.switch.float(...)) raised 'BooleanMath' has no attribute 'switch'. _lift_expr now forces only the leading operand to a socket via socket_expr (a no-op when it is already a socket), so the lifted result is reliably a socket regardless of nodebpy's per-type operator return inconsistency. Right operands stay bare (socket * node is already a socket). One to_python snapshot updated (combine_xyz_1 -> combine_xyz_1.o.vector). Flips Shrinkwrap Hair Curves (_ASSET_ROUNDTRIP_OK 42 -> 43). * fix: matrix/color/rotation captures and boolean ~/^ socket operators Two builder fixes for socket-method round-trip faithfulness: - CaptureAttribute._type_map mapped VECTOR/RGBA/ROTATION/MATRIX to their data_type spellings (FLOAT_VECTOR/FLOAT_COLOR/QUATERNION/FLOAT4X4), but capture_items.new(socket_type=...) takes the socket-type spelling, so capturing those attribute types raised "error with keyword argument socket_type". Only VALUE differs (FLOAT), so the map is now {'VALUE': 'FLOAT'}; the item data_type still reads back correctly. - The boolean socket mixin overrode & and | to return .o.boolean sockets but missed ~ (__invert__) and ^ (__xor__), which fell through to the node-returning OperatorMixin. So ~socket returned a BooleanMath node and (~flag).switch.boolean(...) raised 'BooleanMath' has no attribute 'switch'. Added __invert__/__xor__ to the socket mixin so all four boolean operators return a BooleanSocket. * docs: socket-method faithfulness progress * builder: allow inactive sockets on group nodes; clearer inactive message A group input that is only used behind an internal switch polls as inactive until the group is evaluated — including the input being linked, which would itself activate the socket. tree.link's inactive-socket guard blocked the very link needed to round-trip the original tree, so group nodes (Geometry/Shader/Compositor) now join the allow-list. Also fixed the misleading guard message, which always named socket1 even when socket2 was the inactive one; it now reports the actual inactive socket, its direction, owning node, and the other end of the link. Flips 7 hair assets (_ASSET_ROUNDTRIP_OK 43 -> 50): Clump/Displace/Frizz/ Smooth/Straighten Hair Curves, Hair Curves Noise, Set Hair Curve Profile. * docs: inactive group-input socket fix; asset count 50 * test: round-trip key — always use variable-items prefix:name My earlier canonicalization stripped the items creation counter only when present, but the *first* item of a collection often has no counter at all (CaptureAttribute 'Attribute' vs 'Attribute_001'), so the same logical socket keyed inconsistently. _socket_key now always keys non-group sockets on the counter-stripped prefix plus the socket name. Items are order-independent and codegen never reorders regular multi-input sockets, so the multiset comparison stays faithful (the link's source side still disambiguates). * codegen: pin output accessor when bare reference would resolve elsewhere A bare node reference (g.MenuSwitch.integer(...)) resolves its output by best type-match at link time. A MenuSwitch fed into a boolean Switch would link its per-item boolean is_selected output instead of the main integer Output that was actually linked. _output_expr now forces the .o.<name> accessor when the linked output's type differs from the consumer's and the node has another output of the consumer's exact type (the real ambiguity); same-type first outputs (g.Cube().o.mesh -> geometry) stay bare, so there is no readability churn. Flips Generate and Interpolate Hair Curves (_ASSET_ROUNDTRIP_OK 50 -> 52). * docs: bundled-asset progress (52 passing) and remaining-11 analysis * codegen: identifier lookup for name clashes Node groups can have sockets with clashing names, so we check to see if the name appears more than once and if so we instead use their identifier which is always unique. The identifier isn't usually descriptive so it's best to avoid if possible, but works as a fallback. * chore: remove accidentally committed temp probe file * codegen: refuse Math lift when its dispatcher operand is color/vector A ShaderNodeMath whose first input is fed by a color/vector source (the original treats it as a scalar via implicit conversion) was lifted to 'color - x', but color_socket - x dispatches to VectorMath, not Math. The operator dispatches on its left/first linked operand, so _operator_dispatch_ok now refuses the Math lift when that dispatcher's source type is RGBA or VECTOR, falling back to the g.Math.* constructor which faithfully builds a ShaderNodeMath. Flips Scatter on Surface (_ASSET_ROUNDTRIP_OK 53 -> 54). * docs: bundled-asset progress (54 passing); Array/Scatter/dup-names resolved * codegen: coerce float defaults to int for INT sockets (55 passing) g.AccumulateField(data_type="INT") crashed: its value=1.0 constructor default was assigned to the integer Value socket. _set_input_default_value now casts float→int for INT sockets, mirroring the VECTOR broadcast case. Flips Create Guide Index Map. * codegen: author CaptureAttribute built-in Selection input (56 passing) Blender added a fixed Selection input/output to geometry CaptureAttribute (class signature already updated with selection=). The _ItemsNodeSpec named only Geometry as fixed, so a linked Selection tripped the bail-to-generic guard and an item input was mis-emitted as value=. Name Selection in the spec's fixed tuple. Flips Attach Hair Curves to Surface. * codegen: only treat keyword-only params that are RNA props as settings (57 passing) _non_default_props captured every keyword-only constructor param via getattr(node, name); FloatCurve's nodebpy-only `items` (curve points) is not a bpy property, so getattr returned the bpy_struct.items dict method and codegen emitted `items=<built-in method…>` (SyntaxError). Require keyword-only params to be real RNA properties, mirroring the existing positional-or-keyword rule. Flips Braid Hair Curves. * codegen: refuse VectorMath SCALE lift when Scale operand isn't scalar (58 passing) A SCALE node lifted to `vec * x`; but vec * x only re-creates SCALE when x is a scalar (VALUE/INT) or unlinked float literal. A linked BOOLEAN/VECTOR Scale operand dispatches to MULTIPLY on rebuild (see _dispatch_vector_math), swapping the op and the target socket. _operator_dispatch_ok now refuses the SCALE lift in that case, falling back to g.VectorMath.scale(). Flips Trim Hair Curves. * items: resolve item sockets by identifier when name clashes (59 passing) A CaptureAttribute item named "Selection" collides with the node's new built-in Selection socket, so _item_socket and the link target in _establish_links resolved by name to the wrong socket. _item_socket now falls back to positional resolution (items are the trailing N sockets) on a name clash, and _add_inputs keys returned sockets by identifier so links are established unambiguously. Flips Curve to Tube; also fixes Instance on Elements in isolation (still xfails in-suite on datablock contamination). * items: exact identifier match wins in _find_socket_from_name (60 passing) A CaptureAttribute with items named "Normal" and "Value" gives the Normal item the socket identifier "Value" (capture_items number sockets from the "Value" base), colliding with the "Value" item's name. _add_inputs keys links by identifier, but _find_socket_from_name resolved name-first, so the "Value" key hijacked the socket named "Value" and the Normal item lost its link (one of two duplicate Normal captures silently dropped). Return an exact identifier match before the name passes, matching SocketAccessor's identifier-first strategy. Flips Instance on Elements. * codegen: ClosureZone builder API + zone emitter (61 passing) A NodeClosureInput/NodeClosureOutput pair defines a closure's signature and body; items live on the output node (input_items drive the input node's outputs, output_items drive the output node's inputs), so the plain-constructor path can't express it. Add a builder API — cz.input_item(name, type), cz.output_item(name, type), cz.closure — and a zone emitter modelled on Repeat/Simulation: _emit_closure_input declares the zone and per-item lines and dissolves the input node into input-item reads; the output node reuses _emit_zone_output. Flips Custom Force. * codegen: resolve duplicate-named group inputs by name, not identifier A recursive group with duplicate-named interface inputs (two "Geometry Space": menu + matrix) was keyed in the GroupCall by raw identifier (Socket_6/8). But identifiers are an authoring-history artifact that _build_group reassigns on rebuild, so the call linked the wrong-typed sockets (MENU vs MATRIX error). Route duplicate-named inputs through a new `_named_links` path: (name, value) pairs (emitted in interface order) resolved by name + a type match at link time, falling back to order. _establish_links is refactored to share a single per-input applier (_apply_input) that accepts a name or a resolved socket. Removes the unstable identifier dependence; names round-trip regardless of creation order. Unblocks the MENU/MATRIX error in Hair/Cloth Dynamics (which now hit a separate JoinBundle multi-input gap). * codegen: multi-input iterable linking + subscript output accessor (63/63 assets) The last two "segfault" assets (Hair/Cloth Dynamics) were a chain of plain exceptions, found by tracing the exec in an isolated subprocess: - Multi-input socket fed an iterable (JoinBundle(bundle=(…))): auto-generated multi-input nodes take a single socket param, so the tuple hit the default-value path. _apply_input now links each source into an is_multi_input socket (reversed, as JoinGeometry does); vector/colour default tuples are not multi-input and fall through unchanged. - Output name that isn't a valid identifier (.o.physics_(experimental) for "Physics (Experimental)"): new Subscript IR; _output_expr emits value.o["Physics (Experimental)"] when the normalised name fails str.isidentifier(). All 63 bundled geometry assets now round-trip; full suite 732 passed, 0 xfailed. * codegen: also tround-trip test on shader and compositor assets Setting of menu defaults additionally delayed until end of node tree construction. * round-trip tests for Molecular Nodes assets * skip MN test if not installed * fix testing for MN existing * fix color support for evaluateAtIndex * codegen: faithful best-match simulation + bpy import for datablock defaults Two MN-asset fixes: - _bare_resolves_elsewhere now simulates _best_match's actual ranking (by SOCKET_COMPATIBILITY order, considering every compatible output of a fresh node) instead of only checking for an exact-type match elsewhere. A bare EdgeVertices reference into a float input resolved to "Position 1" (VECTOR outranks INT for a VALUE consumer) instead of the linked "Vertex Index 1"; the accessor is now forced. Visibility is ignored because the rebuilt node is fresh (an author-hidden output like ImageTexture "Alpha" reappears). - A Material/Object/Image socket default renders via repr() as bpy.data.<collection>['name']; the module now emits a bare `import bpy` when any such reference is present. Flips Edge Info; keeps all bundled assets green. * codegen: Viewer extend-socket linking + SetHandleType mode emitter - Viewer data inputs are created on link via a virtual __extend__ socket, so they can't be constructor kwargs. _find_best_socket_pair gains a generic __extend__ fallback (so `geometry >> g.Viewer()` works), and a custom emitter emits `viewer = g.Viewer(...)` + `expr >> viewer` per linked input. - SetHandleType's left/right params map to the `mode` ENUM_FLAG set but aren't RNA-named, so _non_default_props can't see them. A custom emitter appends left/right from node.mode. (Note: nodebpy's SetHandleType() defaults mode to set(), unlike Blender's native {'LEFT','RIGHT'} — a latent library default discrepancy, left as-is.) Flips MN assets Atoms to CA Curves and Edge Info. * codegen: reference colliding variable-items outputs by stable index A captured item named "Selection" collides with CaptureAttribute's built-in "Selection" output, so _output_expr fell back to the item's identifier ("Attribute_2") — but that counter is creation-order-dependent and the rebuilt node reassigns it (the original built items as Position/Normal/Selection, the rebuild as Selection/Position/Normal), linking the wrong item. The output *position* follows item-collection order, which round-trips, so reference a duplicate-named variable-items output by its index (value.o[2]) instead. Fixed-output nodes (Mix's "Result" sockets) keep their stable identifier. Flips the last MN asset (Curve Visualize); full suite 873 passed. * fix: SetHandleType default mode matches Blender native ({'LEFT','RIGHT'}) g.SetHandleType() defaulted left=False/right=False (mode=set()), so a fresh node set no handle types — diverging from Blender's native {'LEFT','RIGHT'} default and from the sibling HandleTypeSelection (which already defaults both True). Default both to True. Also simplify the left/right setters in both classes from the duplicated 4-case match statements to a single set union/difference, add a `mode` property to SetHandleType for parity with HandleTypeSelection, and update the codegen emitter to emit left=/right= only when they differ from the new both-True default. * refactor: extract Bézier handle left/right/mode into _HandleModeMixin SetHandleType and HandleTypeSelection carried byte-identical left/right/mode properties over the `mode` ENUM_FLAG set. Factor them into a shared _HandleModeMixin; both classes now inherit it and keep only their own constructor and handle_type surface. * codegen: optional snapshot_positions to export authored node locations to_python(..., snapshot_positions=True) builds the tree with arrange=None (no auto-layout) and appends a {name: (x, y)} dict + loop restoring each node's authored location. Keyed by node name and applied via tree.tree.nodes.get(name), so nodes a rebuild drops (reroutes) or renames (duplicate-type nodes created in a different order — the known caveat) are skipped rather than erroring. * codegen: snapshot_positions covers nested groups; apply via TreeBuilder property - Nested group classes now restore positions too: each generated _build_group calls tree.disable_arrange() and assigns tree.node_positions, so locations round-trip at every depth (not just the top-level tree). - The apply-loop moved out of the generated script into TreeBuilder: a new read/write `node_positions` property ({name: (x, y)}) applies locations by name (skipping nodes a rebuild dropped). Generated code is now just `tree.node_positions = {...}`. Added `disable_arrange()` too. - Tests: round-trip tests for top-level and nested-group snapshots, plus syrupy snapshot tests showing the generated output. Switched test_field_to_grid_capture_typed to snapshot to_mermaid() instead of the TreeBuilder object (it shouldn't have snapshotted the builder). * codegen: keep_reroutes option to preserve reroute nodes to_python(..., keep_reroutes=True) leaves reroute chains intact instead of collapsing them, emitting each as g.Reroute(input=<socket>). _effective_links / _ordering_edges / _topo_sort gained a keep_reroutes pass-through, and reroutes are no longer filtered from emission. A NodeReroute emitter forces the source to a socket expression (the runtime links it directly — tree.link skips type-compat for reroutes — instead of the colour-typed input= best-matching and failing). keep_reroutes implies arrange=None (top-level and nested groups), since the sugiyama layout dissolves reroutes; combine with snapshot_positions to reproduce the original wire routing. Tests assert on generated source rather than exec-rebuilding: linking a fresh g.Reroute mutates adaptive sockets, which intermittently segfaults in-process Blender under full-suite memory pressure. The rebuilt output is locked by a syrupy snapshot. * builder: NodeGroupBuilder.create_group() classmethod create_group() builds and returns a custom group's node tree without an active TreeBuilder context (it opens its own), reusing an existing tree of the same name. A group can now be pre-built and assigned directly to a node's node_tree instead of being constructed inside a tree. Each Custom*Group declares its inner-tree bl_idname via `_tree_idname`; the shared classmethod replaces the three duplicated `_get_or_create_group` instance methods (and the now-unused `_get_or_create_tree`). The instantiation path (`_setup_node_group`) just calls `create_group()`. * codegen: top_level='class' option to emit the working tree as a group class to_python(tree, top_level="class") renders the top-level tree as a Custom*Group subclass too — so every node group, including the one being exported, becomes a class buildable via ClassName.create_group(). Clean for archiving a set of node groups as plain reusable Python. Reuses the existing group-class machinery (register / _render_group_class), so snapshot_positions and keep_reroutes work inside the class body; TreeBuilder isn't imported when unused. Default stays "with" (the executable `with TreeBuilder(...) as tree:` form), so existing output and the round-trip test infrastructure are unchanged. * codegen: emit nested frames as nested with-blocks Frames inside other frames were dropped (_frame_key bailed when a frame had a parent frame), and pure-container frames that hold only sub-frames never emitted at all — Hair Dynamics recreated only 3 of its 17 frames. _frame_order now models each node's full frame chain (outermost first) and orders nodes hierarchically: at every nesting level the frame clusters are cluster-topologically sorted, a frame on a dependency cycle is dropped one level (frameless at worst), and container frames appear via their descendants' paths. The body assembly tracks an open frame-path stack, opening/closing nested `with g.Frame():` blocks as the path changes. Inlining and line-width budgets account for the nesting depth. Hair Dynamics now round-trips all 17 frames with matching parent hierarchy and node-to-frame assignment. * docs: document to_python options and update changelog for the codegen branch - nodes-to-code.qmd: snapshot_positions and keep_reroutes tuning subsections, a Frames section (incl. nested frames), an "Archiving as Classes" section for top_level="class" + create_group(), and a corrected round-tripping note. - changelog.qmd: headline "Nodes to code (to_python)" enhancement with its options, plus create_group(), node_positions/disable_arrange(), the bundle/ closure item APIs, colour field-evaluation methods, related fixes, and the SetHandleType default-mode breaking change. * codegen: optional ruff formatting of to_python() output to_python(..., format=True) (the new default) runs the generated source through `ruff format` when the optional `ruff` package is installed, tidying lines the generator leaves unwrapped. Invoked via the binary bundled with the ruff Python package (ruff.find_ruff_bin), so it needs no `ruff` CLI on PATH and is a no-op when ruff isn't installed. Added a `format` optional-dependency extra (`pip install nodebpy[format]`). Tests: snapshot assertions and the round-trip helper pin format=False so they keep testing the generator's deterministic output (and avoid a ruff subprocess per round-trip); a dedicated test covers the format=True path. Docs/changelog updated. * test coverage and pytest deprecation * codegen for sign() and other float / int socket methods * list socket methods in codegen * test: cover menu-default deferral and normalized-name ambiguity Add tests for the normalized-name ambiguity RuntimeError in SocketAccessor and the MENU-typed MenuSwitch deferral of empty string item defaults. Mark the unreproducible Blender enum-refresh except branch with pragma: no cover.
1 parent 0cc0c89 commit 583a5d9

54 files changed

Lines changed: 10466 additions & 1093 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

ITEMS_API_PLAN.md

Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,231 @@
1+
# Items API Unification Plan
2+
3+
Design notes for cleaning up zones, capture, and all variable-items nodes —
4+
**do this before building codegen for zones** (see PLAN.md). The goal is one
5+
canonical, code-generatable authoring form, one shared mixin for item
6+
machinery, and handle objects that name socket *roles* instead of plumbing.
7+
8+
## Why now
9+
10+
Codegen (Stage 8) needs a canonical form to emit for zones. The graph state
11+
is fully readable, but several things cannot be faithfully *re-authored*
12+
through the current API (no explicit item names, no unlinked item
13+
declaration). Fixing the API first means codegen targets the good form
14+
instead of baking in today's quirks.
15+
16+
## Current state inventory
17+
18+
### Item-bearing nodes and their machinery
19+
20+
| Node | Mixin? | bpy collection | `_add_socket` style |
21+
|---|---|---|---|
22+
| `Bake` (manual.py ~1053) | DynamicInputsMixin | `bake_items` | `.new(socket_type, name)` |
23+
| `FormatString` (~1347) | DynamicInputsMixin | `format_items` | `.new(socket_type, name)` |
24+
| `CaptureAttribute` (~2114) | DynamicInputsMixin | `capture_items` | `.new(socket_type, name)` |
25+
| `FieldToList` (~2222) | DynamicInputsMixin | `list_items` | **`_add_socket` is a `pass` stub**; real work in a parallel `_new_item` + per-dtype methods |
26+
| `FieldToGrid` (~2355) | DynamicInputsMixin | grid items | `.new(...)` |
27+
| `SimulationZone` in/out (zone.py) | via `BaseZone` | `state_items` (shared, 4 sockets/item) | `items.new(type, name)` via abstract `items` property |
28+
| `RepeatZone` in/out | via `BaseZone` | `repeat_items` (shared) | same |
29+
| `ForEachGeometryElementZone` | via `BaseZone` | **three** collections: `input_items`, `main_items`, `generation_items` | `capture_generated` temporarily monkeypatches `_socket_data_types` + `_add_socket` (zone.py:436–443) |
30+
| `IndexSwitch` (~1747) | **no mixin** | `index_switch_items` | `.new()` (unnamed), own `_create_socket`/`_link_args` |
31+
| `MenuSwitch` (~1982) | **no mixin** | `enum_items` | `.new(name)`, own `_create_socket`/`_link_args`, plus `is_selected` |
32+
| `ClosureZone` / `EvaluateClosure` | no | `input_items`/`output_items` | bespoke `link()` + `sync_signature` |
33+
| `JoinGeometry` / `StringJoin` | no | multi-input socket (not items) | n/a — already handled by codegen tuples |
34+
35+
### DynamicInputsMixin contract today (builder/node.py:208)
36+
37+
`_socket_data_types` tuple + `_type_map` dict + abstract `_add_socket(name,
38+
type)` + `_add_inputs(*args, **kwargs)` (auto-names positional args after
39+
their source socket) + a `_find_best_socket_pair` override that makes `>>`
40+
*implicitly create a new item*.
41+
42+
### Verb/signature drift
43+
44+
- `CaptureAttribute.capture(value) -> SocketLinker`
45+
- `BaseZone.capture(value, domain="POINT")`**`domain` is dead, never used**
46+
(zone.py:59–65)
47+
- `ForEachOutput.capture_generated(value)`**lacks** `domain`, the one place
48+
it matters (generation items have per-item domain; the microscopy usecase
49+
pokes `generation_items[0].domain` via raw bpy)
50+
- `FieldToList.capture(fields: dict) -> list[SocketLinker]` — different
51+
signature entirely
52+
- Constructor kwarg names: `items=` (CaptureAttribute, FormatString),
53+
`fields=` (FieldToList), `*args/**kwargs` (Bake), positional dict
54+
(MenuSwitch), `items=iterable` (IndexSwitch)
55+
- Three implementations of "return the just-added socket":
56+
`outputs[-2]` (BaseZone), index-of-`__extend__` (ForEachInput),
57+
`_latest(suffix)` (ForEachOutput)
58+
59+
### Zone wrapper inconsistencies
60+
61+
- `SimulationZone`/`ForEachGeometryElementZone` unpack via `__getitem__`
62+
(2-tuple); `RepeatZone` instead has `__iter__`/`__next__` yielding a
63+
3-tuple inside a faux one-pass `for` loop — `input, output = repeat_zone`
64+
does not work.
65+
- All zone wrappers use mutable default args (`items: dict = {}`).
66+
- Zone wrappers are plain objects (not BaseNode) — fine, keep that.
67+
68+
## Target design
69+
70+
### 1. One `ItemsMixin` replacing all the bespoke machinery
71+
72+
A single mixin parameterised by declarative class attributes instead of
73+
per-class method overrides:
74+
75+
```python
76+
class ItemsMixin:
77+
_items_collection: str # "capture_items", "bake_items", "repeat_items", …
78+
_socket_data_types: tuple[str, ...]
79+
_type_map: dict[str, str] = {}
80+
81+
@property
82+
def _items_node(self) -> bpy.types.Node: # override for zone inputs → paired_output
83+
return self.node
84+
85+
def _items(self): # the bpy collection
86+
return getattr(self._items_node, self._items_collection)
87+
88+
def _new_item(self, name, type) -> Item: ... # normalises .new() signatures
89+
def _item_socket(self, item, *, output: bool): ... # one "find the socket" impl
90+
def add_items(self, items: Mapping[str, InputLinkable | None]) -> dict[str, Item]
91+
def capture(self, value, *, name: str | None = None) -> Item | Socket
92+
```
93+
94+
- `.new()` signature differences (`(type, name)` vs `()` vs `(name)`) are
95+
normalised inside `_new_item` via a small adapter, not subclass overrides.
96+
- ForEach's dual collections become **two mixin instances' worth of state on
97+
one class** (e.g. a second descriptor `_generation = ItemCollection(
98+
"generation_items", ...)`) — kills the monkeypatch.
99+
- The one "socket for item" implementation replaces `outputs[-2]` /
100+
`__extend__`-index / `_latest`.
101+
- Keep the `>>`-implicit-add behaviour (it's used and liked), implemented
102+
once in the mixin.
103+
- `IndexSwitch` (unnamed positional items) and `MenuSwitch` join the mixin
104+
with `_new_item` adapters; their positional/iterable constructor sugar
105+
stays.
106+
107+
### 2. Item handles (the core new concept)
108+
109+
`add_items`/`capture`/`zone.item()` return handle objects that name roles,
110+
not plumbing:
111+
112+
```python
113+
class Item: # single-node items (CaptureAttribute, Bake, FormatString, …)
114+
name, socket_type
115+
input: Socket # the node's input socket for this item
116+
output: Socket # the node's output socket for this item
117+
118+
class ZoneItem(Item): # repeat/simulation state items (4 sockets per item)
119+
initial: Socket # input-node input — set the starting value
120+
current: Socket # input-node output — read inside the body
121+
next: Socket # output-node input — write the per-iteration result
122+
result: Socket # output-node output — read after the zone
123+
```
124+
125+
Canonical zone authoring becomes:
126+
127+
```python
128+
zone = g.RepeatZone(100)
129+
value = zone.item("value", initial=1.0)
130+
(value.current + 1 / factorial) >> value.next
131+
value.result >> g.ValueToString.float(decimals=10) >> ...
132+
```
133+
134+
Existing spellings (`zone.input.o.value`, constructor `items=` dict,
135+
`capture`) remain as sugar over handles. ForEach gets
136+
`zone.item(...)` (input items), `zone.main_item(...)`, and
137+
`zone.generated_item(name, value, domain=...)`.
138+
139+
### 3. Unified verbs and signatures
140+
141+
- `capture(value, *, name: str | None = None)` everywhere; auto-name from the
142+
source socket only when `name` is omitted. **Explicit names are the single
143+
most important change for codegen round-trip.**
144+
- `capture_generated(value, *, name=None, domain="POINT")` — gains the
145+
`domain` it needs; `BaseZone.capture` loses the dead one.
146+
- Constructor kwarg is `items=` on every items node (keep `fields=`
147+
etc. as deprecated aliases initially).
148+
- `items=` dict values may be `None` → declare the item *unlinked* (needed
149+
to round-trip items whose initial value is the socket default). Type then
150+
comes from… an explicit type when value is None: allow
151+
`items={"value": Float}` socket-class or `"FLOAT"` string. (Pick one;
152+
string matches `_socket_data_types`.)
153+
- All zone wrappers: `__getitem__` 0/1 + 2-tuple `__iter__`; keep
154+
`.input`/`.output`/`.iteration`/`.delta_time`/`.index` properties.
155+
Deprecate (or document as sugar) the 3-tuple `for i, input, output in
156+
RepeatZone(...)` faux-loop.
157+
- Replace mutable default args (`items={}``items=None`).
158+
159+
### 4. Codegen follow-up unlocked by this (Stage 8)
160+
161+
- Canonical zone form to emit: `zone = g.RepeatZone(n, items={...})` /
162+
handle form; body refs `zone.input.o.x` or `item.current`; result links as
163+
`expr >> item.next` statements at the output node's topological position.
164+
- Emission needs: paired-node recognition (`node.paired_output`); zone
165+
declaration emitted at input-node position; output-node incoming links as
166+
deferred `>>` statements; `var_map` entries for both bpy nodes →
167+
`Attr(zone_ref, "input"/"output")`; a `DictExpr` IR node (also unlocks
168+
`FormatString.format({...})`, `MenuSwitch`, `IndexSwitch`, Bake,
169+
CaptureAttribute items).
170+
- `CaptureAttribute` needs no API change for codegen — `g.CaptureAttribute
171+
.point(geo, items={"Position": g.Position()})` is already canonical.
172+
173+
## Suggested implementation order
174+
175+
1. **`ItemsMixin`** in builder/ — port `CaptureAttribute`, `Bake`,
176+
`FormatString` first (simple, one collection each). Verify: existing
177+
tests pass unchanged (the public constructors don't change). ✅ DONE
178+
2. Port `FieldToList`/`FieldToGrid` (removes the `_add_socket` stub) and
179+
`IndexSwitch`/`MenuSwitch` (adapters for their `.new()` signatures).
180+
DONE
181+
3. Port zones onto the mixin; fix `capture` signatures (`name=`, domain
182+
move); replace the ForEach monkeypatch with a second collection
183+
descriptor; unify wrapper unpacking; mutable-default cleanup. ✅ DONE
184+
Implementation notes (deviations from the sketch above):
185+
- Zone item sockets are found by **identifier prefix + collection
186+
index** (`Item_`/`Input_`/`Main_`/`Generation_` via
187+
`_socket_for_item` in zone.py) — names are not unique across a zone
188+
node's fixed sockets and item collections (e.g. a main item and a
189+
generation item can both be called "Position").
190+
- The ForEach second collection is handled by an explicit
191+
`capture_generated` built on `ItemsMixin._resolve_capture` rather
192+
than an `ItemCollection` descriptor — only one node needed it.
193+
- `ItemsMixin.add_items(items) -> dict[str, Socket]` is the dict-based
194+
verb (replaces the `FieldToList`/`FieldToGrid` dict `capture`); step
195+
4 upgrades its return value to `Item` handles.
196+
- Constructor kwarg is now `items=` everywhere (incl. `Bake`);
197+
`FieldToList(fields=...)` kept as a `DeprecationWarning` alias.
198+
4. **Item / ZoneItem handles** + `zone.item()` and `items={name: None/type}`
199+
declaration support. ✅ DONE
200+
Implementation notes:
201+
- `Item` (builder/items.py) stores the item's **collection index**, not
202+
the bpy item — bpy collection item references are invalidated when
203+
the collection grows (segfault). `Item.socket_type` falls back to
204+
`data_type` (capture_items, grid_items spelling).
205+
- `ItemsMixin.add_item(name, value=None, *, type=)` is the single-item
206+
verb; `add_items` now returns `dict[str, Item]`.
207+
- Unlinked declaration uses **socket-type strings as dict values**
208+
(`items={"geo": "GEOMETRY"}`), validated against
209+
`_socket_data_types`/`_type_map` via `_declared_item_type` — a hook
210+
on `DynamicInputsMixin._add_inputs`, so every `items=` constructor
211+
supports it. Bare `None` values are rejected (no type to infer).
212+
- `zone.item(name, initial=...)` lives on `_StateZone`
213+
(Simulation/Repeat wrappers) and returns `ZoneItem`
214+
(initial/current/next/result). `initial` accepts linkables, plain
215+
defaults (python-type inference), or type strings; the constructor
216+
`items=` dict is now sugar over `zone.item()`.
217+
- ForEach wrapper: `zone.item()` / `zone.main_item()` /
218+
`zone.generated_item(name, value, type=, domain=)`; generation
219+
handles resolve sockets with the `Generation_` prefix
220+
(`_GenerationItem`); `capture_generated` delegates to
221+
`add_generated_item`.
222+
5. Then Stage 8 codegen: zone emitters targeting the handle/canonical form,
223+
plus `DictExpr`. ✅ DONE — see PLAN.md Stage 8 for the emitter design
224+
and known gaps. Along the way: `ItemsMixin.add_item` /
225+
`add_generated_item` accept plain default values,
226+
`ForEachGeometryElementZone.generation` exposes the default generation
227+
item handle, and `RepeatZone` pairs before linking `Iterations`
228+
(sockets on unpaired zone nodes are inactive).
229+
230+
Each step: `uv run pytest` green before moving on; the structural round-trip
231+
harness in tests/test_codegen.py is the safety net for codegen stages.

0 commit comments

Comments
 (0)