fix(export): pad the attributes a newer schema appended in the Rust converter (#3271) (#3273)

`convertStepLine` has padded the trailing optional attributes newer schemas
ADD since #1416 -- `PredefinedType` on IfcWall / IfcBeam / IfcOpeningElement,
IfcMaterial's Description and Category, 61 more. Its Rust port never got the
fix, and the Rust port is what `exportStep` runs, so
`ifc-lite export --format step --schema IFC4` on an IFC2X3 source wrote
entities a positional attribute short -- invalid under the FILE_SCHEMA it
declares, which is what #1416 was reported for.

The tables were not the problem: the three entity-rename maps and the IFC2X3
attribute-count map diff byte-for-byte against their TypeScript twins over all
3 + 23 + 36 + 30 entries. The missing piece is the algorithm.

Padding is only safe where the source schema's positional attribute NAME list
is a strict PREFIX of the target's. IfcMaterialProperties goes from [Material]
to [Name, Description, Properties, Material]; a trailing `$` there shoves the
material reference into the Name slot. Rust has no per-schema attribute table
to test that against -- the generated `IfcType` is IFC4X3 alone -- so
`schema_pad` carries the strict-prefix pairs derived from the same generated
buildingSMART tables the TypeScript side reads at run time.

Neither half is trusted to have stayed in step with the other. Both now run
their own converter over one shared fixture
(`rust/export/tests/fixtures/schema_upconvert_sweep.json`), following the
`rooted_type_parity.rs` precedent. WHICH rows that fixture must hold is
re-derived from `schema_pad`'s own tables rather than floored by a count: a
count survives dropping the rows that matter, and stays silent on a type added
to the tables with no row to pin it. A negative control covers the direction
the positive rows cannot -- deleting the strict-prefix restriction and padding
every short line passes all 134 padded rows and fails
`a_reordered_attribute_list_is_never_padded`.

`schema_convert.rs`'s inline test module moves to `schema_convert_tests.rs`
via the `#[path]` convention (`schema_helpers.rs`, `georef.rs`) so the module
stays under the 400-line ratchet.

Refs #3271

Author: BIMvoice

.changeset/rust-upconvert-attribute-padding.md

@@ -0,0 +1,21 @@
+---
+"@ifc-lite/wasm": patch
+---
+
+Pad the attributes a newer schema appended when the Rust STEP converter upgrades a file.
+
+`convertStepLine` in `packages/export` has padded the trailing optional attributes newer schemas ADD since #1416 — `PredefinedType` on `IfcWall` / `IfcBeam` / `IfcOpeningElement`, `IfcMaterial`'s `Description` and `Category`, and 61 more. Its Rust port never got the fix, and the Rust port is what `exportStep` runs, so `ifc-lite export --format step --schema IFC4` on an IFC2X3 source wrote entities one or more positional attributes short. That is an invalid IFC4 file, and strict readers reject it. Verbatim, before:
+
+```
+#1=IFCWALL('0aBcDeFgHiJkLmNoPqRsTu',$,'W1',$,$,$,$,'tag');
+```
+
+and after:
+
+```
+#1=IFCWALL('0aBcDeFgHiJkLmNoPqRsTu',$,'W1',$,$,$,$,'tag',$);
+```
+
+Padding applies only where the source schema's positional attribute NAME list is a strict PREFIX of the target's — the same restriction the TypeScript half enforces at run time. Entities that reorder or insert mid-list (`IfcMaterialProperties` goes from `[Material]` to `[Name, Description, Properties, Material]`) are left untouched, because a trailing `$` there would shove existing values into the wrong and type-invalid slots.
+
+The two implementations are now pinned to one shared fixture, `rust/export/tests/fixtures/schema_upconvert_sweep.json`, whose rows are derived from the generated buildingSMART attribute tables and which NAMES every padded type rather than counting them — a count floor stays silent exactly when a row is dropped.

packages/export/src/schema-upconvert-sweep.parity.test.ts

@@ -0,0 +1,70 @@
+/* This Source Code Form is subject to the terms of the Mozilla Public
+ * License, v. 2.0. If a copy of the MPL was not distributed with this
+ * file, You can obtain one at https://mozilla.org/MPL/2.0/. */
+
+/**
+ * TypeScript half of the schema-upconversion padding parity pin.
+ *
+ * `convertStepLine` pads the trailing optional attributes a newer target
+ * schema APPENDED (#1416). The Rust port in
+ * `rust/export/src/schema_convert.rs` did not — and it is the Rust one that
+ * `ifc-lite export --format step --schema IFC4` runs, through `exportStep` in
+ * the wasm bindings — so the two halves emitted different files for the same
+ * conversion for two months. Both are now held to ONE fixture; the Rust half
+ * is `rust/export/tests/schema_upconvert_parity.rs`. Follows the
+ * `rooted-type-sweep.parity.test.ts` precedent.
+ *
+ * The fixture's padded rows are DERIVED from
+ * `packages/data/src/ifc-schema/generated/entities-*.ts` (the types whose
+ * source attribute NAME list is a strict prefix of the target's — the same
+ * relation `isStrictAttrPrefix` tests at run time), so this side is not
+ * circular: it pins the answer that generated data implies, and a regenerated
+ * schema that changes a count fails here rather than silently changing what
+ * `ifc-lite` writes.
+ *
+ * WHICH rows the fixture must contain is pinned on the Rust side only
+ * (`fixture_names_every_padded_type`), which re-derives the universe from its
+ * own tables. The count check below is a smoke test, not the coverage gate.
+ */
+import { describe, it, expect } from 'vitest';
+import { readFileSync } from 'node:fs';
+import { fileURLToPath } from 'node:url';
+import { convertStepLine, type IfcSchemaVersion } from './schema-converter.js';
+
+interface SweepCase {
+  why: string;
+  from: IfcSchemaVersion;
+  to: IfcSchemaVersion;
+  line: string;
+  expect: string;
+}
+
+// The fixture lives in the Rust crate so `include_str!` can reach it; this side
+// resolves it relative to the source file. NOT guarded by `existsSync`: a
+// missing fixture means the pin is not being enforced, which must fail loudly.
+const fixturePath = fileURLToPath(
+  new URL('../../../rust/export/tests/fixtures/schema_upconvert_sweep.json', import.meta.url),
+);
+const fixture: { cases: SweepCase[] } = JSON.parse(readFileSync(fixturePath, 'utf8'));
+
+describe('convertStepLine matches the shared cross-language upconversion sweep', () => {
+  it('the fixture is not empty or trivial', () => {
+    expect(fixture.cases.length).toBeGreaterThan(100);
+  });
+
+  it('every case agrees with the Rust converter', () => {
+    const mismatches = fixture.cases
+      .map((c) => ({ c, got: convertStepLine(c.line, c.from, c.to) }))
+      .filter(({ c, got }) => got !== c.expect)
+      .map(({ c, got }) => `${c.from}->${c.to} [${c.why}]\n  in     ${c.line}\n  expect ${c.expect}\n  got    ${got}`);
+    expect(mismatches).toEqual([]);
+  });
+
+  it('a reordered attribute list is never padded (negative control)', () => {
+    // IFC2X3 IfcMaterialProperties is [Material]; IFC4 is
+    // [Name, Description, Properties, Material]. Padding would read #8 as Name.
+    expect(convertStepLine('#7=IFCMATERIALPROPERTIES(#8);', 'IFC2X3', 'IFC4')).toBe(
+      '#7=IFCMATERIALPROPERTIES(#8);',
+    );
+  });
+});

rust/export/src/lib.rs

@@ -39,6 +39,8 @@ mod parquet_bos;
 mod rooms;
 pub mod rooted_type;
 mod schema_convert;
+mod schema_pad;
+pub use schema_pad::padded_type_universe;
 mod shades;
 mod step;
 mod step_cow;

rust/export/src/schema_convert.rs

@@ -2,7 +2,8 @@
 //! IFC **schema conversion** for STEP export (Phase 2 P2). Ports
 //! `packages/export/src/schema-converter.ts`: entity-type renames between
 //! IFC2X3 / IFC4 / IFC4X3 / IFC5 (with multi-step chaining), IFC2X3 attribute-count
-//! trimming on downgrade, and a proxy fallback for types with no target representation.
+//! trimming on downgrade, padding of the attributes a newer target schema appended
+//! (`schema_pad`), and a proxy fallback for types with no target representation.
 
 /// Canonicalize a FILE_SCHEMA label to one of the four families we convert between.
 fn canon(s: &str) -> &'static str {
@@ -232,7 +233,7 @@ pub fn convert_step_line(line: &str, from: &str, to: &str, express_id: u32) -> S
         );
     }
 
-    let final_attrs = if cto == "IFC2X3" {
+    let mut final_attrs = if cto == "IFC2X3" {
         match ifc2x3_attr_count(&new_type) {
             Some(max) => trim_attributes(attrs, max),
             None => attrs.to_string(),
@@ -241,6 +242,24 @@ pub fn convert_step_line(line: &str, from: &str, to: &str, express_id: u32) -> S
         attrs.to_string()
     };
 
+    // Pad the trailing optional attributes a newer target schema APPENDED
+    // (#1416). Keyed on the ORIGINAL type, because the source schema is what
+    // decides how many attributes the line already has; the table's target
+    // count already accounts for the rename. Only types whose source
+    // attribute NAME list is a strict prefix of the target's are in it, so
+    // this can never shift a value into a reordered slot -- see
+    // `schema_pad`. An attribute-less line is left alone rather than
+    // fabricated from nothing, matching the TypeScript twin's
+    // `currentCount > 0` guard.
+    if let Some(target_count) = crate::schema_pad::padded_attr_count(cfrom, cto, &entity_type) {
+        let current = crate::schema_pad::count_top_level_attributes(&final_attrs);
+        if current > 0 && current < target_count {
+            for _ in current..target_count {
+                final_attrs.push_str(",$");
+            }
+        }
+    }
+
     format!("{prefix}{new_type}({final_attrs});")
 }
 
@@ -250,136 +269,5 @@ pub fn needs_conversion(from: &str, to: &str) -> bool {
 }
 
 #[cfg(test)]
-mod tests {
-    use super::*;
-
-    #[test]
-    fn entity_type_renames() {
-        assert_eq!(convert_entity_type("IFCBURNERTYPE", "IFC4", "IFC2X3"), "IFCGASTERMINALTYPE");
-        assert_eq!(convert_entity_type("IFCCHIMNEY", "IFC4", "IFC2X3"), "IFCBUILDINGELEMENTPROXY");
-        assert_eq!(convert_entity_type("IFCWALL", "IFC2X3", "IFC4"), "IFCWALL"); // unchanged
-        // chained 4X3 → 2X3 (via 4): IfcFacility → IfcBuilding
-        assert_eq!(convert_entity_type("IFCFACILITY", "IFC4X3", "IFC2X3"), "IFCBUILDING");
-    }
-
-    /// The UPGRADE table (`map_2x3_to_4`) had no test in this crate: the only
-    /// 2X3 → 4 case above is `IFCWALL`, a type that is unchanged in every
-    /// schema, so replacing the whole arm with a pass-through left the entire
-    /// `ifc-lite-export` suite green (confirmed by mutation). The TS twin
-    /// `packages/export/src/schema-converter.test.ts` covers this direction;
-    /// the Rust port is what the CLI, server and wasm actually run.
-    ///
-    /// Concretely, un-renamed output is an INVALID file: none of these three
-    /// type names exists in IFC4.
-    #[test]
-    fn ifc2x3_only_types_are_renamed_on_upgrade() {
-        for (from, want) in [
-            ("IFCELECTRICDISTRIBUTIONPOINT", "IFCELECTRICDISTRIBUTIONBOARD"),
-            ("IFCGASTERMINALTYPE", "IFCBURNERTYPE"),
-            ("IFCEQUIPMENTELEMENT", "IFCBUILDINGELEMENTPROXY"),
-        ] {
-            assert_eq!(convert_entity_type(from, "IFC2X3", "IFC4"), want, "2X3 → 4");
-            // 2X3 → 4X3 and 2X3 → 5 route through the same table.
-            assert_eq!(convert_entity_type(from, "IFC2X3", "IFC4X3"), want, "2X3 → 4X3");
-            assert_eq!(convert_entity_type(from, "IFC2X3", "IFC5"), want, "2X3 → 5");
-        }
-    }
-
-    /// The direct `4X3 → 4` arm was only ever reached by types it does NOT
-    /// rename: `entity_type_renames` chains through it on the way to 2X3, and
-    /// `alignment_becomes_proxy_on_downgrade` hits the proxy branch instead.
-    /// Replacing the arm with a pass-through likewise left the suite green.
-    /// `IFC5 → IFC4` shares the arm, so it is pinned here too — nothing else
-    /// in this crate exercises `canon`'s IFC5 branch at all.
-    #[test]
-    fn ifc4x3_and_ifc5_types_are_renamed_down_to_ifc4() {
-        for (from, want) in [
-            ("IFCBRIDGE", "IFCBUILDING"),
-            ("IFCBRIDGEPART", "IFCBUILDINGSTOREY"),
-            ("IFCPAVEMENT", "IFCSLAB"),
-            ("IFCCAISSONFOUNDATION", "IFCFOOTING"),
-            ("IFCDISTRIBUTIONBOARD", "IFCELECTRICDISTRIBUTIONBOARD"),
-        ] {
-            assert_eq!(convert_entity_type(from, "IFC4X3", "IFC4"), want, "4X3 → 4");
-            assert_eq!(convert_entity_type(from, "IFC5", "IFC4"), want, "5 → 4");
-        }
-        // 4 ↔ 4X3 ↔ 5 carry no renames, and IFCX* canonicalizes to IFC5.
-        assert_eq!(convert_entity_type("IFCWALL", "IFC4", "IFC5"), "IFCWALL");
-        assert_eq!(convert_entity_type("IFCBRIDGE", "IFCX", "IFC4"), "IFCBUILDING");
-        assert!(!needs_conversion("IFC5", "IFCX"));
-    }
-
-    #[test]
-    fn downgrade_trims_attributes() {
-        // IfcWall in IFC4 has 9 attrs (trailing PredefinedType); IFC2X3 keeps 8.
-        let line = "#5=IFCWALL('guid',$,'W1',$,$,#6,#7,'tag',.STANDARD.);";
-        let out = convert_step_line(line, "IFC4", "IFC2X3", 5);
-        assert!(out.starts_with("#5=IFCWALL("), "type kept");
-        assert!(!out.contains(".STANDARD."), "9th attr (PredefinedType) trimmed");
-        // 8 top-level attrs remain → 7 commas.
-        let inner = &out["#5=IFCWALL(".len()..out.len() - 2];
-        assert_eq!(inner.split(',').count(), 8, "trimmed to 8 attrs");
-    }
-
-    #[test]
-    fn nested_attrs_not_split_when_trimming() {
-        // Commas inside a nested list must not count as top-level separators.
-        let line = "#9=IFCWALL('g',$,$,$,$,(#1,#2,#3),#7,'t',.STANDARD.);";
-        let out = convert_step_line(line, "IFC4", "IFC2X3", 9);
-        assert!(out.contains("(#1,#2,#3)"), "nested list preserved intact");
-        assert!(!out.contains(".STANDARD."), "trailing attr trimmed");
-    }
-
-    #[test]
-    fn alignment_becomes_proxy_on_downgrade() {
-        let line = "#3=IFCALIGNMENTHORIZONTAL('g',$,$,$,$,#4);";
-        let out = convert_step_line(line, "IFC4X3", "IFC4", 3);
-        assert!(out.starts_with("#3=IFCPROXY("), "alignment → proxy");
-        assert!(out.contains("'IFCALIGNMENTHORIZONTAL'"), "original type recorded as name");
-    }
-
-    /// PINS the schema-downgrade proxy-GlobalId divergence between the two
-    /// exporters (#3015) AS divergence -- it does not fix it. Which side wins
-    /// is a maintainer decision, not something a test should resolve
-    /// unilaterally.
-    ///
-    /// `placeholder_guid` here derives the id purely from the express id
-    /// (`id as u64 + 0x1000_0000`, base64-stamped). The TypeScript twin
-    /// (`convertStepLine` in `packages/export/src/schema-converter.ts`)
-    /// derives it from `deterministicGlobalId` of the WHOLE source line
-    /// (`ifcproxy:{prefix}{entityType}({attrs})`) -- a different algorithm
-    /// entirely, not just a different seed to the same one. Verified by
-    /// actually running both on the byte-identical input line below;
-    /// `packages/export/src/schema-converter.test.ts`'s
-    /// `placeholder_guid_diverges_from_the_rust_mint_pinned_not_fixed` pins
-    /// the TS side of the same pair.
-    ///
-    /// If this test ever starts failing because the values converged, that
-    /// is good news -- update the doc here (and the TS twin) to say so,
-    /// don't just delete the assertion.
-    #[test]
-    fn placeholder_guid_diverges_from_the_typescript_mint_pinned_not_fixed() {
-        let line = "#42=IFCALIGNMENTSEGMENT('2K5H1$Zs9CQuKQFQKQFQKQ',#1,'A',$,$,#7,#9,$);";
-        let out = convert_step_line(line, "IFC4X3", "IFC4", 42);
-        let guid = out.split('\'').nth(1).expect("IFCPROXY line has a quoted GlobalId");
-        assert_eq!(
-            guid, "00000000000000000G000g",
-            "Rust's placeholder_guid(42) output changed -- update this pin (and check whether \
-             it now agrees with the TS twin, in which case update both docs to say so)"
-        );
-        assert_ne!(
-            guid, "3m5OyAyREn46dEymqijDwc",
-            "this is the TS side's minted value for the byte-identical input line -- if Rust \
-             now matches it, the divergence has been resolved; update both tests' docs instead \
-             of silently dropping this assertion"
-        );
-    }
-
-    #[test]
-    fn no_conversion_is_identity() {
-        let line = "#1=IFCWALL('g',$,$);";
-        assert_eq!(convert_step_line(line, "IFC4", "IFC4", 1), line);
-        assert!(!needs_conversion("IFC4", "IFC4"));
-        assert!(needs_conversion("IFC2X3", "IFC4"));
-    }
-}
+#[path = "schema_convert_tests.rs"]
+mod tests;