feat(thoughtspot): preserve obj_id, ThoughtSpot's portable object handle

`guid`, `fqn` and `obj_id` were grouped together as "instance-local identity
never travels in a portable document". They are not the same kind of thing.

`guid` is a raw cluster UUID. `fqn` is a reference to one -- and this repo's own
schema reference already records that a viz-level `fqn` is DROPPED on import,
leaving the object with no data source. Both are correctly refused.

`obj_id` is the opposite: ThoughtSpot introduced it precisely so objects can be
referenced across environments, and it survives import. It is a readable handle
-- `SampleRetail-Apparel-LH-58435d2b`, the display name plus the GUID's first
segment -- not a bare identifier. Discarding it meant a converted model
re-imported as a NEW object beside the one it came from, rather than updating
it, which breaks the promote-between-environments workflow this converter exists
to serve.

It is now stashed under its own payload key and restored at the document root.
The forbidden-key scan still names `obj_id`, deliberately: that scan stops one
riding along unnoticed inside a block copied wholesale from source TML, while
the stash preserves it on purpose, under a distinct key. The two are not in
conflict and the comment now says so.

`guid` is still stripped at every depth, and the test asserts that.

Observed on a real export while investigating why a converted model could not be
re-imported: the source carried `obj_id: SampleRetail-Apparel-LH-58435d2b` and
the converted document carried nothing.

Mutation-checked. 928 tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: djwaldo

converters/thoughtspot/docs/vendor-payload.md

@@ -38,6 +38,7 @@ Every `custom_extensions` entry this converter writes uses `vendor_name` `THOUGH
 | Key | Scope | Classification | Treatment on the return trip |
 |---|---|---|---|
 | `tml_name` | Shared | shadows_derivable | Restored only if reconstructing it from the live document still agrees with the stashed value (self-verifying — no separate witness key); disagreement re-derives instead. |
+| `tml_obj_id` | Model | information_only | Restored as-is whenever present — nothing on the Ossie side could have diverged from it. |
 | `formula_id` | Field | information_only | Restored as-is whenever present — nothing on the Ossie side could have diverged from it. |
 | `db_column_name` | Field | shadows_derivable | Restored only if its witness companion key still matches the live document's current value; a mismatch means the document changed since the stash was written, so the value is re-derived instead. |
 | `data_type` | Field | shadows_derivable | Restored only if its witness companion key still matches the live document's current value; a mismatch means the document changed since the stash was written, so the value is re-derived instead. |

converters/thoughtspot/src/ossie_thoughtspot/constants.py

@@ -99,6 +99,22 @@
 #: must agree on the exact spelling and nothing else enforces that.
 FIELD_STASH_DB_COLUMN_NAME = "db_column_name"
 
+#: The source TML `obj_id` -- ThoughtSpot's own PORTABLE object handle, e.g.
+#: `SampleRetail-Apparel-LH-58435d2b` (display name, then the first segment of
+#: the GUID). Stashed under a distinct payload key so the forbidden-key scan,
+#: which looks for a literal `obj_id`, does not mistake this deliberate copy for
+#: leaked identity.
+#:
+#: Why this is kept when `guid` and `fqn` are not, though all three were once
+#: grouped as "instance-local identity": they are not the same kind of thing.
+#: `guid` is a raw cluster UUID and `fqn` is a reference to one -- and a
+#: viz-level `fqn` is DROPPED on import, leaving the object with no data source.
+#: `obj_id` is the identifier ThoughtSpot introduced precisely so objects can be
+#: referenced across environments, and it survives import. Discarding it meant a
+#: converted model re-imported as a NEW object rather than updating the one it
+#: came from, which is the ordinary promote-between-environments workflow.
+MODEL_STASH_OBJ_ID = "tml_obj_id"
+
 #: The source TML `formulas[].id` of a computed field or metric, stashed
 #: verbatim. TML's `formulas[].id` and `formulas[].name` are INDEPENDENT -- a
 #: formula renamed after creation keeps its original id -- but the Ossie -> TML
@@ -456,6 +472,8 @@ class StashKeyClass(Enum):
     # is no live value this can diverge FROM. That is also the point of keeping
     # it -- TML's id is independent of its name, so renaming the metric in Ossie
     # must NOT change the id, or every cross-reference written against it breaks.
+    # Ossie has no object-identity concept, so nothing here can diverge from it.
+    MODEL_STASH_OBJ_ID: StashKeyClass.INFORMATION_ONLY,
     FIELD_STASH_FORMULA_ID: StashKeyClass.INFORMATION_ONLY,
     FIELD_STASH_DB_COLUMN_NAME: StashKeyClass.SHADOWS_DERIVABLE,
     FIELD_STASH_DATA_TYPE: StashKeyClass.SHADOWS_DERIVABLE,

converters/thoughtspot/src/ossie_thoughtspot/ossie_to_thoughtspot.py

@@ -72,6 +72,7 @@
 from . import datatypes, formula, identifiers, stash
 from .constants import (
     FIELD_STASH_FORMULA_ID,
+    MODEL_STASH_OBJ_ID,
     DATASET_STASH_ALIAS,
     DATASET_STASH_CONNECTION_NAME,
     DATASET_STASH_SOURCE_PARTS,
@@ -2235,7 +2236,15 @@ def build_model(semantic_model: dict, tables: Sequence[TmlDocument], log: IssueL
         # differently-scoped `joins_with[]` inside the same payload namespace.
         body["joins_with"] = model_joins_with
 
-    return TmlDocument(kind="model", body=body, guid=None)
+    # `guid` is never restored -- it is raw cluster identity. `obj_id` is, and
+    # the difference is the point: it is the handle ThoughtSpot uses to
+    # recognise this as the SAME object on re-import, including in another
+    # environment, so keeping it is what makes the round trip an update
+    # rather than a duplicate.
+    return TmlDocument(
+        kind="model", body=body, guid=None,
+        obj_id=model_payload.get(MODEL_STASH_OBJ_ID),
+    )
 
 
 # ---------------------------------------------------------------------------

converters/thoughtspot/src/ossie_thoughtspot/stash.py

@@ -27,6 +27,13 @@
 from .errors import ConversionError
 
 #: Instance-local identity never travels in a portable document.
+#:
+#: `obj_id` is in this scan but is NOT wholly forbidden: the scan stops it
+#: riding along unnoticed inside a block copied wholesale from source TML, while
+#: `constants.STASH_OBJ_ID` preserves it DELIBERATELY, under its own payload key.
+#: The three are not equivalent -- `guid` is a raw cluster UUID and `fqn` a
+#: reference to one, but `obj_id` is the handle ThoughtSpot introduced so objects
+#: can be referenced ACROSS environments, and unlike an `fqn` it survives import.
 _FORBIDDEN_KEYS = frozenset({"guid", "obj_id", "fqn"})
 
 

converters/thoughtspot/src/ossie_thoughtspot/tml.py

@@ -93,6 +93,10 @@ class TmlDocument:
     kind: str
     body: dict
     guid: str | None
+    #: ThoughtSpot's portable object handle. Read AND written, unlike `guid`:
+    #: it is how a re-import updates the object it came from instead of
+    #: creating a duplicate, and it survives moving between environments.
+    obj_id: str | None = None
     source: str | None = None
 
 
@@ -132,7 +136,10 @@ def load_document(text: str, *, source: str | None = None) -> TmlDocument:
     body = data[kind]
     if not isinstance(body, dict):
         raise ConversionError(f"{source or '<input>'}: {kind} must be a mapping")
-    return TmlDocument(kind=kind, body=body, guid=data.get("guid"), source=source)
+    return TmlDocument(
+        kind=kind, body=body, guid=data.get("guid"),
+        obj_id=data.get("obj_id"), source=source,
+    )
 
 
 def load_document_set(texts: Sequence[tuple[str, str]]) -> DocumentSet:
@@ -171,7 +178,11 @@ def _strip_nested_guids(value: object) -> object:
 def dump_document(document: TmlDocument) -> str:
     """Serialise one document. `guid` is stripped unconditionally, at every depth of
     the body — not only at the document root."""
-    return _yaml.dump({document.kind: _strip_nested_guids(document.body)})
+    payload: dict = {}
+    if document.obj_id:
+        payload["obj_id"] = document.obj_id
+    payload[document.kind] = _strip_nested_guids(document.body)
+    return _yaml.dump(payload)
 
 
 def _safe_filename_component(name: object) -> str:

converters/thoughtspot/src/ossie_thoughtspot/tml_to_ossie.py

@@ -128,6 +128,7 @@
     RELATIONSHIP_STASH_ON_EXPRESSION_WITNESS,
     RELATIONSHIP_STASH_REFERENCING_JOIN,
     RELATIONSHIP_STASH_TYPE,
+    MODEL_STASH_OBJ_ID,
     STASH_TML_NAME,
 )
 from .errors import ConversionError
@@ -2022,6 +2023,10 @@ def convert(document_set: DocumentSet) -> OssieConversion:
             )
     semantic_model: dict = {"name": semantic_model_name, "datasets": []}
     model_stash: dict = {}
+    # ThoughtSpot's portable object handle, so a re-import updates the model
+    # it came from rather than creating a duplicate beside it.
+    if document_set.model.obj_id:
+        model_stash[MODEL_STASH_OBJ_ID] = document_set.model.obj_id
     if semantic_model_name != model_display_name:
         model_stash[STASH_TML_NAME] = model_display_name
 

converters/thoughtspot/tests/test_roundtrip.py

@@ -922,3 +922,38 @@ def test_every_referencing_join_carries_the_compulsory_with_field():
                 assert join.get("with"), (
                     f"{fixture_name}: joins[] entry on {entry['name']!r} has no `with`: {join}"
                 )
+
+
+def test_the_models_obj_id_survives_the_round_trip():
+    """`obj_id` is kept where `guid` and `fqn` are not, and the difference matters.
+
+    All three were once grouped as "instance-local identity". They are not the
+    same kind of thing: `guid` is a raw cluster UUID and `fqn` a reference to
+    one -- and a viz-level `fqn` is dropped on import anyway -- while `obj_id`
+    (`SampleRetail-Apparel-LH-58435d2b`: display name, then the GUID's first
+    segment) is the handle ThoughtSpot introduced so objects can be referenced
+    ACROSS environments, and it survives import.
+
+    Discarding it meant a converted model re-imported as a NEW object rather
+    than updating the one it came from -- which breaks the ordinary
+    promote-between-environments workflow the converter exists to serve.
+    """
+    table = TmlDocument(kind="table", guid=None, body={
+        "name": "t", "db": "D", "schema": "S", "db_table": "T",
+        "connection": {"name": "Conn"},
+        "columns": [{"name": "a", "db_column_name": "A",
+                     "db_column_properties": {"data_type": "DOUBLE"}}]})
+    model = TmlDocument(kind="model", guid="11111111-2222-3333-4444-555555555555",
+                        obj_id="SampleRetail-58435d2b", body={
+        "name": "M", "model_tables": [{"name": "t"}],
+        "columns": [{"name": "a", "column_id": "t::a",
+                     "properties": {"column_type": "ATTRIBUTE"}}]})
+    ossie = tml_to_ossie.convert(DocumentSet(model=model, tables=(table,)))
+    returned = ossie_to_thoughtspot.convert(ossie.model).documents.model
+    assert returned.obj_id == "SampleRetail-58435d2b"
+
+    emitted = tml.dump_document(returned)
+    assert "obj_id: SampleRetail-58435d2b" in emitted
+    # The guid must still never travel.
+    assert "11111111-2222" not in emitted
+    assert "guid:" not in emitted