chore: removed all gretel mentions when appropriate (#219)

# Summary

All "gretel" mentions in `src/` have been removed or replaced with
product-appropriate alternatives ("nss" for Nemo Safe Synthesizer).
External dataset URLs referencing `gretelai/gretel-blueprints` are kept
as-is since they point to a public data source.

## Code Identifiers Renamed

| File | Old | New |
|---|---|---|
| `records/fragment.py` | `gretel_id` (attribute + params) | `record_id`
|
| `records/fragment.py` | `gretel_fragment_ts` | `fragment_ts` |
| `records/fragment.py` | `gretel_fragment_epoch` | `fragment_epoch` |
| `records/fragment.py` | `gretel_fragment_datetime` |
`fragment_datetime` |
| `records/json_record.py` | `remove_gretel_array_markers()` |
`remove_array_markers()` |
| `records/base.py` | `"_gretelarray_"` | `"_nssarray_"` |
| `actions/utils.py` | `"__gretel__idx"` | `"__nss__idx"` |
| `actions/utils.py` | `"__gretel_reject_reason"` |
`"__nss_reject_reason"` |

## Constants Renamed (`pii_replacer/ner/const.py`)

| Old Key / Value | New Key / Value |
|---|---|
| `GRETEL_ID` / `_gretel_id` | `NSS_ID` / `_nss_id` |
| `GRETEL_TS` / `_gretel_ts` | `NSS_TS` / `_nss_ts` |
| `GRETEL_SUB` / `_gretel_subscriber` | `NSS_SUB` / `_nss_subscriber` |
| `ARRAY_POS` / `_gretelarray_` | `ARRAY_POS` / `_nssarray_` |

## Environment Variables Renamed (`pii_replacer/ner/models.py`)

| Old | New |
|---|---|
| `GRETEL_OPT_BUCKET` | `NSS_OPT_BUCKET` |
| `gretel-opt-dev-use2` (default bucket) | `nss-opt-dev-use2` |
| `GRETEL_OPT_CACHE_DIR` | `NSS_OPT_CACHE_DIR` |

## Comments / Documentation Cleaned Up

| File | Change |
|---|---|
| `src/.../record_utils.py` | "gretel client" → "nss client" |
| `src/.../pii_replacer/ner/models.py` | Removed "gretel" from
docstrings about visibility and cache management |
| `src/.../pii_replacer/ner/regexes/age.py` | Replaced internal GitHub
issue URL with descriptive comment |
| `src/.../pii_replacer/ner/regexes/sex_gender.py` | Replaced internal
GitHub issue URL with descriptive comment |
| `src/.../artifacts/analyzers/field_features.py` | Removed internal
GitHub repo URL |
| `src/.../records/fragment.py` | Error message: "different gretel
records" → "different records" |
| `tests/evaluation/conftest.py` | "gretel core arfifact classifier" →
"core artifact classifier" |

## Kept As-Is

- External dataset URLs referencing `gretelai/gretel-blueprints`
(third-party public data source)
- Historical tag references in test comments (`gretel-2025-02-25 tag`)

## Migration Notes

- **Environment variables**: Anyone previously setting
`GRETEL_OPT_BUCKET` or `GRETEL_OPT_CACHE_DIR` must update to
`NSS_OPT_BUCKET` / `NSS_OPT_CACHE_DIR`.



## Pre-Review Checklist

<!-- These checks should be completed before a PR is reviewed, -->
<!-- but you can submit a draft early to indicate that the issue is
being worked on. -->

Ensure that the following pass:

- [x] `make format && make check` or via prek validation.
- [x] `make test` passes locally
- [x] `make test-e2e` passes locally
- [ ] `make test-ci-container` passes locally (recommended)

## Pre-Merge Checklist

<!-- These checks need to be completed before a PR is merged, -->
<!-- but as PRs often change significantly during review, -->
<!-- it's OK for them to be incomplete when review is first requested.
-->

- [ ] New or updated tests for any fix or new behavior
- [ ] Updated documentation for new features and behaviors, including
docstrings for API docs.

## Other Notes

<!-- Please add the issue number that should be closed when this PR is
merged. -->
- Closes #218

---------

Signed-off-by: Sean Yang <seayang@nvidia.com>
Signed-off-by: seayang-nv <seayang@nvidia.com>

Author: seayang-nv

src/nemo_safe_synthesizer/artifacts/analyzers/field_features.py

@@ -140,7 +140,6 @@ def describe_field(field_name: str, data: Series) -> FieldFeatures:
         return features
 
     # See
-    # - https://github.com/Gretellabs/text_research/blob/main/column_detector.py
     # - https://jeffreymorgan.io/articles/identifying-categorical-data/
     diff = non_na_count - unique_count
     diff_percent = diff / non_na_count

src/nemo_safe_synthesizer/data_processing/record_utils.py

@@ -387,7 +387,7 @@ def normalize_dataframe(dataframe: pd.DataFrame) -> pd.DataFrame:
     """
     # HACK: Handle NaN/None/NA values with mixed types by
     # normalizing through pandas csv io format, which will match
-    # the format in reports generated via the gretel client.
+    # the format in reports generated via the nss client.
     try:
         # try without trying to resolve utf-8 issues first
         return pd.read_csv(StringIO(dataframe.to_csv(index=False, quoting=QUOTE_NONNUMERIC)))

src/nemo_safe_synthesizer/data_processing/records/base.py

@@ -32,7 +32,7 @@
 BOOL = "boolean"
 NUMBER = "number"
 NULL = "null"
-ARRAY_POS = "_gretelarray_"
+ARRAY_POS = "_nssarray_"
 NESTING_DELIM = "*#N#*"
 DELIM = "."
 

src/nemo_safe_synthesizer/data_processing/records/fragment.py

@@ -40,8 +40,7 @@ class Metadata:
         field_name -> fragment_name -> metadata_type -> [metadata_items]
     """
 
-    gretel_id: str
-    """Unique identifier for the source record."""
+    record_id: str
 
     fields: dict
     """Nested dict of per-field, per-fragment metadata."""
@@ -65,24 +64,24 @@ class MetadataFragment:
     ``Metadata`` object per record.
 
     Args:
-        gretel_id: Unique identifier for the source record.
-        gretel_fragment_ts: ISO-8601 timestamp string.
-        gretel_fragment_epoch: Unix epoch of the fragment creation.
+        record_id: Unique identifier for the source record.
+        fragment_ts: ISO-8601 timestamp string.
+        fragment_epoch: Unix epoch of the fragment creation.
         fragment_name: Identifier for this annotation pass (e.g., ``"ner"``).
     """
 
-    gretel_id: str
-    gretel_fragment_ts: str
-    gretel_fragment_epoch: float
+    record_id: str
+    fragment_ts: str
+    fragment_epoch: float
     fragment_name: str
 
     def __post_init__(self):
         self.fields = defaultdict(lambda: defaultdict(list))
 
     @property
-    def gretel_fragment_datetime(self) -> datetime:
+    def fragment_datetime(self) -> datetime:
         """Fragment creation time as a ``datetime`` object."""
-        return datetime.fromtimestamp(self.gretel_fragment_epoch)
+        return datetime.fromtimestamp(self.fragment_epoch)
 
     def add_field_data(self, field_name: str, metadata_type: str, field_data: dict | list):
         """Append metadata entries for a field.
@@ -121,30 +120,30 @@ def merge_fragments(*fragments, ts: str | None = None) -> Metadata:
     Raises:
         MetadataError: If the fragments have different ``gretel_id`` values.
     """
-    if len(set([fragment.gretel_id for fragment in fragments])) != 1:
-        raise MetadataError("cannot merge fragments from different gretel records")
+    if len(set([fragment.record_id for fragment in fragments])) != 1:
+        raise MetadataError("cannot merge fragments from different records")
     else:
-        gretel_id = fragments[0].gretel_id
+        record_id = fragments[0].record_id
 
     # todo(dn): there might be a better way to build up this object
     merged_fragment = defaultdict(lambda: defaultdict(lambda: defaultdict(list)))
-    ts = ts or min([f.gretel_fragment_datetime for f in fragments]).isoformat() + "Z"
+    ts = ts or min([f.fragment_datetime for f in fragments]).isoformat() + "Z"
     fragment: MetadataFragment
     for fragment in fragments:
         for field_name, field_data in fragment.fields.items():
             for meta_type, meta_data in field_data.items():
                 merged_fragment[field_name][fragment.fragment_name][meta_type].extend(meta_data)
-    return Metadata(gretel_id=gretel_id, fields=merged_fragment, received_at=ts, entities={})
+    return Metadata(record_id=record_id, fields=merged_fragment, received_at=ts, entities={})
 
 
-def fragment_for_record(gretel_id: str, fragment_name: str) -> MetadataFragment:
+def fragment_for_record(record_id: str, fragment_name: str) -> MetadataFragment:
     """Create a new ``MetadataFragment`` timestamped to the current time."""
     epoch = time.time()
     ts = datetime.fromtimestamp(epoch).isoformat() + "Z"
     return MetadataFragment(
-        gretel_id=gretel_id,
-        gretel_fragment_epoch=epoch,
-        gretel_fragment_ts=ts,
+        record_id=record_id,
+        fragment_epoch=epoch,
+        fragment_ts=ts,
         fragment_name=fragment_name,
     )
 
@@ -216,7 +215,7 @@ def predictions_to_dict(
 def fragment_from_ner_predictions(
     fragment_name: str,
     predictions: list[NERPrediction],
-    gretel_id: str,
+    record_id: str,
 ) -> tuple[MetadataFragment, dict]:
     """Build a ``MetadataFragment`` and entity map from NER predictions.
 
@@ -230,9 +229,9 @@ def fragment_from_ner_predictions(
     """
     epoch = time.time()
     fragment = MetadataFragment(
-        gretel_id=gretel_id,
-        gretel_fragment_ts=datetime.fromtimestamp(epoch).isoformat() + "Z",
-        gretel_fragment_epoch=epoch,
+        record_id=record_id,
+        fragment_ts=datetime.fromtimestamp(epoch).isoformat() + "Z",
+        fragment_epoch=epoch,
         fragment_name=fragment_name,
     )
     preds_by_field, ent_map = predictions_to_dict(predictions)

src/nemo_safe_synthesizer/data_processing/records/json_record.py

@@ -66,7 +66,7 @@ def unpack_level(parent_key, parent_val):
     return raw
 
 
-def remove_gretel_array_markers(data: str) -> tuple[str, int, base.ValuePath]:
+def remove_array_markers(data: str) -> tuple[str, int, base.ValuePath]:
     """Strip array-position markers from a composite key and build a ``ValuePath``.
 
     Returns:
@@ -91,7 +91,7 @@ def convert_flat_dict_to_kv_pairs(data: dict) -> list[base.KVPair]:
     out = []
     for k, v in data.items():
         k = str(k)
-        new_key, array_count, value_path = remove_gretel_array_markers(k)
+        new_key, array_count, value_path = remove_array_markers(k)
         flat = base.KVPair(new_key, v, base.get_type_as_string(v), array_count, value_path)
         out.append(flat)
     return out

src/nemo_safe_synthesizer/pii_replacer/ner/const.py

@@ -14,9 +14,9 @@ def __getattr__(self, key):
 
 const = ConstDict(
     {
-        "GRETEL_ID": "_gretel_id",
-        "GRETEL_TS": "_gretel_ts",
-        "GRETEL_SUB": "_gretel_subscriber",
+        "NSS_ID": "_nss_id",
+        "NSS_TS": "_nss_ts",
+        "NSS_SUB": "_nss_subscriber",
         "TYPE": "type",
         "STR_VALUE": "string_value",
         "INT_VALUE": "int_value",
@@ -48,7 +48,7 @@ def __getattr__(self, key):
         "TRUE": "true",
         "FALSE": "false",
         "NULL": "null",
-        "ARRAY_POS": "_gretelarray_",
+        "ARRAY_POS": "_nssarray_",
         "BOOL": "boolean",
         "DELIM": ".",
         "ENTITY": "entity",

src/nemo_safe_synthesizer/pii_replacer/ner/models.py

@@ -15,11 +15,11 @@
 
 logger = get_logger(__name__)
 
-DEFAULT_BUCKET = os.getenv("GRETEL_OPT_BUCKET", "gretel-opt-dev-use2")
+DEFAULT_BUCKET = os.getenv("NSS_OPT_BUCKET", "nss-opt-dev-use2")
 """Default bucket environment variable. If it's not found, use dev."""
 
-DEFAULT_CACHE_DIR = os.getenv("GRETEL_OPT_CACHE_DIR", ".optcache")
-"""``StorageConfig`` default cache directory. Searches the GRETEL_OPT_CACHE_DIR environment
+DEFAULT_CACHE_DIR = os.getenv("NSS_OPT_CACHE_DIR", ".optcache")
+"""``StorageConfig`` default cache directory. Searches the NSS_OPT_CACHE_DIR environment
 for a cache directory. By default it will fallback to `.optcache`. If this is set to
 ``disabled`` files wont be cached to disk.
 """
@@ -32,10 +32,10 @@ class Visibility(Enum):
     """Packages that are open to the public with a public-read ACL"""
 
     PRIVATE = "priv"
-    """Packages available to customers via "paywall" behind gretel api key"""
+    """Packages available to customers via "paywall" behind an api key"""
 
     INTERNAL = "int"
-    """Only available from internal gretel infrastructure."""
+    """Only available from internal infrastructure."""
 
 
 @dataclass
@@ -95,7 +95,7 @@ class StorageConfig:
     def from_system(cls) -> StorageConfig:
         """Return a default ``StorageConfig`` based on a system's environment variables.
 
-        By convention, it looks for the environment variable ``GRETEL_OPT_BUCKET`` as
+        By convention, it looks for the environment variable ``NSS_OPT_BUCKET`` as
         the bucket location. The default settings from this function are appropriate
         for development without additional configuration.
         """
@@ -113,7 +113,7 @@ def get_cache_manager(storage_config: StorageConfig = None) -> CacheManager:
 
 
 class CacheManager:
-    """Handles downloading model files from the gretel "opt" package repo.
+    """Handles downloading model files from the "opt" package repo.
 
     This class will also optionally cache these files to disk. This is useful for
     environments with local persistent state such as a local development laptop.

src/nemo_safe_synthesizer/pii_replacer/ner/regexes/age.py

@@ -11,7 +11,7 @@
 from ..entity import Entity
 from ..regex import Pattern, RegexPredictor
 
-# https://github.com/Gretellabs/monogretel/issues/190
+# Relevant issue: detection should support descriptive age terms
 HEADERS = ["age", "ages"]
 
 

src/nemo_safe_synthesizer/pii_replacer/ner/regexes/sex_gender.py

@@ -11,7 +11,7 @@
     create_exact_field_matcher,
 )
 
-# https://github.com/Gretellabs/monogretel/issues/190
+# Relevant issue: detection should support sex/gender header patterns
 SEX_HEADERS = [
     create_exact_field_matcher("sex"),
     create_exact_field_matcher("sexo"),

tests/evaluation/conftest.py

@@ -27,7 +27,7 @@ def make_df(seed: int, n: int = 100):
         {
             "num": [random.random() for _ in range(n)],
             "num_Int64": [random.randint(1, 100) for _ in range(n)],
-            # Categorical columns according to gretel core arfifact classifier
+            # Categorical columns according to core artifact classifier
             "num_cat": [random.randint(1, 4) for _ in range(n)],
             "num_cat_Int64": [random.randint(1, 4) for _ in range(n)],
             "small_cat": [random.choice(["foo", "bar", "baz", "biff", "barf"]) for _ in range(n)],