fix: ensure consistent placeholder assignment for identical values (#283)

* fix: ensure consistent placeholder assignment for identical values in Anonymize scanner

- Check vault for existing value matches before creating new placeholders
- Reuse existing placeholder indices for known values
- Calculate next available index for new values to prevent conflicts
- Add batch consistency tracking within single processing runs
- Prevents duplicate vault entries for identical entity values
- Maintains referential integrity for proper deanonymization

Fixes issue where same entity values (e.g., "John Smith") would get
different placeholders ([REDACTED_PERSON_1], [REDACTED_PERSON_2])
across multiple scans, making consistent anonymization impossible.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* style: apply code formatting and fix performance warning in anonymize.py

- Apply ruff code formatting to improve readability
- Optimize try-except logic to avoid performance overhead in loop
- Pre-filter placeholder parts before attempting index extraction

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* test: add comprehensive test for placeholder consistency in Anonymize scanner

- Test ensures identical values get same placeholders across multiple scans
- Verifies no duplicate entries in vault for same entity values
- Confirms proper index assignment and reuse logic
- Validates referential integrity for deanonymization

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* style: apply code formatting to test assertions

Reformat multi-line assert statements to follow consistent style guidelines.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Trigger actions: empty commit

* Empty commit to retrigger git actions

* fix space

* Disable caching and add cleanup

---------

Co-authored-by: Claude <[email protected]>

Author: AnandShivansh

.github/workflows/test.yml

@@ -32,15 +32,37 @@ jobs:
 
     steps:
       - uses: actions/checkout@v4
+
+      - name: Aggressive disk cleanup
+        run: |
+          sudo rm -rf /usr/local/lib/android
+          docker system prune -af
+          sudo apt-get clean
+          sudo rm -rf /var/lib/apt/lists/*
+          sudo rm -rf /tmp/*
+          sudo rm -rf /var/tmp/*
+          rm -rf ~/.cache/huggingface
+          rm -rf ~/.cache/pip
+          df -h
+
       - name: Set up Python ${{ matrix.python-version }}
         uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
-          cache: 'pip'
+          cache: ''   # disables pip cache
+
       - name: Install deps
         run: |
           python -m pip install --upgrade pip
           python -m pip install ".[dev]"
+
+      - name: Free up space & Hugging Face cache
+        run: |
+          rm -rf ~/.cache/huggingface
+          df -h
+
       - name: Test with pytest
+        env:
+          HF_HUB_DISABLE_CACHE: 1   # prevents cache from filling up runner disk
         run: |
           python -m pytest --exitfirst --verbose --failed-first --cov=. --color=yes --code-highlight=yes

llm_guard/input_scanners/anonymize.py

@@ -255,7 +255,9 @@ def _anonymize(
         """
         text_replace_builder = TextReplaceBuilder(original_text=prompt)
 
-        entity_type_counter, new_entity_counter = {}, {}
+        entity_type_counter = {}
+        batch_entity_tracker = {}
+
         for pii_entity in pii_entities:
             entity_type = pii_entity.entity_type
             entity_value = text_replace_builder.get_text_in_position(
@@ -266,25 +268,60 @@ def _anonymize(
                 entity_type_counter[entity_type] = {}
 
             if entity_value not in entity_type_counter[entity_type]:
+                # Check vault for existing exact match first
                 vault_entities = [
                     (entity_placeholder, entity_vault_value)
                     for entity_placeholder, entity_vault_value in vault.get()
-                    if entity_type in entity_placeholder
-                ]
-                entity_placeholder = [
-                    entity_placeholder
-                    for entity_placeholder, entity_vault_value in vault_entities
-                    if entity_vault_value == entity_value
+                    if entity_placeholder.startswith(f"[REDACTED_{entity_type}_")
                 ]
-                if len(entity_placeholder) > 0:
+
+                # Look for exact value match in vault
+                existing_placeholder = None
+                for entity_placeholder, entity_vault_value in vault_entities:
+                    if entity_vault_value == entity_value:
+                        existing_placeholder = entity_placeholder
+                        break
+
+                if existing_placeholder:
+                    # Extract index from existing placeholder
                     entity_type_counter[entity_type][entity_value] = int(
-                        entity_placeholder[0].split("_")[-1][:-1]
+                        existing_placeholder.split("_")[-1][:-1]
                     )
                 else:
-                    entity_type_counter[entity_type][entity_value] = (
-                        len(vault_entities) + new_entity_counter.get(entity_type, 0) + 1
-                    )
-                    new_entity_counter[entity_type] = new_entity_counter.get(entity_type, 0) + 1
+                    # Check if we've already assigned this value in current batch
+                    if entity_type not in batch_entity_tracker:
+                        batch_entity_tracker[entity_type] = {}
+
+                    if entity_value in batch_entity_tracker[entity_type]:
+                        # Reuse the index assigned earlier in this batch
+                        entity_type_counter[entity_type][entity_value] = batch_entity_tracker[
+                            entity_type
+                        ][entity_value]
+                    else:
+                        # Calculate next available index
+                        existing_indices = set()
+
+                        # Add indices from vault
+                        for entity_placeholder, _ in vault_entities:
+                            parts = entity_placeholder.split("_")
+                            if len(parts) >= 3 and parts[-1].endswith("]"):
+                                try:
+                                    index = int(parts[-1][:-1])
+                                    existing_indices.add(index)
+                                except ValueError:
+                                    pass  # Skip invalid indices
+
+                        # Add indices from current batch
+                        for assigned_index in batch_entity_tracker.get(entity_type, {}).values():
+                            existing_indices.add(assigned_index)
+
+                        # Find next available index
+                        next_index = 1
+                        while next_index in existing_indices:
+                            next_index += 1
+
+                        entity_type_counter[entity_type][entity_value] = next_index
+                        batch_entity_tracker[entity_type][entity_value] = next_index
 
         results = []
         sorted_pii_entities = sorted(pii_entities, reverse=True)
@@ -347,6 +384,7 @@ def scan(self, prompt: str) -> tuple[str, bool, float]:
             for entity_placeholder, entity_value in anonymized_results:
                 if not self._vault.placeholder_exists(entity_placeholder):
                     self._vault.append((entity_placeholder, entity_value))
+
             return (
                 self._preamble + sanitized_prompt,
                 False,

tests/input_scanners/test_anonymize.py

@@ -327,3 +327,69 @@ def test_patterns():
                 assert re.search(expression, example) is not None, (
                     f"Test for {name} failed. No match found for example: {example}"
                 )
+
+
+def test_placeholder_consistency():
+    """Test that identical values get the same placeholder across multiple scans."""
+    vault = Vault()
+    scanner = Anonymize(vault, entity_types=["PERSON", "EMAIL_ADDRESS"])
+
+    # First scan with John Smith
+    prompt1 = "Hello John Smith, your email is [email protected]"
+    result1, valid1, score1 = scanner.scan(prompt1)
+
+    # Second scan with same person
+    prompt2 = "John Smith called yesterday"
+    result2, valid2, score2 = scanner.scan(prompt2)
+
+    # Third scan with different person
+    prompt3 = "Mary Johnson replied to [email protected]"
+    result3, valid3, score3 = scanner.scan(prompt3)
+
+    # Fourth scan with both people
+    prompt4 = "John Smith and Mary Johnson are meeting"
+    result4, valid4, score4 = scanner.scan(prompt4)
+
+    # Verify vault contents
+    vault_contents = vault.get()
+
+    # Assert consistent placeholder assignment
+    john_placeholders = [p for p, v in vault_contents if v == "John Smith"]
+    mary_placeholders = [p for p, v in vault_contents if v == "Mary Johnson"]
+    email_placeholders = [p for p, v in vault_contents if v == "[email protected]"]
+
+    # Each unique value should have exactly one placeholder
+    assert len(john_placeholders) == 1, (
+        f"John Smith should have 1 placeholder, got {len(john_placeholders)}: {john_placeholders}"
+    )
+    assert len(mary_placeholders) == 1, (
+        f"Mary Johnson should have 1 placeholder, got {len(mary_placeholders)}: {mary_placeholders}"
+    )
+    assert len(email_placeholders) == 1, (
+        f"[email protected] should have 1 placeholder, got {len(email_placeholders)}: {email_placeholders}"
+    )
+
+    # Verify specific placeholder assignments
+    john_placeholder = john_placeholders[0]
+    mary_placeholder = mary_placeholders[0]
+    email_placeholder = email_placeholders[0]
+
+    # John should be PERSON_1, Mary should be PERSON_2, email should be EMAIL_ADDRESS_1
+    assert john_placeholder == "[REDACTED_PERSON_1]", (
+        f"Expected [REDACTED_PERSON_1], got {john_placeholder}"
+    )
+    assert mary_placeholder == "[REDACTED_PERSON_2]", (
+        f"Expected [REDACTED_PERSON_2], got {mary_placeholder}"
+    )
+    assert email_placeholder == "[REDACTED_EMAIL_ADDRESS_1]", (
+        f"Expected [REDACTED_EMAIL_ADDRESS_1], got {email_placeholder}"
+    )
+
+    # Verify results contain consistent placeholders
+    assert john_placeholder in result1
+    assert john_placeholder in result2
+    assert john_placeholder in result4
+    assert mary_placeholder in result3
+    assert mary_placeholder in result4
+    assert email_placeholder in result1
+    assert email_placeholder in result3