Merge branch 'main' into docs/copilot-instructions-yaml-testing

Author: omri374

docs/development.md

@@ -16,52 +16,54 @@ The project is structured so that:
 - In the project root directory, you will find common code for using, serving and testing Presidio
     as a cluster of services, as well as CI/CD pipelines codebase and documentation.
 
-### Setting up Poetry
+### Setting up uv
 
-[Poetry](https://python-poetry.org/) is Python package manager. It is used to manage dependencies and virtual
-environments for Presidio services.
-Follow these steps when starting to work on a Presidio service with poetry:
+[uv](https://docs.astral.sh/uv/) is the Python package manager used to manage dependencies and virtual
+environments for Presidio services (the CI installs dependencies the same way).
+Follow these steps when starting to work on a Presidio service with uv:
 
-1. Install poetry
+1. Install uv
 
-    - Using Pip
+    - Using the standalone installer (you can inspect [install.sh](https://astral.sh/uv/install.sh) before running it)
 
         ```sh
-        pip install poetry
+        curl -LsSf https://astral.sh/uv/install.sh | sh
         ```
 
     - Using Homebrew (in MacOS)
 
         ```
-        brew install poetry
+        brew install uv
         ```
 
-    Additional installation instructions for poetry: <https://python-poetry.org/docs/#installation>
+    Additional installation instructions for uv: <https://docs.astral.sh/uv/getting-started/installation/>
 
-2. Have poetry create a virtualenv for the project and install all requirements in the pyproject.toml,
+2. Have uv create a virtualenv for the project and install all requirements in the pyproject.toml,
     including dev requirements.
 
     For example, in the `presidio-analyzer` folder, run:
 
     ```
-    poetry install --all-extras
+    uv sync --all-extras --group dev
     ```
 
+    Add `--locked` to install the exact locked dependency graph (matching CI behavior).
+
 3. Run all tests:
 
     ```
-    poetry run pytest
+    uv run pytest
     ```
 
 4. To run arbitrary scripts within the virtual env, start the command with
-    `poetry run`. For example:
-    1. `poetry run ruff check`
-    2. `poetry run pip freeze`
-    3. `poetry run python -m spacy download en_core_web_lg`
+    `uv run`. For example:
+    1. `uv run ruff check`
+    2. `uv run pip freeze`
+    3. `uv run python -m spacy download en_core_web_lg`
 
-    Command 3 downloads the default spacy model needed for Presidio Analyzer.`
+    Command 3 downloads the default spacy model needed for Presidio Analyzer.
 
-#### Alternatively, activate the virtual environment and use the commands using [this method](https://python-poetry.org/docs/basic-usage/#activating-the-virtual-environment).
+#### Alternatively, [activate the virtual environment](https://docs.astral.sh/uv/pip/environments/#using-a-virtual-environment) and run the commands without the `uv run` prefix.
 
 ### Development guidelines
 
@@ -93,9 +95,9 @@ use docker-compose ps:
 ```bash
 >docker-compose ps
 CONTAINER ID   IMAGE                       COMMAND                  CREATED         STATUS         PORTS                    NAMES
-6d5a258d19c2   presidio-anonymizer         "/bin/sh -c 'poetry …"   6 minutes ago   Up 6 minutes   0.0.0.0:5001->5001/tcp   presidio_presidio-anonymizer_1
-9aad2b68f93c   presidio-analyzer           "/bin/sh -c 'poetry …"   2 days ago      Up 6 minutes   0.0.0.0:5002->5001/tcp   presidio_presidio-analyzer_1
-1448dfb3ec2b   presidio-image-redactor     "/bin/sh -c 'poetry …"   2 seconds ago   Up 2 seconds   0.0.0.0:5003->5001/tcp   presidio_presidio-image-redactor_1
+6d5a258d19c2   presidio-anonymizer         "./entrypoint.sh"   6 minutes ago   Up 6 minutes   0.0.0.0:5001->5001/tcp   presidio_presidio-anonymizer_1
+9aad2b68f93c   presidio-analyzer           "./entrypoint.sh"   2 days ago      Up 6 minutes   0.0.0.0:5002->5001/tcp   presidio_presidio-analyzer_1
+1448dfb3ec2b   presidio-image-redactor     "./entrypoint.sh"   2 seconds ago   Up 2 seconds   0.0.0.0:5003->5001/tcp   presidio_presidio-image-redactor_1
 ```
 
 Edit docker-compose.yml configuration file to change the default ports.
@@ -138,7 +140,7 @@ Running the tests locally can be done in two ways:
 1. Using cli, from each service directory, run:
 
     ```sh
-    poetry run pytest
+    uv run pytest
     ```
 
 2. Using your IDE.
@@ -202,7 +204,7 @@ Running the e2e-tests locally can be done in two ways:
 
 Presidio services are PEP8 compliant and continuously enforced on style guide issues during the build process using `ruff`, in turn running `flake8` and other linters.
 
-Running ruff locally, using `poetry run ruff check`, you can check for those issues prior to committing a change.
+Running ruff locally, using `uv run ruff check`, you can check for those issues prior to committing a change.
 
 Ruff runs linters in addition to the basic `flake8` functionality, Presidio uses linters as part as ruff such as:
 

presidio-analyzer/presidio_analyzer/conf/default_recognizers.yaml

@@ -311,6 +311,14 @@ recognizers:
     enabled: false
     country_code: kr
 
+  - name: KrPassportRecognizer
+    supported_languages:
+    - ko
+    - kr
+    type: predefined
+    enabled: false
+    country_code: kr
+
   - name: SeOrganisationsnummerRecognizer
     supported_languages:
     - sv

presidio-analyzer/presidio_analyzer/predefined_recognizers/country_specific/korea/kr_brn_recognizer.py

@@ -58,6 +58,7 @@ def __init__(
         supported_language: str = "ko",
         supported_entity: str = "KR_BRN",
         replacement_pairs: Optional[List[Tuple[str, str]]] = None,
+        name: Optional[str] = None,
     ):
         self.replacement_pairs = replacement_pairs if replacement_pairs else [("-", "")]
 
@@ -68,6 +69,7 @@ def __init__(
             patterns=patterns,
             context=context,
             supported_language=supported_language,
+            name=name,
         )
 
     def validate_result(self, pattern_text: str) -> Union[bool, None]:

presidio-analyzer/presidio_analyzer/predefined_recognizers/country_specific/korea/kr_driver_license_recognizer.py

@@ -70,6 +70,7 @@ def __init__(
         supported_language: str = "ko",
         supported_entity: str = "KR_DRIVER_LICENSE",
         replacement_pairs: Optional[List[Tuple[str, str]]] = None,
+        name: Optional[str] = None,
     ):
         self.replacement_pairs = (
             replacement_pairs if replacement_pairs else [("-", ""), (" ", "")]
@@ -82,6 +83,7 @@ def __init__(
             patterns=patterns,
             context=context,
             supported_language=supported_language,
+            name=name,
         )
 
     def validate_result(self, pattern_text: str) -> bool:

presidio-analyzer/presidio_analyzer/predefined_recognizers/country_specific/korea/kr_passport_recognizer.py

@@ -18,6 +18,12 @@ class KrPassportRecognizer(PatternRecognizer):
     the previous passport number format:
         - one letter 'M' or 'm' or 'S' or 's' or 'R' or 'r' or 'O' or 'o' or 'D' or 'd'
         - eight digits
+
+    :param patterns: List of patterns to be used by this recognizer
+    :param context: List of context words to increase confidence in detection
+    :param supported_language: Language this recognizer supports
+    :param supported_entity: The entity this recognizer can detect
+    :param name: Name of the recognizer
     """
 
     COUNTRY_CODE = "kr"
@@ -48,8 +54,9 @@ def __init__(
         self,
         patterns: Optional[List[Pattern]] = None,
         context: Optional[List[str]] = None,
-        supported_language: str = "kr",
+        supported_language: str = "ko",
         supported_entity: str = "KR_PASSPORT",
+        name: Optional[str] = None,
     ):
         patterns = patterns if patterns else self.PATTERNS
         context = context if context else self.CONTEXT
@@ -58,4 +65,5 @@ def __init__(
             patterns=patterns,
             context=context,
             supported_language=supported_language,
+            name=name,
         )

presidio-analyzer/presidio_analyzer/predefined_recognizers/country_specific/us/us_mbi_recognizer.py

@@ -90,6 +90,7 @@ def __init__(
         context: Optional[List[str]] = None,
         supported_language: str = "en",
         supported_entity: str = "US_MBI",
+        name: Optional[str] = None,
     ):
         patterns = patterns if patterns else self.PATTERNS
         context = context if context else self.CONTEXT
@@ -98,4 +99,5 @@ def __init__(
             patterns=patterns,
             context=context,
             supported_language=supported_language,
+            name=name,
         )

presidio-analyzer/presidio_analyzer/predefined_recognizers/third_party/azure_ai_language.py

@@ -25,6 +25,8 @@ def __init__(
         ta_client: Optional["TextAnalyticsClient"] = None,
         azure_ai_key: Optional[str] = None,
         azure_ai_endpoint: Optional[str] = None,
+        name: Optional[str] = None,
+        context: Optional[List[str]] = None,
     ):
         """
         Wrap the PII detection in Azure AI Language.
@@ -36,15 +38,24 @@ def __init__(
         the client will be created using the key and endpoint.
         :param azure_ai_key: Azure AI for language key
         :param azure_ai_endpoint: Azure AI for language endpoint
+        :param name: Name of the recognizer. Defaults to "Azure AI Language PII".
+        :param context: List of context words to boost confidence score.
 
         For more info, see https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/overview
         """
 
+        # `name` and `context` are appended last so existing positional callers
+        # are unaffected. They exist because RecognizerListLoader passes both
+        # when building a recognizer from a registry configuration, and
+        # RemoteRecognizer.__init__ accepts neither **kwargs nor unknown keys.
+        # Without them, any YAML entry naming this class raises TypeError before
+        # the registry finishes loading (#1457, and again after #1800).
         super().__init__(
             supported_entities=supported_entities,
             supported_language=supported_language,
-            name="Azure AI Language PII",
+            name=name if name else "Azure AI Language PII",
             version="5.2.0",
+            context=context,
         )
 
         is_available = bool(TextAnalyticsClient)

presidio-analyzer/tests/conf/test_azure_ai_language_reco.yaml

@@ -1,9 +1,15 @@
 recognizer_registry:
   global_regex_flags: 26
   recognizers:
+    # Plain form: `name` doubles as the class selector and is also passed to the
+    # constructor. This is the entry shape from #1457.
     - name: MockAzureAiLanguageRecognizer
       type: predefined
-      ta_client: "test" # This is a placeholder for testing purposes
+    # Rename form: `class_name` selects the class, `name` becomes the instance
+    # name. This is the documented reason the loader passes `name` at all.
+    - name: Azure AI Language PII
+      class_name: MockAzureAiLanguageRecognizer
+      type: predefined
 
 
 supported_languages:
@@ -15,4 +21,4 @@ nlp_configuration:
   models:
     -
       lang_code: en
-      model_name: en_core_web_lg
+      model_name: en_core_web_lg

presidio-analyzer/tests/test_analyzer_engine_provider.py

@@ -1,5 +1,7 @@
 # ruff: noqa: D103,D205,E501,F541,F841,W293
 
+import importlib.util
+import os
 import re
 from pathlib import Path
 from typing import List
@@ -17,6 +19,27 @@
     StanzaRecognizer,
 )
 
+def _has_module(name: str) -> bool:
+    # find_spec imports the parent packages of a dotted name, so a missing
+    # intermediate raises ModuleNotFoundError instead of returning None --
+    # "azure.health.deidentification" does exactly that without the ahds extra.
+    try:
+        return importlib.util.find_spec(name) is not None
+    except (ImportError, ValueError):
+        return False
+
+
+# Gate the optional-dependency tests below on the module each one actually
+# imports. `pytest.importorskip` cannot be used for this: evaluated inside a
+# decorator it runs at module import, so a missing extra takes the whole file
+# out of collection instead of skipping the two tests that need it.
+#
+# The bare `azure` name is also too coarse. `azure.*` are namespace packages and
+# the `ahds` extra populates them too, so `import azure` can succeed while
+# `azure.ai.textanalytics` is absent.
+_HAS_TEXT_ANALYTICS = _has_module("azure.ai.textanalytics")
+_HAS_HEALTH_DEID = _has_module("azure.health.deidentification")
+
 
 def get_full_paths(analyzer_yaml, nlp_engine_yaml=None, recognizer_registry_yaml=None):
     this_path = Path(__file__).parent.absolute()
@@ -241,9 +264,15 @@ def test_analyzer_engine_provider_with_files_per_provider():
 
 
 @pytest.mark.skipif(
-    pytest.importorskip("azure"), reason="Optional dependency not installed"
-)  # noqa: E501
-def test_analyzer_engine_provider_with_azure_ai_language():
+    not _HAS_TEXT_ANALYTICS, reason="azure-ai-language extra not installed"
+)
+def test_analyzer_engine_provider_with_azure_ai_language(monkeypatch):
+    # The constructor builds a client when none is injected, falling back to
+    # these variables. Nothing here reaches the network: the SDK client is
+    # constructed locally and `analyze` is overridden below.
+    monkeypatch.setenv("AZURE_AI_KEY", "test-key")
+    monkeypatch.setenv("AZURE_AI_ENDPOINT", "https://example.invalid/")
+
     analyzer_yaml, _, _ = get_full_paths(
         "conf/test_azure_ai_language_reco.yaml",
     )
@@ -261,17 +290,27 @@ def analyze(
 
     analyzer_engine = provider.create_engine()
 
-    azure_ai_recognizers = [
-        rec
+    names = {
+        rec.name
         for rec in analyzer_engine.registry.recognizers
-        if rec.name == "Azure AI Language PII"
-    ]
+        if isinstance(rec, MockAzureAiLanguageRecognizer)
+    }
 
-    assert len(azure_ai_recognizers) == 1
+    # The plain entry takes its name from the YAML key; the `class_name` entry
+    # takes the name configured next to it. Both require the constructor to
+    # accept `name`, which is what regressed after #1800.
+    assert names == {"MockAzureAiLanguageRecognizer", "Azure AI Language PII"}
 
     assert len(analyzer_engine.analyze("This is a test", language="en")) > 0
 
-@pytest.mark.skipif(pytest.importorskip("azure"), reason="Optional dependency not installed") # noqa: E501
+
+# AzureHealthDeidRecognizer.__init__ builds a client when none is passed, and
+# that path reads AHDS_ENDPOINT and raises ValueError without it. The YAML entry
+# supplies no client, so the endpoint is as much a precondition as the package.
+@pytest.mark.skipif(
+    not _HAS_HEALTH_DEID or not os.getenv("AHDS_ENDPOINT"),
+    reason="ahds extra not installed or AHDS_ENDPOINT not set",
+)
 def test_analyzer_engine_provider_with_ahds():
     analyzer_yaml, _, _ = get_full_paths(
         "conf/test_ahds_reco.yaml",

presidio-analyzer/tests/test_azure_ai_language_recognizer.py

@@ -87,3 +87,39 @@ def test_mocked_entities_match_recognizer_results():
         assert expected.offset == actual.start
         assert expected.length == actual.end - actual.start
         assert expected.confidence_score == actual.score
+
+
+def _mock_ta_client():
+    try:
+        importlib.import_module("azure.ai.textanalytics")
+    except ImportError:
+        pytest.skip("Skipping test because 'azure.ai.textanalytics' is not installed")
+
+    from azure.ai.textanalytics import TextAnalyticsClient
+    from azure.core.credentials import AzureKeyCredential
+
+    return TextAnalyticsClient(endpoint="", credential=AzureKeyCredential(key=""))
+
+
+def test_name_defaults_to_the_display_name():
+    recognizer = AzureAILanguageRecognizer(ta_client=_mock_ta_client())
+    assert recognizer.name == "Azure AI Language PII"
+
+
+def test_name_can_be_overridden():
+    # RecognizerListLoader passes `name` for every entry it builds, so the
+    # constructor has to accept it; a `class_name` entry relies on it winning
+    # over the hardcoded display name.
+    recognizer = AzureAILanguageRecognizer(
+        ta_client=_mock_ta_client(), name="Custom Azure PII"
+    )
+    assert recognizer.name == "Custom Azure PII"
+
+
+def test_context_is_accepted():
+    # The kwarg #1457 originally failed on. RemoteRecognizer takes it, so the
+    # subclass has to forward it rather than reject it.
+    recognizer = AzureAILanguageRecognizer(
+        ta_client=_mock_ta_client(), context=["patient", "record"]
+    )
+    assert recognizer.context == ["patient", "record"]