Merge pull request #918 from valory-xyz/feat/open-autonomy-2485-upstream-hooks

feat(api): upstream hooks for open-autonomy fetchai#2485

Author: DIvyaNautiyal07

HISTORY.md

@@ -4,6 +4,11 @@
 
 Framework:
 
+- `aea.configurations.loader`: exposes `parse_service_yaml(file_pointer) -> (head, overrides)` as a public module-level helper that parses a service YAML stream into the head document and the trailing override documents without schema validation. `ConfigLoader._load_service_config` now delegates to it. Lets downstream packagers (e.g. open-autonomy, which needs to substitute environment variables in the head document before validating it through its own `Service` config class) drop a hand-maintained copy of the parse-and-split logic. Raises `ValueError("Service configuration file was empty.")` whenever the head document is not a mapping — covers empty streams, bare `---`, and YAML scalars (`false`, `0`, `""`, `[]`) — so the failure surfaces at parse time rather than as an opaque `AttributeError` from the downstream validator.
+- `aea.cli.scaffold.scaffold_item`: saves `ctx.cwd` on entry and restores it via `try ... finally`. Previously the `--to-local-registry` branch swapped `ctx.cwd` to `<registry_path>/<author>` to make the inner `fingerprint_item` call resolve correctly, but never restored it, leaking the registry path back to callers. The post-fingerprint steps in the same function (notably the `--with-symlinks` block, which builds vendor symlinks from `ctx.cwd`) also see the original value again. Downstream wrappers that had to bracket `scaffold_item` with their own `preserve_cwd = ctx.cwd; ...; ctx.cwd = preserve_cwd` can drop the workaround.
+- `aea.contracts.base.Contract.from_config` and `aea.connections.base.Connection.from_config`: pass the canonical packages-prefixed dotted path (`packages.<author>.contracts.<name>.contract` / `packages.<author>.connections.<name>.connection`) to `load_module` instead of the bare `"contracts"` / `"connection_module"` constants. With the new `sys.modules` registration in `load_module` (below), the bare keys would have collided across contracts and across connections loaded in the same process. Caller-visible behaviour is unchanged because `load_module` always returns a fresh module object.
+- `aea.skills.base._SkillComponentLoader`: makes the dotted module path returned by `_compute_module_dotted_path` the canonical one (`packages.<author>.skills.<name>.<file>`), drops the now-redundant "exclude classes whose `__module__` starts with this skill's dotted path" condition from `_filter_classes`, and updates the unused-class warning filter at `_print_warning_message_for_unused_classes` to keep classes that belong to this skill (both submodule classes and those defined at the skill's own `__init__.py` level) instead of dropping every `packages.`-prefixed class. Cross-skill re-exports (the canonical FSM composition idiom — `class_name: AbciDialogues` in `skill.yaml` bound to an `AbciDialogues` imported from a parent skill) still pass the filter because their `__module__` starts with another skill's prefix, not `aea.`. The legacy `_parse_module` filter (used by `Behaviour.parse_module` / `Handler.parse_module` / `Model.parse_module`) drops the same local-prefix exclusion clause and switches its `load_module` call to the canonical packages-prefixed dotted path so its `sys.modules` cache entries do not collide on bare keys (`"behaviours"` / `"handlers"` / `"models"`) across skills loaded in the same process. The remaining difference vs. `_filter_classes` is the legacy filter's name-allowlist gate (`x[0] in component_names`), which is unchanged. `aea.helpers.base.load_module` now registers the loaded module in `sys.modules` under its dotted path before `exec_module` and rolls the entry back if `exec_module` raises, so (a) a subsequent ordinary `from packages.<author>.skills.<name>.<file> import X` returns the same module object instead of re-executing the file and producing a second copy of every class defined in it, and (b) a transient module-level error does not permanently poison `sys.modules` with a half-initialised stub. Together these eliminate the dual short-name / long-name class objects that downstream code (open-autonomy's `_MetaPayload.registry`) previously had to paper over with a key-rewrite loop. `_compute_module_dotted_path` is no longer a `@classmethod` (it reads `self.skill_dotted_path`) — internal API, no out-of-tree caller.
+
 - `aea.cli` / `aea.helpers.protocols`: removes the two module-top `logging.basicConfig(...)` calls that fired as a side effect of `import aea.cli` and attached a stderr `StreamHandler` to the root logger. That orphan root handler survived the agent's `logging.config.dictConfig` (the config block has no `root:` key) and caught every `aea.agent.*` record a second time via propagation, producing the `] [INFO]` vs `][INFO]` doubling seen in operator captures. Fix moves the `basicConfig` in `aea/cli/generate_all_protocols.py` into the click command body and switches `aea/helpers/protocols.py` from `setup_logger(__name__)` to `logging.getLogger(__name__)`. Behaviour change for operators: third-party library `INFO`/`DEBUG` logs (e.g. from `web3`, `requests` in the ledger plugins) that the orphan handler previously surfaced now go silent under an agent config without a `root:` block in `logging_config`. `WARNING` and above still appear on stderr via Python's `logging.lastResort` handler, though with that handler's bare-message format (just `%(message)s`, no level or logger-name prefix and no timestamp) rather than the formatted one the orphan installed. Operators who want the previous behaviour (timestamped third-party `INFO`/`DEBUG`) can add a `root:` block to `logging_config` in the agent's `aea-config.yaml`. #914 #917
 
 ## 2.2.6 (2026-05-14)

aea/cli/scaffold.py

@@ -244,6 +244,9 @@ def scaffold_item(ctx: Context, item_type: str, item_name: str) -> None:
         )
 
     ctx.clean_paths.append(str(dest))
+    # Save ctx.cwd so the local-registry path swap below does not leak out
+    # of this call.
+    preserve_cwd = ctx.cwd
     try:
         # copy the item package into the agent project.
         src = Path(os.path.join(AEA_DIR, item_type_plural, "scaffold"))
@@ -283,6 +286,12 @@ def scaffold_item(ctx: Context, item_type: str, item_name: str) -> None:
         if to_local_registry:
             ctx.cwd = str(registry_path / author_name)
             fingerprint_item(ctx, item_type, new_public_id)
+            # Restore now (not just in the `finally` below) so the
+            # `--with-symlinks` calls to `create_symlink_vendor_to_local` /
+            # `create_symlink_packages_to_vendor` further down — both of
+            # which read `ctx.cwd` — see the original agent directory
+            # instead of the registry author path.
+            ctx.cwd = preserve_cwd
         else:
             fingerprint_item(ctx, item_type, new_public_id)
 
@@ -314,6 +323,8 @@ def scaffold_item(ctx: Context, item_type: str, item_name: str) -> None:
         )
     except Exception as e:
         raise click.ClickException(str(e))
+    finally:
+        ctx.cwd = preserve_cwd
 
 
 def _scaffold_dm_handler(ctx: Context) -> None:

aea/configurations/loader.py

@@ -29,6 +29,7 @@
     List,
     Optional,
     TextIO,
+    Tuple,
     Type,
     TypeVar,
     Union,
@@ -320,11 +321,7 @@ def _load_agent_config(self, file_pointer: TextIO) -> AgentConfig:
 
     def _load_service_config(self, file_pointer: TextIO) -> PackageConfiguration:
         """Load a service configuration."""
-        configuration_data = yaml_load_all(file_pointer)
-        if not configuration_data:
-            raise ValueError("Service configuration file was empty.")
-
-        service_config, *overrides = configuration_data
+        service_config, overrides = parse_service_yaml(file_pointer)
         self.validate(service_config)
         key_order = list(service_config.keys())
         service = self.configuration_class.from_json(service_config)
@@ -384,6 +381,44 @@ def _process_component_section(
         return component_id
 
 
+def parse_service_yaml(
+    file_pointer: TextIO,
+) -> Tuple[Dict[str, Any], List[Dict[str, Any]]]:
+    """
+    Parse a service configuration YAML stream into its raw documents.
+
+    Returns the head service configuration document and any override
+    documents without applying schema validation or constructing a
+    typed configuration object. Callers that need to apply additional
+    transformations (e.g. environment variable substitution) before
+    validation can use this helper as the parse step.
+
+    :param file_pointer: an open text stream pointing at a service.yaml.
+    :return: a ``(service_config, overrides)`` tuple where ``service_config``
+        is the head document and ``overrides`` is the list of trailing
+        documents.
+    :raises ValueError: if the stream is empty, or if the head document is
+        not a YAML mapping (e.g. a bare ``---``, a scalar, or a sequence).
+    :raises yaml.YAMLError: if the stream contains malformed YAML and the underlying parser fails (propagated as-is so callers can handle parse errors and shape errors distinctly).  # noqa: DAR402
+    """
+    configuration_data = yaml_load_all(file_pointer)
+    if not configuration_data:
+        raise ValueError("Service configuration file was empty.")
+    service_config, *overrides = configuration_data
+    # The head document must be a mapping. `yaml_load_all` happily returns
+    # `[None]` / `[False]` / `[0]` / `[""]` / `[[]]` for malformed YAML
+    # (bare `---`, a scalar, a sequence, etc.) — surface those at parse
+    # time rather than as opaque AttributeErrors from the downstream
+    # validator. Distinct from the empty-stream message so an operator
+    # reading the traceback can tell the two cases apart.
+    if not isinstance(service_config, dict):
+        raise ValueError(
+            "Service configuration head document must be a YAML mapping, "
+            f"got {type(service_config).__name__}."
+        )
+    return service_config, overrides
+
+
 class ConfigLoaders:
     """Configuration Loader class to load any package type."""
 

aea/configurations/utils.py

@@ -34,9 +34,34 @@
     PublicId,
     SkillConfig,
 )
+from aea.configurations.constants import PACKAGES
 from aea.configurations.data_types import PackageIdPrefix
 
 
+def package_dotted_path(
+    author: str,
+    package_type_plural: str,
+    package_name: str,
+    file_stem: str,
+) -> str:
+    """
+    Compose the canonical dotted path string for a file inside an AEA package.
+
+    Returns a string of the form ``packages.<author>.<plural>.<name>.<stem>``.
+    Has no side effects; the loaded module is registered in
+    ``sys.modules`` by ``aea.helpers.base.load_module``.
+
+    :param author: package author (e.g. ``"valory"``).
+    :param package_type_plural: plural package-type token (``"skills"`` /
+        ``"contracts"`` / ``"connections"`` / ``"protocols"``).
+    :param package_name: the package's local name.
+    :param file_stem: source file stem without the ``.py`` suffix
+        (``"contract"`` / ``"connection"`` / ``"behaviours"`` / ...).
+    :return: the composed packages-prefixed dotted path string.
+    """
+    return f"{PACKAGES}.{author}.{package_type_plural}.{package_name}.{file_stem}"
+
+
 @singledispatch
 def replace_component_ids(
     _arg: PackageConfiguration,

aea/connections/base.py

@@ -30,8 +30,10 @@
 
 from aea.components.base import Component, load_aea_package
 from aea.configurations.base import ComponentType, ConnectionConfig
+from aea.configurations.constants import CONNECTIONS
 from aea.configurations.data_types import PublicId
 from aea.configurations.loader import load_component_configuration
+from aea.configurations.utils import package_dotted_path
 from aea.crypto.wallet import CryptoStore
 from aea.exceptions import (
     AEAComponentLoadException,
@@ -284,9 +286,13 @@ def from_config(
                 "Connection module '{}' not found.".format(connection_module_path)
             )
         load_aea_package(configuration)
-        connection_module = load_module(
-            "connection_module", directory / "connection.py"
+        dotted_path = package_dotted_path(
+            configuration.public_id.author,
+            CONNECTIONS,
+            configuration.public_id.name,
+            "connection",
         )
+        connection_module = load_module(dotted_path, directory / "connection.py")
         classes = inspect.getmembers(connection_module, inspect.isclass)
         connection_class_name = cast(str, configuration.class_name)
         connection_classes = list(

aea/contracts/base.py

@@ -30,6 +30,7 @@
 from aea.configurations.base import ComponentType, ContractConfig, PublicId
 from aea.configurations.constants import CONTRACTS
 from aea.configurations.loader import load_component_configuration
+from aea.configurations.utils import package_dotted_path
 from aea.crypto.base import LedgerApi
 from aea.crypto.registries import Registry, ledger_apis_registry, make_ledger_api_cls
 from aea.exceptions import AEAComponentLoadException, AEAException
@@ -123,7 +124,13 @@ def from_config(cls, configuration: ContractConfig, **kwargs: Any) -> "Contract"
             raise ValueError("Configuration must be associated with a directory.")
         directory = configuration.directory
         load_aea_package(configuration)
-        contract_module = load_module(CONTRACTS, directory / "contract.py")
+        dotted_path = package_dotted_path(
+            configuration.public_id.author,
+            CONTRACTS,
+            configuration.public_id.name,
+            "contract",
+        )
+        contract_module = load_module(dotted_path, directory / "contract.py")
         classes = inspect.getmembers(contract_module, inspect.isclass)
         contract_class_name = cast(str, configuration.class_name)
         contract_classes = list(filter(lambda x: x[0] == contract_class_name, classes))

aea/helpers/base.py

@@ -127,9 +127,37 @@ def load_module(dotted_path: str, filepath: Path) -> types.ModuleType:
     :raises ValueError: if the filepath provided is not a module.  # noqa: DAR402
     :raises Exception: if the execution of the module raises exception.  # noqa: DAR402
     """
+    # If the dotted path is already in ``sys.modules``, reuse it instead
+    # of re-executing the file. Re-execution creates a duplicate copy of
+    # every class defined in the source, breaking class identity for
+    # callers that already hold a reference (e.g. via an earlier
+    # ``from ... import X``). ``__file__`` is intentionally NOT compared
+    # — in test fixtures the same logical package may be reached
+    # through different physical files (vendor copy in a tmpdir vs the
+    # source tree) but both register the same dotted path and must
+    # resolve to the same module object across the whole process. An
+    # explicit ``None`` entry is CPython's block-import sentinel: raise
+    # ``ImportError`` to match the standard import semantics rather
+    # than silently overwriting it with a fresh exec.
+    if dotted_path in sys.modules:
+        existing = sys.modules[dotted_path]
+        if existing is None:
+            raise ImportError(f"import of {dotted_path!r} halted; None in sys.modules")
+        return existing
     spec = importlib.util.spec_from_file_location(dotted_path, str(filepath))
     module = importlib.util.module_from_spec(cast(ModuleSpec, spec))
-    spec.loader.exec_module(module)  # type: ignore
+    # Register before exec so a downstream `import` of the same dotted
+    # path during exec resolves to this module. We only reach this point
+    # when the key was absent (the cache-hit and block-import branches
+    # above covered the other two cases), so on exec failure we pop the
+    # half-built stub rather than restoring a prior entry that, by
+    # construction, did not exist.
+    sys.modules[dotted_path] = module
+    try:
+        spec.loader.exec_module(module)  # type: ignore
+    except BaseException:  # pylint: disable=broad-except
+        sys.modules.pop(dotted_path, None)
+        raise
     return module