Skip to content

feat: provider extension points, ProviderFulfilment contract, auth security fixes#253

Merged
fgogolli merged 154 commits into
finos:mainfrom
fgogolli:feat/provider-extension-points
Jun 22, 2026
Merged

feat: provider extension points, ProviderFulfilment contract, auth security fixes#253
fgogolli merged 154 commits into
finos:mainfrom
fgogolli:feat/provider-extension-points

Conversation

@fgogolli

@fgogolli fgogolli commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Description

Adds provider extension points so new providers (Azure, GCP, OCI, etc.) can be added cleanly without leaking provider-specific knowledge into shared services. Scope grew during review: the branch now also includes an auth subsystem refactor with typed sub-configs, four security fixes found during an auth audit, AWS error detail surfacing, provider registration consolidation, a ProviderFulfilment domain contract that fixes weighted-fleet status semantics, an MCP server bootstrap fix, several live-test scheduler/wire-format corrections, and CI/test infrastructure improvements.

Motivation: In-flight provider PRs (#234 OCI, plus #256/#257/#258 — the Azure + GCP work originally proposed in #240, now split into a common-scaffold PR plus one per provider) were each touching 50+ shared files because main lacked proper extension points. This PR adds those extension points and tightens the seams so new provider PRs are small and reviewable.

After this lands, adding a new provider touches: 1 line in _REGISTERED_PROVIDERS + providers/<name>/ package + SDK deps.


Extension points added (8)

  1. ProviderRegistration.default_api — provider declares its default API at registration; ProviderRegistry.get_default_api() exposes it; TemplateDefaultsService delegates instead of reading provider-internal config directly. Removes hardcoded "aws" defaults from api/routers/templates.py.
  2. CLISpecRegistry enforcement — cli/args.py, init_command_handler.py, infrastructure_command_handler.py, provider_config_handler.py all drive provider flags/display/config from the registry. No more hardcoded --aws-profile/--aws-region lines per consumer.
  3. TemplateExtensionRegistryTemplateDTO.provider_config: BaseModel | None typed field populated via the registry. Replaces the getattr(template, "fleet_type", None) string-literal pattern in from_domain. AWS-specific fields move into AWSTemplateDTOConfig.
  4. OperationOutcome typed discriminated union (Accepted | Completed | RequiresFollowUp | Failed). Replaces stringly-typed provider_metadata keys. Provider strategies declare actual operation state explicitly; orchestrator pattern-matches with assert_never() for exhaustiveness.
  5. AuthRegistry enforcement — iam/cognito strategies now registered by the AWS provider in providers/aws/registration.py; api/server.py dispatch is uniform auth_registry.get_strategy(...) with no provider-specific imports.
  6. FieldMappingRegistry for HostFactory scheduler — per-provider field-name mappings, apply_defaults hook, and derive_attributes (cpu/ram lookup) hook. Removes AWS-specific imports from shared scheduler infrastructure.
  7. DefaultsLoaderRegistry — provider defaults loaded via registered loaders; config/loader.py iterates instead of hardcoded dispatch.
  8. ProviderFulfilment contract (NEW) — ProviderPort.check_hosts_status returns CheckHostsStatusResult with ProviderFulfilment(state, message, target_units, fulfilled_units, …). Every provider computes fulfilment in its own API terms (RunInstances: instance count; EC2Fleet/SpotFleet: weighted capacity vs target; ASG: weighted InService sum vs DesiredCapacity). Application's request_status_service trusts the provider verdict — no count math, no AWS-specific keys leak into shared code. Future providers (GCP MIG / Azure VMSS / OCI Pool) plug in identically.

Auth subsystem refactor

  • IAMAuthSubConfig, CognitoAuthSubConfig, BearerTokenAuthSubConfig, ProviderAuthSubConfig — Pydantic models replace bare dicts for auth configuration.
  • from_auth_config(cls, auth_config) classmethod on each strategy — single, typed construction path.
  • AuthRegistry.get_strategy(name, auth_config) unified signature.
  • Removed _build_strategy_kwargs dispatch block from api/server.py.
  • register_aws_auth_strategies() now reachable from bootstrap (was dead code).
  • Registered EnhancedBearerTokenStrategy as bearer_token_enhanced.
  • 22 new auth unit tests (IAM, Cognito, EnhancedBearer).

Security fixes (4 critical/high)

These findings came out of an auth security review; the four issues below are fixed in this PR. Remaining hardening items (IAM admin role ARN allowlist, SecretStr for keys, jti claim, CORS defaults) are tracked in a follow-up epic.

Severity Issue Fix
Critical Cognito strategy broken_get_public_key returned a raw JWK dict; PyJWT could not decode it Convert JWK to RSA public key via cryptography lib
High IAM assume_permissions privilege bypass — config flag granted all permissions without any AWS check Now requires ORB_IAM_ASSUME_PERMISSIONS_DEV_ONLY=true env var; logs CRITICAL when active
Medium EnhancedBearer key-length error inconsistency — logged as warning instead of raising Raises ConfigurationError, consistent with base strategy
Medium Algorithm allowlist not enforced — Pydantic algorithm field accepted arbitrary strings including none Pydantic validator restricts to {HS256, HS384, HS512} with explicit none rejection

Status / fulfilment fixes (post-review)

Area Fix
Acquire status (weighted fleets) Old code compared running_count to request.requested_count for fleets, but for EC2Fleet/SpotFleet/ASG with WeightedCapacity, requested_count is in capacity UNITS, not instance count. Each provider now emits its own fulfilment verdict — request_status_service consumes only the typed ProviderFulfilment.state and never inspects AWS metadata keys.
Return status _update_request_to_terminating writes IN_PROGRESS until all instances actually reach terminated. Combined with terminal-state guard in update_request_status and a query-time short-circuit, prevents premature COMPLETED that drops machines from the response.
Weighted-capacity decrement on release ASG releases now use terminate_instance_in_auto_scaling_group(ShouldDecrementDesiredCapacity=True) per instance — DesiredCapacity decrements by WeightedCapacity, not by 1. EC2Fleet/SpotFleet releases subtract sum(WeightedCapacity) from TotalTargetCapacity via modify_fleet / modify_spot_fleet_request. Symmetric with weighted scale-up.
GetRequestHandler cache-only-when-terminal The 300s response cache previously stored in_progress snapshots, so every subsequent poll for the same request was answered from stale cache and never reached the read-through sync. Cache now only stores entries when request.status.is_terminal() — non-terminal polls always hit the sync. Was masking ~50 live-test polling failures.
check_hosts_status async offload check_hosts_status (synchronous boto3 calls) is now dispatched via asyncio.to_thread from the strategy, preventing the event loop from blocking during provider-status polls under load.
ListReturnRequestsHandler read-through sync Mirrors the pattern in GetRequestHandler / ListActiveRequestsHandler. Without it, callers polling list_return_requests saw IN_PROGRESS forever. CLI has no background sync, so every poll IS the sync.
Query fallback preserves db_machines If the read-through sync throws, GetRequestHandler previously returned an empty machine list. Now returns the persisted machines.
MCP server bootstrap OpenResourceBrokerMCPServer(app=None) never initialised the DI container; every MCP request failed dispatch with "No strategy found for provider". Fixed by bootstrapping Application in both handle_mcp_serve and the live test fixture.
REST template_id KeyError Live test hard-coded snake_case; HF scheduler emits camelCase per IBM Symphony spec. Test now branches on test_case["overrides"]["scheduler"] to use the right key — every scheduler has a single wire format and tests must match it precisely.
Return-completion poll SDK/MCP polled list_return_requests and expected per-request status, but the HF wire format flattens to {machine, gracePeriod} per spec. Both tests now poll get_request_status(return_id) — the correct cross-scheduler API for "is this request complete?".

CI / infrastructure / cleanup

  • lxml bumped to >=6.1.0 (CVE-2026-41066).
  • cryptography bumped to >=46.0.5 (CVE-2026-26007).
  • starlette bumped to >=0.49.1 (GHSA-82w8-qh3p-5jfq).
  • Makefile: test-providers-aws-live now uses --extra api so the venv has uvicorn for the REST server subprocess.
  • Makefile: PYTEST_WORKERS capped at half-CPU (was unbounded; OOM'd dev laptops during live runs).
  • .gitignore: ignore live test artifact directory tests/providers/aws/live/run_templates/; drop obsolete tests/onaws/ rule.
  • Auto-format suggester via reviewdog/action-suggester on pull_request_target so fork PRs get ruff suggestions without write tokens.
  • Ephemeral port assignment for ORB REST server subprocess in live tests (was hard-coded :8000).
  • Reorganised tests/onaws/tests/providers/aws/live/, tests/onmoto/tests/providers/aws/moto/.
  • Live-test boto3 sessions resolve profile_name from ORB_CONFIG_DIR/config.json + AWS_PROFILE env-var fallback so pytest-env's injected fake AWS_ACCESS_KEY_ID=testing cannot mask real credentials.
  • EC2Fleet cleanup forces modify_fleet(TotalTargetCapacity=0) before delete_fleets so maintain/request/instant fleet types delete cleanly.
  • All shipped template examples raise maxNumber default to 100.

Provider PRs this unblocks


Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change
  • Documentation update
  • Performance improvement
  • Code cleanup or refactor
  • Dependencies update (CVE bumps; no new direct deps)
  • CI/CD or build process changes

Related Issues

N/A

How Has This Been Tested?

  • Unit tests added/updated
  • Integration tests added/updated
  • Live AWS smoke (single-test runs per provider type; full suite must be run separately)

Verification (this branch HEAD):

  • uv run pyright src/: 0 errors
  • uv run ruff format --check . / uv run ruff check --select W,F,I --ignore E501: clean
  • uv run pytest --no-cov -q tests/unit tests/providers/aws/unit tests/providers/aws/moto: 5832 unit + 410 integration pass; targeted moto suite green (1 pre-existing failure in test_request_status_capacity::test_return_request_completed_when_all_terminated — wrong expectation in pre-existing test, scheduled for follow-up; 4 other pre-existing failures unrelated to this PR's scope and pre-date this branch on finos/main).
  • New return-machines status test: status stays IN_PROGRESS while instances are shutting-down; transitions to COMPLETED only when all reach terminated.
  • New auth unit tests: 22 tests covering IAM, Cognito, EnhancedBearer.
  • New provider-fulfilment unit tests + per-handler fulfilment tests for RunInstances, EC2Fleet, SpotFleet, ASG.
  • Live AWS suite (tests/providers/aws/live/) — NOT run in CI; run separately with make test-providers-aws-live.

Test Configuration

  • Python version: 3.12 (project targets >=3.10)
  • OS: Darwin 25.3.0 arm64
  • AWS region (live): eu-west-2

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published
  • I have updated the CHANGELOG.md file (semantic-release manages this)
  • I have updated the version number (semantic-release manages this)

Additional Notes

  • Each commit is atomic and independently reviewable; suggested merge order matches git log --oneline finos/main..HEAD --reverse. Will rebase + group commits before merge if requested.
  • Provider-aware Make targets added: make test-no-live, make test-providers-aws[-unit|-moto|-live], make test-architecture.
  • New extension points are reused: the AWS provider registers into CLISpecRegistry, TemplateExtensionRegistry, AuthRegistry, FieldMappingRegistry, DefaultsLoaderRegistry, ProviderRegistry, TemplateAdapterPort, TemplateExampleGeneratorPort, plus the new ProviderPort.check_hosts_status → CheckHostsStatusResult contract.
  • Follow-up epic tracks remaining items: response-formatting consolidation across SDK/REST/CLI/MCP (single port instead of three patterns), additional auth methods (API key, OIDC, service-account JWT, mTLS, password-credential), RBAC enforcement, remaining security hardening, AWS error detail UX polish, provider packaging via pip extras.

Performance Impact

  • No significant performance impact

asyncio.to_thread for blocking persistence and asyncio.timeout for operation dispatch are correctness fixes, not performance changes. Auth registry lookups and provider registration are dict operations with negligible cost. ListReturnRequestsHandler read-through sync now triggers a provider describe per poll, mirroring the existing pattern in ListActiveRequestsHandler — same cost shape.

Security Considerations

  • Security improved

Four auth security issues fixed (see security fixes table above): broken Cognito JWK decoding, IAM assume_permissions privilege bypass, key-length error inconsistency in EnhancedBearer, and missing algorithm allowlist. Plus CVE bumps for cryptography, Starlette, and lxml.

Dependencies

No new runtime dependencies. cryptography is already a transitive dependency via PyJWT; this PR uses it directly in the Cognito strategy for JWK-to-RSA-public-key conversion. CVE-driven version bumps to cryptography, Starlette, lxml.

Migration Guide

For ORB operators: No action required. Existing AWS-only deployments continue to work unchanged.

For provider authors (in-flight PRs):

  1. Rebase onto main after this lands.
  2. Remove provider-specific edits in shared files (CLI args, config loader, infrastructure command handler, field mappings, scheduler, template handlers).
  3. Add 1 line to _REGISTERED_PROVIDERS in providers/registration.py.
  4. Register your provider via the new extension points in providers/<name>/registration.py. See docs/root/developer_guide/adding_a_provider.md.
  5. Replace stringly-typed status logic with OperationOutcome variants from your strategy AND have check_hosts_status return CheckHostsStatusResult(instances, fulfilment) with provider-computed fulfilment.

Test runners: --run-aws continues to work (aliased to --live); prefer --live going forward. Live AWS tests run via make test-providers-aws-live (note: uses --extra api so uvicorn is installed for the REST server subprocess).

Reviewers

@finos/open-resource-broker-maintainers

@fgogolli fgogolli requested a review from a team as a code owner June 15, 2026 17:10
@fgogolli fgogolli force-pushed the feat/provider-extension-points branch 2 times, most recently from 047e784 to 8a683c7 Compare June 15, 2026 21:30
@fgogolli fgogolli changed the title feat: provider extension points and DTO/outcome typing feat: provider extension points, ProviderFulfilment contract, auth security fixes Jun 16, 2026
@fgogolli fgogolli force-pushed the feat/provider-extension-points branch from 62374a4 to 5dae11d Compare June 17, 2026 13:15
@codecov-commenter

codecov-commenter commented Jun 17, 2026

Copy link
Copy Markdown

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

@fgogolli fgogolli force-pushed the feat/provider-extension-points branch 3 times, most recently from 3cadf35 to a05feb8 Compare June 18, 2026 14:23
fgogolli added 21 commits June 22, 2026 12:18
Create orb.providers.aws.value_objects as the canonical public import
path for AWSAllocationStrategy and its helpers. Update callers in
ec2_fleet handler, AWSTemplate aggregate, and tests to import from the
new shallow path rather than the internal domain sub-package.

The class definition stays in providers/aws/domain/template/value_objects;
value_objects.py re-exports it to establish a stable provider-level API.
Wrap execute_operation in asyncio.timeout(dispatch_timeout_seconds)
to prevent unbounded awaits when a provider hangs.  Run
_persist_acquiring via asyncio.to_thread so the blocking DB write
does not hold the event loop between retry attempts.

dispatch_timeout_seconds is read from request config (key
dispatch_timeout_seconds) and defaults to 300 s.  TimeoutError is
caught and returned as a final failure result so the retry loop
can record the outcome cleanly.
- Delete validate_provider_availability (zero callers; bootstrap
  moved to provider_services.py)
- Remove unused ConfigurationPort import from provider_validation_service
- Inject ProviderValidationService via constructor in
  CreateMachineRequestHandler instead of constructing a second
  instance locally, so the registered singleton is used
- Replace hardcoded ensure_provider_type_registered("aws") in
  _load_strategy_defaults with pkgutil discovery of all provider
  subpackages that carry a registration module
- Update tests to pass provider_validation_service as a constructor
  argument and configure AsyncMock return values appropriately
- Add orb.providers to the architecture leak-detection whitelist
  for config/loader.py (intentional bootstrap wiring)
Register IAMAuthStrategy and CognitoAuthStrategy in
providers/aws/registration.py via register_aws_auth_strategies(),
called from initialize_aws_provider().

Remove provider-specific if/elif branches and inline imports
from api/server.py. _create_auth_strategy now delegates uniformly
to AuthRegistry.get_strategy() for all strategy names. Config
extraction is centralised in _build_strategy_kwargs().

No orb.providers.aws.* imports remain in api/server.py.
- Replace `if not registered_types else "aws"` fallback with
  RuntimeError when no providers are registered in
  infrastructure_command_handler._get_active_providers (both
  no-config and all-disabled branches)
- Replace hardcoded Region/Profile display in
  _show_provider_infrastructure with CLISpecRegistry.format_display;
  fall back to raw key/value for unknown provider types
- Replace `getattr(args, "provider_type", "aws")` default in
  handle_provider_add with an explicit error when provider_type is
  absent
- Replace `provider.get("type", "aws")` default in
  handle_provider_update with `provider.get("type") or ""`; CLISpec
  lookup returns None for empty string, handled by existing fallback
- Add unit tests for all changed behaviours
Replace hardcoded --aws-profile/--aws-region in providers add
and providers update subcommands with registry iteration:
CLISpecRegistry.all().values() → spec.add_arguments(parser).

Add register_all_provider_cli_specs() to providers/registration.py
as a lightweight bootstrap (no full strategy init) that build_parser
calls before iterating the registry, so provider-specific flags are
available before any application context exists.

Inject per-provider CLI flags into the init subparser via the same
registry loop, enabling _get_default_config to use spec.extract_config()
instead of hardcoded {"profile": args.profile, "region": args.region},
with fallback to init-level flags for backward compatibility.

Update architecture known-violation lists to whitelist the intentional
cli/args.py → orb.providers.registration import.
Add default_api field to ProviderRegistration so each provider
can declare its default API name at registration time. AWS reads
the value from aws_defaults.json. ProviderRegistry.get_default_api
exposes it via ProviderRegistryPort.

TemplateDefaultsService delegates the provider_api fallback to the
registry instead of reading handlers.defaults.default_handler directly,
removing the AWS-specific config path from the application layer.

Remove hardcoded "aws" default from TemplateCreateRequest.provider_api
and the templates router create path; callers must supply provider_api
or the command handler will reject the request explicitly.
Remove AWS-specific getattr blocks and top-level launch_template_id
from TemplateDTO. Add provider_config: BaseModel | None populated via
TemplateExtensionRegistry.create_extension_config so infra layer stays
provider-agnostic. AWSTemplateDTOConfig captures fleet_type, fleet_role,
percent_on_demand, launch_template_id, abis_instance_requirements.
AWSTemplate.validate_aws_template promotes from provider_config dict for
round-trip compatibility via model_dump. Tests updated accordingly.
Introduce per-provider field-mapping extension points in the
HostFactory scheduler so AWS-specific logic no longer lives in
shared infrastructure:

- Add FieldMappingPort protocol: get_mappings(), apply_defaults(),
  derive_attributes()
- Add FieldMappingRegistry (simple class-variable dict, same pattern
  as CLISpecRegistry)
- Add AWSFieldMapping adapter (providers/aws/scheduler/) that owns
  the AWS field-name dict, apply_aws_defaults logic, and the
  derive_cpu_ram_from_instance_type call
- Remove apply_aws_defaults from HostFactoryFieldMappings (shared
  class) and the unconditional call in hostfactory_strategy.py;
  replace with registry.get(provider_type).apply_defaults(mapped)
- Remove direct orb.providers.aws.utilities.ec2.instances imports
  from field_mapper.py and hostfactory_strategy.py; both now
  delegate to registry.get(provider_type).derive_attributes()
- Change field_mapper.py provider_type default from "aws" to None
- Register AWSFieldMapping in initialize_aws_provider()
- Remove stale known-violations from test_provider_leak_detection
Replace the stringly-typed is_final boolean with an explicit
Accepted | Completed | RequiresFollowUp | Failed discriminated
union.  ProvisioningResult.outcome is set on every return path;
is_final is derived from it via __post_init__ for backward
compatibility.  BaseProviderStrategy gains three abstract async
methods (acquire, return_machines, get_status) that AWSProvider-
Strategy implements, correctly modelling AWS async-acceptance
semantics.  assert_never() guards all match exhaustion points.
Add ProviderDefaultsLoaderPort protocol, DefaultsLoaderRegistry
(class-var dict, same pattern as CLISpecRegistry), and AWSDefaultsLoader
that reads aws_defaults.json via importlib.resources.

Register AWS loader in initialize_aws_provider(). Refactor
ConfigurationLoader._load_strategy_defaults to iterate the registry
instead of reflecting over pkgutil.iter_modules.
TerminateInstances is accepted asynchronously — instances enter
shutting-down, not terminated.  Writing COMPLETED immediately was a lie
that caused ReturnMachinesOrchestrator to exit while machines were still
running.  Replace _update_request_to_completed with
_update_request_to_terminating (IN_PROGRESS) so background sync can poll
and transition to COMPLETED only when all instances reach terminated.

Also adds machine_ids to ReturnMachinesOutput so callers see which
machines were submitted for return.
Remove has_extension guards from _get_extension_defaults and
_get_provider_instance_extension_defaults — get_extension_defaults
already returns {} for unknown providers so the checks are redundant.

Remove hasattr/getattr dynamic validate_{provider_type} dispatch from
validate_template_with_extensions — no Template subclass in the
codebase has a validate_aws() or similar method, making the block
unreachable dead code. Provider-specific validation belongs in the
extension registry, not on template objects.
Move provider-specific tests under tests/providers/<name>/:
- tests/unit/providers/aws/** -> tests/providers/aws/unit/
- tests/onmoto/ -> tests/providers/aws/moto/
- tests/onaws/ -> tests/providers/aws/live/
- tests/unit/providers/test_aws_handlers.py -> tests/providers/aws/unit/

Cross-provider files remain in tests/unit/providers/.

Add provider-aware pytest flags to tests/conftest.py:
- --live (replaces --run-aws; backward-compat alias kept)
- --no-mocked (skip moto subtree)
- --provider <name> (filter to one provider)

Add tests/providers/aws/conftest.py consolidating moto fixtures.
Thin moto/conftest.py re-exports helpers for existing imports.
Live conftest pre-flight gated behind --live flag.

Update makefiles/common.mk TESTS_ONMOTO to new path.
Update pyproject.toml norecursedirs to tests/providers/aws/live.
FollowUpContext types are pure frozen value objects with literal
discriminants — domain concepts, not application concerns.  Moving
them to orb.domain.base.follow_up_context eliminates the
domain→application import violation introduced alongside
OperationOutcome.

orb.application.services.request_follow_up_context is kept as a
re-export shim for backward compatibility.
When _load_strategy_defaults is called before a full DI bootstrap
(e.g. bare config loading or isolated tests), DefaultsLoaderRegistry
is empty because AWSDefaultsLoader is only registered inside
initialize_aws_provider().

Add register_all_defaults_loaders() to orb.providers.registration
(same lightweight pattern as register_all_provider_cli_specs()) and
call it from _load_strategy_defaults when the registry has no entries.
- Remove provider_config_handler aws_region/aws_profile fallback;
  require CLISpec registration for known provider types
- Document HostFactoryFieldMappings.MAPPINGS["aws"] as bootstrap-free
  fallback; canonical AWS mappings live in AWSFieldMapping adapter
- Update onaws → providers/aws/live path stragglers in dev-tools and
  contract test docstrings; keep test_onaws.py target intact
- Add provider-aware Make targets: test-no-live, test-providers,
  test-providers-aws[-unit|-moto|-live], test-architecture
- Whitelist config/loader.py → orb.providers.registration as known
  bootstrap-only import in provider-leak detection
…stry test

test_aws_extension_registration called clear_registry() and registered
AWSTemplateExtensionConfig against the "aws" key, but never restored the
registry.  The production bootstrap registers AWSTemplateDTOConfig against
"aws"; after the test ran, subsequent tests saw AWSTemplateExtensionConfig
instead, causing TemplateDTO.from_domain to produce empty provider_config
for all AWS templates — 147 failures across moto and unit test suites.

Wrap the mutation in try/finally and snapshot-restore both _extensions and
_extension_instances dicts so the registry is identical before and after.
When tests/onmoto/conftest.py was migrated to tests/providers/aws/
conftest.py, the handler construction was accidentally changed from
using cfg_port (the container-resolved ConfigurationPort) to the
config_port parameter (which callers pass as None).  Handlers such
as RunInstancesHandler require a non-None config_port to build launch
parameters; with None they raised AWSConfigurationError, the acquire
returned FAILED, and HostFactory mapped that to "complete_with_error"
— a value not in RequestStatus that wait_for_request never treats as
terminal, causing a 30-second timeout.

Fix: prefer cfg_port when config_port is None, matching the original
onmoto/conftest.py behaviour.
fgogolli added 15 commits June 22, 2026 12:19
…efaults in remaining test files

Pyright misreports AuthConfig(strategy="...") and BearerTokenAuthSubConfig(secret_key=...)
as missing required fields even though all other fields carry Field(...) defaults with
extra="forbid" set. This is a known pyright/pydantic-v2 interaction.

Applied the same # type: ignore[call-arg] suppression already used across the codebase
for identical false-positives.
… infrastructure layer

Per clean architecture rules, mutable global registry state is an
infrastructure concern, not a domain concern.  Aligns with the sibling
DefaultsLoaderRegistry and FieldMappingRegistry, which already live under
infrastructure/registry/.

Protocol interfaces (ProviderCLISpecPort, TemplateExtension) remain in
the domain layer; only the registry implementations move.

Updates the architecture violation counts to reflect the lower number of
domain-layer infrastructure leaks after the move.
ProvisioningResult is shared across providers; AWS-specific nouns on its
field names leaked provider details into application-layer types. The AWS
extractor still populates these fields; the names no longer encode the
source provider.
…acity_based_fulfilment

The parameter was declared in the signature and documented in the
docstring but never referenced in the function body. Both call sites
(ec2_fleet and spot_fleet handlers) already omit it via keyword args,
so removal requires no changes at the call sites.
Replace Any-typed request and status params with concrete types
(Request and RequestStatus). Lift both imports to module level and
remove the five deferred per-method imports of RequestStatus that
were scattered across _update_request_to_in_progress,
_execute_deprovisioning_for_request, _update_request_to_failed,
_update_request_to_terminating, and _update_request_to_completed.
Also remove the now-redundant deferred Request import inside
execute_command and the deferred RequestStatus import in
_handle_dry_run.
…EADME

The example showed `from src.domain.template.extensions import ...`
which had two problems: the `src.` prefix was never valid (the package
root is `orb`), and the class moved to
`orb.infrastructure.registry.template_extension_registry`. Update to
the correct canonical path.
…emove infra import

application layer now depends on TemplateExtensionRegistryPort (Protocol) defined in
the domain layer. Infrastructure registry remains; an adapter satisfies the port via
DI. Removes the only application→infrastructure registry import; aligns with the
DefaultsLoaderPort / FieldMappingPort pattern.

- Add TemplateExtensionRegistryPort (Protocol) to domain/template/ports/
- Add TemplateExtensionRegistryAdapter in infrastructure/registry/ wrapping classmethods
- TemplateDefaultsService constructor accepts TemplateExtensionRegistryPort (Optional)
- DI factory injects TemplateExtensionRegistryAdapter() at construction time
- violation_counts.json: application_forbidden_imports 6 → 5
Large ABIS scenarios with capacity=100 occasionally exceed 60s on a
single REST call because the request_machines /
get_request_status path issues many AWS describe_* calls under load.
The 2 minute ceiling gives headroom without affecting fast scenarios,
which return well under the old limit.
…der_error with read-side alias

Outer envelope key in error_details is renamed to be provider-neutral. The read side
accepts both old and new key names so persisted records written before this change keep
working. Inner AWS-specific dict keys (aws_request_id, aws_error_code, aws_error_message)
remain — they are AWS exception field names, not shared types.
…rio yields <2 instances

EC2Fleet Maintain + spot uses weighted vmTypes, so capacity_to_request=4
can be fulfilled by a single physical instance with WeightedCapacity=4.
The partial-return-reduces-capacity test needs at least 2 physical
instances to terminate one and verify the other survives.

Mirrors the existing skip in test_partial_return_terminates_instance_only
(line 2237).
@fgogolli fgogolli force-pushed the feat/provider-extension-points branch from 40eb900 to 3792bd4 Compare June 22, 2026 11:20
fgogolli added 3 commits June 22, 2026 12:24
Lock had drifted to msgpack 1.1.2 (out-of-bounds read on Unpacker reuse)
during rebase replay.  uv lock --upgrade brings transitive deps in line
with current main and clears the high-severity vulnerability.
ListRequestsHandler now eagerly instantiates RequestDTOFactory in
__init__ (factory hoist).  The previous test patched the factory after
the handler was already constructed, so the patch had no effect and
the real factory accessed attributes the SimpleNamespace mock did
not define (e.g. last_status_check).  Move the construction inside
the patch context and re-target the patch at the import site.
…return tests

EC2Fleet partial path now calls _fleet_has_no_remaining_instances, which
runs _collect_with_next_token in a NextToken loop against a MagicMock
ec2_client.  MagicMock.get(...) returns a truthy MagicMock for the
NextToken key, so the loop never terminates — the test hung indefinitely.

SpotFleet partial path has the same defensive call.  Its retry-based
variant returns a MagicMock whose .get('ActiveInstances', []) yields an
empty iterator, which falsely flags the fleet as empty and triggers an
unintended launch-template cleanup, failing the assertion.

Patch _fleet_has_no_remaining_instances to return False on both
partial-return tests so the production code skips the new defensive
check and the original partial-return semantic is exercised.

@kirillsc kirillsc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants