Skip to content
Open
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -362,7 +362,7 @@ docker run -d --name floci \
| `FLOCI_SERVICES_RDS_DEFAULT_MYSQL_IMAGE` | `mysql:8.0` |
| `FLOCI_SERVICES_RDS_DEFAULT_MARIADB_IMAGE` | `mariadb:11` |
| `FLOCI_SERVICES_MSK_DEFAULT_IMAGE` | `redpandadata/redpanda:latest` |
| `FLOCI_SERVICES_OPENSEARCH_DEFAULT_IMAGE` | `opensearchproject/opensearch:2` |
| `FLOCI_SERVICES_OPENSEARCH_DEFAULT_IMAGE` | *(unset — images resolve per requested `EngineVersion`)* |
| `FLOCI_SERVICES_KINESIS_ANALYTICS_DEFAULT_IMAGE` | _(unset; chosen per RuntimeEnvironment)_ |
| `FLOCI_SERVICES_NEPTUNE_DEFAULT_IMAGE` | `tinkerpop/gremlin-server:3.7.3` |
| `FLOCI_SERVICES_NEPTUNE_DEFAULT_NEO4J_IMAGE` | `neo4j:5-community` |
Expand Down
2 changes: 1 addition & 1 deletion docs/configuration/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -360,7 +360,7 @@ These services spawn Docker containers. They require access to the Docker socket
|---|---|---|
| `FLOCI_SERVICES_OPENSEARCH_ENABLED` | `true` | Enable the OpenSearch service |
| `FLOCI_SERVICES_OPENSEARCH_MOCK` | `false` | When `true`, domains are created instantly without a real container (API only) |
| `FLOCI_SERVICES_OPENSEARCH_DEFAULT_IMAGE` | `opensearchproject/opensearch:2` | Docker image for OpenSearch domains |
| `FLOCI_SERVICES_OPENSEARCH_DEFAULT_IMAGE` | *(unset)* | Optional fixed Docker image for every OpenSearch domain; when unset, images resolve per requested `EngineVersion` |
| `FLOCI_SERVICES_OPENSEARCH_PROXY_BASE_PORT` | `9400` | First port in the OpenSearch proxy range |
| `FLOCI_SERVICES_OPENSEARCH_PROXY_MAX_PORT` | `9499` | Last port in the OpenSearch proxy range |
| `FLOCI_SERVICES_OPENSEARCH_KEEP_RUNNING_ON_SHUTDOWN` | `false` | Keep OpenSearch containers running when Floci stops |
Expand Down
12 changes: 9 additions & 3 deletions docs/configuration/ports.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,21 @@
| `4566` | HTTP | All AWS API calls (every service) | Yes |
| `5100–5199` | HTTP | ECR Registry sidecar — bound directly by the `registry:2` container | **No** (see note) |
| `6379–6399` | TCP | ElastiCache Redis proxy (inside Floci) | Yes |
| `6400–6419` | TCP | MemoryDB proxy (inside Floci) | Yes |
| `6500–6599` | HTTPS | EKS k3s API server — bound directly by each k3s container | **No** |
| `7001–7099` | TCP | RDS proxy (inside Floci) | Yes |
| `8182–8282` | TCP | Neptune Gremlin proxy (inside Floci) | Yes |
| `8700–8799` | HTTP | MWAA Airflow webserver proxy (inside Floci) | Yes |
| `9400–9499` | HTTP | OpenSearch data-plane — bound directly by each OpenSearch container | **No** |
| `12000–12499` | HTTP | Lambda Runtime API (internal, Docker-network only) | **No** |

## Why some ports don't need docker-compose mapping

There are two distinct patterns Floci uses to expose container ports:

### Proxy-in-Floci (ElastiCache, RDS)
### Proxy-in-Floci (ElastiCache, MemoryDB, RDS, Neptune, MWAA)

Floci runs a **TCP proxy process inside its own container**. The proxy listens on the host port and forwards traffic to the backend container.
Floci runs a **proxy process inside its own container**. The proxy listens on the host port and forwards traffic to the backend container.

```
host:6379 → [docker-compose ports mapping] → Floci container:6379 → Redis container:6379
Expand Down Expand Up @@ -147,7 +150,7 @@ host:5100 ←── floci-ecr-registry (registry:2 container, started by Floci

## Exposing Ports in Docker Compose

Only the proxy-based services (ElastiCache and RDS) need port mappings in `docker-compose.yml`. Direct-binding services (ECR, EKS, OpenSearch) bind their ports on the host automatically via Docker:
Only the proxy-based services (ElastiCache, MemoryDB, RDS, Neptune, MWAA) need port mappings in `docker-compose.yml`. Direct-binding services (ECR, EKS, OpenSearch) bind their ports on the host automatically via Docker:

```yaml
services:
Expand All @@ -156,7 +159,10 @@ services:
ports:
- "4566:4566" # All AWS API calls
- "6379-6399:6379-6399" # ElastiCache / Redis proxy (proxy in Floci)
- "6400-6419:6400-6419" # MemoryDB proxy (proxy in Floci)
- "7001-7099:7001-7099" # RDS proxy (proxy in Floci)
- "8182-8282:8182-8282" # Neptune Gremlin proxy (proxy in Floci)
- "8700-8799:8700-8799" # MWAA webserver proxy (proxy in Floci)
volumes:
- /var/run/docker.sock:/var/run/docker.sock
```
Expand Down
16 changes: 8 additions & 8 deletions docs/getting-started/quick-start.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,21 +165,21 @@ Floci emulates ECR with a real OCI registry behind it, so the stock `docker` cli

```bash
# Create the repository (lazy-starts the backing registry container)
aws ecr create-repository --repository-name floci-it/app --endpoint-url $AWS_ENDPOINT
aws ecr create-repository --repository-name floci-it/app --endpoint-url $AWS_ENDPOINT_URL

# Authenticate
aws ecr get-login-password --endpoint-url $AWS_ENDPOINT \
aws ecr get-login-password --endpoint-url $AWS_ENDPOINT_URL \
| docker login --username AWS --password-stdin \
000000000000.dkr.ecr.us-east-1.localhost:5000
000000000000.dkr.ecr.us-east-1.localhost:5100

# Push
docker pull alpine:3.19
docker tag alpine:3.19 000000000000.dkr.ecr.us-east-1.localhost:5000/floci-it/app:v1
docker push 000000000000.dkr.ecr.us-east-1.localhost:5000/floci-it/app:v1
docker tag alpine:3.19 000000000000.dkr.ecr.us-east-1.localhost:5100/floci-it/app:v1
docker push 000000000000.dkr.ecr.us-east-1.localhost:5100/floci-it/app:v1

# Pull from a clean local image store
docker rmi 000000000000.dkr.ecr.us-east-1.localhost:5000/floci-it/app:v1
docker pull 000000000000.dkr.ecr.us-east-1.localhost:5000/floci-it/app:v1
docker rmi 000000000000.dkr.ecr.us-east-1.localhost:5100/floci-it/app:v1
docker pull 000000000000.dkr.ecr.us-east-1.localhost:5100/floci-it/app:v1
```

See the [ECR service docs](../services/ecr.md) for the full action surface, image-backed Lambda integration, and CDK `DockerImageFunction` support.
Expand All @@ -198,7 +198,7 @@ If you want to scope it tighter to just the Lambda Runtime API and the ECR regis

```bash
sudo ufw allow in on docker0 to any port 12000:12499 proto tcp comment 'floci lambda runtime api'
sudo ufw allow in on docker0 to any port 5000:5099 proto tcp comment 'floci ecr registry'
sudo ufw allow in on docker0 to any port 5100:5199 proto tcp comment 'floci ecr registry'
```

**Docker Desktop** (macOS / Windows / Linux) does not need this — it routes container → host through the Docker VM, which Floci's `DockerHostResolver` detects automatically.
Expand Down
84 changes: 84 additions & 0 deletions docs/service-api-parity-todos.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Floci service/API parity TODO inventory

This is the consolidated backlog from the service/API investigations performed during the
current Floci compatibility work. It turns the existing service guides, parity epics, issue
write-ups, and completed branch work into one actionable queue.

This is a planning document. It does not claim that an AWS operation is unsupported merely
because it is not listed here: the service guide and SDK compatibility tests remain the source
of truth for an individual operation. Every item below names the evidence that caused it to be
included and the next verification or implementation step.

## Priority and status

| Priority | Meaning |
| --- | --- |
| P0 | Blocks a demonstrated LZA path, can cause false-green behavior, or affects account/region correctness. |
| P1 | Important public API parity or lifecycle gap with a clear compatibility consumer. |
| P2 | Deliberate capability limit or lower-frequency API surface; schedule after P0/P1. |

`Open` means work remains. `Implemented — verify` means the code is on a feature branch but
needs rollup, compatibility evidence, or release documentation before it can be called done.

## P0: correctness and product decisions

| ID | Service / area | Status | Evidence / gap | Next step |
| --- | --- | --- | --- | --- |
| PAR-001 | Account and region scoping across services | Open | [issues/0009](../issues/0009-epic-account-region-scoping-audit.md) records that the earlier CFN sweep did not cover the remaining service tree or the region-ambient variant. Its static pre-filter identifies 13 async/ambient-account candidates: Amazon MQ, AppSync, Backup, CloudMap, CloudTrail, CodeDeploy, EKS, Floci UI, Kinesis Analytics, MSK, MWAA, OpenSearch, and SQS. | Re-run `sh scripts/static-checks/find-ambient-account-region-candidates.sh`, then perform bounded, evidence-backed service batches. Record each finding in a numbered issue and exclude only the confirmed global-resource exceptions documented by the epic. |
| PAR-002 | LZA CloudFormation replay/idempotency | Implemented — verify | Branch `feature/lza-cloudformation-idempotency` commit `13687fee` adds status filtering, per-resource checkpoints, security-group/custom-resource idempotency, VPC update handling, and a governed-pipeline integration test. | Roll the branch into integration and run the net-new LZA matrix (including the supported 1.14/1.15/1.16 compatibility targets). Preserve exact stack events and restart behavior as the acceptance record. |
| PAR-003 | CodeBuild execution backend | Open investigation | [CodeBuild local-agent epic](services/codebuild-local-agent-investigation-epic.md) separates the AWS-compatible control plane from the execution backend. The published local agent is not the same image as `aws/codebuild/standard:7.0`; image mapping, output translation, cancellation, secrets, artifacts, and restart behavior remain decisions. | Characterize the published agent with deterministic fixtures, pin image digests, define a versioned backend seam, and run differential native-vs-agent tests before selecting preferred, opt-in, oracle-only, or rejected adoption. |
| PAR-004 | CodePipeline V2 | Open follow-up | [CodePipeline V2 epic](services/codepipeline-v2-epic.md) documents that current support is partial and that LZA currently exercises V1. Trigger execution, validation, queued/superseded/parallel isolation, condition providers, artifact lineage, retry/rollback, events, and CloudFormation round-trips remain. | Implement in slices beginning with V2 validation and trigger fixtures; require AWS SDK integration tests and preserve the existing V1/LZA suite as a regression gate. |
| PAR-005 | Local VPC network data plane | Open investigation | [Network data-plane epic](services/network-data-plane-investigation-epic.md) defines the gap between control-plane records and observable local traffic. Route, security-group, Network Firewall, DNS, endpoint, and logging behavior are not implied by the current API models. | Build a bounded privileged prototype behind a reconciler interface, then decide whether to adopt Podman, an appliance backend, an opt-in experiment, or control-plane-only behavior. Do not make normal API use require host privileges before the decision is proven. |

## P1: public API and lifecycle gaps

| ID | Service / area | Status | Evidence / gap | Next step |
| --- | --- | --- | --- | --- |
| PAR-101 | CloudFormation | Open | `docs/services/cloudformation.md` marks `ValidateTemplate`, stack-policy operations, some intrinsic resolution, and update/delete behaviors as stubs or unimplemented. Unsupported resource types intentionally receive stub physical IDs, which can hide missing service provisioners. | Add AWS SDK contract tests for each advertised stub/error; make unsupported resource handling explicit in the service guide and add exact `Ref`/`Fn::GetAtt` assertions for every newly wired provisioner. |
| PAR-102 | Lambda | Open | `docs/services/lambda.md` marks `ListLayers` and `ListLayerVersions` as empty stubs and notes that SQS event-source `MaximumConcurrency` is tracked but not enforced. | Add layer storage and SDK tests, then enforce event-source concurrency with restart-safe state and throttling/error semantics. |
| PAR-103 | RAM | Open | `docs/services/ram.md` states that resource-share APIs such as `CreateResourceShare` and `GetResourceShares` are not implemented; the persistence branch only addresses resource-share state retention. | Define the supported RAM resource/share model, implement the management API through `StorageFactory`, and verify account/region visibility plus persistence. |
| PAR-104 | AWS Batch | Open | `docs/services/batch.md` states that `process` mode, array-child fan-out, `CancelJob`, `TerminateJob`, and full Batch-specific input transformers are not implemented. Capacity, VCPU, and VPC behavior are metadata-only. | Choose a bounded local scheduler contract, implement cancellation/termination first, and add SDK tests that distinguish accepted metadata from executable behavior. |
| PAR-105 | API Gateway | Open | `docs/services/api-gateway.md` contains explicit “Not Implemented” sections for management/data-plane operations. | Convert each listed operation into an AWS SDK compatibility test, then prioritize operations required by LZA and common IaC providers. |
| PAR-106 | EKS | Open | `docs/services/eks.md` lists Phase 1 features as not implemented; current support does not imply a local Kubernetes control/data plane. | Keep unsupported operations AWS-shaped, document the supported IRSA/issuer boundary, and scope any future cluster behavior as a separate product decision. |
| PAR-107 | RUM | Open | `docs/services/rum.md` says event/data-plane, tag, resource-policy, metric-definition, and metric-destination APIs are not implemented; `CwLogEnabled` does not emit logs. | Implement the management subset only if a consumer requires it; otherwise add negative SDK tests and make the control-plane-only boundary explicit. |
| PAR-108 | Config | Open | `docs/services/config.md` notes that external evaluation does not record resource configurations and that rule evaluation is invocation bookkeeping rather than real evaluation. | Define the minimum resource recorder/configuration model, then add deterministic rule evaluation fixtures and AWS-shaped failure semantics. |
| PAR-109 | CloudWatch Logs Insights | Open | `docs/services/cloudwatch.md` documents a supported subset where unsupported commands are skipped with warnings rather than rejected, and data-protection policy behavior is incomplete. | Decide whether compatibility requires strict rejection or documented degradation; add query corpus tests for `stats`, `parse`, field projection, pagination, and data-protection APIs. |
| PAR-110 | Cost Explorer | Open | `docs/services/ce.md` lists reservation/Savings Plans coverage/utilization, cost categories, and anomaly management as zeroed/empty stubs or out of scope. | Keep stubs clearly marked, then implement only from a concrete consumer requirement with AWS SDK response-shape tests. |
| PAR-111 | RDS Data API | Open | `docs/services/rds-data.md` marks `BatchExecuteStatement`, parameter binding, JSON formatting/result options, and generated fields as unsupported. | Add parameter binding and batch execution against the local JDBC boundary, with tests for malformed requests and engine-specific errors. |
| PAR-112 | IoT Core | Open | `docs/services/iot.md` identifies missing TLS/mTLS, dynamic thing groups, fleet indexing, job rollouts/cancellation, S3 documents, and advanced scheduling. | Treat TLS/mTLS and job lifecycle as separate slices; avoid claiming data-plane parity while the embedded broker remains plaintext-only. |

## P2: deliberate capability boundaries worth tracking

| ID | Service / area | Status | Evidence / gap | Next step |
| --- | --- | --- | --- | --- |
| PAR-201 | DocumentDB | Open | `docs/services/docdb.md` leaves snapshot creation/restore out of scope and returns an empty snapshot result. | Add snapshot persistence only when a real compatibility consumer needs it; otherwise add an explicit negative test. |
| PAR-202 | CUR / BCM Data Exports | Open | `docs/services/cur.md` and `docs/services/bcm-data-exports.md` limit output to Parquet and reject CSV/text and compression variants. | Add format/compression support behind shared export fixtures, or keep the limitation explicit and tested. |
| PAR-203 | MWAA | Open | `docs/services/mwaa.md` notes metadata-only updates for several environment fields, stubbed web-login tokens, and no automatic Docker reconnection after restart. | Prioritize restart-safe reconnection and update semantics if LZA or a compatibility suite exercises them; leave hosted Airflow authentication explicit until then. |
| PAR-204 | OpenSearch | Open | `docs/services/opensearch.md` lists cross-cluster connections, VPC endpoints, packages, applications, and data sources as unsupported. | Add only the resources required by local IaC scenarios; preserve `UnsupportedOperationException`/AWS-shaped errors for the remainder. |
| PAR-205 | Transfer Family | Open | `docs/services/transfer.md` emulates management state but explicitly excludes actual SFTP/FTP protocol handling. | Keep the control/data-plane distinction visible; investigate a local protocol backend separately if a consumer requires end-to-end file transfer. |
| PAR-206 | Textract / Transcribe | Open | `docs/services/textract.md` and `docs/services/transcribe.md` are synthetic/stub control-plane implementations rather than OCR/transcription engines. | Document as deterministic test doubles and add negative/fixture tests; do not describe synthetic output as model parity. |

## Completed work that still needs rollup evidence

These are not new implementation TODOs, but they must not be lost during branch consolidation:

| Area | Branch / commit | Required close-out |
| --- | --- | --- |
| KMS grant metadata | `feature/kms-grant-fidelity` / `0b8724a8` | Roll up and run KMS integration plus persistence/reload assertions. |
| SNS / Control Tower prerequisite | `feature/controltower-sns-prerequisite` / `6e7c1155` | Roll up and run the Control Tower/SNS prerequisite slice. |
| CodeBuild image mapping | `feature/codebuild-local-image` / `ad79c668`, docs `f99bdc39` | Verify the selected ARM image digest and keep the curated-vs-public image distinction in docs. |
| CodePipeline retry artifacts | `feature/codepipeline-artifact-retry` / `cbbd9305` | Run retry/rollback and persistence tests before V2 work changes lineage. |
| DynamoDB index persistence | `feature/dynamodb-hybrid-persistence` / `90a9062f` | Run hybrid-storage reload coverage. |
| RAM persistence | `feature/ram-persistence` / `eeb10ad4` | Pair with the open RAM API work above. |
| Network Firewall and Service Catalog | `feature/network-firewall` / `399d454f`; `feature/service-catalog` / `76eed019` | Roll up service and CloudFormation provisioner tests, then update the service count once both are present. |

## Working rules for turning rows into issues

1. One row becomes an issue only after its source guide or compatibility test is named in the
issue body.
2. A TODO is not closed by accepting an AWS-shaped request: the observable response, lifecycle,
persistence, error, and account/region behavior need an SDK-backed test where applicable.
3. Keep intentional stubs explicit. Never silently return success for an unsupported provider,
trigger, resource type, or data-plane action.
4. Re-run the integration rollup and branch-orphan audit after each batch; parity consolidation
comes after branch hygiene, not before.
Loading