Plan: Migrate DevBox CLI Behind a Lambda in PR-Sized Phases

Summary

This file is the living handoff document for the migration.
Goal: migrate the current CLI to a Lambda-backed architecture in a sequence of separate PRs, where each PR ends with a locally invokable, fully working command path against deployed AWS.
Initial migration order: status -> terminate -> launch -> new -> delete-project -> full cleanup/cutover.
CLI rollout model: hybrid by command. Migrated commands go remote; unmigrated commands keep current local behavior until their phase lands.

Locked Decisions

Transport: Lambda Function URL from day one.
Discovery: CLI resolves the Function URL from SSM at ${param_prefix}/cli/functionUrl.
Auth: CLI signs HTTP requests with AWS SigV4.
Response model: NDJSON event stream with progress, warning, result, success, and error.
Python streaming implementation: Lambda Web Adapter.
Packaging: the original shared-image plan is superseded for phase 1. The CLI Lambda gets its own image because Lambda Web Adapter would interfere with the existing handler-based Lambdas if we baked it into the current shared image.
Protocol ownership: shared wire-format constants plus the action and event enums live in src/devbox/cli_protocol.py. Future actions should extend that shared module instead of re-declaring protocol strings in the CLI and Lambda separately.
Lambda routing: the CLI Lambda router uses an explicit dispatch table keyed by the shared action enum. Future actions should add themselves to that dispatch table rather than branching on ad hoc string comparisons.
Command-module layout: command-specific client-side and Lambda-side behavior now lives under src/devbox/commands/<action>.py. cli.py remains the Click entry point, remote_client.py remains generic transport, and cli_lambda/app.py remains the generic dispatch layer.
Python documentation style: new Python code for this migration should use numpydoc-style docstrings for modules, public functions, and non-trivial helpers. Exception: Click entrypoints should keep user-facing docstrings, because Click surfaces those docstrings in --help output and they are part of the CLI UX rather than internal developer documentation.
IAM layout: keep one CLI Lambda IAM statement per action wherever practical, and make each action block self-contained so its required permissions can be reviewed in isolation even when that duplicates shared read permissions.
Phase 1 status Lambda permissions are intentionally EC2-only. The local CLI resolves the Function URL from SSM, and the Lambda-side status handler currently reuses DevBoxManager.list_* inventory helpers that call EC2 Describe* APIs only.
Documentation authority: this file is authoritative for phase 1. Any older standalone CLI migration spec is advisory only until it is reconciled back into this document.
Dependencies: requests may be added to runtime dependencies; responses may be added to test dependencies.
End-of-phase validation: real local CLI invocation against deployed AWS, not just local mocks or container smoke tests.
new is deferred but remains in scope, even though the current CLI references a missing implementation.

Public Contract

CLI request envelope:

{
  "version": "v1",
  "action": "status",
  "request_id": "uuid",
  "param_prefix": "/devbox",
  "payload": {
    "project": null
  }
}

Lambda event envelope:

{
  "type": "progress|warning|result|success|error",
  "action": "status",
  "message": "human-readable text",
  "data": {}
}

The CLI remains responsible for:
- resolving the Function URL from SSM
- signing requests
- reading local files such as --userdata-file
- rendering streamed output
- keeping local confirmation prompts for destructive flows
The Lambda remains responsible for:
- validating the request envelope
- performing AWS-side operations
- emitting versioned event streams
- enforcing server-side safety checks

Phase 1: Foundation and `status`

Milestone: devbox status [project] runs from the local CLI through the deployed CLI Lambda and renders the same tables as today.

Phase 1 `status` Contract

Request envelope:

{
  "version": "v1",
  "action": "status",
  "request_id": "uuid",
  "param_prefix": "/devbox",
  "payload": {
    "project": null
  }
}

payload.project may be a project name string or null.
Success path event sequence:
- exactly one result event with the full status payload
- exactly one terminal success event
status result payload shape:

{
  "instances": [
    {
      "InstanceId": "i-0123456789abcdef0",
      "Project": "my-project",
      "PublicIpAddress": "54.0.0.1",
      "LaunchTime": "2026-03-30T20:30:00+00:00",
      "State": "running",
      "InstanceType": "t3.medium"
    }
  ],
  "volumes": [
    {
      "VolumeId": "vol-0123456789abcdef0",
      "Project": "my-project",
      "State": "in-use",
      "Size": 100,
      "AvailabilityZone": "us-east-1a",
      "IsOrphaned": false
    }
  ],
  "snapshots": [
    {
      "SnapshotId": "snap-0123456789abcdef0",
      "Project": "my-project",
      "Progress": "100%",
      "VolumeSize": 100,
      "StartTime": "2026-03-30T18:00:00+00:00",
      "IsOrphaned": false
    }
  ]
}

The CLI must rehydrate LaunchTime and StartTime to datetime objects before calling ConsoleOutput.
Phase 1 status does not render progress events; warnings and errors may still be surfaced immediately.

Phase 1 Validation Log

2026-03-31: targeted phase 1 pytest suite passed after the command-module refactor.
- Command: pixi run -e dev python -m pytest tests/commands/test_status.py tests/cli_lambda/test_contracts.py tests/cli_lambda/test_app.py tests/test_remote_client.py tests/test_cli.py -q
- Result: 84 passed, 1 warning
2026-03-31: Terraform formatting passed.
- Command: tofu fmt
- Result: passed
2026-03-31: Terraform validation passed in the real deployment environment after the fresh-account/Public ECR auth fixes were applied.
- Command: tofu validate
- Result: passed
End-to-end commands (must be run by DWHS in devbox-deploy/ repo, not for agents)
- tofu plan -out tfplan && tofu apply tfplan to deploy the CLI Lambda and related infra
- devbox status against the deployed CLI Lambda, expecting to see the desired output
Observed outcome: It works!

Phase 1 Handoff Notes

The shared wire contract now lives in src/devbox/cli_protocol.py. Extend CliAction and the shared protocol constants there first whenever a new remote command is added.
tests/cli_lambda/test_contracts.py owns low-level request/event contract coverage. tests/cli_lambda/test_app.py owns dispatch and execution-failure behavior. Keep that split as more actions land.
tests/commands/test_status.py now owns the status command semantics across both CLI-side and Lambda-side behavior. Keep command-specific tests near the command module instead of spreading them across transport and app test files.
The CLI Lambda router in src/devbox/cli_lambda/app.py is organized around ACTION_HANDLERS. Future actions should wire themselves into that table and add parametrized coverage rather than growing one-off branching tests.
status is the template command for the new src/devbox/commands/<action>.py layout. Future simple commands should follow that module pattern first; only extract a shared helper after a second migrated command proves the duplication is real.
The CLI Lambda image now uses python:3.13-slim directly instead of the AWS public mirror public.ecr.aws/docker/library/python:3.13-slim, because the mirrored image returned 403 Forbidden during docker build in a fresh account deployment.
The CLI Lambda image now uses Lambda Web Adapter 1.0.0, matching the current official README example for container images.
The CLI Lambda local-exec build step now logs into Public ECR explicitly with aws ecr-public get-login-password before docker build, because the adapter image pull failed with expired or missing Public ECR auth in a fresh-account deployment.
The CLI Lambda local-exec build step now uses a temporary DOCKER_CONFIG so Terraform does not depend on or mutate the operator's persistent Docker Desktop keychain state on macOS.
The phase 1 IAM policy in modules/cli-lambda/main.tf is intentionally grouped per action. Keep that structure, and prefer each action block to list the full permission set that command needs in isolation even when that duplicates shared EC2 read permissions.
Do not add SSM or DynamoDB permissions to the CLI Lambda just because DevBoxManager can use them elsewhere. Add them only when a migrated Lambda action actually reads those services.
The request envelope still carries param_prefix from the client. That is acceptable in phase 1 because status only reuses EC2 Describe* inventory helpers and the CLI Lambda IAM is EC2-only, so the prefix does not currently select SSM or DynamoDB resources. Before any later action uses prefix-derived SSM parameter names, DynamoDB table names, or other non-EC2 resources, add server-side validation/normalization for the prefix or replace the client-provided value with Lambda-side configuration.
src/devbox/remote_client.py now wraps requests transport failures as RemoteInvocationError, but it still uses a single timeout=30 value. Revisit timeout policy in phase 3 before migrating launch, because long-lived streaming commands may need separate connect/read timeouts or a longer read timeout to avoid aborting a healthy response stream mid-operation.
Remaining phase 1 blockers: none. Phase 1 and the inter-phase command-module refactor are complete.
Next session starts here: begin phase 2 terminate using the src/devbox/commands/<action>.py pattern established by status, and preserve the current remote contract/transport structure.

Inter-Phase Refactor: Command Modules

Milestone: status command-specific behavior is co-located under src/devbox/commands/status.py, while cli.py, remote_client.py, and cli_lambda/app.py remain generic entrypoint, transport, and router layers.

Update this file to record the command-module layout decision and handoff guidance.
Add src/devbox/commands and package it for distribution.
Move status client-side and Lambda-side command behavior into src/devbox/commands/status.py.
Keep src/devbox/remote_client.py generic by removing status-specific helpers.
Keep src/devbox/cli.py as a thin Click wrapper that delegates to the status command module.
Keep src/devbox/cli_lambda/app.py as the generic router and dispatch status through the command module.
Move status command tests to tests/commands/test_status.py.
Run the targeted refactor validation suite and record the result here.

Phase 2: `terminate`

Milestone: devbox terminate <instance-id-or-project> runs from the local CLI through the deployed CLI Lambda and preserves current success and failure behavior.

Phase 2 `terminate` Contract

Request envelope:

{
  "version": "v1",
  "action": "terminate",
  "request_id": "uuid",
  "param_prefix": "/devbox",
  "payload": {
    "identifier": "i-0123456789abcdef0"
  }
}

payload.identifier must be a non-empty string and may be either an instance ID or a project name.
Success path event sequence:
- exactly one result event with the termination payload
- exactly one terminal success event
terminate result payload shape:

{
  "instance_id": "i-0123456789abcdef0",
  "project": "my-project"
}

Phase 2 Validation Log

2026-05-29: targeted phase 2 pytest suite passed.
- Command: pixi run -e dev python -m pytest tests/commands/test_terminate.py tests/cli_lambda/test_contracts.py tests/cli_lambda/test_app.py tests/test_cli.py tests/test_devbox_manager.py -q
- Result: 135 passed, 1 warning
2026-05-29: Terraform formatting passed.
- Command: tofu fmt
- Result: passed
2026-05-29: Terraform validation passed.
- Command: tofu validate
- Result: passed
End-to-end commands (must be run by DWHS in devbox-deploy/ repo, not for agents)
- tofu plan -out tfplan && tofu apply tfplan to deploy the updated CLI Lambda and IAM policy
- devbox terminate i-0123456789abcdef0 against the deployed CLI Lambda, expecting the existing success message
- devbox terminate my-project against the deployed CLI Lambda, expecting project-name resolution to match current behavior
Observed outcome: automated validation passed locally; deployed-AWS end-to-end termination validation is still pending operator run.

Phase 3: `launch`

Milestone: devbox launch ... runs from the local CLI through the deployed CLI Lambda, including userdata handling and current DNS flags.

Phase 4: `new`

Milestone: devbox new ... works end-to-end through the deployed CLI Lambda and is no longer dependent on a missing local implementation.

Phase 5: `delete-project`

Milestone: devbox delete-project ... completes the full confirmation and deletion flow through the deployed CLI Lambda.

Phase 6: Full Cutover and Cleanup

Milestone: the primary CLI surface is fully Lambda-backed, and obsolete direct-AWS command paths are removed.

Handoff Expectations for Every PR

Update this file's phase checklist status.
Add any newly locked decisions and why they were chosen.
Add any contract changes made in the PR.
Add the exact manual validation commands that were run and their results.
Add any known gaps, risks, or follow-up items for the next session.
Leave a short "next session starts here" note naming the next unchecked work item.

Assumptions

Each phase is intended to be small enough to fit in a single PR.
Local validation means the real CLI running from a developer machine against deployed AWS infrastructure for the migrated command.
Unmigrated commands remain local until their dedicated phase lands.
Streaming remains in scope throughout the migration; if Lambda Web Adapter proves unworkable in practice, this file must be updated before implementation continues.

FilesExpand file tree

PLAN-cli-lambda.md

Latest commit

History