Proposal: Deploying the AFM Ballerina Runtime on WSO2 Devant

[Thareesha98]

Proposal: Deploying the AFM Ballerina Runtime on WSO2 Devant

1. Purpose

We want to run an AFM agent in the cloud on WSO2 Devant instead of only on a local
machine. This proposal describes how we would package and deploy the Ballerina AFM
interpreter (plus one example agent) as a hosted service on Devant, reachable over a public
URL. The aim is to validate that AFM agents run well on WSO2's own platform and to document a
repeatable path others can follow.

This document is a plan for review before any deployment work begins.

2. Background

• AFM lets you describe an AI agent in an .afm.md file (YAML frontmatter for model,
  tools, interfaces, authentication + a markdown body for role/instructions). The Ballerina
  interpreter reads that file and serves the agent over HTTP.
• WSO2 Devant is a cloud platform (the integration-focused sibling of Choreo, on the same
  underlying platform). You connect a Git repository, and Devant builds and runs your code as a
  managed component, handling networking, configuration/secrets, scaling, and observability.

These fit together naturally: the Ballerina runtime is a containerized HTTP service, which is
exactly the kind of workload Devant hosts — and it's a WSO2-on-WSO2 story.

3. Goals and non-goals

Goals

• Deploy the Ballerina AFM runtime + one example agent as a running Devant service.
• Expose the agent's webchat HTTP endpoint at a public Devant URL.
• Manage the model-provider API key as a Devant secret (not in the repo).
• Produce a clear, repeatable deployment path.

Non-goals (this phase)

• The Python runtime (separate effort).
• The consolechat interface (it's an interactive terminal interface and cannot be served as a
  web endpoint).
• Building a one-click template/marketplace entry (possible later phase).

4. Scope

Item	Choice
Runtime	Ballerina interpreter (ballerina-interpreter/)
Packaging	Existing Dockerfile (multi-stage; builds afm_ballerina.jar, runs on JRE 21)
Example agent	One simple webchat agent (e.g. an OpenAI-backed assistant)
Interface deployed	webchat → HTTP /chat on port 8085
Platform	WSO2 Devant, Dockerfile-based Service component

5. Architecture

 Git repo (ballerina-interpreter/)
   ├─ *.bal  (the runtime)
   ├─ Dockerfile
   └─ resources/agent.afm.md  (the example agent)
            │
            │  Devant connects the repo & builds the Dockerfile
            ▼
   ┌──────────────────────────────────────────┐
   │  Devant "Service" component               │
   │  (runs the afm_ballerina.jar container)   │
   │                                           │
   │   port 8085  ──►  /chat  (webchat)        │
   │                                           │
   │   reads:  afmFilePath (configurable)      │
   │   reads:  OPENAI_API_KEY (secret)         │
   └──────────────────────────────────────────┘
            │  public endpoint
            ▼
   https://<agent>.<org>.devant.../chat   ──►  end users
            │  outbound HTTPS
            ▼
   Model provider API (e.g. OpenAI/Anthropic)

The deployment unit is the container that runs the runtime with one agent file. Devant
builds it, runs it, gives it a public address, and injects the API key at runtime.

6. Deployment approach

1. Repository — deploy from the ballerina-interpreter directory (it already contains
   Ballerina.toml, the .bal sources, modules/, and a Dockerfile).
2. Agent file — place the example agent.afm.md under resources/ (the Dockerfile already
   copies resources/), and point the runtime at it via the afmFilePath configurable.
3. Component type — create a Devant Service component using the Dockerfile build
   option (build context = ballerina-interpreter, Dockerfile path = ./Dockerfile).
4. Endpoint — declare the service endpoint on port 8085, protocol HTTP, so Devant routes
   /chat to the container. (In the Choreo/Devant family this is described in a component
   descriptor such as .choreo/component.yaml / component.yaml — exact format to confirm.)
5. Build & deploy — let Devant build the image and deploy it; verify the agent answers on the
   generated URL.

Alternative to evaluate: Devant/Choreo also have first-class Ballerina support, so a native
Ballerina component (no Dockerfile) may be simpler. We will compare both and pick one; the
Dockerfile path is the safe default since it already exists and works.

7. Configuration and secrets

The runtime needs two kinds of settings:

Setting	How it's read today	On Devant
port, afmFilePath	Ballerina configurable (Config.toml / env / CLI)	Devant "Configurations"
Model provider API key (e.g. OPENAI_API_KEY)	environment variable	Devant Secret
Any MCP/tool auth secrets	env via ${env:...} in the AFM file	Devant Secrets

API keys and other secrets will be set in Devant's secret store, never committed to the
repo. The AFM file references them with ${env:...}.

Security note: AFM currently substitutes ${env:...} into the file as raw text before
parsing the YAML, so a secret containing special characters (:, newline, etc.) can break
loading or alter the config. Since secrets on Devant are entered through the dashboard, we will
use simple values for the PoC and track the parser fix separately. (Issue noted in the repo.)

8. Exposure and access control

• The webchat endpoint will be exposed publicly so we can demo it.
• For anything beyond a demo, we will protect the endpoint (Devant API
  management / an auth layer), since an open endpoint lets anyone invoke the agent (and spend
  model tokens). This connects to AFM's own authentication support.

9. Observability and scaling

• Use Devant's built-in logs and metrics to confirm requests reach the agent and to debug.
• Start with a single small instance; rely on Devant autoscaling if load grows.
• Watch model-provider cost/usage during testing (the agent makes paid API calls).

10. Plan / milestones

Phase	Outcome
0. Prep	Add a clean example webchat agent under resources/; confirm it runs locally in Docker on port 8085.
1. First deploy (PoC)	Connect repo to Devant, deploy the Dockerfile Service, set the API key secret, get a public /chat URL that answers.
2. Harden	Add endpoint protection, confirm logs/metrics, document the exact Devant steps.
3. (Optional, later)	Reusable template; webhook-interface agent; Python runtime; expose as a managed API.

11. Risks and open questions

• Platform specifics to confirm — exact Devant steps for: Dockerfile build context, endpoint
  declaration (port 8085), and secret injection. (Verify in Devant docs/console.)
• Native Ballerina vs Dockerfile component — decide which build path is simpler on Devant.
• Agent file delivery — bake the .afm.md into the image (chosen) vs. mount it as config
  (more flexible, but needs a config-file mechanism).
• Outbound access — confirm the component can reach external model-provider APIs.
• Cost — model API usage is billed per call; keep the demo small.
• Free-tier limits — confirm Devant free-tier resource/endpoint limits are enough.

12. Success criteria

• A webchat AFM agent, running the Ballerina runtime, is reachable at a public Devant URL
  and returns correct responses.
• The model API key is supplied as a Devant secret, with nothing sensitive in the repo.
• The deployment steps are written down clearly enough for someone else to repeat.