|
| 1 | +# HTTP client callable contract |
| 2 | + |
| 3 | +RustScript exposes buffered HTTP and SSE as bounded host imports. The SSE call keeps one ordinary host call active and invokes a script callable for each event; it exposes no response stream object. |
| 4 | + |
| 5 | +The embedding must configure destination policy and grant each available callable explicitly. HTTP configuration and capability bindings are snapshotted when a call is admitted, so later profile or configuration changes cannot widen an active connection. |
| 6 | + |
| 7 | +Both APIs described below are available with the `http-client` feature. |
| 8 | + |
| 9 | +## Capabilities and profiles |
| 10 | + |
| 11 | +The two imports are independent capabilities: |
| 12 | + |
| 13 | +- `http::client::request` |
| 14 | +- `http::client::sse` |
| 15 | + |
| 16 | +Granting `http::client::request` does not grant `http::client::sse`. A restricted host-function profile must allow every imported callable used by the program. Profiles remain isolated: a grant or configuration in one VM/profile does not authorize another. |
| 17 | + |
| 18 | +Each available API is a host import gated by the HTTP client feature. The two-import contract does not consume or change static builtin IDs. See [Script call frames and callable values](callable-runtime.md) for callable execution and backend behavior. |
| 19 | + |
| 20 | +## Buffered requests |
| 21 | + |
| 22 | +```rust |
| 23 | +use http; |
| 24 | +use bytes; |
| 25 | + |
| 26 | +let response = http::client::request({ |
| 27 | + "method": "POST", |
| 28 | + "url": "https://example.test/v1/messages", |
| 29 | + "headers": {"content-type": "application/json"}, |
| 30 | + "body": bytes::from_utf8("{}"), |
| 31 | +}); |
| 32 | +``` |
| 33 | + |
| 34 | +`http::client::request(request)` accepts a map with: |
| 35 | + |
| 36 | +- `method`: one of `GET`, `POST`, `PUT`, `PATCH`, `DELETE`, `HEAD`, or `OPTIONS`; |
| 37 | +- `url`: an `http` or `https` URL admitted by host policy; |
| 38 | +- `headers`: an optional string-to-string map; |
| 39 | +- `body`: optional bytes or a string. |
| 40 | + |
| 41 | +The response is buffered under the configured response-body limit and returned as: |
| 42 | + |
| 43 | +```rust |
| 44 | +{ |
| 45 | + "status": 200, |
| 46 | + "headers": {"content-type": "application/json"}, |
| 47 | + "body": bytes, |
| 48 | + "url": "https://example.test/v1/messages", |
| 49 | +} |
| 50 | +``` |
| 51 | + |
| 52 | +`url` is the final validated URL after redirects. The request body, response body, response head, redirect count, concurrent connection count, connect phase, and total request duration are bounded. `Host`, `Content-Length`, `Transfer-Encoding`, and `Connection` are client-managed request headers. A limit, policy, transport, TLS, redirect, or timeout failure is a host error and produces no response map. |
| 53 | + |
| 54 | +## Server-sent events |
| 55 | + |
| 56 | +`http::client::sse` is available with the `http-client` feature. |
| 57 | + |
| 58 | +```rust |
| 59 | +fn on_sse(item: map) -> map { |
| 60 | + if item["kind"] == "event" { |
| 61 | + print(item["data"]); |
| 62 | + } |
| 63 | + return {"action": "continue"}; |
| 64 | +} |
| 65 | + |
| 66 | +let result = http::client::sse({ |
| 67 | + "method": "GET", |
| 68 | + "url": "https://example.test/events", |
| 69 | + "headers": {"accept": "text/event-stream"}, |
| 70 | +}, on_sse); |
| 71 | +``` |
| 72 | + |
| 73 | +`http::client::sse(request, on_event)` uses this request map: |
| 74 | + |
| 75 | +| Field | Required | Accepted type and value | Bound or policy | |
| 76 | +| --- | --- | --- | --- | |
| 77 | +| `method` | yes | string: `GET` or `POST` | Other methods are rejected before transport admission | |
| 78 | +| `url` | yes | string containing an `http` or `https` URL | Protocol family and the configured scheme, host, port, and address policy must all admit it | |
| 79 | +| `headers` | no | map from string header names to string values | Names and values must be syntactically valid; client-managed request headers remain forbidden, and `Accept: text/event-stream` is supplied when absent | |
| 80 | +| `body` | no | bytes or string, including for `POST` | Bounded by `max_request_body_bytes` | |
| 81 | +| `timeout_ms` | no | positive integer milliseconds | Caps this optional shortening deadline by `HttpConfig::max_stream_duration` | |
| 82 | + |
| 83 | +The callback schema is `fn(map) -> map`, and the response must have an event-stream content type. The response head remains bounded by the existing HTTP parser. The contract adds no configurable request-header byte accounting. |
| 84 | + |
| 85 | +The callback receives exactly one map at a time, in this order: |
| 86 | + |
| 87 | +```rust |
| 88 | +// The response was accepted; this precedes every event. |
| 89 | +{ |
| 90 | + "kind": "open", |
| 91 | + "status": 200, |
| 92 | + "headers": map, |
| 93 | + "url": string, |
| 94 | +} |
| 95 | + |
| 96 | +// One parsed event. "event" is per-dispatch state, reset to null at every |
| 97 | +// dispatch boundary (including a blank line that dispatches no event); "id" |
| 98 | +// and "retry_ms" are persistent stream state, retaining the last valid |
| 99 | +// values seen so far and null only before any value has been seen. |
| 100 | +{ |
| 101 | + "kind": "event", |
| 102 | + "event": string | null, |
| 103 | + "data": string, |
| 104 | + "id": string | null, |
| 105 | + "retry_ms": int | null, |
| 106 | +} |
| 107 | + |
| 108 | +// Clean EOF, after every preceding event callback completed. |
| 109 | +{"kind": "end"} |
| 110 | +``` |
| 111 | + |
| 112 | +The callback must return one of: |
| 113 | + |
| 114 | +```rust |
| 115 | +{"action": "continue"} |
| 116 | +{"action": "stop"} |
| 117 | +``` |
| 118 | + |
| 119 | +`continue` acknowledges the item and permits the next network poll. `stop` ends the call locally. Any other shape or action is a callback error. |
| 120 | + |
| 121 | +SSE parsing follows the event-stream grammar: |
| 122 | + |
| 123 | +- UTF-8 text may start with one byte-order mark; |
| 124 | +- `\r\n`, `\r`, and `\n` line endings are recognized; |
| 125 | +- repeated `data:` fields are joined with `\n`, with the final join newline removed at dispatch; |
| 126 | +- `event`, `id`, and decimal non-negative `retry` fields are normalized into the event map; |
| 127 | +- comments and unknown fields are ignored; |
| 128 | +- a blank line dispatches only after at least one `data:` field; |
| 129 | +- malformed UTF-8, an over-limit line or event, and cumulative received event-stream application bytes exceeding the call limit are host errors. |
| 130 | + |
| 131 | +There is no automatic reconnection. Values such as a provider's `[DONE]` marker remain ordinary event data. |
| 132 | + |
| 133 | +## Terminal summaries and errors |
| 134 | + |
| 135 | +After callback processing terminates normally, the SSE call returns one summary: |
| 136 | + |
| 137 | +```rust |
| 138 | +{ |
| 139 | + "outcome": "eof" | "stopped", |
| 140 | + "status": int, |
| 141 | + "headers": map, |
| 142 | + "url": string, |
| 143 | + "items": int, |
| 144 | + "bytes_received": int, |
| 145 | + "bytes_sent": int, |
| 146 | +} |
| 147 | +``` |
| 148 | + |
| 149 | +`items` counts delivered callback items. `bytes_received` and `bytes_sent` are observational summary counters. Limit enforcement uses independent entire-call accounting and does not depend on whether or how these counters are displayed. |
| 150 | + |
| 151 | +Transport, parser, destination-policy, timeout, and callback failures stay errors. They are never converted into a successful terminal summary. |
| 152 | + |
| 153 | +## Sequencing, backpressure, and lifecycle |
| 154 | + |
| 155 | +Streaming is a single caller-owned operation: |
| 156 | + |
| 157 | +1. the host polls for one protocol item; |
| 158 | +2. the VM invokes `on_event` in a child script frame; |
| 159 | +3. the callback returns one action; |
| 160 | +4. the host applies that action before polling for another item. |
| 161 | + |
| 162 | +At most one unacknowledged protocol item crosses the host/VM boundary. Decoder scratch space is bounded separately. The network future is not polled while the callback runs, yields, or waits in another async host call. If the callback yields or invokes an ordinary async host function, the callback resumes first; only its final action resumes the outer stream operation. This sequencing supplies backpressure without a background reader or callback queue. |
| 163 | + |
| 164 | +The network future never owns or re-enters the VM. Callback error, protocol completion, configured deadline, VM reset/shutdown/drop, invocation termination, or normal return retires the operation exactly once. The embedding owns pending futures: retiring a call drops its transport and permit, and a late completion cannot re-enter the VM. |
| 165 | + |
| 166 | +`request_timeout` is the total bound for a buffered request and does not apply to SSE. `max_stream_duration` is the host-controlled absolute total-duration bound for each SSE call. SSE computes one admission-time deadline from the smaller of `max_stream_duration` and optional positive `timeout_ms`; the script value can only shorten the call and cannot disable or extend the host maximum. DNS, TCP, TLS, active reads, callback execution, and callback waits all count against the same deadline. Embedding invocation retirement may terminate the call sooner. `stream_idle_timeout` remains a separate wait-for-network-progress bound and resets only after progress; periodic traffic cannot extend the total deadline. Network idle time excludes time spent inside the callback, while callback work remains inside the total deadline. |
| 167 | + |
| 168 | +## Configuration defaults |
| 169 | + |
| 170 | +`HttpConfig` uses explicit bounded defaults. Streaming byte limits and all timeout fields must remain positive: |
| 171 | + |
| 172 | +| Field | Default | Purpose | |
| 173 | +| --- | ---: | --- | |
| 174 | +| `allowed_schemes` | `https` | Scheme allowlist; protocol-family checks still apply | |
| 175 | +| `allowed_hosts` | empty | Destination host allowlist; empty denies every host | |
| 176 | +| `allowed_ports` | empty | Destination port allowlist; empty denies every port | |
| 177 | +| `allow_private_ips` | `false` | Reject private and other special-use addresses | |
| 178 | +| `max_redirects` | 5 | Buffered/SSE redirect bound | |
| 179 | +| `max_request_body_bytes` | 1 MiB | Request body bound | |
| 180 | +| `max_response_body_bytes` | 8 MiB | Buffered response body bound | |
| 181 | +| `connect_timeout` | 10 s | DNS/connect/TLS phase bound | |
| 182 | +| `request_timeout` | 30 s | Buffered request total duration | |
| 183 | +| `max_stream_item_bytes` | 1 MiB | SSE event bound | |
| 184 | +| `max_stream_total_bytes` | 64 MiB | Entire-call cumulative received event-stream byte bound | |
| 185 | +| `max_sse_line_bytes` | 64 KiB | SSE line bound | |
| 186 | +| `max_stream_duration` | 5 min | Host maximum total duration for SSE calls | |
| 187 | +| `stream_idle_timeout` | 30 s | Wait-for-network-data bound | |
| 188 | + |
| 189 | +The shared in-flight connection default is 64. Zero values for streaming byte limits or any timeout are invalid configuration; buffered `max_request_body_bytes` and `max_response_body_bytes` may be zero to prohibit request or response payload bytes. `HttpConfig::default()` allows `https`. Embeddings should set explicit host and port allowlists and add `http` only when cleartext transport is required. Buffered HTTP and SSE accept only `http`/`https`. |
| 190 | + |
| 191 | +## Destination policy and protocol transports |
| 192 | + |
| 193 | +Every protocol uses the same admission, address-pinning, and security policy: |
| 194 | + |
| 195 | +- URLs require a host and reject userinfo; |
| 196 | +- both the protocol's scheme family and the configured scheme allowlist must admit the URL; |
| 197 | +- host and effective port must match their configured allowlists; |
| 198 | +- every DNS result is validated, and the selected validated address is pinned for the connection; |
| 199 | +- when private addresses are disabled, private, loopback, link-local, multicast, unspecified, documentation, transition, reserved, and other special-use IPv4/IPv6 ranges are rejected; IPv4-mapped IPv6 addresses receive the IPv4 checks; |
| 200 | +- the original validated hostname remains the TLS SNI name and HTTP `Host` authority when connecting to a pinned address; |
| 201 | +- buffered HTTP and SSE revalidate every redirect and remove `Authorization` and `Cookie` on a cross-origin redirect; |
| 202 | +- ambient proxy settings are ignored. There is no implicit cookie jar, authentication source, or global proxy state. |
| 203 | + |
| 204 | +The policy snapshot taken at call admission applies for the complete operation. |
| 205 | + |
| 206 | +Buffered HTTP and SSE use direct Hyper HTTP/1 over Tokio/Rustls connections and perform no independent DNS lookup outside the shared admission and pinning path. |
| 207 | + |
| 208 | +## Deliberately absent APIs and semantics |
| 209 | + |
| 210 | +RustScript core provides no script-visible HTTP request ID, response/stream handle, `next`, `next_event`, or `cancel` callable. Streams cannot detach from their caller. There is no multiplexing, background reader, automatic reconnect, provider/model interpretation, agent loop, or platform retry policy. Applications implement provider-specific JSON, `[DONE]`, tool-call deltas, retry rules, and reconnect decisions in RSS or downstream hosts. |
| 211 | + |
| 212 | +## Cancellation migration |
| 213 | + |
| 214 | +PR #13 introduced HTTP-private pending-operation and abort-handle maps, one abort pair per request, HTTP owner routes, request-local runtimes, and HTTP-synthesized cancellation errors. The callable streaming contract supersedes those mechanisms. Buffered requests and SSE submit ordinary futures through the embedding-owned async bridge; HTTP has no private pending map, abort map, operation-ID namespace, token owner route, or cancellation state machine. |
| 215 | + |
| 216 | +The generic `src/builtins/runtime/cancellation.rs` remains for non-HTTP runtime callers. HTTP does not depend on `CancellationToken`, `CancellationReason`, `OperationOwner::Http`, or owner-wide cancellation routing. Embedding-owned retirement of a pending future remains VM lifecycle control and rejects late completion; dropping an `Invocation` also retires active producer/callback waits and returns the VM and connection permit for reuse. This lifecycle cleanup is not an HTTP API-level cancellation facility. |
| 217 | + |
| 218 | +## Target and backend notes |
| 219 | + |
| 220 | +The callable pump follows the ordinary host-call suspension boundary for interpreter, Trace JIT, and whole-program AOT execution. Network futures remain outside VM execution, and callback frames use the same wait/yield continuation rules across backends. |
| 221 | + |
| 222 | +`pd-vm-nostd` retains callable metadata and static builtin IDs without including HTTP transport implementations. WebAssembly and other embeddings can expose host imports only when that embedding supplies the capability, policy configuration, and async driving required by this contract. The contract does not imply an HTTP backend on targets where the host has not provided one. |
0 commit comments