fix(sdk): make the v1 byte types refuse what they used to swallow

Three defects an adversarial pass found in the conversion itself.

Rust's `report_data` kept `#[builder(into)]`, and both `&str` and
`String` implement `Into<Vec<u8>>`. So `.report_data("00ff")` still
compiled and attested the four ASCII bytes of that string -- exactly the
failure the change was made to eliminate, on the one field a caller
sends rather than receives. It is now a `ReportData` newtype that
converts from a `Vec<u8>`, an array or a slice and from nothing else, so
the builder keeps the coercion that made those three ergonomic while a
string becomes a type error. A `compile_fail` doctest keeps it one.

JavaScript decoded with `Buffer.from(value, 'hex')`, which stops at the
first pair it cannot parse and returns the prefix, silently: a corrupted
`app_id` came back as a short `Uint8Array`, `signature_chain:
["aabb","qq"]` came back one link short, and an absent required field
came back as an empty array that is *truthy*, so `if (!info.app_id)`
guards became dead code. Rust, Python and Go all reject these. It now
throws and names the field, and absence of a required field is an error
rather than empty bytes -- `os_image_hash` and `mr_aggregated` still
read as empty, so a degraded `Info` stays parseable.

Python accepted hex with embedded whitespace, which `bytes.fromhex`
skips, while raising a message that said it did not; and it rejected a
response that omits `os_image_hash` or `mr_aggregated`, where Rust has
`#[serde(default)]`. Both aligned.

None of this was covered: JS and Python had zero negative tests for any
v1 byte field, which is why a green suite proved nothing here. Added,
and mutation-checked -- four of the five new JS tests fail against the
old lenient decoder.

Also corrects the CHANGELOG, which said four of the eleven fields had no
Rust decoder where the number is three, and which claimed "the wire is
unchanged" without saying that only the JSON wire is: borsh writes a
`Vec<u8>` as length-prefixed bytes where it wrote a hex `String`, so a
0.5.x blob deserializes without error into the wrong content. Three
comments called `os_image_hash` and `mr_aggregated` `optional` on the
wire; the proto declares them plain `bytes`, and `not_before` and
`not_after` are the only `optional` fields it has. Tolerating their
absence is a client-side choice, and now says so.

Author: kvinwang

CHANGELOG.md

@@ -61,9 +61,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - sdk: the Go SDK's v1 `IssueCert` defaults `usage_server_auth` to true, as the Rust, Python and JavaScript v1 clients already did. Go was the odd one out, so the same argument-free call produced a certificate that could serve TLS in three languages and one that could not in the fourth — and a certificate you cannot serve with is useless to most callers. `WithCertUsageServerAuth(false)` opts out. v0's `GetTlsKey` keeps its `false` default deliberately: that is what the released 0.5.x Go SDK sent, and `DstackClientV0` mirrors released behaviour rather than the better choice
 - sdk: the JavaScript v1 `issueCert` response no longer carries a raw-bytes accessor. `asUint8Array()` is **removed rather than renamed**: it existed to feed the private key into the blockchain adapters, and v1 has no chain-flavoured surface. `IssueCert` returns TLS material, PEM is the form a TLS stack takes, and a caller who genuinely needs DER converts it with a standard library. The Rust, Python and Go v1 clients already returned the PEM string and the chain alone, so all four now agree. v0's `GetTlsKeyResponse.asUint8Array` is untouched — released API, and the viem and solana adapters depend on its truncating behaviour
 - sdk: the JavaScript v1 GPU evidence bundle's `asUint8Array()` accessor is gone, and `evidence` is the vendor's bytes directly, as Go's `Evidence` always was. Byte-exact off the wire, because sha256 over precisely those bytes is what the measured `gpu-attestation` event commits to
-- sdk: every field the `dstack.guest.v1` proto declares `bytes` is now that language's byte type on the v1 clients — Rust `Vec<u8>`, Python `bytes`, JavaScript `Uint8Array`, Go `[]byte` — and the `decode_*` helpers are gone along with the hex strings they decoded. Eleven fields move: `GetKeyResponse`'s `key`, `public_key` and `signature_chain`; `AttestResponse.attestation`; `GpuEvidenceBundle.evidence`; and `InfoResponse`'s `app_id`, `instance_id`, `compose_hash`, `device_id`, `os_image_hash` and `mr_aggregated`. Rust's `AttestConfig.report_data` moves with them, so the public builder and `attest()` finally agree on a type. **The wire is unchanged** — JSON still carries lowercase hex; the encoding moved into the serialization layer, as serde's `hex::serde` in Rust, an annotated pydantic type in Python, and the client's decode step in JavaScript. Go already did this and is untouched.
+- sdk: every field the `dstack.guest.v1` proto declares `bytes` is now that language's byte type on the v1 clients — Rust `Vec<u8>`, Python `bytes`, JavaScript `Uint8Array`, Go `[]byte` — and the `decode_*` helpers are gone along with the hex strings they decoded. Eleven fields move: `GetKeyResponse`'s `key`, `public_key` and `signature_chain`; `AttestResponse.attestation`; `GpuEvidenceBundle.evidence`; and `InfoResponse`'s `app_id`, `instance_id`, `compose_hash`, `device_id`, `os_image_hash` and `mr_aggregated`. Rust's `AttestConfig.report_data` moves with them, so the public builder and `attest()` finally agree on a type. **The JSON wire is unchanged** — it still carries lowercase hex; the encoding moved into the serialization layer, as serde's `hex::serde` in Rust, an annotated pydantic type in Python, and the client's decode step in JavaScript. Go already did this and is untouched.
 
-  The old typing did quiet damage. `docs/guest-api-v1.md` says of the v1 key claim that `public_key` is the raw derived public key, *not* a hex string, and its verification steps rebuild the claim from raw bytes — so a Rust or Python caller passing `response.public_key` straight into a claim builder built it over 66 ASCII characters instead of 33 bytes. No type error, no exception, just a chain that never verifies. `evidence` had the same shape of problem: three separate documents had to keep repeating "hash the decoded bytes, not the string as returned", precisely because the type did not say it. In Go the mistake was unspellable, and now it is unspellable everywhere. Nor was there one line to learn: before this, four of the eleven fields had no decoder at all in Rust, six had none in Python, and JavaScript had decoded three of them for a while.
+  The old typing did quiet damage. `docs/guest-api-v1.md` says of the v1 key claim that `public_key` is the raw derived public key, *not* a hex string, and its verification steps rebuild the claim from raw bytes — so a Rust or Python caller passing `response.public_key` straight into a claim builder built it over 66 ASCII characters instead of 33 bytes. No type error, no exception, just a chain that never verifies. `evidence` had the same shape of problem: three separate documents had to keep repeating "hash the decoded bytes, not the string as returned", precisely because the type did not say it. In Go the mistake was unspellable, and now it is unspellable everywhere -- which is why Rust's `report_data` is a `ReportData` newtype rather than a bare `Vec<u8>`: `&str` and `String` both implement `Into<Vec<u8>>`, so under the builder's `into` coercion `.report_data("00ff")` would still compile and attest the four ASCII bytes of that string. The newtype converts from a `Vec<u8>`, an array or a slice and from nothing else, so the ergonomics survive and the string does not. A `compile_fail` doctest keeps it that way. Nor was there one line to learn: before this, three of the eleven fields had no decoder at all in Rust, six had none in Python, and JavaScript had decoded three of them for a while.
+
+  Decoding got stricter where it was silently lenient. JavaScript relied on `Buffer.from(value, 'hex')`, which stops at the first pair it cannot parse and returns the prefix, so a corrupted `app_id` became a short `Uint8Array` and a signature chain with one bad link came back quietly one link short; it now throws and names the field, as Rust, Python and Go already did. A required field that is absent altogether is an error rather than empty bytes -- `os_image_hash` and `mr_aggregated` are the two exceptions, read as empty so a degraded `Info` stays parseable, which is what Rust's `#[serde(default)]` already did and what Python now does instead of rejecting the response. Python also stops accepting hex with embedded whitespace, which `bytes.fromhex` skips and Rust refuses.
+
+  The `borsh` encoding of these structs does change, since borsh writes a `Vec<u8>` as length-prefixed bytes where it wrote a hex `String` before. Blobs written by 0.5.x deserialize without error into the new types and yield the ASCII of the hex string, so do not read old ones with the new types. Only the JSON wire is compatible.
 
   **v0 deliberately keeps its hex strings and `decode_*` helpers.** That surface mirrors the released 0.5.x SDK so a 0.5.x program keeps working by changing only the class name; retyping every byte field would break that promise on an API that is frozen anyway. The blockchain adapters are v0-typed and unaffected
 - sdk: the v0 modules carry a `_v0` suffix, so the file a reader opens matches the client it holds. Rust's `dstack_sdk::dstack_client` becomes `dstack_sdk::dstack_client_v0` and `dstack_sdk_types::dstack` becomes `dstack_sdk_types::dstack_v0`; Python's `dstack_sdk.dstack_client` becomes `dstack_sdk.dstack_client_v0`; Go's `client.go`/`client_test.go` become `client_v0.go`/`client_v0_test.go`; and the JavaScript `index.ts`, which held both surfaces in one file, splits into `client-v0.ts`, `client-v1.ts` and a `shared.ts`, leaving `index.ts` as a barrel that re-exports exactly the names it always did. Until now the unsuffixed *file* meant v0 while the unsuffixed *class* meant v1, so a reader opening `dstack_client.rs` for the recommended client found the legacy one instead. **There are deliberately no backward-compat module aliases**: 0.6.0 is the loud-break release, and an import of an old module path fails at build time rather than silently binding the frozen surface under a name that now means something else. Package-level exports are untouched in every SDK — `dstack_sdk::DstackClient`, `from dstack_sdk import DstackClientV0` and `@phala/dstack-sdk`'s public surface are exactly what they were; only a deep import of the module path moves. In Go this is file naming alone, since it is all one `package dstack`

sdk/js/src/__tests__/index-v1.test.ts

@@ -266,6 +266,72 @@ describe('DstackClientV1', () => {
     })
   })
 
+  // Every one of these decoded to a shorter-than-asked-for Uint8Array with no
+  // error before the hex decoding was made strict: Node's decoder stops at the
+  // first pair it cannot parse and returns the prefix. A truncated `key` or a
+  // signature chain quietly one link short is a verification failure nobody can
+  // trace back to its cause.
+  describe('malformed hex from the agent', () => {
+    async function withAgentAnswering(body: unknown, fn: (client: DstackClientV1) => Promise<void>) {
+      const server = http.createServer((_req, res) => {
+        res.writeHead(200, { 'Content-Type': 'application/json' })
+        res.end(JSON.stringify(body))
+      })
+      await new Promise<void>(resolve => server.listen(0, '127.0.0.1', () => resolve()))
+      try {
+        const { port } = server.address() as AddressInfo
+        await fn(new DstackClientV1(`http://127.0.0.1:${port}`))
+      } finally {
+        await new Promise<void>(resolve => server.close(() => resolve()))
+      }
+    }
+
+    const identity = {
+      app_id: 'aa'.repeat(20),
+      compose_hash: 'bb'.repeat(32),
+      instance_id: 'cc'.repeat(20),
+      device_id: 'dd'.repeat(32),
+      os_image_hash: 'ee'.repeat(32),
+      mr_aggregated: 'ff'.repeat(48),
+    }
+
+    it('should reject a non-hex character rather than truncate', async () => {
+      await withAgentAnswering({ ...identity, app_id: 'aabbzz' + 'aa'.repeat(17) }, client =>
+        expect(client.info()).rejects.toThrow(/malformed app_id/)
+      )
+    })
+
+    it('should reject an odd-length string rather than drop a digit', async () => {
+      await withAgentAnswering({ ...identity, compose_hash: 'abc' }, client =>
+        expect(client.info()).rejects.toThrow(/malformed compose_hash/)
+      )
+    })
+
+    it('should name the chain link that is malformed', async () => {
+      await withAgentAnswering(
+        { key: 'aa'.repeat(32), public_key: 'bb'.repeat(33), signature_chain: ['aabb', 'qq'] },
+        client => expect(client.getKey('x', 'secp256k1')).rejects.toThrow(/signature_chain\[1\]/)
+      )
+    })
+
+    it('should reject an absent required field instead of returning empty bytes', async () => {
+      const { instance_id: _dropped, ...without_instance_id } = identity
+      await withAgentAnswering(without_instance_id, client =>
+        expect(client.info()).rejects.toThrow(/no instance_id/)
+      )
+    })
+
+    it('should accept an absent optional field as empty', async () => {
+      const { os_image_hash: _a, mr_aggregated: _b, ...older_agent } = identity
+      await withAgentAnswering(older_agent, async client => {
+        const info = await client.info()
+        expect(info.os_image_hash).toEqual(new Uint8Array(0))
+        expect(info.mr_aggregated).toEqual(new Uint8Array(0))
+        expect(info.app_id.length).toBe(20)
+      })
+    })
+  })
+
   describe('info', () => {
     it('should return the flat identity shape', async () => {
       const client = new DstackClientV1()

sdk/js/src/client-v1.ts

@@ -8,15 +8,55 @@
 import { send_rpc_request } from './send-rpc-request'
 import { to_hex, throwOnRpcError, resolveDstackEndpoint } from './shared'
 
+/** An even number of hex digits, and nothing else. */
+const HEX_ONLY = /^(?:[0-9a-fA-F]{2})*$/
+
+/**
+ * Decode a wire hex string, or say which field was malformed.
+ *
+ * Strict on purpose. Node's hex decoder stops at the first pair it cannot
+ * parse and returns the prefix it managed, without error: `Buffer.from(
+ * '0102zz', 'hex')` is two bytes, and an odd-length string loses its last
+ * digit. These fields are private keys, signature chain links and application
+ * identity -- handing back a silently truncated one is worse than throwing,
+ * and Rust, Python and Go all refuse the same input.
+ */
+function decode_hex(value: string, field: string): Uint8Array {
+  if (!HEX_ONLY.test(value)) {
+    throw new Error(
+      `the agent returned a malformed ${field}: expected an even-length hex string`
+    )
+  }
+  return new Uint8Array(Buffer.from(value, 'hex'))
+}
+
+/**
+ * Decode a `bytes` field the proto declares required.
+ *
+ * Absence is an error rather than the empty default: `app_id` and `key` are
+ * answers the agent always has, so a response without one is a response that
+ * did not come from a working agent. An empty *string* still decodes to zero
+ * bytes, which is what every other SDK does with it.
+ */
+function from_hex(value: string | undefined, field: string): Uint8Array {
+  if (value === undefined || value === null) {
+    throw new Error(`the agent returned no ${field}`)
+  }
+  return decode_hex(value, field)
+}
+
 /**
- * Decode a wire hex string into the bytes the field is declared as.
+ * Decode a `bytes` field, treating absence as the empty default.
  *
- * Every `bytes` field in the v1 proto travels as lowercase hex and surfaces
- * here as a `Uint8Array`, so this runs on all of them. A missing field is
- * proto3's empty default, not an error.
+ * `os_image_hash` and `mr_aggregated` are the two that get this. Both are
+ * plain `bytes` in the proto, so a current agent always sends them -- empty
+ * when it could not compute one. Reading a missing key as those same empty
+ * bytes rather than an error keeps a degraded `Info` readable instead of
+ * unparseable; Rust spells the same rule `#[serde(default)]`. It costs
+ * nothing, because neither field means anything unattested anyway.
  */
-function from_hex(value: string | undefined): Uint8Array {
-  return new Uint8Array(Buffer.from(value ?? '', 'hex'))
+function from_optional_hex(value: string | undefined, field: string): Uint8Array {
+  return value === undefined || value === null ? new Uint8Array(0) : decode_hex(value, field)
 }
 
 export interface IssueCertOptionsV1 {
@@ -157,8 +197,9 @@ export interface InfoResponseV1 {
 /**
  * `InfoResponseV1` as it arrives: identity fields hex, everything else final.
  *
- * `os_image_hash` and `mr_aggregated` are `optional` on the wire, so an older
- * agent may omit them entirely rather than send an empty string.
+ * `os_image_hash` and `mr_aggregated` are optional here rather than on the
+ * wire: the proto declares them plain `bytes` and the agent always sends them,
+ * but a response that omits one is read as empty rather than rejected.
  */
 type InfoResponseV1Wire =
   Omit<InfoResponseV1, '__name__' | 'app_id' | 'compose_hash' | 'instance_id'
@@ -190,7 +231,7 @@ function to_gpu_evidence_bundles(
 ): GpuEvidenceBundleV1[] {
   return (bundles ?? []).map(bundle => Object.freeze({
     ...bundle,
-    evidence: from_hex(bundle.evidence),
+    evidence: from_hex(bundle.evidence, 'GPU evidence'),
   }))
 }
 
@@ -284,9 +325,11 @@ export class DstackClientV1 {
       this.endpoint, '/v1/GetKey', payload)
     throwOnRpcError(result)
     return Object.freeze({
-      key: from_hex(result.key),
-      public_key: from_hex(result.public_key),
-      signature_chain: result.signature_chain.map(from_hex),
+      key: from_hex(result.key, 'key'),
+      public_key: from_hex(result.public_key, 'public_key'),
+      signature_chain: result.signature_chain.map((link, i) =>
+        from_hex(link, `signature_chain[${i}]`)
+      ),
       __name__: 'GetKeyResponseV1' as const,
     })
   }
@@ -321,7 +364,7 @@ export class DstackClientV1 {
     throwOnRpcError(result)
     return Object.freeze({
       __name__: 'AttestResponseV1' as const,
-      attestation: from_hex(result.attestation),
+      attestation: from_hex(result.attestation, 'attestation'),
       boottime_gpu_evidence: to_gpu_evidence_bundles(result.boottime_gpu_evidence),
     })
   }
@@ -358,12 +401,12 @@ export class DstackClientV1 {
     throwOnRpcError(result)
     return Object.freeze({
       ...result,
-      app_id: from_hex(result.app_id),
-      compose_hash: from_hex(result.compose_hash),
-      instance_id: from_hex(result.instance_id),
-      device_id: from_hex(result.device_id),
-      os_image_hash: from_hex(result.os_image_hash),
-      mr_aggregated: from_hex(result.mr_aggregated),
+      app_id: from_hex(result.app_id, 'app_id'),
+      compose_hash: from_hex(result.compose_hash, 'compose_hash'),
+      instance_id: from_hex(result.instance_id, 'instance_id'),
+      device_id: from_hex(result.device_id, 'device_id'),
+      os_image_hash: from_optional_hex(result.os_image_hash, 'os_image_hash'),
+      mr_aggregated: from_optional_hex(result.mr_aggregated, 'mr_aggregated'),
       __name__: 'InfoResponseV1' as const,
     })
   }

sdk/python/src/dstack_sdk/dstack_client_v1.py

@@ -17,6 +17,7 @@
 """
 
 import binascii
+import re
 from typing import Annotated
 from typing import Any
 from typing import Dict
@@ -31,14 +32,22 @@
 from .dstack_client_v0 import BaseClient
 from .dstack_client_v0 import call_async
 
+#: An even number of hex digits, and nothing else.
+_HEX_ONLY = re.compile(r"\A(?:[0-9a-fA-F]{2})*\Z")
+
 
 def _decode_hex(value: Any) -> Any:
-    """Turn a wire hex string into bytes, leaving anything else to pydantic."""
+    r"""Turn a wire hex string into bytes, leaving anything else to pydantic.
+
+    The regex is not redundant with ``bytes.fromhex``: that helper skips ASCII
+    whitespace, so ``"aa bb"`` and ``"aa\nbb"`` decode happily while the error
+    message here promises they do not. Rust and Go reject both, and a field
+    one SDK accepts and another refuses is a field verifiers cannot rely on.
+    """
     if isinstance(value, str):
-        try:
-            return bytes.fromhex(value)
-        except ValueError as err:
-            raise ValueError(f"expected a lowercase hex string: {err}") from err
+        if not _HEX_ONLY.match(value):
+            raise ValueError(f"expected an even-length hex string, got {value!r}")
+        return bytes.fromhex(value)
     return value
 
 
@@ -152,8 +161,11 @@ class InfoResponseV1(BaseModel):
     instance_id: HexBytes
     # Identifies the host machine, not this instance.
     device_id: HexBytes
-    os_image_hash: HexBytes
-    mr_aggregated: HexBytes
+    # Plain `bytes` in the proto, so the agent always sends these -- empty when
+    # it could not compute one. A response that omits one is read as those same
+    # empty bytes rather than rejected, as Rust's `#[serde(default)]` does.
+    os_image_hash: HexBytes = b""
+    mr_aggregated: HexBytes = b""
     vm_config: str = ""
     key_provider_info: str = ""
     cloud_vendor: str = ""