Skip to content

Commit 2e7ca1f

Browse files
committed
fix: update realtime model names to match OpenAI GA release
Transcription: gpt-4o-transcribe, gpt-4o-mini-transcribe, whisper-1. Translation: gpt-4o-transcribe, gpt-4o-mini-transcribe. Removed fake models: gpt-realtime-translate, gpt-realtime-whisper. Added dated snapshots to canary known-models set.
1 parent 448c93f commit 2e7ca1f

8 files changed

Lines changed: 58 additions & 30 deletions

File tree

DRIFT.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,7 @@ Uses `describe.skipIf(!GOOGLE_API_KEY)` like other Gemini tests. The Interaction
131131

132132
**GA Realtime Drift Tests**:
133133

134-
- **Model canary** — Verifies all 5 GA models exist (`gpt-realtime-2`, `gpt-realtime-1.5`, `gpt-realtime-mini`, `gpt-realtime-translate`, `gpt-realtime-whisper`) and flags unknown realtime models
134+
- **Model canary** — Verifies GA models exist (`gpt-realtime`, `gpt-realtime-2`, `gpt-realtime-1.5`, `gpt-realtime-mini` and dated snapshots) and flags unknown realtime models
135135
- **Protocol probe** — Connects with both GA and Beta protocol, normalizes event sequences, and verifies consistency
136136
- **Event shape validation** — GA event names (`response.output_text.delta`, `conversation.item.added`, `conversation.item.done`) and nested session config (`session.audio.*`, `session.type`, `session.reasoning`)
137137

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ Run them all on one port with `npx @copilotkit/aimock --config aimock.json`, or
5757
- **Per-Request Strict Mode**`X-AIMock-Strict` header overrides the server-level `--strict` flag per request (`true`/`1` = strict, `false`/`0` = lenient)
5858
- **[Drift Detection](https://aimock.copilotkit.dev/drift-detection)** — Daily CI validation against real APIs
5959
- **[Streaming Physics](https://aimock.copilotkit.dev/streaming-physics)** — Configurable `ttft`, `tps`, and `jitter`
60-
- **[WebSocket APIs](https://aimock.copilotkit.dev/websocket)** — OpenAI Realtime (GA protocol with 5 models: gpt-realtime-2, gpt-realtime-1.5, gpt-realtime-mini, gpt-realtime-translate, gpt-realtime-whisper; transcription/translation session types; image input; commentary phase), Responses WS, Gemini Live
60+
- **[WebSocket APIs](https://aimock.copilotkit.dev/websocket)** — OpenAI Realtime (GA protocol with models: gpt-realtime, gpt-realtime-2, gpt-realtime-1.5, gpt-realtime-mini; transcription/translation via gpt-4o-transcribe, gpt-4o-mini-transcribe, whisper-1; image input; commentary phase), Responses WS, Gemini Live
6161
- **[Prometheus Metrics](https://aimock.copilotkit.dev/metrics)** — Request counts, latencies, fixture match rates
6262
- **[Docker + Helm](https://aimock.copilotkit.dev/docker)** — Container image and Helm chart for CI/CD
6363
- **[Vitest & Jest Plugins](https://aimock.copilotkit.dev/test-plugins)** — Zero-config `useAimock()` with auto lifecycle and env patching

docs/websocket/index.html

Lines changed: 18 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,11 @@ <h3>Supported Models</h3>
130130
</tr>
131131
</thead>
132132
<tbody>
133+
<tr>
134+
<td>gpt-realtime</td>
135+
<td>conversation</td>
136+
<td>Base alias &mdash; resolves to latest GA model</td>
137+
</tr>
133138
<tr>
134139
<td>gpt-realtime-2</td>
135140
<td>conversation</td>
@@ -146,14 +151,19 @@ <h3>Supported Models</h3>
146151
<td>Smaller, faster GA model</td>
147152
</tr>
148153
<tr>
149-
<td>gpt-realtime-translate</td>
150-
<td>translation</td>
151-
<td>Real-time speech translation</td>
154+
<td>gpt-4o-transcribe</td>
155+
<td>transcription, translation</td>
156+
<td>Speech transcription and translation</td>
157+
</tr>
158+
<tr>
159+
<td>gpt-4o-mini-transcribe</td>
160+
<td>transcription, translation</td>
161+
<td>Smaller transcription and translation model</td>
152162
</tr>
153163
<tr>
154-
<td>gpt-realtime-whisper</td>
164+
<td>whisper-1</td>
155165
<td>transcription</td>
156-
<td>Real-time speech transcription</td>
166+
<td>Legacy Whisper transcription model</td>
157167
</tr>
158168
</tbody>
159169
</table>
@@ -166,11 +176,12 @@ <h3>Session Types</h3>
166176
</li>
167177
<li>
168178
<strong>transcription</strong> &mdash; Audio-to-text transcription (requires
169-
<code>gpt-realtime-whisper</code>)
179+
<code>gpt-4o-transcribe</code>, <code>gpt-4o-mini-transcribe</code>, or
180+
<code>whisper-1</code>)
170181
</li>
171182
<li>
172183
<strong>translation</strong> &mdash; Real-time speech translation (requires
173-
<code>gpt-realtime-translate</code>)
184+
<code>gpt-4o-transcribe</code> or <code>gpt-4o-mini-transcribe</code>)
174185
</li>
175186
</ul>
176187

scripts/update-competitive-matrix.ts

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -90,10 +90,11 @@ const FEATURE_RULES: FeatureRule[] = [
9090
],
9191
},
9292
{
93-
rowLabel: "Realtime translate/whisper",
93+
rowLabel: "Realtime transcription/translation",
9494
keywords: [
95-
"gpt-realtime-translate",
96-
"gpt-realtime-whisper",
95+
"gpt-4o-transcribe",
96+
"gpt-4o-mini-transcribe",
97+
"whisper-1",
9798
"realtime.*transcription",
9899
"realtime.*translation",
99100
],

src/__tests__/competitive-matrix.test.ts

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -101,10 +101,11 @@ const FEATURE_RULES: FeatureRule[] = [
101101
],
102102
},
103103
{
104-
rowLabel: "Realtime translate/whisper",
104+
rowLabel: "Realtime transcription/translation",
105105
keywords: [
106-
"gpt-realtime-translate",
107-
"gpt-realtime-whisper",
106+
"gpt-4o-transcribe",
107+
"gpt-4o-mini-transcribe",
108+
"whisper-1",
108109
"realtime.*transcription",
109110
"realtime.*translation",
110111
],

src/__tests__/drift/ws-realtime.drift.ts

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,18 +56,27 @@ describe.skipIf(!OPENAI_API_KEY)("OpenAI Realtime API drift", () => {
5656
const models = await listOpenAIModels(config.apiKey);
5757

5858
const gaModels = [
59+
"gpt-realtime",
5960
"gpt-realtime-2",
61+
"gpt-realtime-2025-08-28",
6062
"gpt-realtime-1.5",
6163
"gpt-realtime-mini",
62-
"gpt-realtime-translate",
63-
"gpt-realtime-whisper",
64+
"gpt-realtime-mini-2025-10-06",
65+
"gpt-realtime-mini-2025-12-15",
6466
];
6567
const knownModels = new Set([
6668
...gaModels,
69+
// Audio models also valid in realtime sessions
70+
"gpt-audio-1.5",
71+
"gpt-audio-mini",
72+
"gpt-audio-mini-2025-10-06",
73+
"gpt-audio-mini-2025-12-15",
6774
// Legacy preview models (may still appear)
6875
"gpt-4o-realtime-preview",
6976
"gpt-4o-mini-realtime-preview",
7077
"gpt-4o-realtime-preview-2024-10-01",
78+
"gpt-4o-realtime-preview-2024-12-17",
79+
"gpt-4o-realtime-preview-2025-06-03",
7180
"gpt-4o-mini-realtime-preview-2024-12-17",
7281
]);
7382

src/__tests__/ws-realtime.test.ts

Lines changed: 13 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1295,14 +1295,14 @@ describe("WebSocket /v1/realtime", () => {
12951295
// Skip session.created
12961296
await ws.waitForMessages(1);
12971297

1298-
// Update session to transcription mode with whisper model
1299-
ws.send(sessionUpdate({ type: "transcription", model: "gpt-realtime-whisper" }));
1298+
// Update session to transcription mode with transcribe model
1299+
ws.send(sessionUpdate({ type: "transcription", model: "gpt-4o-transcribe" }));
13001300

13011301
const updateRaw = await ws.waitForMessages(2);
13021302
const updateEvent = parseEvents(updateRaw.slice(1))[0];
13031303
expect(updateEvent.type).toBe("session.updated");
13041304
expect((updateEvent.session as Record<string, unknown>).type).toBe("transcription");
1305-
expect((updateEvent.session as Record<string, unknown>).model).toBe("gpt-realtime-whisper");
1305+
expect((updateEvent.session as Record<string, unknown>).model).toBe("gpt-4o-transcribe");
13061306

13071307
// Send audio buffer messages
13081308
ws.send(JSON.stringify({ type: "input_audio_buffer.append", audio: "base64data" }));
@@ -1389,13 +1389,13 @@ describe("WebSocket /v1/realtime", () => {
13891389

13901390
await ws.waitForMessages(1); // session.created
13911391

1392-
ws.send(sessionUpdate({ type: "translation", model: "gpt-realtime-translate" }));
1392+
ws.send(sessionUpdate({ type: "translation", model: "gpt-4o-transcribe" }));
13931393

13941394
const raw = await ws.waitForMessages(2);
13951395
const event = parseEvents(raw.slice(1))[0];
13961396
expect(event.type).toBe("session.updated");
13971397
expect((event.session as Record<string, unknown>).type).toBe("translation");
1398-
expect((event.session as Record<string, unknown>).model).toBe("gpt-realtime-translate");
1398+
expect((event.session as Record<string, unknown>).model).toBe("gpt-4o-transcribe");
13991399

14001400
ws.close();
14011401
});
@@ -1425,7 +1425,7 @@ describe("WebSocket /v1/realtime", () => {
14251425

14261426
await ws.waitForMessages(1); // session.created
14271427

1428-
ws.send(sessionUpdate({ type: "translation", model: "gpt-realtime-2" }));
1428+
ws.send(sessionUpdate({ type: "translation", model: "gpt-realtime-mini" }));
14291429

14301430
const raw = await ws.waitForMessages(2);
14311431
const event = parseEvents(raw.slice(1))[0];
@@ -1444,7 +1444,7 @@ describe("WebSocket /v1/realtime", () => {
14441444

14451445
await ws.waitForMessages(1); // session.created
14461446

1447-
ws.send(sessionUpdate({ type: "translation", model: "gpt-realtime-translate" }));
1447+
ws.send(sessionUpdate({ type: "translation", model: "gpt-4o-transcribe" }));
14481448
await ws.waitForMessages(2); // session.updated
14491449

14501450
ws.send(JSON.stringify({ type: "input_audio_buffer.commit" }));
@@ -1539,11 +1539,13 @@ describe("WebSocket /v1/realtime", () => {
15391539

15401540
// ── GA model acceptance tests ───────────────────────────────────────────
15411541
it.each([
1542+
"gpt-realtime",
15421543
"gpt-realtime-2",
1544+
"gpt-realtime-2025-08-28",
15431545
"gpt-realtime-1.5",
15441546
"gpt-realtime-mini",
1545-
"gpt-realtime-translate",
1546-
"gpt-realtime-whisper",
1547+
"gpt-realtime-mini-2025-10-06",
1548+
"gpt-realtime-mini-2025-12-15",
15471549
])("accepts GA model %s via query parameter", async (model) => {
15481550
instance = await createServer(allFixtures);
15491551
const ws = await connectWebSocket(instance.url, `/v1/realtime?model=${model}`);
@@ -1598,7 +1600,7 @@ describe("WebSocket /v1/realtime", () => {
15981600

15991601
it("sets _endpointType to realtime-transcription for transcription sessions", async () => {
16001602
instance = await createServer(allFixtures);
1601-
const ws = await connectWebSocket(instance.url, "/v1/realtime?model=gpt-realtime-whisper");
1603+
const ws = await connectWebSocket(instance.url, "/v1/realtime?model=gpt-4o-transcribe");
16021604

16031605
await ws.waitForMessages(1); // session.created
16041606

@@ -1624,7 +1626,7 @@ describe("WebSocket /v1/realtime", () => {
16241626

16251627
it("sets _endpointType to realtime-translation for translation sessions", async () => {
16261628
instance = await createServer(allFixtures);
1627-
const ws = await connectWebSocket(instance.url, "/v1/realtime?model=gpt-realtime-translate");
1629+
const ws = await connectWebSocket(instance.url, "/v1/realtime?model=gpt-4o-transcribe");
16281630

16291631
await ws.waitForMessages(1); // session.created
16301632

src/ws-realtime.ts

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -485,8 +485,12 @@ async function processMessage(
485485
} | null;
486486

487487
// Validate model+type combinations (rollback on failure)
488-
const transcriptionModels = new Set(["gpt-realtime-whisper"]);
489-
const translationModels = new Set(["gpt-realtime-translate"]);
488+
const transcriptionModels = new Set([
489+
"gpt-4o-transcribe",
490+
"gpt-4o-mini-transcribe",
491+
"whisper-1",
492+
]);
493+
const translationModels = new Set(["gpt-4o-transcribe", "gpt-4o-mini-transcribe"]);
490494

491495
if (session.type === "transcription" && !transcriptionModels.has(session.model)) {
492496
session.model = prevModel;

0 commit comments

Comments
 (0)