diff --git a/PRD.md b/PRD.md index dbc0349..5c6917f 100644 --- a/PRD.md +++ b/PRD.md @@ -379,6 +379,8 @@ Question - Prefer 3–5 most relevant files - Include related files in output - Keep technical labels readable and consistent +- Stream human-readable AI responses progressively +- Keep `--json` buffered so stdout remains exactly one valid JSON document --- @@ -454,6 +456,7 @@ JSON mode rules: - no ANSI colors, Markdown rendering, box drawing, or progress text - runtime errors use a stable `{ "status": "error", "error": "...", "hint": "..." }` shape - `init --json` is non-interactive and requires `GROQ_API_KEY` or existing config +- AI responses are buffered instead of streamed - human-readable output remains the default - package-manager wrappers may still write their own warnings to stderr diff --git a/docs/architecture.md b/docs/architecture.md index 6f3090c..58c3e28 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -502,11 +502,31 @@ Only Groq production models should be used as public defaults. Users can override automatic routing with `devmap config model `. Running `devmap config model auto` restores the defaults above. -Raw provider errors should not be shown directly to users. - ---- - -## Prompt Strategy +Raw provider errors should not be shown directly to users. + +--- + +## Streaming AI Output + +Groq chat completions use server-sent events for human-readable `analyze` and +`ask` output. The provider adapter reconstructs the complete response while +emitting incremental deltas to the output layer. + +Terminal Markdown is buffered to paragraph boundaries before rendering. This +keeps headings, lists, tables, wrapping, and inline formatting readable while +still showing the answer before generation has fully completed. + +Rules: + +* streaming is an optional `AiClient` capability +* commands fall back to regular completion for clients without streaming +* the final reconstructed text is used for snapshot persistence and metadata +* rate-limit retry and model fallback happen before consuming response deltas +* `--json` never streams because stdout must contain one complete JSON document + +--- + +## Prompt Strategy Prompt templates should be centralized. diff --git a/docs/commands.md b/docs/commands.md index d6a5348..391cf26 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -234,7 +234,9 @@ Shared utilities, database access, authentication logic, and helpers. * Do not send the entire project source to AI * Snapshot must be regenerated after analyze * Snapshot must remain compact and deterministic -* Raw provider errors must not be shown directly to users +* Raw provider errors must not be shown directly to users +* New AI interpretation streams progressively in human-readable mode +* Cached interpretation is rendered immediately without a provider request --- @@ -317,7 +319,9 @@ app/api/auth/* * Include related files in output * Keep answer readable * Respond in the same language as the question -* Technical labels can remain in English +* Technical labels can remain in English +* Stream new AI answers progressively in human-readable mode +* Do not stream `--json`; emit one complete JSON document instead ### Output Example @@ -508,6 +512,7 @@ Contract: * stdout contains exactly one JSON document * ANSI codes and terminal decoration are disabled * progress sections and Markdown rendering are omitted +* AI responses are buffered instead of streamed * runtime failures return a JSON object with `status`, `error`, and optional `hint` * `init --json` never prompts and therefore requires `GROQ_API_KEY` or an existing API key diff --git a/docs/development-testing.md b/docs/development-testing.md index d17ddb4..21f6d08 100644 --- a/docs/development-testing.md +++ b/docs/development-testing.md @@ -14,3 +14,22 @@ Parse stdout directly as JSON and verify that it contains no ANSI codes or terminal decoration. When invoking through `npm exec` or another package manager, ignore wrapper-owned stderr warnings and validate DevMap stdout separately. + +## AI Streaming Output + +Focused verification: + +```bash +pnpm --filter devmap exec tsx --test test/ai-client.test.ts test/ask-command.test.ts test/analyze-ai.test.ts test/json-output.test.ts +``` + +With a live Groq key, run: + +```bash +devmap analyze --fresh +devmap ask "explain the main architecture" +devmap ask "explain the main architecture" --json +``` + +Human output should appear progressively without raw Markdown markers. JSON +output should wait for completion and remain one parseable document. diff --git a/docs/for-me-personal/PROGRESS.md b/docs/for-me-personal/PROGRESS.md index cc3f197..e5a6ab0 100644 --- a/docs/for-me-personal/PROGRESS.md +++ b/docs/for-me-personal/PROGRESS.md @@ -1,6 +1,24 @@ # Progress DevMap -Terakhir diperbarui: 2026-06-14 +Terakhir diperbarui: 2026-06-15 + +## Update 2026-06-15 + +### AI Response Streaming + +- Human-readable `devmap ask` dan AI interpretation pada `devmap analyze` + sekarang memakai Groq server-sent events. +- Delta response direkonstruksi menjadi hasil lengkap untuk token metadata, + snapshot persistence, dan cache. +- Output ditampilkan progresif per paragraf agar heading, list, table, wrapping, + dan inline Markdown tetap rapi. +- Provider yang belum memiliki method streaming tetap memakai regular + completion tanpa mengubah public command behavior. +- Retry rate limit dan model fallback tetap berjalan sebelum stream dibaca. +- `--json` sengaja tidak memakai streaming agar stdout tetap satu dokumen JSON. +- Automated test mencakup SSE yang terpecah antar-network chunk, command + streaming, snapshot persistence, dan JSON non-streaming. +- Automated test saat ini berjumlah 65 dan seluruhnya lulus. ## Update 2026-06-14 diff --git a/docs/for-me-personal/TEST.md b/docs/for-me-personal/TEST.md index 738330b..52b9aac 100644 --- a/docs/for-me-personal/TEST.md +++ b/docs/for-me-personal/TEST.md @@ -101,6 +101,39 @@ Expected: - API keys are never included; - packed package E2E verifies JSON output after tarball installation. +## AI Response Streaming + +Focused automated test: + +```powershell +pnpm --filter devmap exec tsx --test test/ai-client.test.ts test/ask-command.test.ts test/analyze-ai.test.ts test/json-output.test.ts +``` + +Coverage penting: + +- SSE event tetap terbaca ketika JSON event terpecah pada network chunk; +- delta dikirim berurutan dan hasil lengkap dikembalikan provider; +- `ask` dan fresh AI interpretation `analyze` memakai streaming jika tersedia; +- hasil lengkap `analyze` tetap disimpan ke snapshot; +- `--json` memakai completion penuh dan tidak memanggil streaming. + +Manual live check: + +```powershell +$env:GROQ_API_KEY="gsk_your_key" +pnpm dev:cli -- analyze --fresh +pnpm dev:cli -- ask "explain the main architecture" +pnpm dev:cli -- ask "explain the main architecture" --json | ConvertFrom-Json +Remove-Item Env:GROQ_API_KEY +``` + +Expected: + +- human output mulai tampil sebelum seluruh AI response selesai; +- Markdown tidak tampil mentah; +- model dan token usage tetap muncul setelah stream selesai; +- JSON baru dicetak setelah response lengkap dan dapat diparse langsung. + ## Urutan Testing Yang Direkomendasikan Untuk development harian: diff --git a/docs/roadmap.md b/docs/roadmap.md index 8955d97..5ef86f8 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -34,7 +34,7 @@ adding AI on top. If the foundation is wrong, AI output will be wrong too. - Groq integration with provider abstraction layer - Prompt templates for analyze and ask - Context Builder — keyword search + file ranking -- Streaming output +- [x] Streaming output for human `analyze` and `ask` responses - Retry logic + model fallback - Token-aware context trimming (max 5 files, max 200 lines each) - Cache integration — skip AI for unchanged files @@ -120,4 +120,4 @@ Not planned. Not scheduled. Revisit when Phase 5 ships. | 1.2.0 | 2 | Express support solidified | | 2.0.0 | 3 | `devmap docs` + `devmap onboard` | | 3.0.0 | 4 | `devmap deadcode` + `devmap flow` + `devmap report` | -| 4.0.0 | 5 | OpenAI + Gemini support | \ No newline at end of file +| 4.0.0 | 5 | OpenAI + Gemini support | diff --git a/packages/cli/src/ai/completion.ts b/packages/cli/src/ai/completion.ts new file mode 100644 index 0000000..cd590b7 --- /dev/null +++ b/packages/cli/src/ai/completion.ts @@ -0,0 +1,42 @@ +import { output } from "../utils/output.js"; +import type { + AiClient, + AiCompletionRequest, + AiCompletionResult +} from "./types.js"; + +export type AiCompletionExecution = { + result: AiCompletionResult; + streamed: boolean; +}; + +export async function completeWithOptionalStreaming( + client: AiClient, + request: AiCompletionRequest, + enabled: boolean, + onStreamStart?: () => void +): Promise { + if (!enabled || !client.stream) { + return { + result: await client.complete(request), + streamed: false + }; + } + + const renderer = output.markdownStream(); + let started = false; + try { + const result = await client.stream(request, (delta) => { + if (!started) { + onStreamStart?.(); + started = true; + } + renderer.write(delta); + }); + renderer.end(); + return { result, streamed: true }; + } catch (error) { + renderer.end(); + throw error; + } +} diff --git a/packages/cli/src/ai/groq.ts b/packages/cli/src/ai/groq.ts index fd694af..060de90 100644 --- a/packages/cli/src/ai/groq.ts +++ b/packages/cli/src/ai/groq.ts @@ -3,6 +3,7 @@ import type { AiClient, AiCompletionRequest, AiCompletionResult, + AiDeltaHandler, AiTokenUsage } from "./types.js"; @@ -64,46 +65,55 @@ export class GroqClient implements AiClient { throw primaryResult.error; } - private async requestModel( + async stream( request: AiCompletionRequest, - model: string - ): Promise { - let response = await this.sendRequest(request, model); + onDelta: AiDeltaHandler + ): Promise { + const primaryResult = await this.requestModelStream( + request, + request.model, + onDelta + ); - for ( - let retryAttempt = 0; - response.status === 429 && retryAttempt < MAX_RATE_LIMIT_RETRIES; - retryAttempt += 1 + if (primaryResult.ok) { + return primaryResult.result; + } + + if ( + primaryResult.modelUnavailable + && request.fallbackModel + && request.fallbackModel !== request.model ) { - const delay = Math.min( - readRetryDelay(response) * (2 ** retryAttempt), - MAX_RATE_LIMIT_DELAY_MS + const fallbackResult = await this.requestModelStream( + request, + request.fallbackModel, + onDelta ); - await this.sleep(delay); - response = await this.sendRequest(request, model); + if (fallbackResult.ok) { + return fallbackResult.result; + } + + throw fallbackResult.error; } + throw primaryResult.error; + } + + private async requestModel( + request: AiCompletionRequest, + model: string + ): Promise { + const response = await this.sendWithRateLimitRetries(request, model, false); + if (!response.ok) { - const providerMessage = await readProviderError(response); - return { - ok: false, - modelUnavailable: isModelUnavailable(response.status, providerMessage), - error: mapGroqError(response.status, providerMessage) - }; + return readFailedRequest(response); } const payload = await readCompletionPayload(response); const content = payload.choices[0]?.message?.content?.trim(); if (!content) { - return { - ok: false, - modelUnavailable: false, - error: new DevmapError( - "Groq returned an empty response.", - "Try the question again or run devmap doctor." - ) - }; + return emptyResponseResult(); } return { @@ -116,9 +126,59 @@ export class GroqClient implements AiClient { }; } + private async requestModelStream( + request: AiCompletionRequest, + model: string, + onDelta: AiDeltaHandler + ): Promise { + const response = await this.sendWithRateLimitRetries(request, model, true); + + if (!response.ok) { + return readFailedRequest(response); + } + + const payload = await readCompletionStream(response, model, onDelta); + if (!payload.content.trim()) { + return emptyResponseResult(); + } + + return { + ok: true, + result: { + content: payload.content.trim(), + model: payload.model, + ...(payload.usage ? { usage: payload.usage } : {}) + } + }; + } + + private async sendWithRateLimitRetries( + request: AiCompletionRequest, + model: string, + stream: boolean + ): Promise { + let response = await this.sendRequest(request, model, stream); + + for ( + let retryAttempt = 0; + response.status === 429 && retryAttempt < MAX_RATE_LIMIT_RETRIES; + retryAttempt += 1 + ) { + const delay = Math.min( + readRetryDelay(response) * (2 ** retryAttempt), + MAX_RATE_LIMIT_DELAY_MS + ); + await this.sleep(delay); + response = await this.sendRequest(request, model, stream); + } + + return response; + } + private async sendRequest( request: AiCompletionRequest, - model: string + model: string, + stream = false ): Promise { try { return await this.fetchImplementation(GROQ_CHAT_COMPLETIONS_URL, { @@ -131,7 +191,11 @@ export class GroqClient implements AiClient { model, messages: request.messages, max_completion_tokens: request.maxCompletionTokens ?? 1200, - temperature: request.temperature ?? 0.2 + temperature: request.temperature ?? 0.2, + ...(stream ? { + stream: true, + stream_options: { include_usage: true } + } : {}) }) }); } catch { @@ -203,10 +267,43 @@ type GroqCompletionPayload = { }; }; +type GroqStreamPayload = { + model?: string; + choices?: Array<{ + delta?: { + content?: string | null; + }; + }>; + usage?: GroqCompletionPayload["usage"]; + x_groq?: { + usage?: GroqCompletionPayload["usage"]; + }; +}; + type GroqRequestResult = | { ok: true; result: AiCompletionResult } | { ok: false; modelUnavailable: boolean; error: DevmapError }; +async function readFailedRequest(response: Response): Promise { + const providerMessage = await readProviderError(response); + return { + ok: false, + modelUnavailable: isModelUnavailable(response.status, providerMessage), + error: mapGroqError(response.status, providerMessage) + }; +} + +function emptyResponseResult(): GroqRequestResult { + return { + ok: false, + modelUnavailable: false, + error: new DevmapError( + "Groq returned an empty response.", + "Try the question again or run devmap doctor." + ) + }; +} + async function readCompletionPayload(response: Response): Promise { try { const payload = await response.json() as Partial; @@ -236,6 +333,104 @@ async function readProviderError(response: Response): Promise { } } +async function readCompletionStream( + response: Response, + requestedModel: string, + onDelta: AiDeltaHandler +): Promise<{ + content: string; + model: string; + usage?: AiTokenUsage; +}> { + if (!response.body) { + throw new DevmapError( + "Groq returned an unreadable streaming response.", + "Try again shortly or run devmap doctor." + ); + } + + const reader = response.body.getReader(); + const decoder = new TextDecoder(); + let buffer = ""; + let content = ""; + let model = requestedModel; + let usage: AiTokenUsage | undefined; + + const consumeEvent = (event: string): boolean => { + const data = event + .split(/\r?\n/) + .filter((line) => line.startsWith("data:")) + .map((line) => line.slice(5).trimStart()) + .join("\n"); + + if (!data) return false; + if (data.trim() === "[DONE]") return true; + + let payload: GroqStreamPayload; + try { + payload = JSON.parse(data) as GroqStreamPayload; + } catch { + throw new DevmapError( + "Groq returned an unreadable streaming response.", + "Try again shortly or check https://status.groq.com." + ); + } + + model = payload.model || model; + const delta = payload.choices?.[0]?.delta?.content; + if (delta) { + content += delta; + onDelta(delta); + } + + const rawUsage = payload.usage ?? payload.x_groq?.usage; + if (rawUsage) { + usage = normalizeUsage(rawUsage); + } + + return false; + }; + + try { + let done = false; + while (!done) { + const readResult = await reader.read(); + buffer += decoder.decode(readResult.value, { stream: !readResult.done }); + + const events = buffer.split(/\r?\n\r?\n/); + buffer = events.pop() ?? ""; + for (const event of events) { + if (consumeEvent(event)) { + done = true; + break; + } + } + + if (readResult.done) { + if (!done && buffer.trim()) { + consumeEvent(buffer); + } + break; + } + } + } catch (error) { + if (error instanceof DevmapError) { + throw error; + } + + throw new DevmapError( + "The Groq response stream ended unexpectedly.", + "Try again or run devmap doctor if the problem continues." + ); + } + + return { + content, + model, + ...(usage ? { usage } : {}) + }; +} + function mapGroqError(status: number, providerMessage: string): DevmapError { if (status === 401 || status === 403) { return new DevmapError( diff --git a/packages/cli/src/ai/types.ts b/packages/cli/src/ai/types.ts index 07e09c8..74d7951 100644 --- a/packages/cli/src/ai/types.ts +++ b/packages/cli/src/ai/types.ts @@ -23,6 +23,12 @@ export type AiCompletionResult = { usage?: AiTokenUsage; }; +export type AiDeltaHandler = (delta: string) => void; + export interface AiClient { complete(request: AiCompletionRequest): Promise; + stream?( + request: AiCompletionRequest, + onDelta: AiDeltaHandler + ): Promise; } diff --git a/packages/cli/src/commands/analyze.ts b/packages/cli/src/commands/analyze.ts index 6b32ec1..a4357a3 100644 --- a/packages/cli/src/commands/analyze.ts +++ b/packages/cli/src/commands/analyze.ts @@ -1,4 +1,5 @@ import { resolve } from "node:path"; +import { completeWithOptionalStreaming } from "../ai/completion.js"; import { DEFAULT_AI_MODELS, GroqClient } from "../ai/groq.js"; import { buildAnalyzeMessages } from "../ai/prompts.js"; import type { AiClient } from "../ai/types.js"; @@ -151,13 +152,14 @@ async function printOrGenerateInterpretation( output.step(`Interpreting architecture with ${model}`); try { - const interpretation = await client.complete({ + const execution = await completeWithOptionalStreaming(client, { messages: buildAnalyzeMessages(snapshot, options.deep), model, fallbackModel: DEFAULT_AI_MODELS.fallback, maxCompletionTokens: options.deep ? 1800 : 1000, temperature: 0.2 - }); + }, !options.json, () => output.section("Architecture")); + const interpretation = execution.result; const updatedSnapshot = { ...snapshot, ai: { @@ -169,8 +171,10 @@ async function printOrGenerateInterpretation( }; await saveSnapshot(projectRoot, updatedSnapshot); - output.section("Architecture"); - output.markdown(interpretation.content); + if (!execution.streamed) { + output.section("Architecture"); + output.markdown(interpretation.content); + } output.note(formatAiMetadata( interpretation.model, interpretation.usage, diff --git a/packages/cli/src/commands/ask.ts b/packages/cli/src/commands/ask.ts index 6010c68..205c799 100644 --- a/packages/cli/src/commands/ask.ts +++ b/packages/cli/src/commands/ask.ts @@ -1,4 +1,5 @@ import { buildQuestionContext } from "../ai/contextBuilder.js"; +import { completeWithOptionalStreaming } from "../ai/completion.js"; import { DEFAULT_AI_MODELS, GroqClient } from "../ai/groq.js"; import { buildAskMessages } from "../ai/prompts.js"; import type { AiClient } from "../ai/types.js"; @@ -104,16 +105,19 @@ async function runAsk( output.step(`Asking Groq with ${model}`); try { - const answer = await client.complete({ + const execution = await completeWithOptionalStreaming(client, { messages: buildAskMessages(context, snapshot.project), model, fallbackModel: DEFAULT_AI_MODELS.fallback, maxCompletionTokens: 1200, temperature: 0.2 - }); + }, !dependencies.json, () => output.section("Answer")); + const answer = execution.result; - output.section("Answer"); - output.markdown(answer.content); + if (!execution.streamed) { + output.section("Answer"); + output.markdown(answer.content); + } output.note(formatUsage(answer.model, answer.usage)); return { status: "ok", diff --git a/packages/cli/src/utils/output.ts b/packages/cli/src/utils/output.ts index 29f429e..89f4f12 100644 --- a/packages/cli/src/utils/output.ts +++ b/packages/cli/src/utils/output.ts @@ -26,6 +26,11 @@ function color(value: string | number, tone: keyof typeof theme): string { return `${theme[tone]}${value}${theme.reset}`; } +export type MarkdownStream = { + write(chunk: string): void; + end(): void; +}; + export const output = { section(title: string): void { if (isJsonOutput()) return; @@ -81,6 +86,35 @@ export const output = { })); }, + markdownStream(): MarkdownStream { + let buffer = ""; + + const renderParagraph = (paragraph: string): void => { + if (isJsonOutput() || paragraph.trim() === "") return; + console.log(renderTerminalMarkdown(paragraph, { + width: process.stdout.columns ?? 80, + colors: true + })); + }; + + return { + write(chunk: string): void { + buffer += chunk.replace(/\r\n?/g, "\n"); + + let boundary = buffer.indexOf("\n\n"); + while (boundary >= 0) { + renderParagraph(buffer.slice(0, boundary)); + buffer = buffer.slice(boundary + 2); + boundary = buffer.indexOf("\n\n"); + } + }, + end(): void { + renderParagraph(buffer); + buffer = ""; + } + }; + }, + json(value: unknown): void { console.log(JSON.stringify(value)); } diff --git a/packages/cli/test/ai-client.test.ts b/packages/cli/test/ai-client.test.ts index a90aefe..3a788d8 100644 --- a/packages/cli/test/ai-client.test.ts +++ b/packages/cli/test/ai-client.test.ts @@ -45,6 +45,52 @@ test("Groq client returns normalized content and token usage", async () => { }); }); +test("Groq client streams split SSE deltas and returns the complete result", async () => { + const deltas: string[] = []; + const encoder = new TextEncoder(); + const client = new GroqClient("gsk_test", { + fetch: async (_url, init) => { + const body = JSON.parse(String(init?.body)) as { + stream?: boolean; + stream_options?: { include_usage?: boolean }; + }; + assert.equal(body.stream, true); + assert.equal(body.stream_options?.include_usage, true); + + return new Response(new ReadableStream({ + start(controller) { + controller.enqueue(encoder.encode( + 'data: {"model":"llama-3.1-8b-instant","choices":[{"delta":{"content":"Auth"}}]}\n' + )); + controller.enqueue(encoder.encode( + '\ndata: {"model":"llama-3.1-8b-instant","choices":[{"delta":{"content":" works."}}],"usage":{"prompt_tokens":10,"completion_tokens":2,"total_tokens":12}}\n\n' + )); + controller.enqueue(encoder.encode("data: [DONE]\n\n")); + controller.close(); + } + }), { + headers: { "content-type": "text/event-stream" } + }); + } + }); + + const result = await client.stream({ + messages: [{ role: "user", content: "Explain auth." }], + model: DEFAULT_AI_MODELS.ask + }, (delta) => { + deltas.push(delta); + }); + + assert.deepEqual(deltas, ["Auth", " works."]); + assert.equal(result.content, "Auth works."); + assert.equal(result.model, DEFAULT_AI_MODELS.ask); + assert.deepEqual(result.usage, { + promptTokens: 10, + completionTokens: 2, + totalTokens: 12 + }); +}); + test("Groq client retries rate limits with exponential backoff", async () => { let requestCount = 0; const delays: number[] = []; diff --git a/packages/cli/test/analyze-ai.test.ts b/packages/cli/test/analyze-ai.test.ts index be9f8d6..835a315 100644 --- a/packages/cli/test/analyze-ai.test.ts +++ b/packages/cli/test/analyze-ai.test.ts @@ -91,6 +91,62 @@ test("analyze stores and reuses AI architecture interpretation", async () => { } }); +test("analyze streams new AI interpretation and persists the complete text", async () => { + const projectRoot = await mkdtemp(join(tmpdir(), "devmap-analyze-stream-")); + let streamCalls = 0; + const client: AiClient = { + async complete(): Promise { + throw new Error("complete should not be used for human output"); + }, + async stream(request, onDelta): Promise { + streamCalls += 1; + onDelta("## Overview\n\n"); + onDelta("The project has one entry point."); + return { + content: "## Overview\n\nThe project has one entry point.", + model: request.model + }; + } + }; + + try { + await writeFile( + join(projectRoot, "package.json"), + JSON.stringify({ name: "analyze-stream-fixture" }), + "utf8" + ); + await writeFile(join(projectRoot, "index.ts"), "export const ready = true;\n", "utf8"); + + const logs = stripAnsi(await captureOutput(() => analyzeCommand( + projectRoot, + { fresh: true }, + { + loadConfig: async () => ({ + provider: "groq", + apiKey: "gsk_fixture", + model: "auto" + }), + createAiClient: () => client + } + ))); + + assert.equal(streamCalls, 1); + assert.match(logs, /Overview\n-+/); + assert.match(logs, /The project has one entry point/); + + const saved = await inspectSnapshot(projectRoot); + assert.equal(saved.status, "valid"); + if (saved.status === "valid") { + assert.equal( + saved.snapshot.ai?.architecture, + "## Overview\n\nThe project has one entry point." + ); + } + } finally { + await rm(projectRoot, { recursive: true, force: true }); + } +}); + test("analyze warns and continues when package.json is malformed", async () => { const projectRoot = await mkdtemp(join(tmpdir(), "devmap-malformed-package-")); diff --git a/packages/cli/test/ask-command.test.ts b/packages/cli/test/ask-command.test.ts index d3de73f..3ad7b82 100644 --- a/packages/cli/test/ask-command.test.ts +++ b/packages/cli/test/ask-command.test.ts @@ -60,6 +60,49 @@ test("ask command uses configured AI client and prints token usage", async () => } }); +test("ask command streams AI paragraphs when the client supports streaming", async () => { + const projectRoot = await createAskProject(); + let completeCalls = 0; + let streamCalls = 0; + const client: AiClient = { + async complete(): Promise { + completeCalls += 1; + throw new Error("complete should not be used for human output"); + }, + async stream(request, onDelta): Promise { + streamCalls += 1; + onDelta("## Authentication\n\n"); + onDelta("Authentication uses `auth.ts`."); + return { + content: "## Authentication\n\nAuthentication uses `auth.ts`.", + model: request.model + }; + } + }; + + try { + const logs = stripAnsi(await captureOutput(() => askCommand( + ["where", "is", "auth"], + { + projectRoot, + loadConfig: async () => ({ + provider: "groq", + apiKey: "gsk_fixture", + model: "auto" + }), + createAiClient: () => client + } + ))); + + assert.equal(streamCalls, 1); + assert.equal(completeCalls, 0); + assert.match(logs, /Authentication\n-+/); + assert.match(logs, /Authentication uses auth\.ts\./); + } finally { + await rm(projectRoot, { recursive: true, force: true }); + } +}); + test("ask command falls back to static context after actionable AI errors", async () => { const projectRoot = await createAskProject(); const client: AiClient = { diff --git a/packages/cli/test/json-output.test.ts b/packages/cli/test/json-output.test.ts index 1c2c017..fb87d29 100644 --- a/packages/cli/test/json-output.test.ts +++ b/packages/cli/test/json-output.test.ts @@ -34,8 +34,11 @@ test("ask --json emits answer, relevant files, model, and usage", async () => { const projectRoot = await createProject("json-ask"); const snapshot = await createProjectMap(projectRoot); await saveSnapshot(projectRoot, snapshot); + let completeCalls = 0; + let streamCalls = 0; const client: AiClient = { async complete(request) { + completeCalls += 1; return { content: "The entry point is index.ts.", model: request.model, @@ -45,6 +48,10 @@ test("ask --json emits answer, relevant files, model, and usage", async () => { totalTokens: 28 } }; + }, + async stream() { + streamCalls += 1; + throw new Error("JSON output must not use streaming"); } }; @@ -69,6 +76,8 @@ test("ask --json emits answer, relevant files, model, and usage", async () => { assert.equal(payload.model, "llama-3.1-8b-instant"); assert.equal(payload.usage.totalTokens, 28); assert.ok(Array.isArray(payload.relevantFiles)); + assert.equal(completeCalls, 1); + assert.equal(streamCalls, 0); } finally { await rm(projectRoot, { recursive: true, force: true }); } diff --git a/readme.MD b/readme.MD index 75a9630..11ea94e 100644 --- a/readme.MD +++ b/readme.MD @@ -105,6 +105,10 @@ The snapshot is the primary output of DevMap. Everything else builds on top of it. +Human `analyze` and `ask` responses stream progressively while preserving +readable terminal Markdown. Agent-facing `--json` output stays buffered as one +complete JSON document. + --- ## Quick Start @@ -126,6 +130,9 @@ devmap doctor devmap ask "explain the main architecture" devmap ask "where is the auth logic?" devmap ask "what external services does this use?" + +# Machine-readable output for AI agents and scripts +devmap ask "where is the auth logic?" --json ``` --- @@ -185,6 +192,10 @@ work immediately One snapshot. Every tool. No repeated explanations. +Use `--json` when an agent or script calls DevMap. Human terminal output streams +AI explanations progressively, while JSON mode returns one complete parseable +document without ANSI or terminal decoration. + > Benchmark results coming — with and without DevMap, same task, measured token usage. > See [docs/benchmarking.md](./docs/benchmarking.md) for methodology.