Skip to content

fix(ai): apply input and output schemas when loading .prompt files#300

Merged
pavelgj merged 4 commits into
mainfrom
fix/dotprompt-file-schemas
Jun 11, 2026
Merged

fix(ai): apply input and output schemas when loading .prompt files#300
pavelgj merged 4 commits into
mainfrom
fix/dotprompt-file-schemas

Conversation

@chrisraygill

Copy link
Copy Markdown
Collaborator

Fixes #299.

Problem

When a prompt is loaded from a .prompt file via promptDir, the input and output schemas in the frontmatter are not applied. The folder loader reads metadata from dotpromptRegistry.parse(source), which returns the raw frontmatter and does not resolve Picoschema (that only happens in _resolveMetadata, reached through render / renderMetadata).

As a result:

  • input.schema was never set on the prompt action, so the action had a null input schema. The Developer UI renders its input form from that schema, so file-based prompts showed no input fields, and input was never validated.
  • output.schema was copied through as raw Picoschema. That raw value was then sent to the model as the response schema. With the Google GenAI plugin it becomes responseJsonSchema, which is not valid JSON Schema, so the request can fail or the schema is ignored.

Both schemas already work when a prompt is defined in code with definePrompt. This was specific to the folder loader.

Changes

In prompt_loader_io.dart:

  • Resolve input.schema and set it on the PromptConfig (and therefore the registered PromptAction) as a SchemanticType.
  • Convert output.schema to JSON Schema before building the output config.
  • Both go through a small _toJsonSchema helper that converts Picoschema and leaves existing JSON Schema untouched.

The conversion does not gate on Picoschema.isPicoschema. That helper does not recognize the type, description form (for example name: string, the person to greet) used throughout the docs, so gating on it would skip the most common case. _isJsonSchema checks for an existing JSON Schema shape (a top-level standard type, or a $schema / $ref / $defs key) and everything else is treated as Picoschema and converted.

Testing

Added three tests to the loadPromptFolder group in prompt_test.dart:

  • input Picoschema (including an optional field) is wired onto the action and converted to JSON Schema with the right required set.
  • plain JSON Schema for input.schema passes through unchanged.
  • output Picoschema is converted to JSON Schema before it reaches the model.

Full dart test for the genkit package passes (368 tests). dart analyze is clean.

Before, for a prompt with output.schema of {category: "string, the predicted category"}:

{format: json, jsonSchema: {category: string, the predicted category}}

After:

{format: json, jsonSchema: {type: object, properties: {category: {type: string, description: the predicted category}}, ...}}

Notes / follow-ups

  • Picoschema.isPicoschema not matching the type, description form looks like a bug in the dotprompt package itself. This PR works around it in the loader. It may be worth fixing upstream too.
  • Input default values from input.default are still not applied at render time. That is a separate render-path change and is out of scope here.

The folder loader read prompt metadata from `parse`, which returns the raw
frontmatter and does not resolve Picoschema. As a result:

- `input.schema` was never wired onto the prompt action, so the action had a
  null input schema. The Developer UI shows no input form and input was not
  validated.
- `output.schema` was copied through as raw Picoschema, so an invalid schema
  was sent to the model as the response schema.

Resolve both schemas in the loader. Input schema is converted to a
SchemanticType and set on the PromptConfig. Output schema is converted to
JSON Schema before building the output config. Conversion does not gate on
Picoschema.isPicoschema, which misses the `type, description` form used in
the docs; it checks for an existing JSON Schema shape instead.

Fixes #299

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for converting input and output schemas defined in Picoschema format within .prompt files into standard JSON Schema. It introduces helper functions to detect existing JSON Schema structures and perform the conversion, ensuring that the Developer UI can render forms and validate inputs correctly. The feedback suggests improving the robustness of the JSON Schema detection in _isJsonSchema by checking for additional keywords like properties or anyOf, and adding a safe type check in _toInputSchema to prevent potential runtime TypeErrors when casting the parsed JSON.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread packages/genkit/lib/src/ai/prompt_loader_io.dart
Comment thread packages/genkit/lib/src/ai/prompt_loader_io.dart
Address review feedback on the .prompt schema loading.

- `_isJsonSchema` now also recognizes JSON Schema that omits the top-level
  `type` and relies on `properties`, `items`, or a `*Of` combinator, so such
  schemas are not mistaken for Picoschema and re-converted.
- The generated input `parse` no longer casts with `as Map`. A prompt can be
  invoked with null input, and `parse` is called with it, so guard the cast
  and return an empty map instead of throwing.
@chrisraygill

Copy link
Copy Markdown
Collaborator Author

Addressed both review comments in 7feb3f3 and did another pass over the change.

Verified edge cases (added tests for the first two):

  • JSON Schema that omits the top-level type and uses properties is kept as JSON Schema, not re-converted.
  • A prompt invoked with null input parses to an empty map instead of throwing.
  • output: { format: json } with no schema produces {format: json} and does not crash.
  • A named schema reference (input: schema: MyType) passes through as {$ref: MyType} without crashing.

One known limitation, out of scope here: a named schema reference is passed through as a raw $ref and is not resolved against schemas registered with defineSchema, so the Developer UI cannot expand it into fields. Resolving named refs for file-loaded prompts would be a separate change. Input default values are also still not applied at render time.

Pass `defineSchema`-registered schemas to Picoschema conversion when
loading `.prompt` files, so input and output schemas can reference
named schemas by name (e.g. `schema: MyAddress`). Without this, prompts
using named schema references could not be resolved.
@pavelgj pavelgj enabled auto-merge (squash) June 11, 2026 15:15
@pavelgj pavelgj merged commit af06185 into main Jun 11, 2026
13 checks passed
@pavelgj pavelgj deleted the fix/dotprompt-file-schemas branch June 11, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dotprompt: input/output schemas in .prompt files are not applied when loaded from promptDir

2 participants