Blueprints - A DSL to define the model's output format

[rlouf]

Overview

This Discussion proposes a Jinja-like Domain Specific Language (DSL), that we call blueprints, for creating LLM output templates. The DSL aims to simplify complex template patterns, especially for function calling formats across different LLM architectures. By leveraging familiar template syntax similar to Jinja2 while adding domain-specific constructs, the DSL aims to make prompt engineering more maintainable and flexible.

Problem Statement

Current approaches to defining output structures for LLMs requires writing regular expressions or using our internal regex DSL, which becomes unwieldy when dealing with complex structures like function calling formats (see Qwen). For example, defining a Hermes function calling format using our regex DSL currently looks like:

from outlines.types import either, one_or_more, newline

outlines = (
    "<|im_start|>assistant"
   + newline
   + one_or_more(
           "<tool_call>" + newline
           + either("schema_1", "schema_2", "schema_3") + newline
           + "</tool_call>" + newline
   )
    + "<|im_end|>"
)

This approach has several limitations:

• It's difficult to read and maintain
• It doesn't scale well with template complexity
• It lacks the flexibility to handle variables and conditional logic easily

Proposed Solution: Blueprints

We propose a Jinja-like DSL that compiles to the existing outlines format (regular expressions or EBNF grammar). The DSL will leverage the power of Jinja2's syntax while adding specialized constructs for LLM prompt templates.

Core Features

• Template Blocks: Define reusable templates with named parameters
• Variable Interpolation: Insert variables into templates
• Control Flow: Use conditional logic and loops within templates
• LLM-specific Constructs: Add specialized syntax for common prompt patterns

Syntax

Basic Template Definition

<|im_start|>assistant
{% repeat min=1 %}
<tool_call>
{% choice %}
    {% option %}{{ schema1|json_schema }}{% endoption %}
    {% option %}{{ schema2|json_schema }}{% endoption %}
    {% option %}{{ schema3|json_schema }}{% endoption %}
{% endchoice %}
</tool_call>
{% endrepeat %}
<|im_end|>

Special Constructs

Repeat Blocks:

{% repeat min=1 max=5 %}...{% endrepeat %}
{% repeat times=3 %}...{% endrepeat %}
{% repeat min=3 %}...{% endrepeat %}
{% repeat max=3 %}...{% endrepeat %}

Choice/Option Blocks:

{% choice %}
    {% option %}option1{% endoption %}
    {% option %}option2{% endoption %}
{% endchoice %}

Optional Blocks:

{% optional %}
  This may or may not appear
{% endoptional %}

Variable Interpolation with Type Annotation:

{{ variable | json_schema }}
{{ variable | regex }}

Anything Constructs (match arbitrary content):

{{ anything() }}

With explicit delimiter:

{{ anything(until="</think>") }}

Implementation Details

The DSL will be implemented by:

• Using Jinja2's parser as a foundation
• Extending it with custom tags and filters
• Compiling the templates to the AST already used for the regex DSL
• Converting the AST to the target format (regex/EBNF)

Example Implementation Pseudocode

# Initialize Jinja environment with custom extensions
env = Environment(extensions=[OutlinesExtension])

# Parse template
template = env.from_string(template_string)

# Render with context
rendered = template.render(context)

# Process custom DSL constructs
tokens = tokenize_dsl(rendered)
ast = parse_tokens(tokens)

# Compile to target format
regex = ast.to_regex()

Templating for Different LLM Models

The DSL can be used to define templates for various LLM function calling formats:

Hermes Format

<|im_start|>assistant
{% repeat min=1 %}
<tool_call>
{% choice %}
    {% option %}{{ schema1|json_schema }}{% endoption %}
    {% option %}{{ schema2|json_schema }}{% endoption %}
    {% option %}{{ schema3|json_schema }}{% endoption %}
{% endchoice %}
</tool_call>
{% endrepeat %}
<|im_end|>

DeepSeek Format

<|im_start|>assistant
<think>
{{ anything() }}
</think>
{{ schema|json_schema }}
<|im_end|>

User interface

from outlines import Blueprint

template_str = "{{ var }}{% anything %}"
blueprint = Blueprint.from_string(template_str)

# We can also load from file, just like `Template`
blueprint = Blueprint.from_file("file_with_blueprint.txt")

outline = blueprint("Hello")

Benefits

• Readability: Templates are more explicit and closer to the final output
• Maintainability: Separation of template structure from control flow.
• Flexibility: Easy to adapt to different LLM architectures
• Extensibility: New constructs can be added as needed

Conclusion

By leveraging familiar syntax and adding domain-specific constructs, blueprints make output templates more readable, maintainable, and flexible. Blueprints can be implemented on top of the existing regex DSL and thus require minimal development effort.

Example

# Define schemas
schema1 = JsonSchema("schema_1")
schema2 = JsonSchema("schema_2")
schema3 = JsonSchema("schema_3")

# Define template
template = """
<|im_start|>assistant
{% repeat min=1 %}
<tool_call>
{% choice %}
  {% option %}{{ schemas[0] | json_schema }}{% endoption %}
  {% option %}{{ schemas[1] | json_schema }}{% endoption %}
  {% option %}{{ schemas[2] | json_schema }}{% endoption %}
{% endchoice %}
</tool_call>
{% endrepeat %}
<|im_end|>
"""

blueprint = Blueprint.from_string(template)
template = Template.from_file("prompts/api.txt")

outline = blueprint(schemas=[schema1, schema2, schema3]))
prompt = template(*input_data)

model(prompt, outline)

[rlouf]

Also linked to #1457