From 7537b7e2a23a0fd15d0feb5b79c804987e2c4fcc Mon Sep 17 00:00:00 2001 From: Philip Bontrager Date: Tue, 8 Apr 2025 22:01:16 -0400 Subject: [PATCH] Update prompt_format.md The image encoding examples in the readme are impossible to parse with so many <|patch|> tokens. I removed the intermediate ones for an ellipses in the style of [llama.com](https://www.llama.com/docs/model-cards-and-prompt-formats/llama4_omni/#-image-prompt-format-) --- models/llama4/prompt_format.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/models/llama4/prompt_format.md b/models/llama4/prompt_format.md index 698571093..bc67a49ce 100644 --- a/models/llama4/prompt_format.md +++ b/models/llama4/prompt_format.md @@ -57,7 +57,7 @@ This example passes an image that is smaller than the tile size, to show the til ``` <|begin_of_text|><|header_start|>user<|header_end|> -<|image_start|><|image|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|image_end|>Describe this image in two sentences<|eot|><|header_start|>assistant<|header_end|> +<|image_start|><|image|><|patch|>...<|patch|><|image_end|>Describe this image in two sentences<|eot|><|header_start|>assistant<|header_end|> ``` @@ -84,7 +84,7 @@ Here is an example of how to pass an image to the model ``` <|begin_of_text|><|header_start|>user<|header_end|> -<|image_start|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|tile_x_separator|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|tile_y_separator|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|tile_x_separator|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|tile_y_separator|><|image|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|patch|><|image_end|>Describe this image in two sentences<|eot|><|header_start|>assistant<|header_end|> +<|image_start|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_y_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_y_separator|><|image|><|patch|>...<|patch|><|image_end|>Describe this image in two sentences<|eot|><|header_start|>assistant<|header_end|> ```