A synthetic test set for font family identification: a word or a short text, set in a known font, printed or painted on real surfaces and in real scenes, with the exact font, the text and the position of every letter recorded for each image.
Made by WhatFontIs, the font finder that identifies fonts from images, to measure how well a tool can find the font in a real-looking photo.
Official page: whatfontis.com/WhatFontIs-Bench.html
Paper: A. Cuibari, WhatFontIs-Bench: A Synthetic Benchmark for Font Family Identification in Real-Looking Images, 2026. doi:10.5281/zenodo.22876579
| Version | Fonts | Images | Data | Images download |
|---|---|---|---|---|
| v2.0 | 3,000 | 60,000 | v2/ |
release v2.0 |
| v1.0 | 600 | 11,995 | v1/ |
release v1.0 |
Each version is a separate folder and a separate release, so results on different versions are not mixed. This page describes v2.0 first, then v1.0.
| Fonts | 3,000: 1,059 sans-serif, 1,000 serif, 500 slab serif, 300 script, 141 monospaced |
| Font sources | 1,402 Adobe Fonts, 713 Creative Fabrica, 595 Google Fonts, 290 DaFont (commercial-use license) |
| Images | 60,000 (20 per font) |
| Text | a short text per image, from 1 to 27 characters; its length and case follow real searches on WhatFontIs |
| Size | long side never over 2000 px |
In v1.0 every image holds one word of 7–12 letters. Real searches are shorter: a logo, a few capitals, a single letter. In v2.0 the length and the case of the text follow the statistics of real searches on WhatFontIs, so the set measures the case that matters most in practice: identifying a font from only a few letters.
- Different letters in the text: 1 in 4.5 % of the images, 2 in 6.2 %, 3 in 14.5 %, 4 in 16.0 %, 5 or 6 in 31.8 %, 7 or more in 27.0 %. Repeated letters count once: ANNA has 2 different letters, Anna 3.
- Case: all capitals 48.0 %, capitalised 18.2 %, mixed 18.5 %, lower case 10.8 %, digits only 4.5 %. Digits
appear in 9.7 % of the texts and
&in 1.5 %, only with fonts that draw them. - Script fonts: 300, a category v1.0 does not have.
- Five times more fonts: 3,000 instead of 600.
Each font has 20 images, with the same plan as v1.0:
| Type | Easy | Medium | Hard | Per font | In the set |
|---|---|---|---|---|---|
| texture: the text printed or painted straight on a close-up photo of a real surface (wood, wall, concrete, brick, metal, fabric, leather, paper, cardboard) | 3 | 2 | 5 | 10 | 30,000 |
| scene: the text painted inside a real photo of a place | 1 | 2 | 3 | 6 | 18,000 |
| object: the text on a printed object (label, box, book cover, poster, menu, shop sign, packaging, business card, painted wall) | 1 | 1 | 2 | 4 | 12,000 |
Difficulty levels:
- easy: capital height 130–220 px, little blur and noise, JPEG quality 88–95, almost even light
- medium: capital height 100–180 px, blur σ 0.2–0.8, noise 1–3.5, JPEG 78–90, uneven light
- hard: capital height 100–140 px, blur σ 0.5–1.3, noise 2–6, JPEG 65–80, strong uneven light, and often a cast shadow over part of the text (16,986 images), a glare spot (14,015) or light print wear (1,804)
As in v1.0, the images are frontal; texture and scene images add only a small camera deviation (corners moved at most 1.5 %, rotation at most 1°), saved in the labels. Text contrast is at least 70 grey levels (50 on the hard level), no letter is clipped, and fonts that only have capitals (480 of them) are written in capitals.
The WhatFontIs API on all 60,000 images, October 2026, in its two modes: the default
one and mode=pro. The API searches its whole catalogue of over 1.2 million fonts, not only the 3,000 fonts of
this set. As for v1.0, a result is correct when it is the right font family, in any weight or from any source.
| Images | Default: correct at 1 | in top 5 | in top 20 | mode=pro: correct at 1 | in top 5 | in top 20 | |
|---|---|---|---|---|---|---|---|
| All images | 60,000 | 82.2 % | 88.0 % | 89.9 % | 88.1 % | 94.3 % | 95.6 % |
| texture | 30,000 | 82.4 % | 88.1 % | 89.9 % | 88.9 % | 95.0 % | 96.1 % |
| scene | 18,000 | 79.6 % | 85.6 % | 87.7 % | 86.0 % | 92.4 % | 94.0 % |
| object | 12,000 | 85.7 % | 91.4 % | 93.0 % | 89.2 % | 95.4 % | 96.8 % |
| easy | 15,000 | 83.0 % | 88.1 % | 89.7 % | 89.2 % | 95.1 % | 96.5 % |
| medium | 15,000 | 81.4 % | 86.9 % | 88.6 % | 87.6 % | 94.0 % | 95.3 % |
| hard | 30,000 | 82.3 % | 88.5 % | 90.6 % | 87.8 % | 94.0 % | 95.3 % |
| sans-serif | 21,180 | 76.7 % | 83.1 % | 86.0 % | 84.1 % | 90.5 % | 92.5 % |
| serif | 20,000 | 84.7 % | 90.7 % | 92.1 % | 88.9 % | 95.7 % | 96.9 % |
| slab serif | 10,000 | 86.4 % | 91.1 % | 92.0 % | 91.2 % | 97.3 % | 97.9 % |
| script | 6,000 | 86.8 % | 91.5 % | 92.3 % | 93.7 % | 97.4 % | 97.9 % |
| monospaced | 2,820 | 81.8 % | 87.5 % | 91.0 % | 89.4 % | 94.9 % | 96.8 % |
| Median time per image | 4.4 s | 9.0 s |
By the number of different letters in the text, the part where v2.0 differs from v1.0:
| Different letters | Images | Default: correct at 1 | in top 5 | in top 20 | mode=pro: correct at 1 | in top 5 | in top 20 |
|---|---|---|---|---|---|---|---|
| 1 | 2,697 | 43.2 % | 53.5 % | 59.9 % | 48.2 % | 62.4 % | 69.1 % |
| 2 | 3,713 | 64.7 % | 73.7 % | 78.6 % | 73.8 % | 85.9 % | 89.8 % |
| 3 | 8,695 | 77.3 % | 83.6 % | 86.1 % | 86.2 % | 93.4 % | 95.1 % |
| 4 | 9,625 | 82.6 % | 88.5 % | 90.4 % | 89.6 % | 95.7 % | 96.9 % |
| 5-6 | 19,056 | 86.8 % | 92.0 % | 93.2 % | 92.0 % | 97.2 % | 97.8 % |
| 7 or more | 16,214 | 89.8 % | 94.4 % | 95.2 % | 93.5 % | 97.8 % | 98.2 % |
Images without an answer count as misses: 280 in the default mode, where no letters were found, and 50 with
mode=pro (37 where the request failed, 13 where no letters were found). Times are the median per image on the
live API, one request at a time, measured on 100 of the images.
The images are in the release v2.0 as 15 zip files of 4,000 images each:
| File | Size | Images |
|---|---|---|
| WhatFontIs-Bench_v2.0_images_part01.zip | 477 MB | 00000–04002 |
| WhatFontIs-Bench_v2.0_images_part02.zip | 525 MB | 04003–08008 |
| WhatFontIs-Bench_v2.0_images_part03.zip | 455 MB | 08009–12013 |
| WhatFontIs-Bench_v2.0_images_part04.zip | 455 MB | 12014–16019 |
| WhatFontIs-Bench_v2.0_images_part05.zip | 455 MB | 16020–20024 |
| WhatFontIs-Bench_v2.0_images_part06.zip | 481 MB | 20025–24027 |
| WhatFontIs-Bench_v2.0_images_part07.zip | 478 MB | 24028–28031 |
| WhatFontIs-Bench_v2.0_images_part08.zip | 460 MB | 28032–32038 |
| WhatFontIs-Bench_v2.0_images_part09.zip | 455 MB | 32039–36044 |
| WhatFontIs-Bench_v2.0_images_part10.zip | 433 MB | 36045–40055 |
| WhatFontIs-Bench_v2.0_images_part11.zip | 455 MB | 40056–44068 |
| WhatFontIs-Bench_v2.0_images_part12.zip | 493 MB | 44069–48081 |
| WhatFontIs-Bench_v2.0_images_part13.zip | 791 MB | 48082–52081 |
| WhatFontIs-Bench_v2.0_images_part14.zip | 795 MB | 52082–56081 |
| WhatFontIs-Bench_v2.0_images_part15.zip | 787 MB | 56082–60081 |
With the GitHub CLI: gh release download v2.0 --repo whatfontis/WhatFontIs-Bench
Unzip all of them into v2/; they create v2/scenes/NNNNN.jpg. The text crops come from
python tools/make_crops.py v2.
| File | What it holds |
|---|---|
v2/labels.jsonl |
one line per image, with the same fields as v1.0, plus query_spec (the number of different characters and the case the text was drawn with) |
v2/annotations.json |
the same set in COCO format: images, categories (one per font), annotations (word box, text, letters) |
v2/fonts.json |
the 3,000 fonts: title, family, source, category, capitals-only flag, WhatFontIs page, number of images |
v2/backgrounds.json |
the 153 background photos used, with source page, author and license; 1,510 object images have no photo (a painted wall fills the frame) |
v2/text_pools.json |
the strings the texts were drawn from, by case and number of different characters |
v2/quick1200.txt |
1,200 image ids that give the same score as the whole set (see below) |
Font files are not included.
v2/quick1200.txt lists 1,200 images, one font each, with the same mix of type, number of letters and category as
the whole set. On every system we measured, its score was within about one point of the score on all 60,000 images
(for the WhatFontIs API: 87.9 % against 88.1 % correct at 1 with mode=pro, 82.4 % against 82.2 % in the default
mode), so it is a quick first check for a new model; use the whole set for final numbers.
- 513 of the 600 fonts of v1.0 are also in v2.0; do not use one version to train and the other to test.
- A single letter is often drawn almost the same way in several fonts, so one-letter images have no single possible answer; read their results as a lower bound.
- The scene images reuse a limited number of real scenes, each with a different text, font, position and light.
- Some fonts in the set are close look-alikes of other fonts that exist under different names.
| Fonts | 600: 200 sans-serif, 200 serif, 100 slab serif, 100 monospaced |
| Font sources | 215 Adobe Fonts, 191 Creative Fabrica, 183 Google Fonts, 11 DaFont (commercial-use license) |
| Images | 11,995 (20 per font; Colt Soft Regular has 15, see v1.0 known gaps) |
| Text | one word per image, 7–12 letters, taken from COCO-Text v2 |
| Size | long side 442–1998 px (never over 2000), short side at least 301 px |
Each font has 20 images:
| Type | Easy | Medium | Hard | Per font |
|---|---|---|---|---|
| texture: the word printed or painted straight on a close-up photo of a real surface (wood, plaster, concrete, brick, metal, fabric, leather, paper, cardboard) | 3 | 2 | 5 | 10 |
| scene: the word painted on a wall inside a real photo (rooms, shops, facades, workshops) | 1 | 2 | 3 | 6 |
| object: the word on a printed object (label, box, book cover, poster, menu, shop sign, packaging, business card, painted wall) over a real photo | 1 | 1 | 2 | 4 |
Difficulty levels:
- easy: capital height 160–220 px, little blur and noise, JPEG quality 88–95, almost even light
- medium: capital height 130–180 px, blur σ 0.3–0.8, noise 1.5–3.5, JPEG 78–90, uneven light
- hard: capital height 100–140 px, blur σ 0.7–1.3, noise 3–6, JPEG 65–80, strong uneven light, and often a cast shadow over part of the word (3,354 images), a glare spot (2,747) or light print wear (357)
When a scene wall is too small for a long word at the level's size, that image uses the next smaller size range, so a few easy scene images have capitals of 130–160 px and a few medium ones of 100–130 px.
The images are frontal, with no slant or perspective, so an upright font never looks italic. Texture and scene images add only a small camera deviation, as in a real photo taken straight on: corners moved at most 1.5 %, rotation at most 1°. The exact transform is saved in the labels.
Text contrast is at least 70 grey levels (50 on the hard level). No letter is clipped. Fonts that only have capitals (107 of them) are written in capitals.
The WhatFontIs API on all 11,995 images, October 2026, in its two modes: the default
one and mode=pro, which takes about two and a half times as long per image and finds more. The API searches its
whole catalogue of over 1.2 million fonts, not only the 600 fonts of this set.
A result counts as correct when it is the right font family, in any weight or from any source: an image in Roboto Bold answered with Roboto Regular is correct, answered with Arial Bold it is wrong.
| Images | Default: correct at 1 | in top 5 | in top 20 | mode=pro: correct at 1 | in top 5 | in top 20 | |
|---|---|---|---|---|---|---|---|
| All images | 11,995 | 83.2 % | 92.2 % | 94.6 % | 89.6 % | 97.1 % | 98.5 % |
| texture | 5,997 | 83.1 % | 92.3 % | 94.4 % | 90.1 % | 97.4 % | 98.8 % |
| scene | 3,599 | 81.2 % | 90.2 % | 93.0 % | 87.6 % | 95.6 % | 97.5 % |
| object | 2,399 | 86.6 % | 95.2 % | 97.2 % | 91.5 % | 98.5 % | 99.4 % |
| easy | 2,995 | 86.0 % | 94.1 % | 95.8 % | 91.3 % | 97.9 % | 99.1 % |
| medium | 3,000 | 82.8 % | 91.1 % | 93.3 % | 89.8 % | 97.0 % | 98.4 % |
| hard | 6,000 | 82.1 % | 91.9 % | 94.5 % | 88.7 % | 96.7 % | 98.3 % |
| sans-serif | 4,000 | 75.2 % | 87.5 % | 91.6 % | 84.1 % | 94.7 % | 97.3 % |
| serif | 4,000 | 83.8 % | 94.3 % | 96.0 % | 88.8 % | 97.6 % | 99.0 % |
| slab serif | 1,995 | 92.6 % | 95.2 % | 95.8 % | 98.1 % | 99.4 % | 99.5 % |
| monospaced | 2,000 | 89.0 % | 94.7 % | 96.3 % | 93.9 % | 98.4 % | 99.2 % |
| Median time per image | 5.2 s | 12.8 s |
Images without an answer count as misses: 20 in the default mode, where no letters were found, and 31 with
mode=pro, where the request failed. Times are the median per image on the live API, measured on 1,200 of the
images. The previous measurement, September 2026, had a single mode: 83.7 % correct at 1, 93.3 % in top 5,
96.5 % in top 20.
The images are in the release v1.0 as four zip files (they are release downloads, not files in the repository):
| File | Size | Images |
|---|---|---|
| WhatFontIs-Bench_v1.0_images_part1.zip | 466 MB | 00000–02998 |
| WhatFontIs-Bench_v1.0_images_part2.zip | 470 MB | 02999–05997 |
| WhatFontIs-Bench_v1.0_images_part3.zip | 460 MB | 05998–08996 |
| WhatFontIs-Bench_v1.0_images_part4.zip | 604 MB | 08997–11994 |
With the GitHub CLI: gh release download v1.0 --repo whatfontis/WhatFontIs-Bench
Unzip all four into v1/; they create v1/scenes/NNNNN.jpg. To also get the text crops
(the word cut out of each image, unscaled), run:
pip install pillow
python tools/make_crops.py v1
This writes v1/crops/NNNNN.png.
| File | What it holds |
|---|---|
v1/labels.jsonl |
one line per image, with everything about it (see below) |
v1/annotations.json |
the same set in COCO format: images, categories (one per font), annotations (word box, text, letters) |
v1/fonts.json |
the 600 fonts: title, family, source, category, capitals-only flag, WhatFontIs page, number of images |
v1/backgrounds.json |
the 153 background photos used, with source page, author and license |
words.txt |
the word list the texts were drawn from (shared by all versions) |
v1/specimens/ |
ten sheets showing all 600 fonts, 60 per sheet |
tools/make_crops.py |
cuts the text crops out of the images |
Font files are not included.
The main fields of each line:
- Image and font
id,image(scenes/NNNNN.jpg, relative tov1/),width,heighttext: the word in the imagetitle,family,source,category,caps_only,url: the fonttype(texture/scene/object),difficulty(easy/medium/hard)
- Where the text is
crop_box:[x0, y0, x1, y1]of the text crop- texture and scene images:
ink_quad(the word's four corners) andchar_quads(four corners per letter) - object images:
ink_box,word_boxandchars(letter boxes), plusobject_box
- How it was made
cap_height_px,font_px,ink_rgb,contrast,blur_sigma,noise_sigma,jpeg_qualityeffects/hard_effects: the hard-level problems appliedcamera:corner_jitter,rotation_deg,homography
- Background
- texture images:
surface,texture,texture_source,texture_license - scene images:
scene_source_file,scene_source,scene_page,scene_license - object images:
scene(object kind),material,extras,background(file,source,page,license)
- texture images:
The background identifiers match the id field in backgrounds.json.
- Colt Soft Regular has 15 images instead of 20: it is so wide that a 7-letter word with 160 px capitals does not fit in 2000 px, so 5 easy images are missing.
- The scene images reuse a limited number of real scenes (each scene appears many times, with a different word, font, position and light).
- Some fonts in the set are close look-alikes of other fonts that exist under different names.
@misc{cuibari2026whatfontisbench,
title = {WhatFontIs-Bench: A Synthetic Benchmark for Font Family Identification in Real-Looking Images},
author = {Cuibari, Alexandru},
year = {2026},
doi = {10.5281/zenodo.22876579},
url = {https://doi.org/10.5281/zenodo.22876579}
}- Backgrounds: all CC0 1.0. Surface photos from Poly Haven and
ambientCG; scene photos from Wikimedia Commons, found through
Openverse; panorama views from Poly Haven. Per-file details are in each version's
backgrounds.json. - Words and texts: from COCO-Text v2 (CC BY 4.0), Veit et al., COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images, 2016.
- Fonts: the typefaces belong to their designers and foundries. The images show text rendered in them; the font files are not part of this set.