[RFC] Deep-check for monospacity

[MoSal]

Before this change, a font is considered monospace if fontdb flags it
as such. fontdb checks the post table for this property.

But some fonts don't set that property there.

Most notably, "Noto Sans Mono" is among these fonts. Monospace as
a property is said to be communicated in other places like OS/2's
panose, but that's not set in the Noto font either.

Loosely based on a fontconfig function called
FcFreeTypeSpacing(), this commit adds an additional check against
fonts that are not set as monospaced by fontdb. The horizontal
advances of all glyphs of a cmap unicode table are checked to see
if they are monospace. Proportionality with double-width and
treble-width advances is taken into consideration. Treble width
advances exist in the aforementioned Noto font.

The checks should be efficient, but the overhead is not in the noise.
So these extra checks are only run if the "monospace_fallback" crate
feature is enabled.

This change also requires library users to check monospacity with
FontSystem::is_monospace() instead of FaceInfo::monospaced from
fontdb to be in-sync with cosmic-text's view. This requirement was
probably coming in the future anyway for when cosmic-text adds support
for variable fonts.

---

This depends on harfbuzz/ttf-parser#174 for making the check more efficient.

How many fonts have this issue?

Running this on my system returns 220 such fonts:

use ttf_parser::Face;
use ttf_parser::cmap::Format;
use walkdir::{WalkDir, DirEntry, Error as WDError};

use std::fs;
use std::path::Path;

fn mono_proportional(font_path: &Path) {
    let font_path_str = font_path.file_name().unwrap().to_str().unwrap();
    let font_data = match fs::read(font_path) {
        Err(e) => {
            eprintln!("reading file '{font_path_str}' failed: {e}");
            return;
        },
        Ok(font_data) => font_data,
    };

    let font_count = ttf_parser::fonts_in_collection(&font_data).unwrap_or(1);

    let faces_res = (0..font_count)
        .map(|i| Face::parse(&font_data, i))
        .collect::<Result<Vec<_>, _>>();

    let faces = match faces_res {
        Ok(faces) => faces,
        Err(e) => {
            eprintln!("Parsing Error: {}.", e);
            return;
        },
    };


    let mono_proportional = |face: &Face| {
        const MAX_ADVANCES: usize = 3;
        let mut advances = Vec::with_capacity(MAX_ADVANCES);
        let mut glyph_count = 0u32;

        let cmap = face.tables().cmap.as_ref()?;
        let subtable12 = cmap.subtables.into_iter()
            .find(|subtable| subtable.is_unicode() && matches!(subtable.format, Format::SegmentedCoverage(_)));
        let subtable4 = cmap.subtables.into_iter()
            .find(|subtable| subtable.is_unicode() && matches!(subtable.format, Format::SegmentMappingToDeltaValues(_)));
        let unicode_subtable = subtable12.or(subtable4)?;
        unicode_subtable.is_unicode().then_some(())?;

        let mut advances_maxed = false;
        unicode_subtable.codepoints(|cp| {
            glyph_count += 1;
            if advances_maxed { return; }
            if let Some(glyph_id) = unicode_subtable.glyph_index(cp) {
                match face.glyph_hor_advance(glyph_id) {
                    Some(advance) if advance != 0 => {
                        match advances.binary_search(&advance) {
                            Err(_) if advances.len() == MAX_ADVANCES => advances_maxed = true,
                            Err(i) => advances.insert(i, advance),
                            Ok(_) => (),
                        }
                    },
                    _ => (),
                }
            }
        });

        if advances_maxed {
            return Some((false, glyph_count, advances));
        }

        let mut advances_iter = advances.iter().copied();
        let smallest = advances_iter.next()?;
        for advance in advances_iter {
            if advance % smallest > 0 {
                return Some((false, glyph_count, advances));
            }
        }
        Some((true, glyph_count, advances))
    };

    let mono_check = |face: &Face| {
        if face.is_monospaced() {
            //println!("{}: monospaced", args[1]);
        } else if let Some((mono_proportional, glyph_count, advances)) = mono_proportional(face) {
            if mono_proportional {
                //println!("{font_path_str}: [{glyph_count} glyphs, advances: {advances:?}]");
                let font_name = face
                    .names()
                    .into_iter()
                    .find_map(|name| (name.name_id == 1).then(|| name.to_string()).flatten())
                    .unwrap_or(font_path_str.to_owned());
                println!("{font_name}: [{glyph_count} glyphs, advances: {advances:?}]");
            }
        }
    };

    faces.iter().for_each(mono_check);
}

fn main() {
    let per_f = |dir_entry_res: Result<DirEntry, WDError>| match dir_entry_res {
        Err(e) => eprintln!("dir walking error @ '{:?}': {e}", e.path()),
        Ok(dir_entry) => if dir_entry.file_type().is_file() {
            mono_proportional(dir_entry.path());
        },
    };

    WalkDir::new("/usr/share/fonts")
        .follow_links(true)
        .into_iter()
        .for_each(per_f);
}

Sorted output:

Details

Monaspace Argon ExtraBold: [487 glyphs, advances: [1240]] Monaspace Argon ExtraBold: [487 glyphs, advances: [1240]] Monaspace Argon ExtraLight: [487 glyphs, advances: [1240]] Monaspace Argon ExtraLight: [487 glyphs, advances: [1240]] Monaspace Argon Light: [487 glyphs, advances: [1240]] Monaspace Argon Light: [487 glyphs, advances: [1240]] Monaspace Argon Medium: [487 glyphs, advances: [1240]] Monaspace Argon Medium: [487 glyphs, advances: [1240]] Monaspace Argon SemiBold: [487 glyphs, advances: [1240]] Monaspace Argon SemiBold: [487 glyphs, advances: [1240]] Monaspace Argon SemiWide ExtraBold: [487 glyphs, advances: [1398]] Monaspace Argon SemiWide ExtraBold: [487 glyphs, advances: [1398]] Monaspace Argon SemiWide ExtraLight: [487 glyphs, advances: [1398]] Monaspace Argon SemiWide ExtraLight: [487 glyphs, advances: [1398]] Monaspace Argon SemiWide Light: [487 glyphs, advances: [1398]] Monaspace Argon SemiWide Light: [487 glyphs, advances: [1398]] Monaspace Argon SemiWide Medium: [487 glyphs, advances: [1398]] Monaspace Argon SemiWide Medium: [487 glyphs, advances: [1398]] Monaspace Argon SemiWide SemiBold: [487 glyphs, advances: [1398]] Monaspace Argon SemiWide SemiBold: [487 glyphs, advances: [1398]] Monaspace Argon SemiWide: [487 glyphs, advances: [1398]] Monaspace Argon SemiWide: [487 glyphs, advances: [1398]] Monaspace Argon SemiWide: [487 glyphs, advances: [1398]] Monaspace Argon SemiWide: [487 glyphs, advances: [1398]] Monaspace Argon Wide ExtraBold: [487 glyphs, advances: [1555]] Monaspace Argon Wide ExtraBold: [487 glyphs, advances: [1555]] Monaspace Argon Wide ExtraLight: [487 glyphs, advances: [1555]] Monaspace Argon Wide ExtraLight: [487 glyphs, advances: [1555]] Monaspace Argon Wide Light: [487 glyphs, advances: [1555]] Monaspace Argon Wide Light: [487 glyphs, advances: [1555]] Monaspace Argon Wide Medium: [487 glyphs, advances: [1555]] Monaspace Argon Wide Medium: [487 glyphs, advances: [1555]] Monaspace Argon Wide SemiBold: [487 glyphs, advances: [1555]] Monaspace Argon Wide SemiBold: [487 glyphs, advances: [1555]] Monaspace Argon Wide: [487 glyphs, advances: [1555]] Monaspace Argon Wide: [487 glyphs, advances: [1555]] Monaspace Argon Wide: [487 glyphs, advances: [1555]] Monaspace Argon Wide: [487 glyphs, advances: [1555]] Monaspace Argon: [487 glyphs, advances: [1240]] Monaspace Argon: [487 glyphs, advances: [1240]] Monaspace Argon: [487 glyphs, advances: [1240]] Monaspace Argon: [487 glyphs, advances: [1240]] Monaspace Krypton ExtraBold: [488 glyphs, advances: [1240]] Monaspace Krypton ExtraBold: [488 glyphs, advances: [1240]] Monaspace Krypton ExtraLight: [488 glyphs, advances: [1240]] Monaspace Krypton ExtraLight: [488 glyphs, advances: [1240]] Monaspace Krypton Light: [488 glyphs, advances: [1240]] Monaspace Krypton Light: [488 glyphs, advances: [1240]] Monaspace Krypton Medium: [488 glyphs, advances: [1240]] Monaspace Krypton Medium: [488 glyphs, advances: [1240]] Monaspace Krypton SemiBold: [488 glyphs, advances: [1240]] Monaspace Krypton SemiBold: [488 glyphs, advances: [1240]] Monaspace Krypton SemiWide ExtraBold: [488 glyphs, advances: [1398]] Monaspace Krypton SemiWide ExtraBold: [488 glyphs, advances: [1398]] Monaspace Krypton SemiWide ExtraLight: [488 glyphs, advances: [1398]] Monaspace Krypton SemiWide ExtraLight: [488 glyphs, advances: [1398]] Monaspace Krypton SemiWide Light: [488 glyphs, advances: [1398]] Monaspace Krypton SemiWide Light: [488 glyphs, advances: [1398]] Monaspace Krypton SemiWide Medium: [488 glyphs, advances: [1398]] Monaspace Krypton SemiWide Medium: [488 glyphs, advances: [1398]] Monaspace Krypton SemiWide SemiBold: [488 glyphs, advances: [1398]] Monaspace Krypton SemiWide SemiBold: [488 glyphs, advances: [1398]] Monaspace Krypton SemiWide: [488 glyphs, advances: [1398]] Monaspace Krypton SemiWide: [488 glyphs, advances: [1398]] Monaspace Krypton SemiWide: [488 glyphs, advances: [1398]] Monaspace Krypton SemiWide: [488 glyphs, advances: [1398]] Monaspace Krypton Wide ExtraBold: [488 glyphs, advances: [1555]] Monaspace Krypton Wide ExtraBold: [488 glyphs, advances: [1555]] Monaspace Krypton Wide ExtraLight: [488 glyphs, advances: [1555]] Monaspace Krypton Wide ExtraLight: [488 glyphs, advances: [1555]] Monaspace Krypton Wide Light: [488 glyphs, advances: [1555]] Monaspace Krypton Wide Light: [488 glyphs, advances: [1555]] Monaspace Krypton Wide Medium: [488 glyphs, advances: [1555]] Monaspace Krypton Wide Medium: [488 glyphs, advances: [1555]] Monaspace Krypton Wide SemiBold: [488 glyphs, advances: [1555]] Monaspace Krypton Wide SemiBold: [488 glyphs, advances: [1555]] Monaspace Krypton Wide: [488 glyphs, advances: [1555]] Monaspace Krypton Wide: [488 glyphs, advances: [1555]] Monaspace Krypton Wide: [488 glyphs, advances: [1555]] Monaspace Krypton Wide: [488 glyphs, advances: [1555]] Monaspace Krypton: [488 glyphs, advances: [1240]] Monaspace Krypton: [488 glyphs, advances: [1240]] Monaspace Krypton: [488 glyphs, advances: [1240]] Monaspace Krypton: [488 glyphs, advances: [1240]] Monaspace Neon ExtraBold: [487 glyphs, advances: [1240]] Monaspace Neon ExtraBold: [487 glyphs, advances: [1240]] Monaspace Neon ExtraLight: [487 glyphs, advances: [1240]] Monaspace Neon ExtraLight: [487 glyphs, advances: [1240]] Monaspace Neon Light: [487 glyphs, advances: [1240]] Monaspace Neon Light: [487 glyphs, advances: [1240]] Monaspace Neon Medium: [487 glyphs, advances: [1240]] Monaspace Neon Medium: [487 glyphs, advances: [1240]] Monaspace Neon SemiBold: [487 glyphs, advances: [1240]] Monaspace Neon SemiBold: [487 glyphs, advances: [1240]] Monaspace Neon SemiWide ExtraBold: [487 glyphs, advances: [1398]] Monaspace Neon SemiWide ExtraBold: [487 glyphs, advances: [1398]] Monaspace Neon SemiWide ExtraLight: [487 glyphs, advances: [1398]] Monaspace Neon SemiWide ExtraLight: [487 glyphs, advances: [1398]] Monaspace Neon SemiWide Light: [487 glyphs, advances: [1398]] Monaspace Neon SemiWide Light: [487 glyphs, advances: [1398]] Monaspace Neon SemiWide Medium: [487 glyphs, advances: [1398]] Monaspace Neon SemiWide Medium: [487 glyphs, advances: [1398]] Monaspace Neon SemiWide SemiBold: [487 glyphs, advances: [1398]] Monaspace Neon SemiWide SemiBold: [487 glyphs, advances: [1398]] Monaspace Neon SemiWide: [487 glyphs, advances: [1398]] Monaspace Neon SemiWide: [487 glyphs, advances: [1398]] Monaspace Neon SemiWide: [487 glyphs, advances: [1398]] Monaspace Neon SemiWide: [487 glyphs, advances: [1398]] Monaspace Neon Wide ExtraBold: [487 glyphs, advances: [1555]] Monaspace Neon Wide ExtraBold: [487 glyphs, advances: [1555]] Monaspace Neon Wide ExtraLight: [487 glyphs, advances: [1555]] Monaspace Neon Wide ExtraLight: [487 glyphs, advances: [1555]] Monaspace Neon Wide Light: [487 glyphs, advances: [1555]] Monaspace Neon Wide Light: [487 glyphs, advances: [1555]] Monaspace Neon Wide Medium: [487 glyphs, advances: [1555]] Monaspace Neon Wide Medium: [487 glyphs, advances: [1555]] Monaspace Neon Wide SemiBold: [487 glyphs, advances: [1555]] Monaspace Neon Wide SemiBold: [487 glyphs, advances: [1555]] Monaspace Neon Wide: [487 glyphs, advances: [1555]] Monaspace Neon Wide: [487 glyphs, advances: [1555]] Monaspace Neon Wide: [487 glyphs, advances: [1555]] Monaspace Neon Wide: [487 glyphs, advances: [1555]] Monaspace Neon: [487 glyphs, advances: [1240]] Monaspace Neon: [487 glyphs, advances: [1240]] Monaspace Neon: [487 glyphs, advances: [1240]] Monaspace Neon: [487 glyphs, advances: [1240]] Monaspace Radon ExtraBold: [487 glyphs, advances: [1240]] Monaspace Radon ExtraBold: [487 glyphs, advances: [1240]] Monaspace Radon ExtraLight: [487 glyphs, advances: [1240]] Monaspace Radon ExtraLight: [487 glyphs, advances: [1240]] Monaspace Radon Light: [487 glyphs, advances: [1240]] Monaspace Radon Light: [487 glyphs, advances: [1240]] Monaspace Radon Medium: [487 glyphs, advances: [1240]] Monaspace Radon Medium: [487 glyphs, advances: [1240]] Monaspace Radon SemiBold: [487 glyphs, advances: [1240]] Monaspace Radon SemiBold: [487 glyphs, advances: [1240]] Monaspace Radon SemiWide ExtraBold: [487 glyphs, advances: [1398]] Monaspace Radon SemiWide ExtraBold: [487 glyphs, advances: [1398]] Monaspace Radon SemiWide ExtraLight: [487 glyphs, advances: [1398]] Monaspace Radon SemiWide ExtraLight: [487 glyphs, advances: [1398]] Monaspace Radon SemiWide Light: [487 glyphs, advances: [1398]] Monaspace Radon SemiWide Light: [487 glyphs, advances: [1398]] Monaspace Radon SemiWide Medium: [487 glyphs, advances: [1398]] Monaspace Radon SemiWide Medium: [487 glyphs, advances: [1398]] Monaspace Radon SemiWide SemiBold: [487 glyphs, advances: [1398]] Monaspace Radon SemiWide SemiBold: [487 glyphs, advances: [1398]] Monaspace Radon SemiWide: [487 glyphs, advances: [1398]] Monaspace Radon SemiWide: [487 glyphs, advances: [1398]] Monaspace Radon SemiWide: [487 glyphs, advances: [1398]] Monaspace Radon SemiWide: [487 glyphs, advances: [1398]] Monaspace Radon Wide ExtraBold: [487 glyphs, advances: [1555]] Monaspace Radon Wide ExtraBold: [487 glyphs, advances: [1555]] Monaspace Radon Wide ExtraLight: [487 glyphs, advances: [1555]] Monaspace Radon Wide ExtraLight: [487 glyphs, advances: [1555]] Monaspace Radon Wide Light: [487 glyphs, advances: [1555]] Monaspace Radon Wide Light: [487 glyphs, advances: [1555]] Monaspace Radon Wide Medium: [487 glyphs, advances: [1555]] Monaspace Radon Wide Medium: [487 glyphs, advances: [1555]] Monaspace Radon Wide SemiBold: [487 glyphs, advances: [1555]] Monaspace Radon Wide SemiBold: [487 glyphs, advances: [1555]] Monaspace Radon Wide: [487 glyphs, advances: [1555]] Monaspace Radon Wide: [487 glyphs, advances: [1555]] Monaspace Radon Wide: [487 glyphs, advances: [1555]] Monaspace Radon Wide: [487 glyphs, advances: [1555]] Monaspace Radon: [487 glyphs, advances: [1240]] Monaspace Radon: [487 glyphs, advances: [1240]] Monaspace Radon: [487 glyphs, advances: [1240]] Monaspace Radon: [487 glyphs, advances: [1240]] Monaspace Xenon ExtraBold: [487 glyphs, advances: [1240]] Monaspace Xenon ExtraBold: [487 glyphs, advances: [1240]] Monaspace Xenon ExtraLight: [487 glyphs, advances: [1240]] Monaspace Xenon ExtraLight: [487 glyphs, advances: [1240]] Monaspace Xenon Light: [487 glyphs, advances: [1240]] Monaspace Xenon Light: [487 glyphs, advances: [1240]] Monaspace Xenon Medium: [487 glyphs, advances: [1240]] Monaspace Xenon Medium: [487 glyphs, advances: [1240]] Monaspace Xenon SemiBold: [487 glyphs, advances: [1240]] Monaspace Xenon SemiBold: [487 glyphs, advances: [1240]] Monaspace Xenon SemiWide ExtraBold: [487 glyphs, advances: [1398]] Monaspace Xenon SemiWide ExtraBold: [487 glyphs, advances: [1398]] Monaspace Xenon SemiWide ExtraLight: [487 glyphs, advances: [1398]] Monaspace Xenon SemiWide ExtraLight: [487 glyphs, advances: [1398]] Monaspace Xenon SemiWide Light: [487 glyphs, advances: [1398]] Monaspace Xenon SemiWide Light: [487 glyphs, advances: [1398]] Monaspace Xenon SemiWide Medium: [487 glyphs, advances: [1398]] Monaspace Xenon SemiWide Medium: [487 glyphs, advances: [1398]] Monaspace Xenon SemiWide SemiBold: [487 glyphs, advances: [1398]] Monaspace Xenon SemiWide SemiBold: [487 glyphs, advances: [1398]] Monaspace Xenon SemiWide: [487 glyphs, advances: [1398]] Monaspace Xenon SemiWide: [487 glyphs, advances: [1398]] Monaspace Xenon SemiWide: [487 glyphs, advances: [1398]] Monaspace Xenon SemiWide: [487 glyphs, advances: [1398]] Monaspace Xenon Wide ExtraBold: [487 glyphs, advances: [1555]] Monaspace Xenon Wide ExtraBold: [487 glyphs, advances: [1555]] Monaspace Xenon Wide ExtraLight: [487 glyphs, advances: [1555]] Monaspace Xenon Wide ExtraLight: [487 glyphs, advances: [1555]] Monaspace Xenon Wide Light: [487 glyphs, advances: [1555]] Monaspace Xenon Wide Light: [487 glyphs, advances: [1555]] Monaspace Xenon Wide Medium: [487 glyphs, advances: [1555]] Monaspace Xenon Wide Medium: [487 glyphs, advances: [1555]] Monaspace Xenon Wide SemiBold: [487 glyphs, advances: [1555]] Monaspace Xenon Wide SemiBold: [487 glyphs, advances: [1555]] Monaspace Xenon Wide: [487 glyphs, advances: [1555]] Monaspace Xenon Wide: [487 glyphs, advances: [1555]] Monaspace Xenon Wide: [487 glyphs, advances: [1555]] Monaspace Xenon Wide: [487 glyphs, advances: [1555]] Monaspace Xenon: [487 glyphs, advances: [1240]] Monaspace Xenon: [487 glyphs, advances: [1240]] Monaspace Xenon: [487 glyphs, advances: [1240]] Monaspace Xenon: [487 glyphs, advances: [1240]] Monoisome: [1262 glyphs, advances: [1024, 2048]] Noto Sans Mono Black: [3490 glyphs, advances: [600, 1200, 1800]] Noto Sans Mono ExtraBold: [3490 glyphs, advances: [600, 1200, 1800]] Noto Sans Mono Medium: [3490 glyphs, advances: [600, 1200, 1800]] Noto Sans Mono SemiBold: [3490 glyphs, advances: [600, 1200, 1800]] Noto Sans Mono Thin: [3490 glyphs, advances: [600, 1200, 1800]] Noto Sans Mono: [3490 glyphs, advances: [600, 1200, 1800]] Noto Sans Mono: [3490 glyphs, advances: [600, 1200, 1800]] Noto Sans SignWriting: [679 glyphs, advances: [1000]] ProggyCleanTTSZ: [226 glyphs, advances: [448]]

All of these font files come from Archlinux packages. Most matches belong to Monaspace. But others include "Noto Sans Mono", "Noto Sans SignWriting", "ProggyCleanTTSZ", and Monoisome (a part of Monoid).

Performance Considerations

Timings listed in seconds are (pre_cache, caching, total). Caching timings are noisy a little bit.

No mono_proportional check

564/3422 mono fonts (without monaspace): 0.000-0.001 0.594 0.595
564/3632 mono fonts (with monaspace):    0.000-0.001 0.597 0.598

With mono_proportional check

574/3632 mono fonts (without monaspace): 0.038 0.608 0.647
784/3422 mono fonts (with monaspace):    0.041 0.673 0.714

So, with 220 extra fonts detected, only ~0.04s is spent on mono_proportional checking, and ~0.11s is spent on caching the extra fonts.