When dnstwist is invoked with fuzzer option homoglyph it uses the map glyphs_unicode (https://github.com/elceef/dnstwist/blob/master/dnstwist.py#L615) as a basis for finding visually equivalent Unicode characters for corresponding ascii characters.
The Unicode consortium publishes https://unicode.org/Public/security/latest/confusables.txt
which lists codes which are visually confusable. The mapping used by dnstwist can be enhanced
to make use of the visually confsuable characters from the above document.
Additionally, some of the Unicode characters that are visually similar to ascii codes (which are mentioned in the above document)
aren't IDNA complaint, but given the evolving threat landscape it might be good to include those Unicode characters
too in the fuzzed output generated (when the fuzzer is homoglyph).
Below script compares the visually matching glyphs (to ascii characters) derived from "confusables.txt"
(https://unicode.org/Public/security/latest/confusables.txt) (Unicode version 17.0.0) with Unicode glyph mapping
used by "dnstwist" for some of the well known domains.
Code from dnstwist version 20250130 was used for below script.
The resulting output shows a very low percentage of overlap between glyphs from confusable and glyphs dnstwist uses.
import string
from pathlib import Path
from pprint import pprint
import dnstwist
def glyphs_from_confusables():
# confusables.txt downloaed from:
# https://unicode.org/Public/security/latest/confusables.txt
confusables_path = Path("confusables.txt")
confusable_glyphs = {}
for c in string.ascii_lowercase + string.digits:
confusable_glyphs[c] = []
with open(confusables_path, mode="r", encoding="UTF-8") as fr:
for line in fr.readlines():
line = line.strip()
if not line or line.startswith("#"):
continue
src, target, _ = line.split(";", 2)
for t in [x for x in target.split(" ") if x]:
try:
t_character = chr(int(t, 16))
except ValueError:
continue # skip malformed entries
# char_to_index now has entries for all ascii chars, so we are interested
# in adding unicode entries that can be confused with these.
if t_character in confusable_glyphs:
s_character = chr(int(src.strip(), 16))
# s_character is the unicode char that could be confused with ascii char (t_character)
confusable_glyphs[t_character].append(s_character)
# pprint(confusable_glyphs, compact=True)
return confusable_glyphs
def validate():
dnstwist_glyphs = dnstwist.Fuzzer.glyphs_unicode
confusable_glyphs = glyphs_from_confusables()
results = {}
test_domains = [
"go.com",
"fb.com",
"paypal.com",
"stripe.com",
"revolut.com",
"apple.com",
"google.com",
"microsoft.com",
"amazon.com",
"instagram.com",
"twitter.com",
"linkedin.com",
"github.com",
]
for domain in test_domains:
for char in set(domain.replace(".", "")):
dt_set = set(dnstwist_glyphs.get(char, []))
cf_set = set(confusable_glyphs.get(char, []))
overlap = dt_set & cf_set
dt_only = dt_set - cf_set
cf_only = cf_set - dt_set
results[char] = {
# uncomment to see the overlapping and non-overlapping charsets
# 'dnstwist_chars': dt_set,
# 'confusables_chars': cf_set,
# 'overlap_chars': overlap,
# 'dnstwist_only_chars': dt_set - cf_set,
# 'confusables_only_chars': cf_set - dt_set,
# 'dnstwist_count': len(dt_set),
# 'confusables_count': len(cf_set),
# 'overlap_count': len(overlap),
"dnstwist_only_count": len(dt_only),
"confusables_only_count": len(cf_only),
"overlap_pct": (
len(overlap) / len(dt_set | cf_set) if dt_set | cf_set else 0
),
}
pprint(results,indent=2)
if __name__ == "__main__":
validate()
Output from above script
{ 'a': { 'confusables_only_count': 34,
'dnstwist_only_count': 13,
'overlap_pct': 0.020833333333333332},
'b': { 'confusables_only_count': 40,
'dnstwist_only_count': 5,
'overlap_pct': 0.021739130434782608},
'c': { 'confusables_only_count': 29,
'dnstwist_only_count': 5,
'overlap_pct': 0.05555555555555555},
'd': { 'confusables_only_count': 31,
'dnstwist_only_count': 6,
'overlap_pct': 0.075},
'e': { 'confusables_only_count': 26,
'dnstwist_only_count': 12,
'overlap_pct': 0.02564102564102564},
'f': { 'confusables_only_count': 27,
'dnstwist_only_count': 1,
'overlap_pct': 0.034482758620689655},
'g': { 'confusables_only_count': 19,
'dnstwist_only_count': 7,
'overlap_pct': 0.07142857142857142},
'h': { 'confusables_only_count': 22,
'dnstwist_only_count': 9,
'overlap_pct': 0.06060606060606061},
'i': { 'confusables_only_count': 54,
'dnstwist_only_count': 12,
'overlap_pct': 0.05714285714285714},
'k': { 'confusables_only_count': 15,
'dnstwist_only_count': 7,
'overlap_pct': 0.0},
'l': { 'confusables_only_count': 194,
'dnstwist_only_count': 3,
'overlap_pct': 0.010050251256281407},
'm': { 'confusables_only_count': 0,
'dnstwist_only_count': 5,
'overlap_pct': 0.0},
'n': { 'confusables_only_count': 50,
'dnstwist_only_count': 9,
'overlap_pct': 0.016666666666666666},
'o': { 'confusables_only_count': 135,
'dnstwist_only_count': 11,
'overlap_pct': 0.020134228187919462},
'p': { 'confusables_only_count': 35,
'dnstwist_only_count': 2,
'overlap_pct': 0.05128205128205128},
'r': { 'confusables_only_count': 44,
'dnstwist_only_count': 10,
'overlap_pct': 0.05263157894736842},
's': { 'confusables_only_count': 28,
'dnstwist_only_count': 7,
'overlap_pct': 0.05405405405405406},
't': { 'confusables_only_count': 23,
'dnstwist_only_count': 6,
'overlap_pct': 0.03333333333333333},
'u': { 'confusables_only_count': 34,
'dnstwist_only_count': 16,
'overlap_pct': 0.0196078431372549},
'v': { 'confusables_only_count': 35,
'dnstwist_only_count': 5,
'overlap_pct': 0.024390243902439025},
'w': { 'confusables_only_count': 27,
'dnstwist_only_count': 8,
'overlap_pct': 0.027777777777777776},
'y': { 'confusables_only_count': 32,
'dnstwist_only_count': 6,
'overlap_pct': 0.09523809523809523},
'z': { 'confusables_only_count': 22,
'dnstwist_only_count': 6,
'overlap_pct': 0.0967741935483871}}
When dnstwist is invoked with fuzzer option homoglyph it uses the map glyphs_unicode (https://github.com/elceef/dnstwist/blob/master/dnstwist.py#L615) as a basis for finding visually equivalent Unicode characters for corresponding ascii characters.
The Unicode consortium publishes https://unicode.org/Public/security/latest/confusables.txt
which lists codes which are visually confusable. The mapping used by dnstwist can be enhanced
to make use of the visually confsuable characters from the above document.
Additionally, some of the Unicode characters that are visually similar to ascii codes (which are mentioned in the above document)
aren't IDNA complaint, but given the evolving threat landscape it might be good to include those Unicode characters
too in the fuzzed output generated (when the fuzzer is homoglyph).
Below script compares the visually matching glyphs (to ascii characters) derived from "confusables.txt"
(https://unicode.org/Public/security/latest/confusables.txt) (Unicode version 17.0.0) with Unicode glyph mapping
used by "dnstwist" for some of the well known domains.
Code from dnstwist version 20250130 was used for below script.
The resulting output shows a very low percentage of overlap between glyphs from confusable and glyphs dnstwist uses.
Output from above script