混合文字系统检测

What is Mixed-Script Detection?

Mixed-script detection is a security technique that identifies text strings containing characters from more than one Unicode script, flagging them as potentially deceptive. Because legitimate text in most contexts is written in a single script — English in Latin, Russian in Cyrillic, Arabic in the Arabic script — the presence of multiple scripts in a single identifier, domain name, or username is a strong signal of a spoofing attempt.

Unicode Technical Report #39 (Unicode Security Mechanisms) formalizes mixed-script detection as one of the primary defenses against homoglyph and confusable attacks.

Unicode Scripts

Unicode organizes characters into scripts — named systems of writing associated with particular languages and cultures. Every Unicode character (except a small set of "Common" and "Inherited" script characters) belongs to exactly one script. The Unicode Character Database includes a Scripts.txt property file assigning each code point to its script.

Examples of scripts: Latin, Cyrillic, Greek, Armenian, Hebrew, Arabic, Devanagari, Bengali, CJK (Han), Hiragana, Katakana, Thai, Georgian.

A handful of characters — digits (0–9), punctuation like ., -, @ — have script property Common and are allowed in any script context without triggering mixed-script detection.

How Mixed-Script Detection Works

The algorithm examines all characters in a string and collects the set of scripts represented:

1. Characters with script Common or Inherited are ignored for mixing purposes
2. Characters with a specific script (Latin, Cyrillic, etc.) are added to the script set
3. If the resulting set contains more than one script, the string is mixed-script

For example:

• paypal — all Latin — single script, clean
• раураl — Cyrillic р, а, у, р, а + Latin l — mixed script, flagged
• münchen — Latin + Common (no mixing concern) — single script, clean
• аррlе — Cyrillic а, р, р + Latin l, е — mixed script, flagged

Augmented Script Sets

Unicode TR39 defines the concept of augmented script sets to handle characters that are legitimately used across scripts. For example, Han characters are used in both Japanese (combined with Hiragana and Katakana) and Chinese. The augmented script sets expand the "allowed combinations" to prevent false positives for legitimate multilingual text such as Japanese.

This means Japanese text containing Hiragana, Katakana, and Han characters is not flagged as mixed-script because all three are in Japan's augmented script set. Only truly suspicious combinations — Latin mixed with Cyrillic, for example — trigger the detection.

Spoof Checks Defined in TR39

Unicode TR39 defines four formal spoof check levels:

1. Single-script confusable: A single-script string that is confusable with another single-script string (e.g., all-Cyrillic lookalike of a Latin word)
2. Mixed-script confusable: A string mixing scripts where replacing characters would produce a confusable string in a single script
3. Whole-script confusable: An entire string in script X that is confusable with a string in script Y
4. Any-case confusable: The above checks applied case-insensitively

Implementation in Practice

Browser implementations use mixed-script detection to decide whether to display an internationalized domain name as Unicode or fall back to Punycode. Domain registrars apply it to block registration of mixed-script domains. Programming language toolchains use it to warn about suspicious identifiers.

Python 3 uses a variant of this check for source code identifiers. The unicodedata module exposes script information, and third-party libraries like confusable-homoglyphs implement full TR39 checks.

Quick Facts