Decomposition

What Is a Decomposition Mapping?

A decomposition mapping tells you how a Unicode character can be broken down into a sequence of simpler characters. There are two kinds:

• Canonical decomposition: the character is identical in meaning and rendering to its decomposed sequence. For example, U+00E9 LATIN SMALL LETTER E WITH ACUTE (é) canonically decomposes to U+0065 LATIN SMALL LETTER E + U+0301 COMBINING ACUTE ACCENT.
• Compatibility decomposition: the character is only compatible (semantically similar, possibly different appearance) with its decomposed sequence. For example, the ligature U+FB01 fi (fi) compatibility-decomposes to U+0066 f + U+0069 i, and U+00B2 ² (superscript two) decomposes to U+0032 2.

Normalization Forms

The four Unicode Normalization Forms are defined in terms of decomposition and canonical composition:

import unicodedata

samples = [
    ("\u00E9", "é  e+acute"),        # canonical
    ("\u00C5", "Å  A+ring"),         # canonical
    ("\uFB01", "fi  fi ligature"),    # compatibility
    ("\u00B2", "²  superscript 2"),  # compatibility
    ("\u2126", "Ω  OHM SIGN"),       # canonical → U+03A9 GREEK CAPITAL OMEGA
]

for char, label in samples:
    raw = unicodedata.decomposition(char)
    nfd = unicodedata.normalize("NFD", char)
    nfkd = unicodedata.normalize("NFKD", char)
    nfc = unicodedata.normalize("NFC", nfd)
    print(f"  {label}")
    print(f"    decomposition() raw : {raw!r}")
    print(f"    NFD  : {[f'U+{ord(c):04X}' for c in nfd]}")
    print(f"    NFKD : {[f'U+{ord(c):04X}' for c in nfkd]}")
    print(f"    NFC  : {[f'U+{ord(c):04X}' for c in nfc]}")

The unicodedata.decomposition() function returns a raw string from UnicodeData.txt. A leading tag in angle brackets like <compat>, <font>, <circle>, <wide>, etc. indicates a compatibility decomposition; no tag means canonical.

Practical Implications

Search and indexing: NFKC normalization lets you match file against file or ２ against 2. Many search engines apply NFKC before indexing. Security: Compatibility decomposition can reveal confusable characters—U+2126 Ω and U+03A9 Ω look identical and are canonically equivalent, so an application that compares usernames should normalize first. Identifiers: Python 3 uses NFKC for identifier normalization (PEP 3131).

Quick Facts