Case Folding

What is Case Folding?

Case folding is a Unicode operation that converts text to a form suitable for case-insensitive comparison. It is defined in the Unicode Standard and supported by the CaseFolding.txt data file in the Unicode Character Database. Case folding is closely related to, but distinct from, simple lowercasing: while lowercasing converts a string to its lowercase representation for display, case folding converts a string to a canonical form specifically optimized for string comparison regardless of case.

The practical difference is that case folding handles edge cases that simple lowercasing misses — particularly in languages with complex case mapping behavior.

CaseFolding.txt: The Data File

The Unicode Consortium publishes CaseFolding.txt as part of the Unicode Character Database. It maps each character to its case-folded form using one of four status codes:

Simple vs. Full Case Folding

Simple case folding maps every character to at most one character — a one-to-one mapping. It is suitable for environments where string length must be preserved.

Full case folding allows one character to map to a sequence of multiple characters. The classic example is the German sharp S:

• ß (U+00DF, Latin Small Letter Sharp S)
• Simple fold: ß → ß (no change — no uppercase in simple mapping)
• Full fold: ß → ss (two characters)

This means that a full case-fold comparison of "STRASSE" and "Straße" would correctly identify them as equal (both fold to "strasse"), while a simple lowercase comparison would not.

# Python uses full case folding via str.casefold()
"STRASSE".casefold() == "Straße".casefold()  # True
"STRASSE".lower() == "Straße".lower()         # False

# The key difference
"Straße".casefold()  # "strasse"
"Straße".lower()     # "straße"  ← ß preserved

Python's str.casefold() implements full Unicode case folding, while str.lower() implements Unicode simple lowercasing.

Locale-Sensitive Folding: The Turkish Problem

The most significant locale-specific case folding issue involves the Turkish and Azerbaijani I. In most languages:

• Uppercase I → lowercase i
• Uppercase İ does not exist (or is rare)

In Turkish and Azerbaijani: - Uppercase İ (U+0130, Latin Capital Letter I with Dot Above) → lowercase i (U+0069) - Uppercase I (U+0049, Latin Capital Letter I) → lowercase ı (U+0131, Latin Small Letter Dotless I)

The T (Turkic) status entries in CaseFolding.txt provide the Turkic-specific mappings. Standard Unicode case folding without the T entries is incorrect for Turkish text: it would map I → i rather than I → ı, causing "KISA" and "kısa" (meaning "short") to compare as unequal while "KISA" and "kisa" would compare as equal — the wrong result.

# Correct Turkish case comparison requires locale awareness
import locale
# Python's str.casefold() uses C-locale folding (non-Turkic)
# For Turkish: use icu-python or a locale-aware library

How Case Folding Differs from Lowercasing

Quick Facts