Folds text to a comparable form for search and deduplication: lowercase, diacritics removed, punctuation collapsed — while keeping the text in its own script.
$normalizer->normalize('Crème Brûlée'); // "creme brulee"
$normalizer->normalize('مَدْرَسَة'); // "مدرسه" — still ArabicMost PHP libraries in this space produce an ASCII slug for URLs. This one produces a comparison key in the original script. Pick accordingly:
| You want | Use |
|---|---|
A URL slug (crème → creme) |
cocur/slugify or symfony/string |
Transliteration into Latin (مدرسة → madrasa) |
ext-intl Transliterator |
A comparison key that stays Arabic (مَدْرَسَة → مدرسه) |
this package |
The distinction matters for search. Transliterating Arabic to Latin collapses unrelated roots onto the same consonant skeleton and produces a key you cannot display, highlight, or feed back into an Arabic index.
composer require cleatsquad/php-text-normalizerPHP 8.2+. Uses ext-intl when present, and falls back to
symfony/polyfill-intl-normalizer otherwise.
use CleatSquad\TextNormalizer\TextNormalizer;
$normalizer = new TextNormalizer();
$normalizer->normalize('Quelle est la MÉTÉO à Rabat ?');
// "quelle est la meteo a rabat"
$normalizer->tokenize('token-2024');
// ['token', '2024']$result = $normalizer->analyze(' Météo à RABAT !!! ');
$result->normalized; // "meteo a rabat"
$result->original; // " Météo à RABAT !!! "
$result->wasModified(); // true
$result->length(); // 13
$result->profileName; // "arabic_search_latin"use CleatSquad\TextNormalizer\NormalizerProfile;
new TextNormalizer(NormalizerProfile::latin()); // Latin only
new TextNormalizer(NormalizerProfile::arabic()); // Arabic search mode (ة -> ه)
new TextNormalizer(NormalizerProfile::arabic(searchEquivalences: false)); // Arabic strict mode (preserves ة)
new TextNormalizer(NormalizerProfile::cyrillic()); // Cyrillic (ё -> е, і/ї -> i)
new TextNormalizer(NormalizerProfile::greek()); // Greek (ς -> σ, tonos stripped)
new TextNormalizer(NormalizerProfile::all()); // default (all scripts)
new TextNormalizer(new NormalizerProfile()); // punctuation only, folds nothingProfiles are scoped: a Latin profile leaves Arabic harakat exactly where they are. Compose your own, or extend a shipped one:
$profile = NormalizerProfile::latin()->merge(
new NormalizerProfile(characterMap: ['ij' => 'ij'])
);Diacritics — by Unicode canonical decomposition, not a table. Košice,
Ṣāliḥ, Đà Nẵng, Ĝangalo all fold correctly, in every script, because NFD
reaches every decomposable letter. A hand-written table only ever covers the
ones someone remembered.
Letters that carry no mark — by table, since decomposition cannot reach
them: æ œ ß ø ł đ ð þ ħ ı ŋ ŧ ƶ.
Arabic orthographic equivalences — by table, because Unicode considers them distinct letters and no normalization form unifies them:
| Fold | Why |
|---|---|
أ إ آ ٱ → ا |
Alif variants |
ة → ه |
Ta Marbuta, as search indexes conventionally do |
ى ی ې ۍ → ي |
Alef Maksura, and Persian/Urdu/Pashto Yeh |
ک ګ → ك |
Keheh (Persian/Urdu Kaf) |
ہ ھ → ه |
Heh Goal, Heh Doachashmee |
٠-٩ and ۰-۹ → 0-9 |
Arabic-Indic and Extended Arabic-Indic digits |
tatweel ـ removed |
decorative elongation, never lexical |
| ZWNJ/ZWJ removed | invisible, and Persian puts them inside words |
Without these, علي typed on an Arabic keyboard and علی typed on a Persian
one are two different strings, and your index answers nothing.
Idempotent. Normalizing an already-normalized string returns it unchanged.
Output stays in NFC whenever valid UTF-8 Unicode normalization succeeds, making it safe to store and compare byte-wise. Malformed UTF-8 inputs are returned untouched rather than converted to an empty string.
Combining marks a profile does not claim are preserved, attached to their letter rather than treated as word boundaries.
No clock, no I/O, no configuration files. One object, two methods.
composer install
composer test # PHPUnit
composer analyse # PHPStan, max levelMIT. See LICENSE.