The Turkish uppercase trap: i, I and the dotted İ in code

A search box returns nothing for a name that is definitely in the database. A file upload rejects a .gif that is plainly a gif. A navigation menu set in capitals reads as a typo to every Turkish speaker who sees it. All three come from the same place: Turkish pairs the letters i and I differently from every other Latin alphabet, and almost every case function in every stack has an opinion about that which it never states out loud.

What actually happens

Turkish has four i letters, not two. There is a dotted pair, i and İ, and a dotless pair, ı and I. Uppercasing i gives İ, and lowercasing I gives ı. In every other Latin locale, i and I are a pair and the other two letters do not exist.

Two consequences follow, and both of them bite.

The first is that case folding stops being reversible:

'ısırgan'.toUpperCase()              // 'ISIRGAN'
'ısırgan'.toUpperCase().toLowerCase() // 'isirgan'  the dotless i is gone
'title'.toLocaleUpperCase('tr')      // 'TİTLE'
'gif'.toLocaleUpperCase('tr')        // 'GİF'

The second is worse because it is invisible. Lowercasing İ outside the Turkish locale does not give i. Unicode keeps the dot as a separate combining mark:

'İ'.toLowerCase().length             // 2
JSON.stringify('İ'.toLowerCase())    // '"i̇"'
'İ'.toLowerCase() === 'i'            // false
'İ'.toLowerCase().normalize('NFC') === 'i'  // still false

So a value that was lower cased in one place and compared against a value lower cased in another place can differ by a character nobody can see, and normalisation will not rescue it.

The rules are not the same across languages either. In JavaScript, toUpperCase is locale independent and toLocaleUpperCase is not. In Java, the plain toUpperCase is the locale sensitive one and uses the default locale of the process. In C# it is ToUpper that follows the current culture and ToUpperInvariant that does not. Carrying the habit from one language into another is how this bug travels.

The four places it shows up

• Identifiers. ext.toLocaleUpperCase() === 'GIF' is false on a Turkish machine, because the left side is GİF. The same applies to header names, enum values, currency codes and anything else you normalise before comparing.
• Slugs. A title lower cased on a Turkish server turns ISTANBUL into ıstanbul, and the step that strips non ASCII characters then deletes the leading letter. The slug becomes stanbul, the old link 404s, and the value is not even stable between machines.
• Display. A menu item iletişim under text-transform: uppercase with no lang renders ILETISIM. Turkish readers see it as a misspelling, in the same way an English reader would read PARlS.
• Comparison in the database. A Turkish collation separates dotted from dotless on purpose, so a LIKE search that works against one column silently returns nothing against another.

How to see it

Run the mappings in the language you actually use, on the machine you actually deploy to:

const s = 'İstanbul ısırgan';
console.log(s.toUpperCase());           // İSTANBUL ISIRGAN
console.log(s.toLowerCase());           // i̇stanbul ısırgan   (note the extra dot)
console.log(s.toLocaleUpperCase('tr')); // İSTANBUL ISIRGAN
console.log(s.toLocaleLowerCase('tr')); // istanbul ısırgan
console.log(Intl.DateTimeFormat().resolvedOptions().locale);

That last line is the one people forget. If it prints a Turkish locale in production and an English one on your laptop, every toLocaleUpperCase() call with no argument is a different function in the two places.

On the database side, ask the server directly rather than reading documentation about it:

/* MySQL: a Turkish collation deliberately splits the two i letters */
SELECT 'i' = 'I' COLLATE utf8mb4_turkish_ci AS dotless,
       'i' = 'İ' COLLATE utf8mb4_turkish_ci AS dotted;

/* PostgreSQL: upper() follows the collation of its input */
SHOW lc_ctype;
SELECT upper('i'), upper('i' COLLATE "tr-TR-x-icu");

A column with a Turkish collation and a column with a general one cannot be compared directly in MySQL; you get an illegal mix of collations error, which is the polite version of this bug. The impolite version is a comparison that succeeds and returns the wrong rows, and it will also stop an index from being used, which turns into a slow query with a confusing plan.

The fix

Separate two operations that look identical and are not. One is folding for comparison, which must be stable everywhere. The other is changing case for display, which must follow the reader's language.

For comparison, fold explicitly and never through a locale aware function:

// identifiers: map the Turkish letters yourself, then use the invariant lowercase
const FOLD = { 'İ': 'i', 'I': 'i', 'ı': 'i', 'Ş': 's', 'ş': 's', 'Ğ': 'g', 'ğ': 'g',
               'Ü': 'u', 'ü': 'u', 'Ö': 'o', 'ö': 'o', 'Ç': 'c', 'ç': 'c' };

export const slug = (s) =>
  [...s].map((c) => FOLD[c] ?? c).join('')
    .toLowerCase()
    .normalize('NFKD').replace(/[̀-ͯ]/g, '')
    .replace(/[^a-z0-9]+/g, '-')
    .replace(/^-|-$/g, '');

For user facing text, do not fold at all. Ask a collator, which knows that dotted and dotless are different letters and that case is not:

const same = new Intl.Collator('tr', { sensitivity: 'base', usage: 'search' });
same.compare('İstanbul', 'istanbul');  // 0   the same word
same.compare('ISIRGAN', 'ısırgan');    // 0   the same word
same.compare('isirgan', 'ısırgan');    // not 0, two different words
same.compare('ISTANBUL', 'istanbul');  // not 0, because İSTANBUL is the Turkish capital form

For display, let CSS do it. text-transform: uppercase on an element inside lang="tr" produces the right letters and leaves the DOM text alone, so copy, find in page and screen readers all still see what you wrote. That is also why the lang attribute is worth setting carefully: the same attribute decides whether the browser can hyphenate a justified column.

Then put a rule in the linter: toLocaleUpperCase and toLocaleLowerCase are never called without an argument. That single rule removes most of the surface.

How to check it worked

Force the locale for one run of the test suite. A machine dependent bug becomes a deterministic failure:

LANG=tr_TR.UTF-8 LC_ALL=tr_TR.UTF-8 npm test

grep -rn "toLocaleUpperCase()\|toLocaleLowerCase()" src/ | wc -l
#        0

Add four fixtures and keep them forever: İstanbul, ısırgan, ILIK, iyi. They cover both pairs in both directions. Assert on the slug output and on the comparison result, not on the intermediate case folded string, because the intermediate value is allowed to change and the behaviour is not.

What to watch out for

• Deduplication by lower cased value will let a duplicate through: a row saved as i plus a combining dot and a row saved as plain i are two different strings with a unique index on the column.
• Container images usually default to the POSIX locale, so the bug shows up only on a developer machine, or only in production, depending on which side sets LANG. Pin it in both.
• text-transform: uppercase on lang="tr" also applies to English words inside that element, so a menu with Blog and iletişim in it will render BLOG correctly and İLETİŞİM correctly, but a borrowed word like title becomes TİTLE. Mark the odd ones with lang="en".
• Search engines and analytics lower case URLs on their own side. A slug containing ı will come back from some of them as something else, which is one more reason to keep slugs ASCII.
• The same instinct applies to every other Turkish specific encoding question, including what a single Turkish character does to the size of a text message.

Case looks like a property of a string and is really a property of a language, and the default that every platform ships is the English one wearing a neutral coat. The practical rule I keep coming back to is that any value the machine compares should be folded by code I can read, and any value a person reads should be cased by the engine with the language written next to it. Once those two paths are separate, the Turkish i stops being a special case and becomes what it always was: one language's letters, handled by the part of the system that knows the language.