omeryanbas.com

Ömer Yanbaş

General Manager, Ticofab Yazılım

WebPractice

Justified Turkish text without a hyphenation dictionary

Chrome ships no Turkish hyphenation dictionary, so justified Turkish opens rivers of white space. Insert soft hyphens at build time instead.

A Turkish column set with text-align: justify looks wrong before you can say why. The word spaces are uneven from line to line, and every few lines the gaps line up into a pale vertical channel running down the paragraph. The CSS is right and hyphens: auto is set. Chrome simply never breaks a Turkish word, because it has no Turkish hyphenation dictionary, and Turkish words are long.

What actually happens

Justification works by taking the space left over at the end of a line and pushing it into the word spaces of that line. If the line holds ten words, the leftover is divided nine ways and nobody notices. If a long word was pushed to the next line and only four words remain, the same leftover is divided three ways and each gap triples.

Hyphenation is what normally prevents that. The browser is allowed to break a word across lines, but only where the language says a break is legal, and it learns that from a set of patterns loaded per language. The language is read from the lang attribute on the element, not from the characters in the text. Chrome loads those pattern sets for a limited list of languages, and Turkish has never been on it. Safari leans on the system dictionaries, so on Apple platforms the result is better. You cannot design a column around a behaviour that only one engine has.

Turkish makes the gaps worse than they would be in English, because it is agglutinative. Suffixes stack, and a perfectly ordinary word in running prose can be sixteen or twenty letters long. One of those at the wrong moment empties a line.

There is a second mode of the same property that nobody uses. hyphens: manual is the default, and it means the browser will break a word at a soft hyphen, U+00AD, and nowhere else. No dictionary is involved. Every engine has supported it for years. All you need is something that knows where Turkish syllables end.

How to see it

Put one paragraph of Turkish in a narrow box with hyphens: auto, then count the lines under two different lang values. If the engine had Turkish patterns, the count would differ:

const box = document.querySelector('#probe');
const lineHeight = parseFloat(getComputedStyle(box).lineHeight);
const lines = (lang) => {
  box.lang = lang;
  return Math.round(box.getBoundingClientRect().height / lineHeight);
};
console.log('tr', lines('tr'), 'en', lines('en'));
// Chrome: tr 14 en 13

The English patterns are wrong for Turkish, so lang="en" produces bad breaks, but it produces breaks. That is the proof that the code path works and only the Turkish data is missing. Never ship lang="en" on Turkish text to get this effect: it breaks screen reader pronunciation and language detection, which is one of the accessibility mistakes worth fixing first.

The second probe is the fix in miniature. Put the hyphens in by hand and watch the break happen:

<p style="width: 7ch; text-align: justify">de&shy;ğer&shy;len&shy;dir&shy;me</p>

Turkish syllables are a rule, not a dictionary

A Turkish syllable has exactly one vowel, and the boundary between two syllables is decided entirely by how many consonants sit between two vowels:

  • two vowels in a row split between them: V-V
  • one consonant between vowels goes forward: V-CV
  • two consonants split: VC-CV
  • three consonants keep the first two and send the last forward: VCC-CV

Every one of those reduces to the same sentence: the last consonant of the cluster starts the next syllable. That is a loop over the characters, not a dictionary.

const VOWELS = new Set([...'aeıioöuüâîûAEIİOÖUÜÂÎÛ']);
const SHY = '­';

function syllabify(word) {
  const chars = [...word];
  const isVowel = chars.map((c) => VOWELS.has(c));
  const breaks = [];
  let prev = -1;
  for (let i = 0; i < chars.length; i++) {
    if (!isVowel[i]) continue;
    if (prev >= 0) {
      const cluster = i - prev - 1;           // consonants since the previous vowel
      breaks.push(cluster === 0 ? i : i - 1); // before the vowel, or before the last consonant
    }
    prev = i;
  }
  // never strand one or two letters at either end of the word
  const ok = new Set(breaks.filter((b) => b >= 2 && chars.length - b >= 2));
  return chars.map((c, i) => (ok.has(i) ? SHY + c : c)).join('');
}

The fix

Wrap the syllabifier in a pass over words, skip anything short, and keep an exception list:

const LETTER = /[A-Za-zÇçĞğİıÖöŞşÜüÂâÎîÛû]/;
// loanwords whose clusters are borrowed onsets, plus English terms used untranslated
const SKIP = new Set(['elektrik', 'kontrol', 'santral', 'backend', 'request', 'blockchain']);

export function hyphenate(text) {
  let out = '';
  let word = '';
  const flush = () => {
    const w = word.length >= 7 && !SKIP.has(word.toLocaleLowerCase('tr')) ? syllabify(word) : word;
    word = '';
    return w;
  };
  for (const ch of text) {
    if (LETTER.test(ch)) word += ch;
    else out += flush() + ch;
  }
  return out + flush();
}

The locale argument on toLocaleLowerCase is not decoration. Without it the result depends on the machine the build runs on, which is the uppercase i trap arriving through the back door.

Run this over the content fields at build time, before they go anywhere near a template. Then the CSS stays boring:

.prose {
  text-align: justify;
  hyphens: manual;        /* the default: break only at a soft hyphen */
  overflow-wrap: break-word;
}

The cost is small and worth stating. On one page this pass inserted 295 soft hyphens. That is 590 raw bytes, 256 after gzip and 212 after brotli. Turkish characters are expensive in other places, notably in the encoding of a text message, but in an HTML response they disappear into the compressor.

How to check it worked

Count what landed in the output, and confirm none of it landed somewhere structural:

node -e 'const s = require("node:fs").readFileSync(process.argv[1], "utf8");
  console.log("soft hyphens:", [...s].filter((c) => c === "­").length);
  console.log("inside a tag:", (s.match(/<[^>]*­[^>]*>/g) || []).length);' dist/tr/index.html
# soft hyphens: 295
# inside a tag: 0

The second number is the one that matters. If it is anything but zero, the pass ran over HTML instead of over text, and there is now a soft hyphen inside an attribute value or a tag name.

Then reload the page and look at the right edge of a paragraph. The fix is working when the word spaces on consecutive lines look the same width, and hyphens appear at the ends of maybe one line in four.

When to set ragged right instead

Justification is not free, and there are cases where the honest answer is to stop:

  • The column is narrower than about sixty characters. There is nowhere for the leftover space to hide, and hyphenation alone will not save it.
  • The text is user supplied. You cannot promise that it is Turkish, and running Turkish syllable rules over a German or Arabic name produces a break in the wrong place, permanently, in the markup.
  • The content contains long unbreakable tokens: URLs, identifiers, file paths. Those defeat justification whatever you do.
  • You cannot run a build step. Inserting soft hyphens in the browser after paint causes a reflow of the whole column and is far more visible than a ragged edge.

What to watch out for

  • Soft hyphens travel with copied text in some browsers. Keep them out of code samples, commands, e-mail addresses and anything numeric.
  • If you have a site search, strip U+00AD before you index and before you match, or a search for a word will miss the hyphenated copy of that same word on the page.
  • The rule mis-splits borrowed clusters. When you see a bad break, add the word to the exception list rather than trying to teach the rule about phonotactics. The list stays at a few dozen entries and it is readable; the clever version is neither.
  • Do not apply the pass twice. Running it over already hyphenated text is harmless only because a soft hyphen is not a letter, and that is an accident you should not rely on.

Automatic hyphenation looks like a browser feature and is actually a data problem, and the data for smaller languages arrives late or never. When the platform has no answer for your language, it is worth checking whether the language itself has a rule simple enough to encode, because orthographies that were designed in the twentieth century usually do. Twenty lines of build step bought a typographic feature that no amount of CSS was going to produce. The same question is worth asking before you accept any "not supported" answer: supported by whom, and is the underlying rule really that hard.

Questions and answers

Why does hyphens: auto do nothing for Turkish text?
Automatic hyphenation needs a set of language specific patterns, and the browser picks the set from the lang attribute on the element. Chrome loads those patterns for a limited list of languages and Turkish is not on it, so the property is accepted and then has nothing to work with. You can prove it by switching the same paragraph to lang="en" and watching the line count change.
Is a soft hyphen safe to put in HTML?
Yes, U+00AD is invisible until the line breaks there, and it is two bytes in UTF-8 that compress to almost nothing. The risk is not rendering, it is everything downstream: search indexes, copied text and string comparisons all see a character that the reader never sees. Strip it before indexing and never insert it into code, URLs or anything a person will copy and paste.
Which Turkish words does a rule based syllabifier get wrong?
Loanwords that carry a consonant cluster Turkish does not normally allow at the start of a syllable. The rule splits elektrik as elekt-rik and kontrol as kont-rol, where readers expect e-lek-trik and kon-trol. A list of twenty or thirty such words, plus the English technical terms you use untranslated, covers almost every case you will see.
Should I justify text on a phone?
Usually not. Justification only looks good when the line is long enough to absorb the leftover space, which means roughly sixty characters or more. On a narrow column, set ragged right and let the right edge be uneven, because uneven word spacing inside the line is far more distracting than an uneven edge.