Found while auditing Stacks for 1.0 (stacksjs/stacks#3112). Reproduced on 0.1.2, the latest release:
| input | slugify() | expected |
|---|---|---|
日本語のタイトル | "" | 日本語のタイトル |
中文标题 | "" | 中文标题 |
한국어 제목 | "" | 한국어-제목 |
Rock—Paper—Scissors (em dashes) | rockpaperscissors | rock-paper-scissors |
Pages 1–10 (en dash) | pages-110 | pages-1-10 |
What it does downstream in Stacks, which calls slugify for CMS categories, tags and pages and in uniqueSlug:
- Two Japanese tag names both get the slug
"", so they collide on a unique index or become one tag. uniqueSlug('日本語のタイトル')hands out"", then-2, then-3, so pages land at/blog/and/blog/-2.
Proposal
- Letters the character map cannot transliterate are kept rather than removed. A Unicode slug is valid in a URL (percent-encoded on the wire), and WordPress does the same. Transliterating CJK would need a dictionary for each language.
- Dash punctuation (
\p{Pd}: em dash, en dash, figure dash and the rest) separates words, as a hyphen does.
This changes output for those inputs, so a minor release (0.2.0) suits it better than a patch, so that ^0.1.x installs don't change their slugs without asking.