Skip to content

Cover CJK and four-byte characters in the filename tests - #39

Merged
jakejackson1 merged 1 commit into
mainfrom
test/cjk-and-4-byte-filenames
Aug 24, 2026
Merged

Cover CJK and four-byte characters in the filename tests#39
jakejackson1 merged 1 commit into
mainfrom
test/cjk-and-4-byte-filenames

Conversation

@jakejackson1

Copy link
Copy Markdown
Member

Nothing under tests/ carried a Han ideograph, kana or Hangul syllable, and the only four-byte character was an emoji in FilenameTest::provideInvalidUtf8ToSanitize — the display sanitizer's provider — so no four-byte character was ever driven through setName().

Adds 26 tests. No source change.

providerSetNameSanitizing — runs in every mbstring configuration

Row Case
82 聊天(PC)-201711201615.xlsx, CJK mixed with the ASCII punctuation the rewriter does touch
83 Katakana ファイル名.txt
84 Hangul 파일이름.txt
85 CJK Extension B 𠮟𪚲.txt, four-byte
86 photo😀.png, four-byte
87 文字(v2).文 — an extension is ASCII alphanumeric or discarded whole
88, 89 A three- and a four-byte character ending exactly on the 250 byte budget, kept

Row 82 is the shape a client name off a CJK desktop arrives in: three-byte characters either side of parentheses the rewriter turns into -. Nothing tested that combination.

providerSetNameSanitizingRequiringMbstring@group mbstring

Every offset at which a wider character can straddle the truncation cut: pads 248 and 249 for a three-byte character, 247 through 249 for a four-byte one. The provider had one two-byte case at pad 249.

This is the half with a branch behind it. forceValidUtf8() matches an incomplete tail with a separate alternative per width — [\xE0-\xEF][\x80-\xBF]? and [\xF0-\xF4][\x80-\xBF]{0,2} — and the one existing case exercised neither. The rows land on different code by configuration:

  • ext-mbstring loadedmb_strcut() refuses to split the character, so they test its boundary handling.
  • mbstring polyfill job — that job runs without --exclude-group, and the polyfill ships no mb_strcut(), so finalize() falls to substr() and the repair strips the partial sequence.

Rows 88 and 89 are in the plain provider on purpose: a character ending on a byte boundary survives a byte cut unchanged, so the expected value holds with no repair and every configuration can run them. Verified against the substr() path rather than assumed.

Checks

  • Suite 628 → 654 tests, 1270 → 1322 assertions, still exactly one skip, so cross-file-system's --fail-on-skipped guard is intact.
  • lint, phpstan (level 9) and check-syntax clean.
  • Non-vacuity checked by mutation: mb_strcutmb_substr fails 16 tests, adding \x80-\xFF to the unsafe-character class fails 62.

No CHANGELOG.md entry — test-only, and tests/ is export-ignored.

Run locally with ext-mbstring loaded. The polyfill and no-mbstring configurations are CI-only.

🤖 Generated with Claude Code

No test anywhere under tests/ carried a Han ideograph, kana or Hangul
syllable, and the only four-byte character was an emoji in the display
sanitizer's provider, so nothing drove one through setName().

The straddle rows are the ones with a branch behind them.
forceValidUtf8() matches an incomplete tail with a separate alternative
per width, and a byte cut can leave a three-byte character with one or
two bytes and a four-byte one with up to three. The single two-byte case
that was there exercised one of those alternatives. The rows land on
mb_strcut() where ext-mbstring is loaded and on substr() plus the repair
in the polyfill job, which runs the mbstring group rather than excluding
it and ships no mb_strcut().

Rows 88 and 89 sit in the provider every configuration runs: a character
ending on the budget survives a byte cut unchanged, so the expected value
holds without the repair.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jakejackson1
jakejackson1 merged commit 7920e5a into main Aug 24, 2026
36 checks passed
@jakejackson1
jakejackson1 deleted the test/cjk-and-4-byte-filenames branch August 24, 2026 03:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant