Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 28, 2026

The toWellFormed function in fallback/utf32.js incorrectly preserved consecutive surrogate codepoints when they formed what would be a valid pair in UTF-16. UTF-32 does not use surrogate pairs—all values in the surrogate range (0xD800-0xDFFF) are invalid and must be replaced individually.

Changes

  • fallback/utf32.js: Simplified surrogate handling to unconditionally replace all codepoints in 0xD800-0xDFFF range with replacement character (0xFFFD)
  • Removed pair-checking logic that was incorrectly ported from UTF-16 implementation

Example

// Before: consecutive surrogates incorrectly preserved
const input = new Uint32Array([0x41, 0xD800, 0xDC00, 0x42])
toWellFormed(input)  // [0x41, 0xD800, 0xDC00, 0x42] ❌

// After: all surrogates replaced as required by UTF-32
toWellFormed(input)  // [0x41, 0xFFFD, 0xFFFD, 0x42] ✓
Original prompt

This section details on the original issue you should resolve

<issue_title>Review the codebase and identify mistakes</issue_title>
<issue_description>Ignore tests/wpt and other vendored tests

Note: wif using synchronous path for async implementation is not an issue, it's done for perf, as underlying method is always sync (again, for perf).

Pay close attention to corner cases</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: ChALkeR <291301+ChALkeR@users.noreply.github.com>
Copilot AI changed the title [WIP] Review codebase and identify mistakes Fix UTF-32 surrogate handling in toWellFormed Jan 28, 2026
Copilot AI requested a review from ChALkeR January 28, 2026 17:24
Copilot stopped work on behalf of ChALkeR due to an error January 28, 2026 17:37
@ChALkeR
Copy link
Collaborator

ChALkeR commented Jan 28, 2026

Ok this makes sense but will do that separately
UTF-32 is not shipped yet and was not carefully tested

@ChALkeR ChALkeR closed this Jan 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Review the codebase and identify mistakes

2 participants