Skip to content

lib: map Latin1 labels to iso-8859-1 instead of Windows-1252#58890

Closed
lytovka wants to merge 7 commits intonodejs:mainfrom
lytovka:lytovka/56542-windows-1252-encoding
Closed

lib: map Latin1 labels to iso-8859-1 instead of Windows-1252#58890
lytovka wants to merge 7 commits intonodejs:mainfrom
lytovka:lytovka/56542-windows-1252-encoding

Conversation

@lytovka
Copy link

@lytovka lytovka commented Jun 29, 2025 •
edited
Loading

Fixes: #56542

This PR updates all Latin1 labels to point to the iso-8859-1 encoding instead of Windows-1252. The iso-8859-1 encoding will now use the decodeLatin1 fast path when calling the decode method. The Windows-1252 encoding will not trigger the decodeLatin1 fast path; instead, it will follow the standard path for obtaining the converter from the simdutf library.

A new test file has been added to verify the decoded Unicode values of bytes 0x7F-0x9F when the Windows-1252 encoding is selected.

NB: Fixing Latin1 label mappings will cause unexpected behavior if TextDecoder is called with any Latin1 label and attempts to decode bytes in the 0x80-0x9F range, since decoding for any of these labels will now follow the iso-8859-1 encoding.

Refs:

lytovka added 4 commits June 29, 2025 14:21
@lytovka
lib: map Latin1 labels to iso-8859-1 encoding
50aa8bc 
Latin1 is incorrectly mapped to the Windows-1252 encoding,
which defines mappings for bytes 0x80–0x9F, unlike Latin1 (ISO-8859-1),
where these bytes are control characters. Fixing this discrepancy can
cause unexpected behavior if TextDecoder is called with any
latin1 label and attempts to decode bytes in the 0x80–0x9F range,
since the decoding will now follow ISO-8859-1 encoding.

Fixes: nodejs#56542
@lytovka
test: fix label assertions in existing tests
1bf1c92
@lytovka
test: decode 0x7F-0x9F bytes with windows-1252 encoding
8819222
@lytovka
test: revert removal of windows-1254 labels
2813ea4
@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Jun 29, 2025

Review requested:

  • @nodejs/web-standards

@nodejs-github-bot nodejs-github-bot added encoding Issues and PRs related to the TextEncoder and TextDecoder APIs. needs-ci PRs that need a full CI run. labels Jun 29, 2025
@lytovka lytovka marked this pull request as ready for review June 29, 2025 19:56
@jasnell
Copy link
Member

jasnell commented Jun 29, 2025

I do have some slight concerns over whether this may be considered a breaking change. My preference would be to handle it as a bug fix, however. I'd like some feedback from @nodejs/tsc

@lytovka
Copy link
Author

lytovka commented Jun 29, 2025

I do have some slight concerns over whether this may be considered a breaking change. My preference would be to handle it as a bug fix, however. I'd like some feedback from @nodejs/tsc

I had similar thoughts. I've updated the PR description with a note explaining why this could be considered a breaking change:

NB: Fixing Latin1 label mappings will cause unexpected behavior if TextDecoder is called with any Latin1 label and attempts to decode bytes in the 0x80-0x9F range, since decoding for any of these labels will now follow the iso-8859-1 encoding.

@codecov
Copy link

codecov bot commented Jun 30, 2025 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.10%. Comparing base (4c65776) to head (43fbb04).
⚠️ Report is 1360 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main   #58890      +/-   ##
==========================================
- Coverage   90.10%   90.10%   -0.01%     
==========================================
  Files         640      640              
  Lines      188431   188427       -4     
  Branches    36956    36959       +3     
==========================================
- Hits       169783   169776       -7     
+ Misses      11362    11359       -3     
- Partials     7286     7292       +6
Files with missing lines Coverage Δ
lib/internal/encoding.js 98.87% <100.00%> (-0.65%) ⬇️

... and 46 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@targos
Copy link
Member

targos commented Jun 30, 2025

I think we should land as a bug fix.

@lytovka
Copy link
Author

lytovka commented Jun 30, 2025

Hi all! Just to confirm - aside from addressing the linting errors (43fbb04), is there anything else you'd like me to do before this can land? Thanks!

@lytovka
Copy link
Author

lytovka commented Jul 13, 2025

@jasnell @targos @nodejs/tsc – bumping this up, let me know if there's anything missing. Thank you in advance!

@jasnell jasnell added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 13, 2025
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 15, 2025
@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Jul 15, 2025

CI: https://ci.nodejs.org/job/node-test-pull-request/67956/

@aduh95 aduh95 added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. request-ci Add this label to start a Jenkins CI on a PR. labels Nov 8, 2025
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Nov 8, 2025
@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Nov 8, 2025

CI: https://ci.nodejs.org/job/node-test-pull-request/70103/

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Nov 9, 2025

CI: https://ci.nodejs.org/job/node-test-pull-request/70125/

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Nov 12, 2025

CI: https://ci.nodejs.org/job/node-test-pull-request/70154/

@jazelly
Copy link
Member

jazelly commented Nov 13, 2025

Hi @lytovka looks like the CI failure is related

@ChALkeR ChALkeR mentioned this pull request Nov 29, 2025
Closed
@avivkeller avivkeller added stalled Issues and PRs that are stalled. and removed author ready PRs that have at least one approval, no pending requests for changes, and a CI started. labels Jan 9, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Jan 9, 2026

This issue/PR was marked as stalled, it will be automatically closed in 30 days. If it should remain open, please leave a comment explaining why it should remain open.

@github-actions github-actions bot removed the stalled Issues and PRs that are stalled. label Jan 10, 2026
@ChALkeR
Copy link
Member

ChALkeR commented Jan 17, 2026 •
edited
Loading

This can be closed, #61093 landed, which removed the broken codepath.

@mcollina mcollina closed this Jan 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jasnell jasnell jasnell approved these changes

@lpinca lpinca lpinca approved these changes

Assignees

No one assigned

Labels

encoding Issues and PRs related to the TextEncoder and TextDecoder APIs. needs-ci PRs that need a full CI run.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

TextDecoder incorrectly decodes 0x92 and several other characters for Windows-1252

10 participants

@lytovka @nodejs-github-bot @jasnell @targos @jazelly @ChALkeR @lpinca @mcollina @aduh95 @avivkeller