Skip to content

buffer: add isLatin1 - #66298

Open
jasnell wants to merge 1 commit into
nodejs:mainfrom
jasnell:jasnell/islatin1
Open

jasnell wants to merge 1 commit into
nodejs:mainfrom
jasnell:jasnell/islatin1

Conversation

@jasnell

@jasnell jasnell commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Implements a fast check to determine if a string is Latin1 a byte string Latin1.

Can be many times faster than the equivalent regex check.

input isByteString regex loop
short, one-byte 232M 122M 55M
short, two-byte 207M 130M 56M
short, invalid 106M 48M 53M
long, one-byte 221M 93M 230k
long, two-byte 6.7M 913k 186k
long, invalid 75M 914k 234k

(the larger perf gap there is with concatenated strings)

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/performance

@nodejs-github-bot nodejs-github-bot added buffer Issues and PRs related to the buffer subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Sep 25, 2026
@aduh95

aduh95 commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

I don't think isLatin1 is a good name, \u0100 can definitely be a 2-byte latin1 string

@jasnell

jasnell commented Sep 25, 2026

Copy link
Copy Markdown
Member Author

Yeah, I'm going to rename to isByteString ... it matches the webidl usage better.

@jasnell jasnell changed the title buffer: add isLatin1 buffer: add isByteString Sep 25, 2026
@jasnell jasnell added the semver-minor PRs that contain new features and should be released in the next minor version. label Sep 25, 2026
@jasnell
jasnell requested a review from aduh95 September 25, 2026 19:28
Comment thread src/node_buffer.cc Outdated
@jasnell jasnell added author ready PRs with CI started, the required approvals, and no outstanding review comments. request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. and removed request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. labels Sep 25, 2026
@nodejs-github-bot

nodejs-github-bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

@codecov

codecov Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.46154% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.35%. Comparing base (3641c36) to head (799740c).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
src/node_buffer.cc 76.92% 0 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #66298      +/-   ##
==========================================
- Coverage   90.36%   90.35%   -0.01%     
==========================================
  Files         792      792              
  Lines      275386   275412      +26     
  Branches    52770    52786      +16     
==========================================
+ Hits       248859   248860       +1     
- Misses      16936    16973      +37     
+ Partials     9591     9579      -12     
Files with missing lines Coverage Δ
lib/buffer.js 99.74% <100.00%> (+<0.01%) ⬆️
src/node_buffer.cc 70.48% <76.92%> (+0.06%) ⬆️

... and 25 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jasnell jasnell added the commit-queue PRs queued for automated landing through the Commit Queue. label Sep 26, 2026

@addaleax addaleax left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isLatin1String would be much better naming because it describes what the method actually tests for. The WebIDL ByteString type describes a legacy type that stems from the lack of commitment to a specific encoding in early HTTP versions other than vague ASCII backwards compatibility, and it's not actually used widely outside of HTTP-specific APIs for that (very good) reason.

I don't think isLatin1 is a good name, \u0100 can definitely be a 2-byte latin1 string

This is not a statement that makes sense, \u0100 cannot be represented in Latin-1, and there are, by definition, no 2-byte Latin-1 strings to begin with -- maybe you're referring to the idea that the byte sequence 01 00 can be Latin-1, but this method takes a string as an argument, i.e. a character sequence, not a byte sequence

@addaleax addaleax removed the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Sep 26, 2026
@addaleax addaleax removed the commit-queue PRs queued for automated landing through the Commit Queue. label Sep 26, 2026
@jasnell

jasnell commented Sep 26, 2026

Copy link
Copy Markdown
Member Author

The challenge with isLatin1 is that there are two interpretations for latin1 (ISO/IEC 8859-1 or Windows-1252 and we're not really verifying against either.... may it should just be isOneByteString(...)?

@addaleax

Copy link
Copy Markdown
Member

may it should just be isOneByteString(...)?

Without knowledge of the V8 OneByteString/TwoByteString terminology, I'd probably assume that this means one-byte when encoded as UTF-8 (i.e. ASCII).

The challenge with isLatin1 is that there are two interpretations for latin1 (ISO/IEC 8859-1 or Windows-1252 and we're not really verifying against either

That's a fair point, although in the context of Node.js APIs, latin1 already has the meaning that this PR uses, and that consistency combined with being relatively "close" to the actual mechanics of this method still makes me think it's the best choice for naming (that isn't excessively verbose).

@jasnell

jasnell commented Sep 26, 2026

Copy link
Copy Markdown
Member Author

Fair... ok... renaming it back to isLatin1 :-)

And if @aduh95 disagrees we'll have you both play rock/paper/scissors at nodeconf to decide.

@jasnell jasnell changed the title buffer: add isByteString buffer: add isLatin1 Sep 26, 2026
@jasnell
jasnell requested a review from addaleax September 26, 2026 23:23

@panva panva left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add the missing typings here? rather than doing it in a follow up #66328, stringLengthUtf8 is missing too.

Implements a fast check to determine if a string is
a valid byte string (only chars <= 0xff).

Signed-off-by: James M Snell <jasnell@gmail.com>
@jasnell

jasnell commented Sep 27, 2026 •

Copy link
Copy Markdown
Member Author

Typings added. If you can give it another stamp I'd appreciate it.

@jasnell jasnell added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 27, 2026
@panva panva added author ready PRs with CI started, the required approvals, and no outstanding review comments. and removed author ready PRs with CI started, the required approvals, and no outstanding review comments. labels Sep 27, 2026
@jasnell jasnell removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 27, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@jasnell jasnell added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. buffer Issues and PRs related to the buffer subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. semver-minor PRs that contain new features and should be released in the next minor version.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants