Skip to content

crypto: avoid copying buffers to UTF-8 strings in crypto.hash() #59067

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

crypto: avoid copying buffers to UTF-8 strings in crypto.hash() #59067

wants to merge 1 commit into from
+11 −14

Conversation

Renegade334
Copy link
Contributor

Fixes: #59057

Testing in the wild confirms this to fix the associated downstream issue. Not sure a unit test is feasible.

cc: @nodejs/releasers – given the impact on the ecosystem, this would ideally be a candidate for a patch release.

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/crypto

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. crypto Issues and PRs related to the crypto subsystem. needs-ci PRs that need a full CI run. labels Jul 14, 2025
@panva panva added request-ci Add this label to start a Jenkins CI on a PR. and removed request-ci Add this label to start a Jenkins CI on a PR. labels Jul 14, 2025
@panva panva self-requested a review July 14, 2025 09:02
@panva
Copy link
Member

panva commented Jul 14, 2025 •
edited
Loading

This likely will have to wait for after the announced security releases.

@Renegade334 Renegade334 force-pushed the crypto-oneshotdigest-no-stringify-buffers branch from c631a71 to 2a23cf2 Compare July 14, 2025 09:17
@codecov Codecov
Copy link

codecov bot commented Jul 14, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.06%. Comparing base (049664b) to head (2a23cf2).
Report is 20 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main   #59067   +/-   ##
=======================================
  Coverage   90.06%   90.06%           
=======================================
  Files         645      645           
  Lines      189130   189130           
  Branches    37094    37085    -9     
=======================================
+ Hits       170339   170349   +10     
+ Misses      11511    11507    -4     
+ Partials     7280     7274    -6
Files with missing lines Coverage Δ
src/crypto/crypto_hash.cc 70.65% <100.00%> (+0.29%) ⬆️

... and 29 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@panva panva added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. request-ci Add this label to start a Jenkins CI on a PR. labels Jul 14, 2025
joyeecheung
joyeecheung approved these changes Jul 14, 2025
View reviewed changes
Copy link
Member

@joyeecheung joyeecheung left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. On a side note according to #59057 this primarily happens on Linux so is likely attributed to fragmentation in glibc's malloc implementation (#21973), and now it seems to affect sync calls with a lot of buffers as well, which is somewhat concerning as the performance issue can lead to OOM..

@Renegade334
Copy link
Contributor Author

Renegade334 commented Jul 15, 2025 •
edited
Loading

On a side note according to #59057 this primarily happens on Linux so is likely attributed to fragmentation in glibc's malloc implementation

FWIW, I see exactly the same behaviour on Windows. It doesn't look like this is a fragmentation issue, or even a heap issue per se – it looks like the process of running the equivalent of %TypedArray%.prototype.toString() on the input buffer is producing some intermediate that's hitting kMaxFixedArrayCapacity, if the input buffer is large enough.

Edit: https://issues.chromium.org/issues/432212877, for academic interest.

@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 15, 2025
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/67945/

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/67962/

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/67968/

@panva panva added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 18, 2025
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 18, 2025
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/68003/

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/68006/

@Renegade334
Copy link
Contributor Author

🎲🎲

panva pushed a commit that referenced this pull request Jul 18, 2025
@Renegade334 @panva
crypto: avoid copying buffers to UTF-8 strings in crypto.hash()
fc4a8af 
PR-URL: #59067
Fixes: #59057
Reviewed-By: Filip Skokan <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
@panva
Copy link
Member

panva commented Jul 18, 2025

Landed in fc4a8af

@panva panva closed this Jul 18, 2025
@Aditi-1400 Aditi-1400 mentioned this pull request Jul 18, 2025
Closed
@Renegade334 Renegade334 deleted the crypto-oneshotdigest-no-stringify-buffers branch July 18, 2025 11:06
aduh95 pushed a commit that referenced this pull request Jul 21, 2025
@Renegade334 @aduh95
crypto: avoid copying buffers to UTF-8 strings in crypto.hash()
135fca5 
PR-URL: #59067
Fixes: #59057
Reviewed-By: Filip Skokan <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
@gengjiawen
Copy link
Member

ping @nodejs/releasers

@aduh95
Copy link
Contributor

aduh95 commented Jul 24, 2025

Releases are currently blocked on nodejs/build#4105

@github-actions github-actions bot mentioned this pull request Jul 28, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@panva panva panva approved these changes

@jasnell jasnell jasnell approved these changes

@addaleax addaleax addaleax approved these changes

@joyeecheung joyeecheung joyeecheung approved these changes

Assignees
No one assigned
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. c++ Issues and PRs that require attention from people who are familiar with C++. crypto Issues and PRs related to the crypto subsystem. needs-ci PRs that need a full CI run.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Node 24.4.0 OOMs with pnpm
8 participants
@Renegade334 @nodejs-github-bot @panva @gengjiawen @aduh95 @jasnell @addaleax @joyeecheung