Skip to content

Update OpenBLAS and raise NUM_THREADS on Windows - #287

Open
ndabas wants to merge 2 commits into
MacPython:mainfrom
ndabas:ndabas-openblas-overflow-fix-and-threads
Open

ndabas wants to merge 2 commits into
MacPython:mainfrom
ndabas:ndabas-openblas-overflow-fix-and-threads

Conversation

@ndabas

@ndabas ndabas commented Oct 28, 2026

Copy link
Copy Markdown
  • I updated the package version in pyproject.toml and made sure the first 3 numbers match ./openblas_commit.txt. If I did not update ./openblas_commit.txt, I incremented the wheel build number (i.e. 0.3.29.0.0 to 0.3.29.0.1)

Note: update ./openblas_commit.txt with cd OpenBLAS; git describe --tags --abbrev=8 > ../openblas_commit.txt

Summary

Update OpenBLAS to v0.3.34-497-g791b02a1

This picks up OpenMathLib/OpenBLAS#6105 ("Default allocator speedup"), which fixes a memory-corruption crash. If more buffers are in use at once than the main table holds (2 * NUM_THREADS * NUM_PARALLEL), blas_memory_alloc() falls back to an overflow area. Each overflow allocation then mapped a new buffer instead of reusing the slot's buffer. It also called blas_release_register() with no bound check, so after NUM_BUFFERS + NEW_BUFFERS mappings it wrote past the end of new_release_info. This was seen on Windows ARM64 with many concurrent BLAS callers. The same PR also fixes two overflow-area races.

I pinned the #6105 merge commit rather than develop HEAD to keep the bump focused. PRs merged after that are mostly Reference-LAPACK syncs. Let me know if you would prefer to move to HEAD instead.

Raise NUM_THREADS from 24 to 64 on Windows

This matches the Linux/macOS builds (tools/build_steps.sh already uses 64). With 24, the main buffer table has only 48 slots, so applications with more concurrent callers reach the overflow path much sooner. That is the path that was crashing. I know a higher NUM_THREADS was declined before over memory use (#97). Linux and macOS have used 64 for a while, and the main buffers are only mapped on first use.

Related: #283 (win_amd64 crash with concurrent dgemm + dgemv).

Note: #248 removes build_steps_win_arm64.bat and moves NUM_THREADS=24 into build_steps_windows.sh. Whichever PR merges second will need to keep 64.

@mattip

mattip commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator

How is this related to #283, which was solved by #284?

@ndabas

ndabas commented Oct 28, 2026

Copy link
Copy Markdown
Author

Perhaps 'related' isn't the correct term here -- they're both Windows concurrency bugs, I was looking through previous issues to see if there was already a fix for this one and found #283. They're not actually related in terms of the code or fixes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants