FIX intermittent Windows module enumeration failure (#217)#219
Open
ogrisel wants to merge 9 commits into
Open
Conversation
Use CreateToolhelp32Snapshot for atomic module discovery on Windows, with GetModuleFileNameExW for long paths and graceful per-module fallbacks when handles go stale. Add Windows integration tests for paths longer than MAX_PATH. Co-authored-by: Cursor <cursoragent@cursor.com>
ogrisel
commented
Jul 9, 2026
- Run repeated-init stress test on all platforms - Drop redundant OpenBLAS glob pattern and win32 guard in utils - Remove MAX_PATH tests from SAFE_SKIPPED_TESTS whitelist Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Dugal Harris <dugal@leftfield.online> Co-authored-by: Cursor <cursoragent@cursor.com>
Correct numpy.libs glob to use the site-packages sibling directory, add conda-forge Library/bin patterns, and fall back to the loaded OpenBLAS filepath from ThreadpoolController when globs find nothing. Co-authored-by: Cursor <cursoragent@cursor.com>
Use extended-length paths when copying and loading test DLLs so LoadLibrary works beyond MAX_PATH, and pass the module buffer directly to EnumProcessModulesEx to match its ctypes signature. Co-authored-by: Cursor <cursoragent@cursor.com>
Use libopenblas-prefixed destination DLL names so conda builds are recognized, normalize extended-length paths for comparisons, prefer shipped libscipy_openblas sources, and detect truncation when the resolved path fills the GetModuleFileNameExW buffer. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
ogrisel
commented
Jul 9, 2026
dugalh
reviewed
Jul 9, 2026
| assert ThreadpoolController().info() == original_info | ||
|
|
||
|
|
||
| def test_threadpool_controller_repeated_init(): |
There was a problem hiding this comment.
It's probably worthwhile running this as is, but in my testing I have only seen its code fail when it is run after import cv2 in a Windows conda-forge environment.
| kernel_32 = self._get_windll("kernel32") | ||
| self._setup_windows_module_apis(ps_api, kernel_32) | ||
|
|
||
| h_process = kernel_32.OpenProcess( |
There was a problem hiding this comment.
You could use kernel_32.GetCurrentProcess() here. It returns a pseudo-handle that doesn't need to be closed.
Comment on lines
+1076
to
+1082
| filepath = self._resolve_module_filepath( | ||
| ps_api, | ||
| h_process, | ||
| h_module, | ||
| snapshot_path, | ||
| max_path, | ||
| path_buf, |
There was a problem hiding this comment.
I don't think this is necessary here. In my testing, I only see _snapshot_loaded_modules() succeed and return paths when they are all complete and shorter than MAX_PATH. If any path of a loaded DLL is longer than MAX_PATH (actually 255), CreateToolhelp32Snapshot() fails.
| """Return the full path for a module, or None if it should be skipped.""" | ||
| from ctypes.wintypes import MAX_PATH | ||
|
|
||
| if ps_api.GetModuleFileNameExW(h_process, h_module, path_buf, max_path): |
There was a problem hiding this comment.
I suggest using kernel_32.GetModuleFileNameW(). It is ~10x faster.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
OSError("GetModuleFileNameEx failed")on Windows when DLLs are loaded or unloaded during library discovery (reported with conda-forge OpenCV).EnumProcessModulesEx+GetModuleFileNameExWscan with a snapshot-first approach usingCreateToolhelp32Snapshot, keeping long-path support viaGetModuleFileNameExWand graceful per-module fallbacks.MAX_PATH, inspired by the local repro documented in ENH Support path length > MAX_PATH=260 on Windows #189 (comment).Closes #217
Test plan
py311_conda_forge_openblas): new teststest_windows_library_path_longer_than_max_pathandtest_windows_library_path_exceeds_internal_limitpassAI assistance disclosure
This pull request was prepared with assistance from an AI coding agent (Cursor / Claude).
Made with Cursor