Add asynchronous OpenMP batch matching - #6
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds asynchronous batch matching APIs and updates the Linux x64 prebuild to use GNU OpenMP for intra-call parallelism, while keeping source builds and non-Linux builds sequential. It also updates native ownership/lifetime handling to keep compiled regex/set state valid across Node workers and async work, and expands validation/benchmarks to cover the new behavior.
Changes:
- Introduces
testManySync()/testManyAsync()onRE2andRE2Set, keepingtestMany()as a synchronous compatibility alias. - Implements async batch input handling with safe-by-default byte snapshotting and an
{ unsafe: true }borrowed mode. - Switches Linux prebuild parallelism from a custom pool to dynamically-linked GNU OpenMP, plus new planner/observed parallelism reporting and benchmark coverage.
Reviewed changes
Copilot reviewed 34 out of 34 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| type-tests/index.ts | Adds type-level coverage for the new sync/async batch APIs and async-only unsafe option. |
| test/release.test.js | Asserts release artifacts/inputs explicitly disable OpenMP except for the Linux prebuild path. |
| test/native-boundary.test.js | Extends native boundary tests for observed parallelism and async context mismatch handling; updates tarball contents list. |
| test.js | Adds JS-level tests for testManySync/testManyAsync, safe snapshot semantics, unsafe borrowed semantics, worker termination behavior, and OpenMP linkage checks. |
| src/re2.ts | Adds testManySync() and testManyAsync(); keeps testMany() as a compatibility alias. |
| src/re2-set.ts | Adds testManySync() and testManyAsync(); keeps testMany() as a compatibility alias. |
| src/index.ts | Exports TestManyAsyncOptions type. |
| src/binding.ts | Extends native binding interface with async batch methods and observed parallelism probe. |
| src/binary.ts | Adds TestManyAsyncOptions and shared option normalization for sync vs async (unsafe validation). |
| scripts/build-darwin-prebuild.sh | Forces NODE_RE2_OPENMP=0 for Darwin prebuild builds. |
| release.sh | Exports NODE_RE2_OPENMP=0 by default to prevent accidental OpenMP enabling outside Linux prebuild. |
| README.md | Documents new sync/async APIs, safe snapshot vs unsafe borrowing semantics, and OpenMP/Linux-prebuild behavior. |
| native/thread-pool.h | Removes the custom native thread pool interface (superseded by OpenMP on Linux prebuild). |
| native/thread-pool.cc | Removes the custom native thread pool implementation. |
| native/text-batch.h | Introduces a packed snapshot representation for safe async batch input bytes. |
| native/set-binding.cc | Adds async batch matching for RE2Set and factors shared matching helpers for sync/async paths. |
| native/regex-binding.cc | Adds async batch matching for RE2 and switches regex ownership to shared_ptr via a tagged context wrapper. |
| native/parallel-for.h | Replaces thread-pool-based parallel loop with an OpenMP-backed implementation (guarded by NODE_RE2_OPENMP). |
| native/openmp-runtime.h | Adds OpenMP submission mutex and observed team-size probing helper. |
| native/napi-utils.h | Adds GetTextBatch, GetBoolean, and RejectDeferred declarations used by async paths. |
| native/napi-utils.cc | Implements packed snapshot batching, shared array input collection, boolean option parsing, and deferred rejection helper. |
| native/batch-plan.h | Updates batch planning to use OpenMP runtime thread limits (when enabled). |
| native/batch-binding.cc | Exposes batch_observed_parallelism alongside planned parallelism. |
| native/async-batch.h | Defines async batch input container supporting safe snapshots vs unsafe borrowing with N-API refs. |
| native/async-batch.cc | Implements safe snapshotting and unsafe borrowed input retention/release. |
| native/addon-lifecycle.h | Removes addon environment lifecycle hook API (previously used for thread-pool lifecycle). |
| native/addon-lifecycle.cc | Removes addon environment lifecycle hook implementation. |
| fixtures/unsafe-batch.js | Adds fixture validating safe snapshot vs unsafe borrowed mutation outcomes under worker-pool load. |
| fixtures/terminate-match-worker.js | Adds worker fixture to queue async match then terminate, exercising teardown resilience. |
| fixtures/batch-worker.js | Updates worker fixture to exercise both sync and async batch APIs (including alternating unsafe mode). |
| Dockerfile | Enables OpenMP for Linux prebuild build step and asserts the prebuild dynamically links libgomp.so.1. |
| binding.gyp | Adds NODE_RE2_OPENMP build toggle and OpenMP compile/link flags for the Linux prebuild configuration. |
| binding.cc | Removes addon lifecycle registration; registers regex/set/batch bindings (including new exports). |
| benchmark.js | Expands benchmarking to compare scalar/sync modes, reports requested vs observed OpenMP threads, and measures safe vs unsafe async admission/settlement. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
18
to
37
| @@ -32,9 +36,8 @@ void ParallelFor(size_t size, size_t total_bytes, size_t requested_batch_size, F | |||
| error = std::current_exception(); | |||
| } | |||
Comment on lines
+22
to
+37
| inline size_t ObserveBatchParallelism(const BatchPlan& plan) { | ||
| #ifdef NODE_RE2_OPENMP | ||
| if (plan.thread_count > 1) { | ||
| std::lock_guard submission_lock(OpenMpSubmissionMutex()); | ||
| size_t observed = 1; | ||
|
|
||
| #pragma omp parallel num_threads(plan.thread_count) | ||
| { | ||
| #pragma omp single | ||
| observed = static_cast<size_t>(omp_get_num_threads()); | ||
| } | ||
| return observed; | ||
| } | ||
| #endif | ||
| return plan.thread_count; | ||
| } |
Comment on lines
+315
to
320
| bool GetTextBatch(napi_env env, napi_value value, TextBatch* texts) { | ||
| std::vector<napi_value> inputs; | ||
| inputs.reserve(input_count); | ||
| for (uint32_t index = 0; index < input_count; ++index) { | ||
| napi_value input; | ||
| if (!Check(env, napi_get_element(env, value, index, &input), "Failed to read input")) { | ||
| return false; | ||
| } | ||
| inputs.push_back(input); | ||
| if (!GetInputValues(env, value, &inputs)) { | ||
| return false; | ||
| } | ||
|
|
Comment on lines
+27
to
+29
| # The published Linux binary intentionally relies on Debian's GNU OpenMP | ||
| # runtime instead of embedding another thread-pool implementation. | ||
| RUN ldd prebuilds/linux-x64/@nxtedition+re2.glibc.node | grep -F libgomp.so.1 |
Comment on lines
+669
to
+676
| () => { | ||
| const bindingPath = nodeGypBuild.path(import.meta.dirname) | ||
| const result = spawnSync('ldd', [bindingPath], { encoding: 'utf8' }) | ||
|
|
||
| assert.equal(result.signal, null) | ||
| assert.equal(result.status, 0, result.stderr) | ||
| assert.match(result.stdout, /libgomp[.]so[.]1/) | ||
| } |
Comment on lines
+22
to
+26
| Patterns may be strings, Buffers, TypedArrays, or DataViews. Input must be a Buffer, TypedArray, or DataView. Both APIs operate on bytes; optional `byteOffset` and `byteLength` values select the input range. Negative values clamp to zero, values past the view clamp to its bounds, and fractional values are truncated. | ||
|
|
||
| The published Linux x64 prebuild uses a process-wide pool when a batch contains enough input bytes to amortize dispatch. It selects at most half of the container-affinity CPUs, including the caller, so matching leaves capacity for other work. Helpers block when idle and the pool shuts down after the last Node.js environment; source builds and other platforms match the batch sequentially regardless of `batchSize`. | ||
| `testManySync()` blocks until matching completes. Without intra-call parallelism it runs inline; with parallelism the caller joins the OpenMP team and blocks until every chunk completes. `testManyAsync()` returns a Promise and performs matching from the Node.js worker pool, optionally entering OpenMP from that worker. It snapshots all input bytes before returning by default, so later mutation, resizing, transfer, or detachment cannot affect the result. `{ unsafe: true }` skips that copy; every backing store must then remain attached, the same size, and unmodified until the Promise settles. SharedArrayBuffer-backed views follow the same rule. `testMany()` remains a synchronous compatibility alias for `testManySync()`. | ||
|
|
||
| All batch methods accept up to 1,048,576 inputs and preserve their outer result order. Their optional `batchSize` is the number of inputs per native scheduling chunk. `Infinity` or a value greater than or equal to the input count disables intra-call OpenMP: synchronous matching stays on the caller thread, while asynchronous matching stays on one Node.js worker. Omitting `batchSize` keeps automatic scheduling at about 16 chunks per requested OpenMP thread. |
| "-pthread" | ||
| ], | ||
| "conditions": [ | ||
| ["node_re2_openmp==1", { |
| assert.match(dockerfile, /NODE_RE2_OPENMP=1 NODE_RE2_MARCH=znver3/) | ||
| assert.match(dockerfile, /ldd .* \| grep -E 'libgomp\[\.\]so\[\.\]1 => \/'/) | ||
| assert.match(binding, /node_re2_openmp%.*NODE_RE2_OPENMP === '1'/) | ||
| assert.match(binding, /node_re2_openmp==1.*-fopenmp/s) |
Comment on lines
+34
to
+41
| napi_status ReleaseAsyncBatch(napi_env env, AsyncBatch* batch) { | ||
| if (batch->borrowed_inputs == nullptr) { | ||
| return napi_ok; | ||
| } | ||
| const napi_ref reference = batch->borrowed_inputs; | ||
| batch->borrowed_inputs = nullptr; | ||
| return napi_delete_reference(env, reference); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
testManySync()and non-blockingtestManyAsync()toRE2andRE2Set, while retainingtestMany()as the synchronous compatibility alias{ unsafe: true }retains and borrows normalized binary views until settlementshared_ptracross Node workers and async workRuntime behavior
testManySync()executes inline when parallelism is disabled. With Linux-prebuild parallelism, the caller participates in the OpenMP team and blocks at the region barrier.testManyAsync()uses one N-API/libuv worker and may enter the same OpenMP region there. The safe default makes one contiguous byte snapshot before queueing. Unsafe mode avoids that byte copy, but backing stores must remain attached, unchanged, and the same size until the Promise settles.The Linux planner requests at most half of
uv_available_parallelism(), including the initiating thread, and about 16 scheduling chunks per requested thread.InfinityorbatchSize >= inputCountdisables intra-call OpenMP. A per-addon submission lock avoids overlapping OpenMP teams across concurrent Node environments. libgomp owns worker lifecycle.Linux prebuilds dynamically link
libgomp.so.1. Source builds, macOS, and other platforms do not compile or link OpenMP. The deployednxtedition/node:26.5.0image was verified to load and execute the prebuild without installing additional packages.Benchmark
Measured in a container on an AMD EPYC 9355P with a 16-CPU cpuset. Parallel rows requested and observed 8 OpenMP threads. Medians use 10 runs after 3 warmups.
4,096 inputs x 4,096 bytes
Selected async settlement medians:
Cold
compileAsync()for 10,000 patterns settled in 17.39 ms; a native cache hit settled in 1.33 ms. Eight concurrent identical calls caused one compilation and seven native in-flight deduplications, settling in 16.41 ms.Production-derived Deepstream workload
This uses the 77 exact listener patterns from
deepstream/benchmark/regex.jsover 65,536 balanced synthetic Deepstream-style inputs averaging 26 bytes. Medians use 12 runs after 3 warmups.This is why the dependent listener-backfill change uses async matching with
batchSize: Infinity: it leaves the event loop without paying OpenMP overhead for one cheap regex. The multi-pattern RE2Set path remains a candidate for automatic parallelism.Validation
libgompdependencyOMP_THREAD_LIMIT=1: planner requested 1 and observed 1nxtedition/node:26.5.0: prebuild loaded and completed async matching with requested/observed 2 threads on a 4-CPU cpusetRE2andRE2Setcalls