Repository navigation
refactor openmp task-based internal use + add policy interface to factorization - #86
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #86 +/- ##
==========================================
+ Coverage 97.19% 98.26% +1.06%
==========================================
Files 80 191 +111
Lines 5132 11457 +6325
==========================================
+ Hits 4988 11258 +6270
- Misses 144 199 +55 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
PierreMarchand20
force-pushed
the
improve_task_based_parallelism
branch
from
July 21, 2026 16:22
c62ada4 to
e292614
Compare
PierreMarchand20
force-pushed
the
improve_task_based_parallelism
branch
from
October 6, 2026 16:29
e292614 to
2469205
Compare
PierreMarchand20
force-pushed
the
improve_task_based_parallelism
branch
5 times, most recently
from
October 8, 2026 08:44
fd9ae43 to
7fc9a34
Compare
- Recompute the L0 cached in omp_task_policy when it does not belong to the factorized HMatrix (HMatrixTaskDependencies::is_L0_of) or when max_number_of_nodes changed. - Build tasks only use values captured when they are created (L0 node, leaves to compute, low-rank generator), so that another build with the same builder or policy can start before they complete; atomic update of the number of false positives. - Run the build and factorizations sequentially when L0 is reduced to the root. - Document omp_task_policy inside and outside a parallel region, and factor the creation of the parallel region into run_in_task_region. - Restore the default L0 size to 64 and remove the debug L0 output of the build test. - Tests: policy reused for another HMatrix, two builds in a row, L0 reduced to the root, Cholesky built with the policy, trivial n1=10 case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- ldlt_factorization with an execution policy (seq, par, omp_task_policy), the existing overload using seq. - task_based_ldlt_factorization, following sequential_ldlt_factorization: the copies W(q) used by the Schur complement are created with their block tree and filled by tasks, and freed by a task once the products using them complete. - task_based_internal_triangular_ldlt_hmatrix_hmatrix_solve, sharing the recursion of the task-based triangular solve with another leaf solve. - task_based_internal_apply_ldlt_diagonal, with one task per node of L0. - HMatrix::set_low_rank_data, counterpart of set_dense_data. - Tests: task-based LDLt triangular solve, LDLt factorization with seq, par and omp_task_policy, build and LDLt in a parallel region. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
As for the task-based build tests, LeakSanitizer reports allocations made by libgomp when creating tasks with dependencies (136-byte objects, most likely dependency hash tables) as leaked, intermittently. It happens with any GCC from 11 to 15, without htool code involved, and in CI it was triggered by the trivial n1=10 case. Address errors are still checked, and leaks are still checked by the sequential factorization tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When A is a dense diagonal block and B is hierarchical, the triangular HMatrix-HMatrix solves densified B. With an unbalanced cluster tree (a leaf cluster next to a non-leaf one, e.g. GeometricSplitting on unevenly distributed points), a consistent block tree has such blocks, and in the task-based factorizations the densification happened in a task while the thread creating the tasks still used the children of B: heap use after free in task-based Cholesky and LDLt. Since the cluster of a dense diagonal block of a consistent block tree is a leaf, the children of B keep it and are now solved with A, which also keeps the compression of B. B is only densified for block trees that are not consistent. Test LU, Cholesky and LDLt with omp_task_policy on an unbalanced cluster tree, for double and complex double. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The builds counted false positives in the HMatrixTreeBuilder, which the task-based build updated from its tasks: in a parallel region, they could still run during a following build with the same builder, which reset the count. The builds now store the number of admissible blocks in the HMatrix information, known before the blocks are computed, and get_hmatrix_information and get_distributed_hmatrix_information deduce the number of false positives from it and from the number of low-rank blocks. HMatrixTreeBuilder::get_false_positive is removed. Also document that the HMatrixTreeBuilder must stay alive until the tasks of a build from a VirtualGenerator complete, since it holds the internal generator wrapping it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PierreMarchand20
force-pushed
the
improve_task_based_parallelism
branch
from
October 9, 2026 08:33
7fc9a34 to
f56d3f0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Improve the OpenMP task-based HMatrix build and factorizations: an execution policy interface for the LU, Cholesky and LDLt factorizations, a task-based LDLt, and several fixes to
omp_task_policy, including a use of freed blocks in task-based factorizations with unbalanced cluster trees.Changes
Execution policies for factorizations
lu_factorization,cholesky_factorizationandldlt_factorizationacceptseq,parandomp_task_policy. The overloads without a policy useseq, so existing callers (including the DDM local solvers) are unchanged.omp_task_policystores the L0 used as task dependencies inHMatrixTaskDependencies. A factorization keeps it if it is a cut of the block tree of the factorized HMatrix (e.g. the L0 set by its build, or one assigned directly), and recomputes it otherwise.omp_task_policy.Task-based LDLt
task_based_ldlt_factorization, followingsequential_ldlt_factorization. The copiesW(q)used by the Schur complement are created with their block tree, filled by tasks, and freed per pivot step.task_based_internal_triangular_ldlt_hmatrix_hmatrix_solve, sharing the recursion of the task-based triangular solve.task_based_internal_apply_ldlt_diagonal.HMatrix::set_low_rank_data, counterpart ofset_dense_data.Python
HTOOL_WITH_PYTHON_INTERFACEmacro, so that threading is available through the Python interface.Fixes
GeometricSplittingon unevenly distributed points), also affecting task-based Cholesky onmain. The triangular HMatrix-HMatrix solves densified a hierarchical right-hand side whose triangular matrix is a dense block, inside a task, while the thread creating the tasks still used its children. They now solve its children with the dense block, which keeps the block structure and the compression. Densification only remains for block trees that are not consistent.omp_task_policyfactorizations no longer reuse the L0 of another HMatrix. In a parallel region, pending tasks are completed before L0 is recomputed.shared_ptrdata member of a class template given tofirstprivatein a task while tasks still use it. Fixed in GCC 12.3.API changes
ldlt_factorization,HMatrix::set_low_rank_data.HMatrixTreeBuilder::get_false_positive. The number of false positives is now computed from the HMatrix information (get_hmatrix_information,get_distributed_hmatrix_information): number of admissible blocks stored by the build minus number of low-rank blocks. This makes it right for task-based builds too.HTOOL_WITH_PYTHON_INTERFACEmacro.Tests
omp_task_policy(double, complex symmetric and Hermitian, both triangles, full and symmetric storage), including the trivialn1 = 10case (root made of dense blocks).CI
ASAN_OPTIONS=detect_leaks=0), as it already was for the task-based build tests. LeakSanitizer intermittently reports allocations made by libgomp inGOMP_taskfor tasks with dependencies, with any GCC from 11 to 15 and without htool code involved. Address errors are still checked, and leaks are still checked by the sequential tests.Follow-ups
ldlt_factorizationwith and without a policy (htool_python).docs/htoolin htool_documentation, whose C++ pages already document LDLt andomp_task_policy.omp_task_policyfor the HMatrix-vector and HMatrix-matrix products, task-based solves, and a policy interface for the DDM local solvers.max_number_of_nodes. First measurements give about 256 nodes as the best value for LDLt and Cholesky on 8 threads.🤖 Generated with Claude Code