DAOS-19288 control: Add allow-numa-imbalance flag to config generate#18621
DAOS-19288 control: Add allow-numa-imbalance flag to config generate#18621tanabarr wants to merge 1 commit into
Conversation
|
Test stage Unit Test completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18621/1/testReport/ |
|
Ticket title is 'Add config generate flag to ignore NVMe NUMA affinity' |
b5ad1ca to
9f076cd
Compare
|
Test stage Unit Test completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18621/2/testReport/ |
Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>
9f076cd to
50d69c8
Compare
|
Test stage Build on EL 9 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18621/3/execution/node/547/log |
|
Test stage Build on EL 8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18621/3/execution/node/595/log |
|
Test stage Build on Leap 15 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18621/3/execution/node/641/log |
Add --allow-numa-imbalance/-n flag to dmg and daos_server config
generate commands. This flag distributes NVMe devices equally across
engines regardless of NUMA affinity.
Without this flag, config generation respects NUMA affinity and balances
device assignment by giving entrance the lowest common denominator
SSD count across NUMA nodes within the cluster.
With this flag enabled:
- All available SSDs are collected from all NUMA nodes
- SSDs are distributed equally across all engines
- Distribution ignores NUMA node boundaries
- If total SSDs not evenly divisible, uses maximum divisible number
- Remainder SSDs are not included in generated config
- Notice logged when SSDs unused due to remainder
- Generated config incl. allow_numa_imbalance: true to allow boot
This maximizes device utilization in heterogeneous environments but may
result in suboptimal performance due to cross-NUMA memory access.
Features: control
Signed-off-by: Tom Nabarro tom.nabarro@intel.com
Steps for the author:
After all prior steps are complete: