[SYCL] Fix device image lifetime for jit_compiler - #22785
Conversation
Signed-off-by: Tikhomirova, Kseniya <kseniya.tikhomirova@intel.com>
Signed-off-by: Tikhomirova, Kseniya <kseniya.tikhomirova@intel.com>
Signed-off-by: Tikhomirova, Kseniya <kseniya.tikhomirova@intel.com>
|
@intel/llvm-gatekeepers hi, could you please consider merging this. 1 job is failing but it seems to be an infrastructure issue. Thanks. |
kernel_bundle feature uses createSYCLImages that adds image data to ProgramManager while device binaries are kept in jit_compiler singleton. jit_compiler instance was not attached to GlobalHandler and though can be destroyed earlier than ProgramManager resources are released. This PR aligns lifetime of jit_compiler and ProgramManager and adds regression E2E test for failing scenario. assisted by GH Copilot. --------- Signed-off-by: Tikhomirova, Kseniya <kseniya.tikhomirova@intel.com>
|
@KseniyaTikhomirova We're seeing the below crash in the nightly, is it possible this PR is related? I didn't see any other changes to sycl-jit, 9084ec7 was in the previous nightly which passed. https://github.com/intel/llvm/actions/runs/30603364779/job/91074055301 Seeing this on multiple GPUs, all Linux though. |
this test was added in my PR so it is definitely related. Strange that precommit didn't show this. I will investigate. |
|
thanks! |
|
revert commit is #22855 |
…22855) This reverts commit 5d03593. This fix is still valid and should be reapplied but jit_compiler uses sycl_jit library and we need to fix sycl_jit resources lifetime as well. sycl-jit uses singleton approach to store its resources. it the current state sycl_jit resources are destroyed before test static cache destruction happens and before sycl_jit library is unloaded. There is no perfect solution to make sycl_jit resources to live longer. Possible fixes are: introduce init/close to sycl_jit library to be able to control obj lifetime or use linux ctor/dtor priorities (as it is done for SYCL RT) where sycl-jit resources will be allocated/deallocated.
kernel_bundle feature uses createSYCLImages that adds image data to ProgramManager while device binaries are kept in jit_compiler singleton. jit_compiler instance was not attached to GlobalHandler and though can be destroyed earlier than ProgramManager resources are released.
This PR aligns lifetime of jit_compiler and ProgramManager and adds regression E2E test for failing scenario.
assisted by GH Copilot.