Skip to content

[SYCL] Fix device image lifetime for jit_compiler - #22785

Merged
sarnex merged 4 commits into
intel:syclfrom
KseniyaTikhomirova:issue_77164
Jul 30, 2026
Merged

[SYCL] Fix device image lifetime for jit_compiler#22785
sarnex merged 4 commits into
intel:syclfrom
KseniyaTikhomirova:issue_77164

Conversation

@KseniyaTikhomirova

Copy link
Copy Markdown
Contributor

kernel_bundle feature uses createSYCLImages that adds image data to ProgramManager while device binaries are kept in jit_compiler singleton. jit_compiler instance was not attached to GlobalHandler and though can be destroyed earlier than ProgramManager resources are released.
This PR aligns lifetime of jit_compiler and ProgramManager and adds regression E2E test for failing scenario.

assisted by GH Copilot.

Signed-off-by: Tikhomirova, Kseniya <kseniya.tikhomirova@intel.com>
@KseniyaTikhomirova
KseniyaTikhomirova requested review from a team and cperkinsintel as code owners July 28, 2026 12:56
Signed-off-by: Tikhomirova, Kseniya <kseniya.tikhomirova@intel.com>
Signed-off-by: Tikhomirova, Kseniya <kseniya.tikhomirova@intel.com>
@KseniyaTikhomirova

Copy link
Copy Markdown
Contributor Author

@intel/llvm-gatekeepers hi, could you please consider merging this. 1 job is failing but it seems to be an infrastructure issue. Thanks.

@sarnex
sarnex merged commit 5d03593 into intel:sycl Jul 30, 2026
30 of 31 checks passed
crystarm pushed a commit to crystarm/intel-llvm that referenced this pull request Jul 31, 2026
kernel_bundle feature uses createSYCLImages that adds image data to
ProgramManager while device binaries are kept in jit_compiler singleton.
jit_compiler instance was not attached to GlobalHandler and though can
be destroyed earlier than ProgramManager resources are released.
This PR aligns lifetime of jit_compiler and ProgramManager and adds
regression E2E test for failing scenario.

assisted by GH Copilot.

---------

Signed-off-by: Tikhomirova, Kseniya <kseniya.tikhomirova@intel.com>
@sarnex

sarnex commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

@KseniyaTikhomirova We're seeing the below crash in the nightly, is it possible this PR is related? I didn't see any other changes to sycl-jit, 9084ec7 was in the previous nightly which passed.

  FAIL: SYCL :: KernelCompiler/sycl_rtc_bundle_cache_teardown.cpp (1921 of 2565)
    ******************** TEST 'SYCL :: KernelCompiler/sycl_rtc_bundle_cache_teardown.cpp' FAILED ********************
    Exit Code: -11
    
    Command Output (stdout):
    --
    # RUN: at line 11
    /__w/llvm/llvm/toolchain/bin/dpclang++ -Werror -D_GLIBCXX_USE_CXX11_ABI=0 --offload-new-driver  -fsycl -fsycl-targets=spir64  /__w/llvm/llvm/llvm/sycl/test-e2e/KernelCompiler/sycl_rtc_bundle_cache_teardown.cpp -o /__w/llvm/llvm/build-e2e/KernelCompiler/Output/sycl_rtc_bundle_cache_teardown.cpp.tmp.out
    # executed command: /__w/llvm/llvm/toolchain/bin/dpclang++ -Werror -D_GLIBCXX_USE_CXX11_ABI=0 --offload-new-driver -fsycl -fsycl-targets=spir64 /__w/llvm/llvm/llvm/sycl/test-e2e/KernelCompiler/sycl_rtc_bundle_cache_teardown.cpp -o /__w/llvm/llvm/build-e2e/KernelCompiler/Output/sycl_rtc_bundle_cache_teardown.cpp.tmp.out
    # note: command had no output on stdout or stderr
    # RUN: at line 12
    env ONEAPI_DEVICE_SELECTOR=opencl:cpu  /__w/llvm/llvm/build-e2e/KernelCompiler/Output/sycl_rtc_bundle_cache_teardown.cpp.tmp.out
    # executed command: env ONEAPI_DEVICE_SELECTOR=opencl:cpu /__w/llvm/llvm/build-e2e/KernelCompiler/Output/sycl_rtc_bundle_cache_teardown.cpp.tmp.out
    # .---command stderr------------
    # | PLEASE submit a bug report to https://software.intel.com/en-us/support/priority-support and include the crash backtrace and instructions to reproduce the bug.
    # `-----------------------------
    # error: command failed with exit status: -11
    
    --

https://github.com/intel/llvm/actions/runs/30603364779/job/91074055301

Seeing this on multiple GPUs, all Linux though.

@KseniyaTikhomirova

Copy link
Copy Markdown
Contributor Author

@KseniyaTikhomirova We're seeing the below crash in the nightly, is it possible this PR is related? I didn't see any other changes to sycl-jit, 9084ec7 was in the previous nightly which passed.

  FAIL: SYCL :: KernelCompiler/sycl_rtc_bundle_cache_teardown.cpp (1921 of 2565)
    ******************** TEST 'SYCL :: KernelCompiler/sycl_rtc_bundle_cache_teardown.cpp' FAILED ********************
    Exit Code: -11
    
    Command Output (stdout):
    --
    # RUN: at line 11
    /__w/llvm/llvm/toolchain/bin/dpclang++ -Werror -D_GLIBCXX_USE_CXX11_ABI=0 --offload-new-driver  -fsycl -fsycl-targets=spir64  /__w/llvm/llvm/llvm/sycl/test-e2e/KernelCompiler/sycl_rtc_bundle_cache_teardown.cpp -o /__w/llvm/llvm/build-e2e/KernelCompiler/Output/sycl_rtc_bundle_cache_teardown.cpp.tmp.out
    # executed command: /__w/llvm/llvm/toolchain/bin/dpclang++ -Werror -D_GLIBCXX_USE_CXX11_ABI=0 --offload-new-driver -fsycl -fsycl-targets=spir64 /__w/llvm/llvm/llvm/sycl/test-e2e/KernelCompiler/sycl_rtc_bundle_cache_teardown.cpp -o /__w/llvm/llvm/build-e2e/KernelCompiler/Output/sycl_rtc_bundle_cache_teardown.cpp.tmp.out
    # note: command had no output on stdout or stderr
    # RUN: at line 12
    env ONEAPI_DEVICE_SELECTOR=opencl:cpu  /__w/llvm/llvm/build-e2e/KernelCompiler/Output/sycl_rtc_bundle_cache_teardown.cpp.tmp.out
    # executed command: env ONEAPI_DEVICE_SELECTOR=opencl:cpu /__w/llvm/llvm/build-e2e/KernelCompiler/Output/sycl_rtc_bundle_cache_teardown.cpp.tmp.out
    # .---command stderr------------
    # | PLEASE submit a bug report to https://software.intel.com/en-us/support/priority-support and include the crash backtrace and instructions to reproduce the bug.
    # `-----------------------------
    # error: command failed with exit status: -11
    
    --

https://github.com/intel/llvm/actions/runs/30603364779/job/91074055301

Seeing this on multiple GPUs, all Linux though.

this test was added in my PR so it is definitely related. Strange that precommit didn't show this. I will investigate.

@sarnex

sarnex commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

thanks!

@KseniyaTikhomirova

Copy link
Copy Markdown
Contributor Author

revert commit is #22855

sarnex pushed a commit that referenced this pull request Aug 3, 2026
…22855)

This reverts commit 5d03593.

This fix is still valid and should be reapplied but jit_compiler uses
sycl_jit library and we need to fix sycl_jit resources lifetime as well.
sycl-jit uses singleton approach to store its resources.
it the current state sycl_jit resources are destroyed before test static
cache destruction happens and before sycl_jit library is unloaded.
There is no perfect solution to make sycl_jit resources to live longer.
Possible fixes are: introduce init/close to sycl_jit library to be able
to control obj lifetime or use linux ctor/dtor priorities (as it is done
for SYCL RT) where sycl-jit resources will be allocated/deallocated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants