to_executorch segfaults when constant tensors are in device memory

@JacobSzwejbka

Serializing a program with CUDA parameters segfaults the emitter

to_executorch() crashes with a segmentation fault when a program's constant
tensors live in device memory. The emitter reads a constant's storage through a
host pointer cast, with no check of where that storage actually is:

# exir/emit/_emitter.py, _tensor_spec_to_evalue
buffer_data = bytes(
    ctypes.cast(
        typing.cast(torch.UntypedStorage, spec.storage).data_ptr(),
        ctypes.POINTER(spec_array_type),
    ).contents
)

data_ptr() on a CUDA storage is a device address, so ctypes.cast reads device
memory from the host.

Reproducer, on a machine with a CUDA build of torch:

import torch
from torch.export import export
from executorch.exir import to_edge_transform_and_lower

model = torch.nn.Sequential(torch.nn.Linear(4, 3), torch.nn.ReLU()).eval().cuda()
example = torch.ones(2, 4).cuda()
program = to_edge_transform_and_lower(export(model, (example,))).to_executorch()

The faulthandler traceback points straight at the cast:

Fatal Python error: Segmentation fault
Current thread ...:
  File ".../executorch/exir/emit/_emitter.py", line 460 in _tensor_spec_to_evalue

The same script with host parameters succeeds, which is the workaround: keep
parameters on the host and let the partitioner see the device only through the
example input.

Suggested fix: copy the storage to the host before serializing when it is not
already there, or fail with a clear message instead of reading device memory
from the host.

Found while running an end-to-end delegate export on Linux aarch64. Not specific
to any one backend, since the crash is in the shared serialization path.

cc @JacobSzwejbka @angelayi @Gasoonjia @digantdesai

@shoumikhin

Hello @shoumikhin happy to pick this up if it's not already in flight; the fix looks small and well-scoped.

Root cause: _tensor_spec_to_evalue dereferences spec.storage.data_ptr() host-side via ctypes.cast(...).contents, but for any non-CPU storage (CUDA, and the same class of problem for MPS/XPU) data_ptr() is a device address, so the host read faults. Not CUDA-specific.

Proposed fix (emit-site, minimal):

storage = typing.cast(torch.UntypedStorage, spec.storage)
if storage.device.type != "cpu":
    storage = storage.cpu()  # transient host copy, bounded by the largest constant
buffer_data = bytes(
    ctypes.cast(storage.data_ptr(), ctypes.POINTER(spec_array_type)).contents
)

Copy-to-host is lossless here: the .pte constant segment is raw host bytes regardless of the exporting device, and the runtime/delegate re-places them at load. So I'd lean toward copying silently rather than raising, raising would force every device-side export workflow to manually .cpu() the module first. Can add a debug-level log on the copy.

Two questions before opening the PR:

Fix at the read site only (safety net), or add a normalization pass in to_executorch() that moves all constant storages to CPU up front and keeps the emitter allocation-free? Happy to do both (pass + assert at emit).
Are there other raw data_ptr() host reads in the serialization path (e.g. the external/named-data constants flow) worth patching in the same PR? I'll grep, but flagging in case you know offhand.

Tests: the repro from the issue gated on torch.cuda.is_available() (parametrized over available accelerators), plus a byte-equality check that the same weights exported from CPU vs device produce an identical program buffer.

@shoumikhin

Hi @shoumikhin thanks for your question but i don't think

model = torch.nn.Sequential(torch.nn.Linear(4, 3), torch.nn.ReLU()).eval().cuda()
example = torch.ones(2, 4).cuda()
program = to_edge_transform_and_lower(export(model, (example,))).to_executorch()

is a legal way to export:

We didn't design for exporting from cuda model and cuda input for our regular path
This code snippet is trying to lower a cuda model onto portable library, which we can't support. We can try it on cuda backend and it works:

(et0) [gasoonjia@devvm38968.vll0 ~]$ python
Python 3.11.15 (main, Jun 11 2026, 15:20:16) [GCC 14.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> from torch.export import export
>>> from executorch.exir import EdgeCompileConfig, to_edge_transform_and_lower
W0803 00:01:15.075000 2749909 site-packages/torch/utils/_pytree.py:630] <enum 'KernelPreference'> is an Enum subclass and is now natively supported by torch.compile as an opaque value type. Calling register_constant() on Enum subclasses is deprecated and will be an error in a future release.
>>> from executorch.backends.cuda.cuda_backend import CudaBackend
>>> from executorch.backends.cuda.cuda_partitioner import CudaPartitioner
>>> model = torch.nn.Sequential(
...     torch.nn.Linear(4, 3),
...     torch.nn.ReLU(),
... ).eval().cuda()
>>> 
>>> example = torch.ones(2, 4).cuda()
>>> 
>>> partitioner = CudaPartitioner([
...     CudaBackend.generate_method_name_compile_spec("forward")
... ])
<stdin>:1: ExperimentalWarning: This API and all of cuda backend related functionality are experimental.
>>> 
>>> program = to_edge_transform_and_lower(
...     export(model, (example,)),
...     partitioner=[partitioner],
...     compile_config=EdgeCompileConfig(_check_ir_validity=False),
... ).to_executorch()
/home/gasoonjia/.conda/envs/et0/lib/python3.11/copyreg.py:105: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
  return cls.__new__(cls, *args)
/home/gasoonjia/.conda/envs/et0/lib/python3.11/copyreg.py:105: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
  return cls.__new__(cls, *args)
/home/gasoonjia/.conda/envs/et0/lib/python3.11/copyreg.py:105: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
  return cls.__new__(cls, *args)
/home/gasoonjia/.conda/envs/et0/lib/python3.11/copyreg.py:105: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
  return cls.__new__(cls, *args)
/home/gasoonjia/.conda/envs/et0/lib/python3.11/site-packages/torch/_inductor/compile_fx.py:320: UserWarning: TensorFloat32 tensor cores for float32 matrix multiplication available but not enabled. Consider setting `torch.set_float32_matmul_precision('high')` for better performance.
  warnings.warn(
/home/gasoonjia/.conda/envs/et0/lib/python3.11/copyreg.py:105: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
  return cls.__new__(cls, *args)
>>> 
>>> print(len(program.buffer))
396192

and also hi @AxelNoun thank you for your enthusiasm! It is so nice having your support! Let me have a better understanding of shoumikhin's concern and we can further collberate on this issue!

Agreed that the snippet in my report is not a supported path; lowering a CUDA module onto the portable library is user error. My ask is only about how it fails: today it is a bare segfault (exit 139) with no Python exception, no message naming the device, and no stack. The cause is a host read of a device pointer in the emitter, where _tensor_spec_to_evalue does ctypes.cast(spec.storage.data_ptr(), ...) on CUDA storage. A small guard that raises InternalError when spec.storage.device.type != "cpu" would turn the crash into an actionable message without changing any supported behavior.

Separately, I could not reproduce the partitioner side-by-side on my hardware, but for an unrelated reason: it stops earlier in Inductor with NoValidChoicesError (a Triton GEMM template gap for this device), not in the emitter.

@shoumikhin

@shoumikhin I opened #24202 with the guard you suggested: _tensor_spec_to_evalue now raises an InternalError naming the device when a non-empty constant's storage is not on CPU. I ran the reproducer from this issue on CUDA (RTX 3060, torch 2.14.0+cu130): main segfaults (exit 139) in _tensor_spec_to_evalue, and the PR raises the InternalError instead. This replaces the host copy I proposed on 2 Aug, since this export path is not supported. @Gasoonjia, since this issue is assigned to you, I will adapt or close the PR if you had another fix in mind.

I used Claude Code to prepare this comment. (Edited: CUDA run added.)

Navigation Menu

Repository navigation

to_executorch segfaults when constant tensors are in device memory #21545

Serializing a program with CUDA parameters segfaults the emitter

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

to_executorch segfaults when constant tensors are in device memory #21545

Description

Serializing a program with CUDA parameters segfaults the emitter

Activity

AxelNoun commented on Aug 2, 2026

Gasoonjia commented on Aug 3, 2026

shoumikhin commented on Aug 3, 2026

AxelNoun commented on Oct 27, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions