[pytorchbot]

What is wrong today

When a delegate runs on a GPU, PropagateDevicePass copies its inputs to the device (_h2d_copy) and its outputs back (_d2h_copy). skip_h2d_for_method_inputs and skip_d2h_for_method_outputs remove those copies at the method's edges.

1. A transposed parameter is stored contiguous, but a delegate reading it is compiled for its original strides. to_edge does this silently.
2. When a host op writes a delegate output in place, the next delegate on that device reads the old value. Such a relu_ makes memory planning fail.
3. A lowered module returning one tensor, called from another module, gets no _d2h_copy.
4. With the output skip flag, a returned output that something else reads comes back on the host.
5. A constant that feeds a device delegate makes to_executorch fail. With the input skip flag, it arrives as host memory marked as device memory.
6. With the input skip flag, an input with more than one reader is refused.
7. A delegate targeting the CPU gets copies that fail at run time.

What this change does

Numbered as above:

1. to_edge makes such a parameter contiguous before partitioning, and warns for each tensor whose layout it changes.
2. The host op writes the host copy of the output. The copies stay when it writes between two delegates, also through a view.
3. The output gets its _d2h_copy.
4. Returned outputs stay on the device. One that a host op writes in place is returned on the host, with a warning.
5. The constant gets a device copy, with a warning. One neither contiguous nor channels-last is refused.
6. The input stays on the device of its first delegate. Shape-only ops read it there. Other readers share one _d2h_copy of it, with a warning. An input written in place on the host is refused.
7. The delegate gets no copies.

Also:

• A tensor that several delegates on one device read is copied there once, unless written in between.
• New option PropagateDeviceConfig(error_on_per_call_copies=True) turns the warnings in 4 to 6 into errors when the matching skip flag is set.

What was tested

• Device pass tests and new exir/tests/test_passes.py tests pass. All new tests but one fail without their fix.
• backends/cuda/tests/test_device_placement.py matches eager PyTorch with real CUDA delegates on an H100. The unittest-cuda job now runs when these passes change, and fails if that test skips.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24210

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 8 Pending, 7 Unclassified Failures

As of commit 24a83cc with merge base 4db778e ():

NEW FAILURES - The following jobs have failed:

• Build Aarch64 Linux CUDA Wheels / pytorch/executorch / build-wheel-py3_11-cuda-aarch6413_2-aarch64 (gh)
• Build Aarch64 Linux CUDA Wheels / pytorch/executorch / upload / upload-wheel-py3_11-cuda-aarch6413_2-aarch64 (gh)
  Unable to download artifact(s): Artifact not found
• Test ROCm AOTI / unittest-rocm-gfx950-rocm7.2 / linux-job (gh)
  backends/cuda/tests/test_autotune_inputs.py::SdpaKvLenAutotuneTest::test_compiled_pick_is_fastest_at_runtime_kv_lengths

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

• Test CUDA Windows Export and E2E / test-model-cuda-windows-e2e (facebook, dinov2-small-imagenet1k-1-layer, non-quantized) / windows-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  executorch
• Test CUDA Windows Export and E2E / test-model-cuda-windows-e2e (mistralai, Voxtral-Mini-3B-2507, non-quantized) / windows-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  executorch
• Test CUDA Windows Export and E2E / test-model-cuda-windows-e2e (mistralai, Voxtral-Mini-3B-2507, quantized-int4-weight-only) / windows-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  executorch
• Test CUDA Windows Export and E2E / test-model-cuda-windows-e2e (mistralai, Voxtral-Mini-4B-Realtime-2602, quantized-int4-tile-packed) / windows-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  executorch
• Test CUDA Windows Export and E2E / test-model-cuda-windows-e2e (nvidia, parakeet-tdt, non-quantized) / windows-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  executorch
• Test CUDA Windows Export and E2E / test-model-cuda-windows-e2e (nvidia, parakeet-tdt, quantized-int4-weight-only) / windows-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  executorch
• Test CUDA Windows Export and E2E / unittest-cuda-runtime-windows / windows-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  executorch

This comment was automatically generated by Dr. CI and updates every 15 minutes.