Skip to content

Pin OpenVINO CI to the 2026.4.1 release instead of a deleted nightly - #24212

Open
pytorchbot wants to merge 1 commit into
release/1.6from
cherry-pick-23502-by-pytorch_bot_bot_
Open

pytorchbot wants to merge 1 commit into
release/1.6from
cherry-pick-23502-by-pytorch_bot_bot_

Conversation

@pytorchbot

Copy link
Copy Markdown
Collaborator

What is wrong today

The OpenVINO test-harness jobs have failed at setup on every run since September 9, on main and release/1.5. The setup script downloads a pinned nightly that OpenVINO has since deleted. The server answers the missing file with an HTML page and HTTP 200, so curl --fail does not catch it, and the job dies in tar:

gzip: stdin: not in gzip format

The YOLO26 test script and its README pin a deleted nightly wheel too. The openvino pip extra, the docs and the runtime error message still cap OpenVINO below 2026.0.0, although CI runs the backend on 2026 releases.

What this change does

  • Both Linux OpenVINO jobs install one pinned release, 2026.4.1, the latest. Before, the harness used that nightly and test-openvino-linux used 2026.0.
  • The download is checked against a pinned SHA-256, so a missing or replaced archive stops before tar.
  • --nightly stays for quick tests only. It installs the newest nightly from OpenVINO's latest.txt, checked against its published .sha256. No CI job passes it.
  • Pull requests that change setup-openvino.sh or test_backend.sh also run the harness.
  • The YOLO26 script and README install 2026.4.1 from PyPI.
  • The extra, docs and error message allow openvino>=2025.1.0,<2027.0.0. The extra still skips Python 3.15 (no build) and Linux aarch64 with Python 3.14 (builds need glibc 2.35, the ExecuTorch wheel 2.28).

Not in this change: nncf stays at 3.1.0, and release/1.5 still has the deleted pin.

resnet50, inception_v3 and wide_resnet50_2 fail the harness tolerance, not from this change: on bfloat16 CPUs OpenVINO runs float32 models in bfloat16 unless given the f32 hint. On the same inputs, 2026.0.0, 2026.1.0 and 2026.4.1 give identical results.

What was tested

On Linux x86_64 (AMD EPYC, bfloat16):

  • The release and --nightly paths download, pass the check and extract; the deleted nightly URL and a swapped archive stop at the checksum.
  • Harness models: 29 of 35 pass; the 6 failures are those three models, static and dynamic. Operator sub-tests: 778 of 798 pass, same failures as 2026.1.0.
  • test-openvino-linux gives the same results with 2026.0.0 and 2026.4.1.

Also:

  • A dry pip resolve of the extra picks 2026.4.1 on Linux x86_64, on Windows, and on Linux aarch64 with glibc 2.35 (Python 3.11 and 3.13).
  • With 2026.4.1, YOLO26 exports to the OpenVINO delegate (fp32, on macOS arm64). CI does not run the YOLO26 script.

…23502)

## What is wrong today

The OpenVINO test-harness jobs have failed at setup on every run since
September 9, on main and `release/1.5`. The setup script downloads a
pinned nightly that OpenVINO has since deleted. The server answers the
missing file with an HTML page and HTTP 200, so `curl --fail` does not
catch it, and the job dies in `tar`:

```
gzip: stdin: not in gzip format
```

The YOLO26 test script and its README pin a deleted nightly wheel too.
The `openvino` pip extra, the docs and the runtime error message still
cap OpenVINO below 2026.0.0, although CI runs the backend on 2026
releases.

## What this change does

- Both Linux OpenVINO jobs install one pinned release, 2026.4.1, the
latest. Before, the harness used that nightly and `test-openvino-linux`
used 2026.0.
- The download is checked against a pinned SHA-256, so a missing or
replaced archive stops before `tar`.
- `--nightly` stays for quick tests only. It installs the newest nightly
from OpenVINO's `latest.txt`, checked against its published `.sha256`.
No CI job passes it.
- Pull requests that change `setup-openvino.sh` or `test_backend.sh`
also run the harness.
- The YOLO26 script and README install 2026.4.1 from PyPI.
- The extra, docs and error message allow
`openvino>=2025.1.0,<2027.0.0`. The extra still skips Python 3.15 (no
build) and Linux aarch64 with Python 3.14 (builds need glibc 2.35, the
ExecuTorch wheel 2.28).

Not in this change: `nncf` stays at 3.1.0, and `release/1.5` still has
the deleted pin.

`resnet50`, `inception_v3` and `wide_resnet50_2` fail the harness
tolerance, not from this change: on bfloat16 CPUs OpenVINO runs float32
models in bfloat16 unless given the f32 hint. On the same inputs,
2026.0.0, 2026.1.0 and 2026.4.1 give identical results.

## What was tested

On Linux x86_64 (AMD EPYC, bfloat16):

- The release and `--nightly` paths download, pass the check and
extract; the deleted nightly URL and a swapped archive stop at the
checksum.
- Harness models: 29 of 35 pass; the 6 failures are those three models,
static and dynamic. Operator sub-tests: 778 of 798 pass, same failures
as 2026.1.0.
- `test-openvino-linux` gives the same results with 2026.0.0 and
2026.4.1.

Also:

- A dry pip resolve of the extra picks 2026.4.1 on Linux x86_64, on
Windows, and on Linux aarch64 with glibc 2.35 (Python 3.11 and 3.13).
- With 2026.4.1, YOLO26 exports to the OpenVINO delegate (fp32, on macOS
arm64). CI does not run the YOLO26 script.

(cherry picked from commit 209be4b)
@pytorch-bot

pytorch-bot Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24212

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Unrelated Failure

As of commit a5bac68 with merge base 4db778e (image):

NEW FAILURE - The following job has failed:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 27, 2026

This branch was previously deployed

1 inactive deployment
cadence — a5bac68e Deployed Oct 27, 2026 by pytorchbot via hifi-op-test / hifi4 #34883
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants