[JakeStevens]

Summary:

Add examples/audio/export_hf_whisper.py to export WhisperForConditionalGeneration directly through Transformers into one .ptn with three methods: encoder maps mel features to hidden states, cross_kv projects per-layer cross-attention keys then values, and decoder consumes token IDs, explicit cache positions, and those projections to produce logits. Self-attention uses zero-initialized mutable static-cache buffers. Positions and the causal mask derive from cache_position, so the runner needs no attention-mask input.

--dump_dir writes runner inputs and an eager greedy transcript with a fresh decoder, preserving the exported cache buffers. An unspecified forced language token is resolved from the example features through Transformers language detection, retaining subsequent forced-token positions. Sparse forced prefixes are rejected. Generation is capped at max_target_positions and skips the unused decoder call after the last requested token.

The export format is independent of the runtime engine. The exporter and the existing LLM example share lowering reports through examples/hf_llm/hf_utils.py. Tests validate native delegation, zero-initialized decoder caches, three-method packaging, and eager numerics. Runtime execution support and runner integration follow in later diffs.

Random-weight loading uses checkpoint generation settings when present, warns when the file is absent, and propagates malformed-config errors. Vector generation supports an unset EOS token by using an empty stop set.

Authored with assistance from Codex.

Reviewed By: digantdesai

Differential Revision: D125295057

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23901

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures, 2 Unrelated Failures

As of commit eccf758 with merge base e683360 ():

NEW FAILURES - The following jobs have failed:

• pull / test-llama-runner-linux-android / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• pull / unittest-buck / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3
• trunk / test-arm-backend-vkml (test_pytest_ops_vkml) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 56
• trunk / test-models-linux-aarch64 (linear, xnnpack-quantization-delegation, mt-l-arm64g4-16-62) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• trunk / test-models-macos-cpu (mobilebert, xnnpack-quantization-delegation) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / test-qnn-buck-build-linux / linux-job (gh) (trunk failure)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 3
• pull / unittest-buck / linux / linux-job (gh) (trunk failure)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@JakeStevens has exported this pull request. If you are a Meta employee, you can view the originating Diff in D125295057.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[digantdesai]

Review automatically exported from Phabricator review in Meta.