Quacomm AI Engine Direct - Enable FCB feature to Qualcomm HTP

[quic-boyuc]

QNN Flexible Context Binary (FCB) multi-SoC HTP support

Adds offline FCB export for shipping one ExecuTorch artifact containing QNN HTP
contexts for multiple known Snapdragon SoCs. Requires QNN SDK >= 2.48.

User-facing API

• generate_qnn_executorch_compiler_spec() now accepts paired lists:
  soc_model=[...] and backend_options=[...].
• Supplying two or more distinct HTP targets enables FCB; scalar inputs retain
  the existing single-SoC behavior.
• Adds fcb_reference_weight_sharing=True by default. It shares referenced
  weights while host AOT appends target contexts to the FCB DLC.
• QnnQuantizer and make_quantizer() accept FCB target lists and validate
  against the least-capable requested HTP architecture.
• FCB rejects online preparation, non-HTP targets, mismatched target/options
  lists, duplicate SoCs, and DLBC when reference-weight sharing is enabled.

Architecture

• Extends the QNN compiler-spec schema with paired FCB target records and
  reference-weight-sharing configuration.
• Creates and caches one QNN manager per (backend type, SoC) during lowering.
• Validates operator support against every requested target.
• Reuses the same AOT op-wrapper graph sequentially for each target; graph
  registration state is reset after every compile, including failures.
• Adds Python adaptor operations to create an FCB DLC, append each compiled
  context, retrieve its bytes, and free the DLC.
• Uses QNN system DLC APIs at runtime to locate the compatible HTP context and
  deserialize its graph metadata.
• Adds compiler-spec and manager-lifecycle coverage plus FCB ResNet50 and
  multi-SoC weight-sharing examples.

Usage

Build the Android runner:

 # Build with QNN SDK >= 2.48.
./backends/qualcomm/scripts/build.sh


# Measure FCB and HTP weight-sharing combinations across multiple targets:

python -m examples.qualcomm.util_scripts.fcb_multi_soc_weight_sharing_demo \
  --soc_models SM8650 SM8750 SM8850 SM8550 \
  --devices adb-host-a:<serial-a> adb-host-b:<serial-b> \
  --build_folder build-android

#  Export and optionally validate a floating-point FCB artifact:

python -m examples.qualcomm.util_scripts.fcb_resnet50 \
  --soc_models SM8650 SM8750 \
  --devices <serial-on-SM8650> <serial-on-SM8750> \
  --build_folder build-android

# For devices attached to separate ADB servers, use host:serial values:

python -m examples.qualcomm.util_scripts.fcb_resnet50 \
  --soc_models SM8650 SM8750 \
  --devices adb-host-a:<serial-a> adb-host-b:<serial-b> \
  --build_folder build-android

# run FCB with quantized reset50   

python -m examples.qualcomm.util_scripts.fcb_resnet50_quantized \
     --soc_models SM8550 SM8850 SM8750 SM8650 \
     --dataset ~/executorch/imagenet-mini-val/val \
     --output_dir /tmp/qnn_fcb_resnet50_quantized \
     --devices adb-host-a:<serial-a> adb-host-b:<serial-b> \
     --build_folder build-android

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22521

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit b873d9b with merge base c9eee94 ():

NEW FAILURE - The following job has failed:

• pull / test-qnn-passes-linux / linux-job (gh)
  RuntimeError: Command docker exec -t fceff7e6749c67a3cb3c88fc766eee6ecb697b79128526a75e2ea63e1f5b6fc7 /exec failed with exit code 139

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[shewu-quic]

Hi @psiddh,

This draft PR supports FCB in QNN ExecuTorch. It is currently under internal review, and we will send it to the community ASAP.
If you have any questions, please let us know. Thanks

[psiddh]

Hi @psiddh,

This draft PR supports FCB in QNN ExecuTorch. It is currently under internal review, and we will send it to the community ASAP. If you have any questions, please let us know. Thanks

@quic-boyuc Thanks for all the hard-work. This is great!
Just to confirm, is this phase 1 , single .pte solution that we discussed in our meetings ? I will share this internally , we will do some testing , provide feedback....

Also in terms qnn libs that should be present , do we have any limitations in terms of native QNN libs ? (what libs needs to be present on device, since the .pte now supports multiple architectures)

[shewu-quic]

Just to confirm, is this phase 1 , single .pte solution that we discussed in our meetings ? I will share this internally , we will do some testing , provide feedback....

Thanks! Yes, this is the phase 1 solution, and PTE contains multiple context binaries for specified multi-SoC.

Also in terms qnn libs that should be present , do we have any limitations in terms of native QNN libs ? (what libs needs to be present on device, since the .pte now supports multiple architectures)

No additional lib needs to be introduced. Just note that this applies with qnn version 2.48+. We will create another PR to uplevel qnn version in community.
You can follow this guideline to use FCB feature.
https://github.com/CodeLinaro/executorch/blob/c6ebe28bb4b1b0a075fc64678f4747170c13ed8d/docs/source/backends-qualcomm.md#flexible-context-binary-for-multiple-htp-socs

[psiddh]

Few conflicts on the PR.. pls resolve. Thanks

[psiddh]

Few more questions. the single.pte part is clear... but how abt other artifacts. Readutils.py:321-326, the host/DSP libraries are still staged per HTP arch:

unsigned/libQnnHtpV{htp_arch}Skel.so # DSP
libQnnHtpV{htp_arch}Stub.so

So for the target set in demo, SM8750 (v79), SM8850 (v81) , afaict. is 1 .pte + 4 Stub/Skel pairs, plus libQnnHtp.so and libQnnSystem.so. Could you confirm or correct that?

Specifically:

1. Does FCB change the Stub/Skel requintext binary?
2. Is the fan-out per HTP arch (so SoCs sharing an arch share a pair) or per SoC?
3. Does FCB add a new runtime dependen DLC APIs, on targets that didn't needit before?
4. Does QAIRT 2.48 offer anything for the skel, or is bundling every arch theonly supported route?
5. Can you confirm FCB doesn't change the Stub/Skel requirement?

Thinking aloud here with an example: with a concrete 3-arch example (v73 / v75 / v79).

Today we build one bundle per arch :

bundle A: model.pte (v73) + libQnnHtpV73Stub.so + libQnnHtpV73Skel.so + libQnnHtp.so + libQnnSystem.so
bundle B: model.pte (v75) + ...V75Stub/Skel + same 2 shared libs
bundle C: model.pte (v79) + ...V79Stub/Skel + same 2 shared libs

With FCB the model side collapses:

1x model.pte (contexts for v73 + v75 + v79)

• Stub/Skel (still per arch)
• libQnnHtp.so, libQnnSystem.so (shared)

did I get this right ??

[shewu-quic]

Few more questions. the single.pte part is clear... but how abt other artifacts. Readutils.py:321-326, the host/DSP libraries are still staged per HTP arch:

unsigned/libQnnHtpV{htp_arch}Skel.so # DSP
libQnnHtpV{htp_arch}Stub.so

So for the target set in demo, SM8750 (v79), SM8850 (v81) , afaict. is 1 .pte + 4 Stub/Skel pairs, plus libQnnHtp.so and libQnnSystem.so. Could you confirm or correct that?

For AOT compilation, the Skel and Stub libraries are not required.

At runtime, the required artifacts depend on the target SoC. You only need to provide the corresponding Stub/Skel pair for the HTP architecture of the device you're running on, together with libQnnHtp.so and libQnnSystem.so, which is the same flow as the existing single-SoC deployment.

So, for example, if the application is running on an SM8750 (v79), only the v79 Stub/Skel pair is needed. If it is running on an SM8850 (v81), only the v81 Stub/Skel pair is needed. They can share the same pte with FCB.

1. Does FCB change the Stub/Skel requintext binary?

Same as the current flow. FCB does not change the Stub/Skel binaries. The runtime still uses the Stub/Skel pair corresponding to the target device's HTP architecture.

2. Is the fan-out per HTP arch (so SoCs sharing an arch share a pair) or per SoC?

It is based on the SoC(s) specified during AOT compilation, not solely on the HTP architecture.

3. Does FCB add a new runtime dependen DLC APIs, on targets that didn't needit before?

No. FCB does not introduce any new runtime dependency on DLC APIs.

The FCB APIs are provided through libQnnSystem.so, which is already a dependency in the existing flow. Since libQnnSystem.so is already required and used today, no additional libraries are needed when enabling FCB.

4. Does QAIRT 2.48 offer anything for the skel, or is bundling every arch theonly supported route?
5. Can you confirm FCB doesn't change the Stub/Skel requirement?

Yes. For Phase 1, FCB does not change the Stub/Skel requirements.

With FCB the model side collapses:

1x model.pte (contexts for v73 + v75 + v79)

Stub/Skel (still per arch)
libQnnHtp.so, libQnnSystem.so (shared)
did I get this right ??

Yes, that's correct.

With FCB, the model side collapses into a single .pte that can contain contexts for multiple target SoCs/HTP architectures. The Stub/Skel requirement remains unchanged and is still determined by the device on which the application is running. At runtime, you use the Stub/Skel pair corresponding to the target device's HTP architecture, together with libQnnHtp.so and libQnnSystem.so.