Add Snapdragon NPU acceleration with SoC-specific Gemma 4 E2B models

[animeshahilya]

Why

Backtalk on-device AI runs Gemma 4 E2B/E4B on CPU/GPU only. LiteRT-LM 0.18.0 (already used) ships Backend.NPU plus ModelInfo supportedBackends/npuBrand/socName, and litert-community/gemma-4-E2B-it-litert-lm publishes vision-capable Qualcomm Hexagon NPU builds:

• gemma-4-E2B-it_qualcomm_sm8750.litertlm (Snapdragon 8 Elite, 3016294400 bytes)
• gemma-4-E2B-it_qualcomm_qcs8275.litertlm (QCS8275, 3294593024 bytes)

NPU gives lower latency plus less battery and heat for Describe image and Describe screen, which is what a screen reader needs most. The AI Edge Gallery uses the same Backend.NPU(nativeLibraryDir) pattern but still lists only gpu,cpu for E2B, so this puts Backtalk ahead.

Note: Gemma3-1B-IT_q4_ekv1280_sm8xxx NPU files are text-only and unsuitable for vision, so they are not added. E4B has no Qualcomm build published.

What this PR does

• Adds litert-npu-runtime-qualcomm 2.3.0 (phone-only, about 0.5 MB dispatch runtime).
• New NpuDetector: normalizes Build.SOC_MODEL (API 31+) with fallback to ro.soc.model, ro.board.platform, ro.hardware to sm8750, qcs8275, etc.
• LocalModel: new socModel and requiresNpu fields, plus 2 NPU entries with Hugging Face X-Linked-Size and X-Linked-ETag (ETag matches SHA256, validated against the existing generic E2B entry).
• LiteRtLmLocalLlm: useNpu and nativeLibraryDir options select Backend.NPU for both LLM and vision backends, with NPU load failure surfaced for GPU/CPU fallback.
• Plumbs NPU flag through LocalModelService (KEY_USE_NPU), RemoteLocalLlm, LocalModelManager (LlmKey). Portable models ignore the switch.
• OnDeviceAiSettings: PREF_NPU (default on), SoC-filtered modelsFor, NPU-first defaultModel, new Support.UNSUPPORTED_SOC.
• OnDeviceAiFragment plus strings: Use the NPU (Snapdragon) toggle, only visible on NPU phones, GPU disabled for NPU files.

Verification

• Sizes and hashes from HF resolve headers (curl -sI on .../resolve/main/...).
• Backend NPU and ModelInfo confirmed present in litertlm-android-0.18.0.aar.
• Existing OnDeviceAiSettingsTest unaffected (null SoC filters NPU out, defaults fall back to E2B).
• Needs on-device test on SM8750/QCS8275: download NPU file, check Describe image and screen latency vs GPU, and fallback with NPU off.

Docs

• https://developers.google.com/edge/litert/next/litert_lm_npu (Qualcomm AI Engine Direct)
• https://developers.google.com/edge/litert/next/qualcomm

[trypsynth]

Thanks for this! NPU support sounds really useful, especially since the Snapdragon 8 Elite is in the Galaxy S25, so a fair number of users could benefit.

Before we can merge it, though:

• It needs to be tried on a real device. Nobody has run it on an SM8750 phone yet, and the whole PR depends on the NPU runtime, the per-chip model and the fallback to GPU/CPU working together. Could you, or someone with an S25 or another 8 Elite phone, try it? Please check that Describe image and Describe screen work, compare their speed with the GPU, and confirm it falls back cleanly when the NPU fails to load or is turned off.
• Please make Use the NPU off by default until it's been proven on devices. Users can opt in for now, and we can flip the default once we know it works.
• The Qualcomm runtime is added to every phone build, including non-Snapdragon phones. It's small, so that's fine if it's only loaded when an NPU model is used, but please confirm that phones without a Snapdragon chip never touch it.

Happy to look again once it has been tested on a device.

[valiant8086]

Hi

My Galaxy S26 seems ideally suited to test this? I think I have the Android build toolchain, but I'm not cloned in anywhere and don't know the full procedure for building yet. If you just want to toss a build at me, I can give it a spin.

[devinprater]

If the person who made the issue doesn’t have an S25, I’ve got an S25 Plus I can test in a good 4 hours.

…

On Oct 12, 2026, at 10:49 AM, Quin Gillespie ***@***.***> wrote: trypsynth left a comment (trypsynth/backtalk#162) <#162 (comment)> Thanks for this! NPU support sounds really useful, especially since the Snapdragon 8 Elite is in the Galaxy S25, so a fair number of users could benefit. Before we can merge it, though: It needs to be tried on a real device. Nobody has run it on an SM8750 phone yet, and the whole PR depends on the NPU runtime, the per-chip model and the fallback to GPU/CPU working together. Could you, or someone with an S25 or another 8 Elite phone, try it? Please check that Describe image and Describe screen work, compare their speed with the GPU, and confirm it falls back cleanly when the NPU fails to load or is turned off. Please make Use the NPU off by default until it's been proven on devices. Users can opt in for now, and we can flip the default once we know it works. The Qualcomm runtime is added to every phone build, including non-Snapdragon phones. It's small, so that's fine if it's only loaded when an NPU model is used, but please confirm that phones without a Snapdragon chip never touch it. Happy to look again once it has been tested on a device. — Reply to this email directly, view it on GitHub <#162?email_source=notifications&email_token=ADUMTTUACN7ATCAMGLFTKKL5TT4QPA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTMMJSGQ3TMMZXG42KM4TFMFZW63VKON2WE43DOJUWEZLEUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#issuecomment-6124763774>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADUMTTQ5WCCYU6QRQCF4X435TT4QPAVCNFSNUABGKJSXA33TNF2G64TZHMYTGOBYGA2DQMRUGY5US43TOVSTWNJYGA4TGMRZGM4TPILWAI>. You are receiving this because you are subscribed to this thread.