Skip to content

fix(flux): warn on unsupported ip_adapter_masks#13956

Closed
akshan-main wants to merge 1 commit into
huggingface:mainfrom
akshan-main:fix-flux-ip-adapter-masks
Closed

fix(flux): warn on unsupported ip_adapter_masks#13956
akshan-main wants to merge 1 commit into
huggingface:mainfrom
akshan-main:fix-flux-ip-adapter-masks

Conversation

@akshan-main

@akshan-main akshan-main commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Addresses the flux review (#13584), Issue 1: FluxIPAdapterAttnProcessor accepts ip_adapter_masks but never applies them, so region-specific IP-Adapter conditioning silently has no effect (the repro in the issue prints 0.0 between masked and unmasked runs).

I looked at implementing the masks properly (the SD IPAdapterAttnProcessor2_0 pattern), but:

  • no Flux pipeline plumbs ip_adapter_masks through, so there's no caller to exercise it, and
  • the SD path applies masks spatially, while the Flux processor only sees packed-sequence latents and doesn't have the spatial shape needed to downsample/map a mask correctly.

So rather than ship a risky reimplementation against Flux's packed layout with no caller, this stops the silent no-op: passing ip_adapter_masks now emits a warning_once that it's unsupported and ignored. Full regional-mask support for Flux would be a follow-up (and needs spatial info plumbed in) — happy to take that on if you'd like it.

Before submitting

Who can review?

@hlky @sayakpaul @yiyixuxu

@github-actions github-actions Bot added size/S PR with diff < 50 LOC models labels Jun 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Hi @akshan-main, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked. See the contribution guide for more details. If this PR intentionally does not fix a tracked issue, a maintainer can add the no-issue-needed label to silence this reminder.

ip_hidden_states: list[torch.Tensor] | None = None,
ip_adapter_masks: torch.Tensor | None = None,
) -> torch.Tensor:
if ip_adapter_masks is not None:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can just remove this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

models size/S PR with diff < 50 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants