Dynamic-shape input validation only enforces total numel ≤ upper_bound

@larryliu0820

🐛 Describe the bug

For a method with DYNAMIC_BOUND inputs, the only shape validation performed at set_input is a single scalar check, numel(new_sizes) <= numel_bound_. The per-dimension lower bounds (Dim(min=…)) and divisibility / derived-dim constraints (e.g. 32*Dim(...)) declared at torch.export time are not enforced at runtime. Inputs that violate them are silently accepted and produce wrong-shaped / silently-incorrect outputs - no error is raised.

Where it happens - runtime/core/portable_type/tensor_impl.cpp, TensorImpl::internal_resize_contiguous(), the DYNAMIC_BOUND case:

case TensorShapeDynamism::DYNAMIC_BOUND:
case TensorShapeDynamism::DYNAMIC_UNBOUND: {
  auto new_numel = safe_numel(new_sizes.data(), dim_).get();
  ET_CHECK_OR_RETURN_ERROR(
      static_cast<size_t>(new_numel) <= numel_bound_,   // <-- the ONLY guard
      NotSupported,
      "Attempted to resize a bounded tensor with a maximum capacity of %zu elements to %zu elements.",
      numel_bound_, new_numel);
  ...
}

DYNAMIC_UNBOUND falls through to the same case (the // TODO(T175194371) treats unbounded as upper-bounded), so even "unbounded" dynamic tensors are silently numel-capped and never per-dim validated.

Root cause - schema/program.fbs, table Tensor, serializes only the upper-bound sizes for a dynamic dim (field comment: "dynamism == DYNAMIC_BOUND: sizes field represents the upper bound shape") plus a shape_dynamism enum. The Dim lower bound and divisibility are dropped during AOT serialization, so the runtime has nothing to validate against except the precomputed numel_bound_ ceiling.

Reproduction - a model exported with a dynamic square input H, W = 32*Dim(min=8, max=40) (valid set = multiples of 32 in [256, 1280]); detect output is [1, 134, H/4, W/4]. Observed at set_input/execute:

Input	Expected	Actual
1280×1280 (max, valid)	OK	✅ OK → `[1,134,320,320]`
640×640 (valid)	OK	✅ OK → `[1,134,160,160]`
256×256 (min, valid)	OK	✅ OK → `[1,134,64,64]`
96×96 (below min)	reject	⚠️ runs → `[1,134,24,24]`
300×300 (not ÷32)	reject	⚠️ runs → `[1,134,80,80]` - note 80 ≠ 300/4 = 75; the internal /4 downsample ratio is broken, so downstream geometry (coordinate decode, grid_sample, etc.) is silently wrong
1312×1312 (above max)	reject	✅ `set_input()` returns `Error::NotSupported` - but only because `numel` exceeds the bound, not because a per-dim max is checked

So in practice only the upper limit is guarded. The off-divisor case returns a normal-looking tensor whose dim/divisor relationship is wrong, and no error surfaces.

Expected behavior: set_input / resize_tensor should reject (or the framework should provide a supported way to reject) shapes that violate the declared per-dim min/max or divisibility, instead of silently accepting any shape whose total numel fits the bound. (Closely related: there is currently no API to query these bounds either)

Versions

PyTorch version: 2.12.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 26.2 (arm64)
GCC version: Could not collect
Clang version: 21.0.0 (clang-2100.1.1.101)
CMake version: version 3.31.10
Libc version: N/A

Python version: 3.10.19 (main, Oct 27 2025, 17:05:40) [Clang 17.0.0 (clang-1700.3.19.1)] (64-bit runtime)
Python platform: macOS-26.2-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
Caching allocator config: N/A

CPU:
Apple M4 Pro

Versions of relevant libraries:
[pip3] executorch==1.3.1+10f535e
[pip3] flake8==6.1.0
[pip3] flake8-breakpoint==1.1.0
[pip3] flake8-bugbear==24.4.26
[pip3] flake8-comprehensions==3.14.0
[pip3] flake8-plugin-utils==1.3.3
[pip3] flake8-pyi==23.5.0
[pip3] mypy==1.14.1
[pip3] mypy_extensions==1.1.0
[pip3] numpy==2.2.6
[pip3] onnx==1.20.1
[pip3] onnx-ir==0.2.0
[pip3] onnxruntime==1.23.2
[pip3] onnxscript==0.6.2
[pip3] optree==0.19.0
[pip3] pytorch_tokenizers==1.3.0
[pip3] tf2onnx==1.17.0
[pip3] torch==2.12.0
[pip3] torchao==0.17.0+git02105d46c
[pip3] torchaudio==2.11.0
[pip3] torchdata==0.11.0+cpu
[pip3] torchsr==1.0.4
[pip3] torchtune==0.0.0
[pip3] torchvision==0.27.0

cc @larryliu0820 @JacobSzwejbka @lucylq

@Gasoonjia

This is a fair issue. Its an uncommon case to have a non trivial minimum size lets think about a non painful way to serialize this. Maybe just throw it in extra tensor info? cc @Gasoonjia

@JacobSzwejbka

@JacobSzwejbka @benITo47 Hello ! Here’s the plan I’d like to execute, with the two design points that need sign-off flagged below.

Root cause recap: the DYNAMIC_BOUND branch of TensorImpl::internal_resize_contiguous() guards a single scalar, numel(new_sizes) <= numel_bound_. Everything else from torch.export‘s range_constraints per-dim min/max and derived-dim relations like 32*Dim(...) is dropped at serialization (program.fbs keeps only the upper-bound sizes + shape_dynamism), so the runtime can’t enforce what it never receives. Concretely, 2560×640 passes today on a model bounded at 1280×1280 (same numel), and 300×300 silently breaks the /4 downsample geometry.

PR 1 : per-dim upper bound, no schema change. The bound shape is already available (it’s the serialized sizes the tensor is constructed with), but sizes_ is mutated in place on resize so snapshot it at construction (or keep a pointer to the flatbuffer-backed sizes, which outlive the Method) and add new_sizes[i] <= bound_sizes[i] to the DYNAMIC_BOUND case. This strictly tightens validation; if anyone relies on numel-trading across dims, it can go behind a strict-validation flag.

PR 2 : serialize min + divisibility. Append-only, BC-safe fields on table Tensor: dim_min_sizes:[int] plus per-dim (divisor, offset), emitted from the ExportedProgram’s range_constraints in the exir serializer; enforce in resize_tensor with actionable errors. DYNAMIC_UNBOUND keeps its current fall-through until T175194371 lands.

PR 3 : introspection. Expose per-dim min/max/divisor via MethodMeta/TensorInfo so callers can query the valid input space instead of probing it.
Tests: unit coverage on the resize path + the repro matrix from the issue (below-min, non-÷32, per-dim-max violated with numel under bound the case the current check misses).
Needs sign-off: (a) strict per-dim check on by default vs opt-in; (b) preferred schema encoding for derived dims (divisor, offset) pairs vs a small expression table.

I can start on PR 1 right away Mb.

If it were up to me, I'd go for strict per-dim check being an opt-out rather than opt-in.
As for the schema - no preference here

@AxelNoun

Happy to see @AxelNoun again! Thanks for sharing your thoughts!

I think the impl plan looks good to me! Thanks for doing that.

strict per-dim check on by default vs opt-in;

I prefer to default turning it on as well; by default the check should turn on otherwise the runtime output maybe UDB

(b) preferred schema encoding for derived dims (divisor, offset) pairs vs a small expression table.

I;m thinking if we can do the third way by using transaction.

Most derived dims in range_constraints we see in practice are affine: d2 = d1 / 4, d = 32 * d_blocks, d1 = d0, etc. We don't need a full expression table for d1 = f(d0, d1).

We can keep the per-dim simplicity of (A) but get ∼99% of (B) by canonicalizing to root + affine transform + divisibility:

Proposed per-dim fields (still O(1) per dim, append-only on Tensor):

root_dim: int  // -1 if root
mul, div, add: int // dim = (root_dim * mul + add) / div
divisor, remainder: int // dim % divisor == remainder

At serialization time in exir, we topologically sort the derived dims. If dim = (parent * a + b) // c, we propagate its root and compute its tightest divisor via gcd. So:

H = 32 * h_blocks, H_down = H / 4 with H % 32 == 0 => we infer and store H_down: root=h_blocks, mul=8, div=1, divisor=8.

Then at runtime in resize_tensor, check is still just integer arithmetic, no AST evaluator. 300x300 fails on dim % 32 != 0 with an actionable message, so the /4 geometry is protected transitively.

Complex non-affine cases like d0 + d1 can't be canonicalized - for those we can fallback to storing the tightest implied divisor, or make the exporter error until we need a full table.

This gives us the simplicity / BC story of A, but covers the downsample case that motivated B.

WDYT?

@Gasoonjia

@Gasoonjia thanks I like the affine canonicalization. It subsumes the flat (divisor, offset) idea, keeps the O(1)-per-dim / append-only story of (A), and protects the transitive downsample case that motivated (B), with resize_tensor staying pure integer arithmetic. +1 on the direction.

On (a): sounds like we're all aligned strict per-dim check on by default. I'll ship PR 1 default-on with an explicit escape hatch (flag restoring the numel-only behavior) plus a release note, since models accidentally relying on numel-trading will start failing loudly at resize instead of producing UB. The flag can be dropped after a deprecation cycle.

On the encoding : one thing to pin down: what does root_dim index?

In range_constraints the root is a symbol, and it's often never materialized as a dim of the tensor being resized:

H = 32 * Dim("h_blocks") → only H exists as a dim; h_blocks is purely symbolic.
Relations routinely span different method inputs: image with H = 32k and mask with H' = 8k, or a batch symbol shared by every input.

So a same-tensor dim index can't express the general case and resize_tensor, which sees one tensor at a time, couldn't enforce it anyway. Suggestion: keep the canonicalization exactly as you describe, but split what it produces into two layers (computed in the same exir pass):

Unary projection, per dim : your divisor/remainder plus min/max, obtained by pushing the root's range and divisibility through the affine map (gcd for divisibility — carrying the add offset so d = 2k + 1 projects to d % 2 == 1 — and interval arithmetic for the range). This is what resize_tensor enforces: PR 2 as planned. 300×300 fails here transitively with the % 32 message, exactly as in your example.
Relational layer : (symbol_id, mul, add, div) per dim, with a small per-method symbol table (symbol_id → {min, max}). Not checkable inside resize_tensor, but cheap to verify at method level (on set_input / before execute: recover the symbol per dim via (dim * div − add) / mul, require integrality + agreement across dims), and it's the right thing to surface in MethodMeta/TensorInfo for PR 3 so callers can query the joint valid input space rather than per-dim boxes. Enforcement can land as a follow-up to PR 2 with no schema churn if we reserve the fields now.

Concretely (append-only on table Tensor; defaults are sentinels so an old .pte decodes to "unconstrained" and keeps exactly PR 1 semantics this also folds dim_min_sizes into the same table instead of a parallel array):

dim_constraints: [DimConstraint];  // parallel to sizes; absent => PR 1 behavior

table DimConstraint {
  // unary — enforced in resize_tensor (PR 2)
  min: int = 0;         // 0 => no minimum; sizes[] stays the per-dim max
  divisor: int = 0;     // 0 => no divisibility check
  remainder: int = 0;   // dim % divisor == remainder

  // relational — dim = (symbol * mul + add) / div  (method-level + MethodMeta, PR 3)
  symbol_id: int = -1;  // -1 => none; index into per-method symbol table
  mul: int = 1;
  add: int = 0;
  div: int = 1;         // emitted only when division is provably exact
}

Two semantics worth fixing now:

Exact division only: emit div > 1 only when export can prove the numerator divisible (as in H % 32 == 0 ⇒ H/4); otherwise treat the dim as non-affine.
Non-affine fallback (d0 + d1, …): project + warn, don't hard-error. The range always projects via interval arithmetic and gcd yields a (possibly trivial) divisor, so we still emit the unary layer and log that the relation itself won't be runtime-enforced. Hard-erroring the exporter would break models that export fine today, which seems wrong for a validation feature.

On top of the issue's repro matrix I'll add tests for: chained derivation (H = 32k, H_down = H/4, feed 300 → actionable % 32 error), an offset case (2k + 1), and a shared batch symbol across two inputs for the method-level check.

If that works for you both, I'll start PR 1 now and put the .fbs layout up as a short design PR so we can settle the encoding asynchronously before PR 2's implementation.

@AxelNoun

Hi @AxelNoun, have you worked by any chance on this one?

@AxelNoun

Hi @AxelNoun, have you worked by any chance on this one?

Hi ! As mentioned earlier in the discussion, I previously suggested a few approaches and have begun working on them locally. Please feel free to reach out if you need any assistance or additional context. I'm happy to help.

@AxelNoun

Hi @AxelNoun sry for the late response on your proposal!

Overall i love your proposal; only thing in my mind is where we do the check. My recommendation is instead of updating resize_tensor(), i would recommend to do the check in set_input() for single dimension regulation and set_inputs()for cross dimension regulation, which will be better fit for the semantics and limit the impact to the right place (resize_tensor used everywhere in the system.)

@msluszniak

@msluszniak A patch is ready; happy to hand it over instead of duplicating effort if you'd rather take it.

@JacobSzwejbka asking for your go-ahead before I push and open the PR.

@Gasoonjia I took the recommendation into account and it seems relevant to me, it serves the perimeter but I don't think it's a bad thing

What the change does

runtime/executor/method.cpp only, +33/−1. The existing input_tensor_meta() call is hoisted above resize_tensor (its Result is now checked rather than dereferenced inside is_memory_planned()), and then, only when the serialized plan declares DYNAMIC_BOUND and the ranks match, each dimension is compared against its bound. tensor_impl.cpp, program.fbs, the exir emitter, export_program.py and the build files are untouched.

This is not new policy: schema/program.fbs:163-165 already states the per-dimension constraint explicitly. Only the first sentence of that contract was being enforced; the patch enforces the second.

Verified by execution

	Result
Repro on unmodified main (`c8d5189`)	`set_input((2,6))` returns `Ok` — test passes, bug reproduced in-tree
Build	clean, logging enabled so `-Wformat` validated the message
`method_test`	22/22, every table case covered
`lintrunner -a`	`No lint issues.`, no patch applied
Binary size	`size_test` `.text` 38,881 → 39,099 (+218 B, +0.56%); `.data`/`.bss` unchanged

Not verified: the ATen-mode build. OSS cmake doesn't compile method.cpp in that variant (that's Buck or the pybind preset). I checked the types by hand int64_t where portable uses ssize_t, signed/signed comparison — but that's reasoning, not a compile; CI will cover it. The Buck build wasn't run either.

Two things worth knowing before review

DYNAMIC_UNBOUND has no test coverage, and structurally can't: tensor_parser_portable.cpp:56-60 rejects those programs at load time (T175194371), so in a portable build which is what method_test uses no UNBOUND input can reach set_input, whatever model is added. That path is only live in ATen mode. The guard is still needed.
The pre-existing SetInputRejectsOverflowingSizes now trips the new guard before resize_tensor's. It still passes, but its error code changes from NotSupported to InvalidArgument. That feeds question 1 below.

Three open design questions

Error code. The new guard returns InvalidArgument; resize_tensor returned NotSupported on the overflow path. Do you want InvalidArgument uniformly for bound violations, or should the overflow case keep NotSupported?
Fallback switch. Should there be an opt-out (build flag or runtime toggle) for anyone currently relying on the lenient behaviour, or is a hard error from the start acceptable?
Release note. Does turning a silently-accepted input into a hard error warrant a release note?

I've deliberately left all three to you rather than picking defaults.

The PR description will read "Part of #20546", not Fixes, so merging won't close the issue. Ready to push as soon as you confirm.

@JacobSzwejbka

@JacobSzwejbka I asked for your go-ahead on 4 Sep and have not heard back, so I opened #24203 to make the change easy to review. I will rework or close it if you would rather handle this differently. @msluszniak if you had started on this, tell me and we can sort out which one to keep.

It adds the per-dimension upper-bound check in set_input(), as @Gasoonjia recommended, and is marked "Part of #20546". The three questions from my 4 Sep comment (error code, opt-out, release note) are still open and repeated in the PR. On 4 Aug I said the first PR would include an opt-out flag and a release note; it has neither yet, pending those answers. The cross-input check in set_inputs(), lower bounds and divisibility, and introspection through MethodMeta are not part of it.

Nah, I haven't stared work on this one so go ahead :))

Navigation Menu

Repository navigation

Dynamic-shape input validation only enforces total `numel ≤ upper_bound` #20546

🐛 Describe the bug

Versions

What the change does

Verified by execution

Two things worth knowing before review

Three open design questions

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Dynamic-shape input validation only enforces total numel ≤ upper_bound #20546

Description

🐛 Describe the bug

Versions

Activity

JacobSzwejbka commented on Jul 14, 2026

AxelNoun commented on Aug 2, 2026

benITo47 commented on Aug 3, 2026

Gasoonjia commented on Aug 3, 2026

AxelNoun commented on Aug 4, 2026

msluszniak commented on Sep 2, 2026

AxelNoun commented on Sep 2, 2026

Gasoonjia commented on Sep 2, 2026

AxelNoun commented on Sep 4, 2026

What the change does

Verified by execution

Two things worth knowing before review

Three open design questions

AxelNoun commented on Oct 27, 2026

msluszniak commented on Oct 27, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Dynamic-shape input validation only enforces total `numel ≤ upper_bound` #20546