Repository navigation
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Follow-up to #1526: the native-autograd normalization path handles replacement tensors, but mistakes same-object in-place edits for observation hooks. For example, returning
value.mul_(2)fromhook_normalizedorhook_scalehas no effect, while returningvalue * 2changes the result and emits the expected edit-fallback warning.This change detects tensor writes as well as replacements at both intermediate hooks. A scale edit recomputes the normalized activation before
hook_normalized; either edit selects reconstruction from the hooked values. Mutations also propagate when the callback returnsNone, consistent with the existing Python normalization path.Hooks receive grad-connected clones so an in-place scale edit does not invalidate the original square-root output saved by autograd. Normal execution uses version counters for mutation detection. Inference tensors have no version counter, so inference-mode callbacks use an exact, NaN-aware value snapshot instead. Pure observers still select HF's native output, preserving exact forward and backward numerics, and no-hook execution avoids cloning.
The existing edit-fallback and backward-hook warnings remain unchanged. No public signatures or model-loading behavior change. The native-path docstring now includes in-place edits.
Type of change
Validation
Before the fix, the added edit/observation matrix had 36 failed / 6 passed. Coverage includes scale/normalized hooks, returned tensors and
None, LayerNorm/RMSNorm/offset RMSNorm, grad/no-grad/inference execution, input and affine-parameter gradients, exact observation parity, and NaN/Inf observations under inference mode.A real config-built tiny Llama also checks layer-internal and final normalization interventions, HF/Bridge and cache parity, equivalent replacement edits and hook cleanup. No pretrained weights are downloaded.
mypy .: 389 source files passed, including the final incremental check.git diff --check: passed.The complete unit suite and GPU/MPS hardware tests were not run.
Checklist