Let memoize's ignore set match a positional arg by name - #368
Open
afonsojanu wants to merge 1 commit into
Open
Conversation
ignore={'session'} only worked when session was passed as a keyword.
Call the same function with session passed positionally instead and
diskcache tried to build a cache key that included the raw argument,
which is exactly the footgun described in grantjenks#240: a caller has to know
whether they always call with keywords, or ignore has to spell out the
positional index too.
memoize() now grabs the wrapped function's parameter names once via
inspect.signature and passes them through to args_to_key, which checks
a positional argument's name against ignore in addition to its index.
Keyword matching was already fine and is untouched. args_to_key's new
arg_names parameter defaults to an empty tuple, so the two other
callers (DjangoCache.memoize, the memoize_stampede recipe) keep their
existing index-only behavior unless they're updated separately.
Added a regression test using an unpicklable value for the ignored
argument, so a caller passing it positionally raises immediately if it
leaks into the key instead of silently caching under a wrong key.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #240.
ignore={'session'} only filters the argument out of the cache key when session is passed as a keyword. Call the same function with session passed positionally and diskcache builds a key that includes the raw value, so two calls that should be treated the same end up as two different cache entries (or, if the value isn't picklable, a crash). That's the footgun the issue describes: whether ignore actually works depends on how the caller happens to invoke the function, not on what you wrote in the decorator.
This grabs the wrapped function's parameter names once, in
memoize's decorator, viainspect.signature, and passes them intoargs_to_key. A positional argument now gets filtered if either its index or its parameter name is inignore. Keyword matching already worked and isn't touched.args_to_key's newarg_namesparameter defaults to(), so the two other places that call it directly (DjangoCache.memoizeand thememoize_stampederecipe) keep their current index-only behavior. Happy to extend the fix there too if that's wanted, just didn't want to change more than the reported case in one PR.Test uses an unpicklable value (a
threading.Lock) for the ignored argument, so if it ever leaks into the key the test fails with a clear pickling error rather than silently comparing cache-hit counts.Ran the full
test_core.pyandtest_recipes.pysuites locally, both green (88 and 5 passing respectively).