Skip to content

perf(python): avoid full registry scans for submodule probes - #2532

Open
DeusData wants to merge 9 commits into
mainfrom
fix/python-module-prefix-v2
Open

DeusData wants to merge 9 commits into
mainfrom
fix/python-module-prefix-v2

Conversation

@DeusData

@DeusData DeusData commented Oct 3, 2026 •

Copy link
Copy Markdown
Owner

Unresolved Python module attributes scan the full function registry for every submodule probe. Reuse an arena-owned sorted function-name index for both duplicate detection and prefix lookup, with binary search for indexed entries and a scan for appended entries or an unavailable index. Exact lookup order, fallback behavior, and duplicate-binding handling are preserved.

Adds six focused registry/resolver regressions and one pipeline operation-count regression with assertions for corpus population, real LSP call controls, and nonzero counters. The index retains extra arena storage through each walk or shared-registry lifetime; no measured speedup or memory claim is made.

Depends on #2531 and its prerequisite stack. This PR targets main, so its diff includes those dependencies until they land. Python re-export resolution remains a separate outstanding change.

Parallel resolver error slots use matching tracked allocation and cleanup. Import-filter failures share cleanup while preserving the full-registry fallback. The existing raw-allocation allowances are unchanged.

Validation: source review, existing-test preservation review, formatting, and diff/integrity checks. Local tests, builds, compiler/syntax checks, sanitizers, revert checks, and local CI were not run under the session waiver. GitHub CI is the validation gate; no runtime or performance result is claimed.

Carry declared instance-field types through extraction and result
compaction into the shared Python type registry so chained calls through
imported classes resolve their receiver types.

Refs #1277

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Copy inherited field-type pointer arrays into the writable registry arena
before Python refines an existing field. Keep shared type objects intact
and avoid retaining per-file scratch types in the sealed fallback.

Add deterministic shared-state and later-overlay coverage, retaining the
sequential Python field regression and adding a forced-parallel variant.

Refs #1277

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Memoize C expression evaluation, index repeated scope and registry queries,
and report incomplete walks explicitly on memo or depth failure.

Refs #1527

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Preserve the arena-owned raw Module caller name so top-level resolutions
join the extractor in both pipeline drivers. Add occurrence and ownership
controls, including package modules and named lambdas.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Resolve symbols declared in __init__.py under their package while retaining
the raw module identity for file and top-level caller nodes. Carry explicit
language through registry callers, select Python package imports, preserve
long exact names, and report scope allocation failures without partial output.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Capture ordered Python namespace facts and retain them through compact, spill,
and versioned LSP surfaces. Reject unavailable metadata before incremental
reuse and rebuild on Python namespace changes or deletion.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Reuse a stable arena-owned function QN order for ambiguity marking and
submodule prefix lookup, preserving exact lookup policy and appended tails.
Add prefix/index failure regressions and a pipeline operation-count case.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Use tracked zeroed storage for the parallel LSP failure array and
release it with the matching memory class after worker joins.
Share import-filter failure cleanup without changing fallback state.
Keep existing raw-allocation allowances unchanged.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

DeusData commented Oct 3, 2026

Copy link
Copy Markdown
Owner Author

Checkpoint and handover (2026-10-03 UTC)

Published head: 6561da09190fb8877901a4344948f17655e98d04. The remote head was verified.

The function-prefix index is published across seven files with seven new tests, unrun locally. Prerequisite: #2531. It preserves ordering/ambiguity and includes the failure seam; the sorted arena-owned index borrows qualified names and uses a binary prefix range plus the post-snapshot tail. No runtime speedup or memory reduction has been measured. #2535 is the next dependent PR.

Hosted snapshot at 2026-10-03 21:54:22 UTC: 3 queued. Confirm the required checks on this exact head before treating it as ready.

No additional local build, test, lint, sanitizer, benchmark or CI runs were performed at this checkpoint, as requested. Earlier executed evidence remains historical; prepared tests and the newer source-reviewed changes must still be validated by the hosted gate.

The campaign is paused at the maintainer’s request. Local monitoring has stopped; hosted jobs remain running. No merge was performed. Thanks for reviewing this change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant