MAINT: Remove sys.path mutations from build_scripts

[Roman Lutz (romanlutz)]

Description

Scripts in build_scripts/ reached their siblings in three different ways, none of them sound. gen_api_md.py and example_index.py did sys.path.insert(0, <script dir>) followed by a bare import validate_docs, and compose_docs_dist.py imported generate_pages_manifest by bare name with no guard at all, working purely by accident of direct execution.

Two concrete problems. First, mixing import validate_docs with from build_scripts import validate_docs creates two distinct module objects for the same file, which is why #2282 needed a regression test to pin it down. Second, inserting the script directory at sys.path[0] pollutes the path for the entire process. Under pytest that means the whole test session, so a future build_scripts/types.py or build_scripts/json.py would silently shadow a stdlib module for every test in the run.

This makes build_scripts/ a real package and imports siblings as from build_scripts import <module>, removing the sys.path mutations entirely.

Invocation changes, and why it was unavoidable. Package-qualified imports do not resolve under direct script execution: with python build_scripts/gen_api_md.py, the script's own directory is sys.path[0] and the repo root is not on the path at all, so the import raises ModuleNotFoundError. I verified this empirically rather than assuming it. So every call site moves to python -m build_scripts.<name>: the Makefile, the pre-commit hooks, the docs workflow, the Dockerfile, and the usage examples in module docstrings and contributing docs. The upside is one simple rule, always -m from the repo root, which also stops a future contributor from reintroducing this by adding a sibling import to a path-invoked script.

Worth flagging for contributors: python build_scripts/evaluate_scorers.py and friends no longer work; use python -m build_scripts.evaluate_scorers. This is a developer workflow change only. build_scripts is not part of the distributed package (packages.find only includes pyrit and pyrit.*), so nothing changes for people who install PyRIT, and I confirmed against a built sdist that build_scripts is absent from it.

One non-obvious detail in the docs workflow. It uses sparse-checkout with sparse-checkout-cone-mode: false, which makes the patterns gitignore-style. Those blocks previously enumerated individual script files, which is fragile now that these scripts import each other: a missed entry breaks CI at runtime with ModuleNotFoundError. They now match the directory wholesale as /build_scripts/. The leading slash matters. A bare build_scripts has no separator, so under gitignore rules it matches at any depth and also drags in tests/unit/build_scripts/. The paths: trigger is likewise collapsed to build_scripts/**. That does mean the docs build now triggers on unrelated build_scripts/ changes, but the workflow already triggers on all of pyrit/** and doc/**, and an occasional extra run is much cheaper than a silently missed one.

Tests and Documentation

Added tests/unit/build_scripts/test_import_hygiene.py, an AST-based guard that scans every build_scripts/*.py and fails on any sys.path mutation or any bare top-level import of a sibling module. Source-based rather than runtime sys.path inspection, which would be order-dependent and flaky. I confirmed it is not vacuous by injecting a sys.path.insert plus a bare sibling import into validate_docs.py and watching both guards fire.

Also de-hacked two existing tests that were doing the same thing: test_compose_docs_dist.py replaced its sys.path.insert plus spec_from_file_location fixture with a plain import, and tests/unit/memory/test_migration.py dropped its sys.path.insert. The test_docs_scripts_share_validate_docs_module regression test from #2282 still passes unchanged.

Verification performed:

• Full unit suite: 15280 passed, 6 skipped.
• Docs pipeline end to end via pydoc2json then gen_api_md. Prints Indexed examples: 236 symbols across 74 pages and [OK] All documentation validations passed!, with a clean git diff on doc/api/, so generated output is byte identical to before.
• pre-commit run --all-files passes, which exercises all 8 converted hooks.
• Built a real non-cone sparse checkout of this branch and ran the versions and deploy jobs against it in a clean venv with only PyYAML installed and no editable install, mirroring CI. resolve_docs_matrix, compose_docs_dist, and inject_version_picker all succeed.
• Confirmed no sys.path entries are added on import, that all modules resolve to a single shared object, and that build_scripts is absent from a built sdist.

Documentation: updated the usage examples in doc/contributing/10_release_process.md, doc/contributing/11_memory_models.md, and the doc/code/scoring/4_scorer_metrics jupytext pair. The notebook hits are all inside markdown cells, so the .py and .ipynb halves received identical textual edits and no re-execution was needed. I verified the pair stays in sync by round-tripping the .ipynb through jupytext and diffing against the .py.