Python 3.11 adoption opportunities: claim one improvement at a time #3166

Is your feature request related to a problem? Please describe.

PyRIT now requires Python >=3.11 (currently <3.15), and Ruff targets py311. We can use Python 3.11 features consistently across supported versions, simplify older compatibility patterns, and improve typing, diagnostics, and async lifecycle management. Some features are already partially adopted; the opportunities below are investigations, not instructions to replace every existing pattern.

Describe the solution you'd like

How to participate: ONE opportunity at a time

Claim exactly ONE checklist item at a time by commenting on this issue thread with its ID (for example, P311-01) and a short description of what you intend to investigate. Do not start multiple items or claim a bundle. Wait for confirmation here before treating the item as reserved. Claims are processed in comment order, and this issue is checked approximately every three hours.

• You must open a PR within five days (120 hours) of your claim comment. The table below records the due date. If no PR is opened by then, the claim expires, the item becomes available, and your issue assignment is removed if you have no other active contribution on this issue. You can claim it again if it is still available.
• Opening a PR within the deadline retains your claim while that PR is active. You cannot claim another item until your current PR is merged or the current contribution is explicitly released/closed.
• Link this issue and the item ID in your PR description. Keep the PR focused on that one opportunity. Follow repository instructions and add relevant behavior/type-checking coverage.
• Investigate first. Preserve public contracts and intended behavior. If a feature is not a good fit, explain why on this thread rather than forcing a change. An investigation can conclude that no code change is appropriate.
• A PR opened without a prior claim will be highlighted to Roman Lutz (@romanlutz). If the item is unclaimed, we can link the PR and assign the item to its author. If someone else has already claimed it or another PR overlaps, Roman Lutz (@romanlutz) will assess how to proceed; no existing claim will be displaced automatically.
• Checkboxes mean completed, not claimed. Check an item only after its PR is merged, or after a maintainer explicitly accepts a no-change investigation. Ownership and PR links are tracked separately below.

Opportunity checklist

Typing

• P311-01: Extend typing.Self adoption. Investigate older same-concrete-class factories and fluent APIs, such as pyrit/common/yaml_loadable.py:YamlLoadable.from_yaml_file. Replace bound-TypeVar boilerplate only where it really models Self; retain independent generic parameters. Merged scope: #3194 covers YAML/seed and scorer-metrics loaders, strategy-context copying, and technique collections, with type-regression coverage. Additional merged scope: #3201 covers the three Message factories and AtomicAttackIdentifier.build, including subclass type-regression tests. Additional merged scope: #3266 covers SeedDataset.from_dict, MessageScorable.from_message, and ContentScorable.from_message, with subclass runtime and assert_type coverage and unchanged validation. Additional merged scope: #3383 covers AttackTechniqueSeedGroup.from_system_prompt and from_messages, with unchanged cls construction/validation and subclass runtime/assert_type coverage. Additional merged scope: #3411 covers MessagePiece.adversarial_placeholder with unchanged cls construction and subclass runtime/assert_type coverage. Additional merged scope: #3470 covers TagQuery.all, any_of, and none_of, with subclass runtime/assert_type coverage and unchanged predicates/validation. Additional merged scope: #3520 covers ScenarioRunSizeEstimate.unavailable, preserving cls construction, unavailable status, defaults, dataset/limit handling and validation, with subclass runtime/assert_type and input/default coverage. Additional merged scope: #3529 covers HarmDefinition.from_yaml, with subclass runtime/assert_type coverage and unchanged YAML loading, path resolution, field conversion, source-path recording and validation. Additional merged scope: #3577 annotates JsonResponseConfig.from_metadata with Self and adds subclass runtime/assert_type coverage; the factory runtime body, schema parsing, defaults, validation and metadata round trips are unchanged. Additional merged scope: #3626 annotates WebSearchConfig.from_metadata with Self, with subclass runtime/assert_type coverage of absent and explicit policies. Construction, read-only Mapping inputs, default/disabled/required modes, frozen/extra-forbid validation, errors, non-mutation and metadata/JSON round trips are unchanged. Remaining focused case (available for a new claim): investigate AdversarialPrompts.from_metadata in pyrit/models/seeds/adversarial_prompts.py. Its absent path returns None and explicit path calls cls.model_validate(...), but the annotation is AdversarialPrompts | None. Consider Self | None with subclass runtime/assert_type checks, preserving absence, read-only Mapping inputs, non-mutation, validation and seed semantics. Keep the claim to this one factory and coupled tests. Another later candidate is AttackResultMetadata.from_metadata in pyrit/models/results/attack_result_metadata.py, which returns cls(...); retain its defaults, legacy role fallback and invalid child-position handling. These are typing-precision follow-ups, not regressions from MAINT Return Self from WebSearchConfig.from_metadata (P311-01) #3626 or a blanket migration. Preserve fixed-base TagQuery.__and__/__or__, independent generics and factories such as SeedPrompt.from_value_with_required_parameters that can return an existing base input.
• P311-02: Improve dictionary contracts with TypedDict, Required, and NotRequired. Investigate internal metadata/provider payloads currently typed as dict[str, Any]. Distinguish absent keys from nullable values. RawAnalyticsProfile already provides prior art. Do not replace Pydantic runtime validation.
• P311-03: Add exhaustive dispatch checks with assert_never / Never. Investigate closed enum/union dispatch for outcomes, states, modalities, or capabilities so added cases surface during type checking. Apply only where handling is genuinely exhaustive, not to intentionally open extension points. Merged scope: #3441 makes AttackStrategy._log_attack_outcome exhaustive across AttackOutcome, preserving existing log messages and rejecting unknown values. Additional merged scope: #3456 explicitly handles ScenarioResultView.CONVERSATIONS and guards both CLI dispatchers with assert_never, preserving HTML full-report override, default view, limits and lazy imports. Additional merged scope: #3472 makes AttackResultAnalytics._option exhaustive for all four AttackAnalyticsValueKind members, preserving display labels and stored keys, with an actual-source enum-extension type regression. Additional merged scope: #3504 explicitly handles all three PuzzleType members in PuzzledConverter.convert_async and adds an assert_never guard with a runtime regression. Constructor validation, crossword fallback, RNG behavior, Unicode handling, clues and output are preserved. Additional merged scope: #3601 covers all five CodeAttackConverter._encode encodings with a type-only assert_never guard, preserving the runtime invalid-state ValueError, constructor validation, template pairing and output. Per-encoding, invalid-state and actual-source type checks cover the change. Additional merged scope: #3654 adds an exhaustive assert_never guard for the validated RepeatTokenConverter mode dispatch, preserving its string-Literal API, None defaulting to split, validation errors/messages, punctuation, spacing, output and identifiers. The unreachable fallback uses the existing coverage exclusion pattern; changed-line coverage passed at 100%. Current focused case (claimed by Slai.life (@huong8373tt-beep)): investigate PromptShieldTarget._input_parser in pyrit/prompt_target/prompt_shield_target.py. PromptShieldEntryField is Literal[None, 'userPrompt', 'documents']; the two named fields have explicit cases, but the fallback also handles future typed fields without a type error. Consider a small type-only exhaustive guard after explicitly accounting for None, patterned after CodeAttackConverter._encode. The constructor does not reject unknown runtime fields, so preserve their existing parsing fallback rather than adding rejection or an unconditional runtime assert_never. Preserve forced-field payloads, None parsing, trailing document text and identifiers. Use the existing parser tests in tests/unit/prompt_target/target/test_prompt_shield_target.py; keep verification proportionate. This is preventive typing coverage, not a present supported-input defect or a blanket migration. Keep open string/custom-callable dispatch such as CodeChameleon and provider/extension handling unchanged.
• P311-04: Add type-regression checks with assert_type. Investigate registry factories, generic results, decorators, and subclass-returning APIs. Ensure checks run through the type checker; assert_type is not a runtime assertion. Additional merged scope: #3625 adds assert_type(results, list[int]) for gather_with_cleanup_async and checks ordered values in tests/unit/test_async_utils.py. The canonical make ty checks tests/unit; the generic helper, public signature and cancellation/cleanup behavior are unchanged. Additional merged scope: #3686 adds concrete and protocol-consumer assert_type checks for DefaultInstanceRegistry[T].get, get_entry and get_all_instances, retaining T | None, RegistryEntry[T] | None and list[RegistryEntry[T]]. Present/missing retrieval, identity and sorted order are checked; the protocol consumer uses a helper parameter to avoid concrete assignment narrowing. Production signatures and runtime behavior are unchanged. Remaining focused case (available for a new claim): investigate a small type-regression check for Registry.get_registry_singleton in pyrit/registry/registry.py. This factory already returns Self; existing tests in tests/unit/registry/test_registry.py check same-class singleton identity and unrelated subclass isolation but do not assert the retrieved static types. Consider adding assert_type for the existing SlowConstructingRegistry and OtherSlowConstructingRegistry results before runtime identity/isinstance checks. Preserve synchronization, cleanup, singleton identity, subclass isolation and public signatures. Keep this test-focused and proportionate, without runtime redesign, dependencies or a new type-test framework. This is a selective typing follow-up, not a requirement to annotate every registry API.
• P311-05: Investigate LiteralString at static-template boundaries. Consider developer-authored SQL or command templates where the type checker can enforce the distinction. Confirm checker support. This is not runtime sanitization and never replaces parameterized SQL.
• P311-06: Investigate TypeVarTuple / Unpack for variadic types. Look for a concrete heterogeneous-tuple or numerical/multimodal API that benefits. Do not add variadic complexity without a real need or assume this automatically checks NumPy shapes. Features standardized after 3.11 still need compatible backports.
• P311-07: Investigate generic TypedDict / NamedTuple containers. Identify payload containers whose content type should be parameterized. Generic TypedDict in analytics result analysis provides existing prior art.

Async execution and diagnostics

• P311-08: Enrich propagated exceptions with BaseException.add_note. Add useful execution, technique, dataset, or conversation context while preserving exception types and tracebacks. Notes appear in formatted tracebacks, not str(error). Exclude secrets and sensitive prompt content. Merged scope: #3175 adds dataset/provider notes to concurrent provider-fetch failures. Additional merged scope: #3277 adds scenario and atomic-attack names to caught worker-pool exceptions in Scenario._execute_atomic_attacks_parallel_async, preserving the original exceptions and existing single/grouped failure behavior. Regression coverage checks grouped notes and single-failure identity. Additional merged scope: #3281 labels cached-backend and analytics close failures in close_services_async, preserving the original exceptions, cancellation and single/grouped behavior. Additional merged scope: #3333 adds safe static phase notes for scenario-service shutdown and memory disposal in RuntimeLifecycle._shutdown_runtime_async. Existing notes, original exception identity/type, cleanup order, cancellation and single/grouped behavior remain intact. Additional merged scope: #3351 adds safe static session-release, active-client-stop and failed-startup-client-stop notes in GitHubCopilotTarget._cleanup_owned_resources_async, with regression coverage for phase labels and original errors. Complete: the identified dataset/provider, scenario-worker, backend service-close, runtime-shutdown and Copilot-cleanup context gaps are addressed. Current-code assessment found no further concrete, worthwhile propagation-context gap requiring another contribution for this item. Other examined groups already describe their operation or preserve ordered/chained causes; this is not a requirement to annotate every exception. Retain original exception identity/type/tracebacks, prior notes, cancellation, cleanup/retry ownership and single/grouped behavior; never add secrets or sensitive prompt content.
• P311-09: Use scoped operation timeouts with asyncio.timeout. Investigate multi-await operations that should share one timeout scope instead of restarting a timeout for every step. Existing analytics, objective-generation, observation, and A2A usage provides prior art. Preserve cancellation and cleanup behavior. Merged scope: #3205 gives each Copilot receive loop one overall budget across streamed messages, with separate handshake/response budgets and cancellation/cleanup coverage. Completed: #3239 gives WebSocketTarget._initialize_connection_async one separate receive budget across all discarded initial messages, with aggregate-budget and cancellation/connection-cleanup coverage. Initialization sends, restoration, and normal responses retain their separate scopes. Current timeout-site assessment found no further worthwhile shared-budget migration: remaining wait_for calls bound individual operations, explicitly per-attempt probes, or already use a shared remaining deadline. Do not mechanically replace those calls.
• P311-10: Propagate absolute deadlines with asyncio.timeout_at. Investigate nested operations/retries that should share one monotonic-clock deadline. Distinguish operation deadlines from per-step timeouts and keep cleanup semantics explicit. Merged scope: #3610 bounds _run_invocation_async in pyrit/_code_execution/docker_client.py with asyncio.timeout_at(deadline) during shielded Docker client startup and passes that same monotonic deadline into invocation handling. Capture and attached-client waiting already deducted elapsed time before this change; the new bound is startup. Startup-specific errors and cleanup ownership remain intact. Assessment: sufficiently addressed; no further worthwhile production migration was demonstrated in the inspected current timeout/retry surfaces. Existing A2A polling, WebSocket initialization, Copilot receive streaming and objective-generation execution already use shared scoped budgets, while QueryControl already computes one absolute deadline. Preserve independently bounded control commands and cancellation/container/workspace cleanup outside the expired invocation budget. The added test checks deadline wiring; behavioral expiry/cancellation regression coverage is an optional test follow-up, not an additional production migration or active reservation.
• P311-11: Investigate structured concurrency with asyncio.TaskGroup. Identify parent-scoped child tasks with appropriate fail-fast semantics. The message-send service already uses it. It is not a blanket gather replacement or a concurrency limiter: sibling cancellation and exception shapes differ. Preserve ordered results and the deliberate cancellation/cleanup behavior in pyrit/common/task_utils.py.
• P311-12: Investigate ExceptionGroup / except* handling. Preserve multiple failures and, where useful, handle matching subsets without losing other errors. Groups already exist in scenario execution and service shutdown. Carefully assess public exception contracts and cancellation (BaseExceptionGroup).
• P311-13: Investigate explicit task context with asyncio.create_task(..., context=...). Identify background operations that need selected tracing/execution ContextVars or a clean context. Default task creation already copies the current context; do not change it gratuitously.
• P311-14: Investigate asyncio.Runner for synchronous entry points. Consider multiple top-level calls sharing one managed loop/context in a shell or lifecycle harness. This does not permit nested loops inside an already-running notebook event loop. Preserve resource and loop ownership.

Standard-library simplifications

• P311-15: Parse project TOML with tomllib. Investigate replacing the version-extraction regex in build_scripts/stamp_compatibility.py with structured access to project.version. Preserve validation/provenance behavior. tomllib reads TOML; it does not write it. Completed: #3208 replaces the version regex with structured project.version parsing and covers table ordering, quoting, invalid metadata, and provenance errors.
• P311-16: Simplify ISO timestamp parsing. Investigate removing trailing-Z compatibility rewrites, such as _RemoteDatasetLoader._parse_datetime, because 3.11 parses valid Z timestamps directly. Preserve timezone and accepted-input contracts. Do not edit immutable historical Alembic revisions merely for cleanup. Completed: #3176 removes the active loader shim and preserves invalid/non-string fallback behavior. Post-merge search found no other actionable ISO-parsing compatibility shims in pyrit or build_scripts; the remaining rewrite in historical migration d7e9f1a3b5c6 is intentionally immutable.
• P311-17: Assess enum.StrEnum for appropriate enums. It is already used for harm categories and message-send models. Existing str, Enum migrations can change str(member) and formatting, and Ruff's automatic replacement is deliberately disabled. Focus on cases where string behavior is intended and compatibility is understood.
• P311-18: Investigate atomic regex groups and possessive quantifiers. Consider (?>...), *+, or ++ in a concrete scorer/parser pattern with unnecessary backtracking on large model outputs. Measure behavior and performance; these constructs change matching semantics and are not a universal regex-safety fix. Current contribution (assigned to Muhammad Waqar Anwar (@WaqarAnwar); #3702): atomic groups in the destructive-shell scorer pattern, with matching-equivalence and long-input performance checks. Opened without a preceding claim; the item was free.
• P311-19: Investigate hashlib.file_digest for whole-file hashes. Consider classifier artifact verification or other complete-file checks. Keep blocking I/O off async paths. Do not replace bounded evidence-reading loops that enforce budgets/cancellation and retain content. Merged scope: #3392 streams the two classifier compute_dataset_hashes() helpers through binary hashlib.file_digest handles, preserving SHA-256 values and dictionary order; multi-buffer regression tests cover current file contents. Assessment: sufficiently addressed. Keep _load_training_rows() hashing the same bytes it subsequently parses; keep LocalFileSurfaceSource._read_entry bounded/cancellable with retained evidence. Async serializer hashing supports storage backends that may be remote, memory media hashing reuses copied content, and FileDocumentStorage._compute_version receives bytes already in memory. No further worthwhile direct whole-file replacement identified; do not add synchronous local-file reads to async storage paths.
• P311-20: Investigate inspect.getmembers_static for safe introspection. Identify registry inspection that should avoid invoking descriptors/properties. It intentionally bypasses dynamic lookup and lazy exports, so verify the discovery contract before adoption. Merged scope: #3531 statically inspects discover_in_directory modules, resolves only explicitly declared missing __all__ exports dynamically, and logs failed declared exports without losing other classes. Registration-outcome tests cover declared lazy exports with/without __dir__, unrelated dynamic names and failed exports; the earlier lazy-export regression is addressed. Additional merged scope: #3580 statically inspects InitializerRegistry._module_defined_initializers for single-file discovery and custom scripts, with targeted resolution of declared lazy exports. Own-module, base-class and abstract filters remain intact. Best-effort discovery logs failed declared exports; strict script loading and custom-script validation still reject them. Registration and strict/best-effort entry-point regressions cover these contracts. Assessment: the registry inspection opportunity is sufficiently addressed by fix: use static member discovery in registry inspection #3531 and FIX avoid dynamic attribute evaluation in single-file initializer discovery #3580; no further worthwhile blanket migration was found. Keep Registry._discover's intentional lazy-export materialization dynamic.

SQLite-specific memory opportunities

These must stay in the SQLite-specific memory/driver layer and preserve portable SQL Server behavior. Check what SQLAlchemy/aiosqlite exposes before choosing an implementation.

• P311-21: Improve SQLite error handling using extended error codes/names. Analytics already checks sqlite_errorcode for plain SQLITE_BUSY. Investigate other real cases where structured codes are preferable to error-message matching, including distinctions such as stale snapshots.
• P311-22: Investigate SQLite connection getlimit / setlimit. Identify a real need for connection-level runtime limits and how limits interact with pooling and workloads. Avoid arbitrary limits that break existing behavior.
• P311-23: Investigate SQLite custom window functions. Assess whether an actual analytics use case benefits from create_window_function, with SQLite runtime support, driver integration, and portability considered. Prefer existing SQL when sufficient.
• P311-24: Investigate incremental SQLite BLOB I/O. Assess whether any actual large-BLOB storage path benefits from blobopen. Do not redesign media storage just to use this API; a no-change conclusion may be appropriate.
• P311-25: Investigate SQLite database serialization/deserialization. Assess a concrete snapshot/test/export use case for serialize / deserialize, including runtime availability, memory cost, concurrency, and driver boundaries.

Claim and PR tracking

The tracker is authoritative for per-item ownership; GitHub issue assignees are the aggregate set of active contributors. Available items have no active tracker row. Preserve released/expired rows as history when updating the tracker.

Describe alternatives you've considered, if relevant

Keep existing implementations when they already express the required behavior clearly. Avoid a repo-wide modernization sweep: each contributor investigates and opens a focused PR for ONE opportunity at a time. Separate changes within an item should remain cohesive; coordinate here before splitting or expanding scope.

Additional context

• Python 3.11 release notes
• Article motivating this inventory
• Existing adoption includes datetime.UTC, Self, NotRequired, generic TypedDict, StrEnum, TaskGroup, timeouts, exception groups, and SQLite extended error codes.
• Better traceback locations and interpreter improvements arrive automatically on 3.11+; they do not need modernization PRs, and model-call performance is often network-bound.
• typing.override, typing.TypeAliasType, the type Alias = ... statement, and class Foo[T] syntax are 3.12+, not 3.11. Keep needed typing_extensions imports while supporting 3.11.
• Other 3.11 APIs such as dataclass_transform can be considered when there is a concrete fitting abstraction, but should not motivate new abstractions on their own.