[Evaluation] Configure adaptive native AOAI output result page size - #49034
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Azure Pipelines: Successfully started running 1 pipeline(s). 10 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Copilot encountered an error: Your billing is not configured or you have Copilot licenses from multiple standalone organizations or enterprises. To use premium requests, select a billing entity via the GitHub site, under Settings > Copilot > Features.
|
Azure Pipelines: Successfully started running 1 pipeline(s). 10 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Copilot encountered an error: Your billing is not configured or you have Copilot licenses from multiple standalone organizations or enterprises. To use premium requests, select a billing entity via the GitHub site, under Settings > Copilot > Features.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot encountered an error: Your billing is not configured or you have Copilot licenses from multiple standalone organizations or enterprises. To use premium requests, select a billing entity via the GitHub site, under Settings > Copilot > Features.
|
Azure Pipelines: Successfully started running 1 pipeline(s). 10 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Copilot encountered an error: Your billing is not configured or you have Copilot licenses from multiple standalone organizations or enterprises. To use premium requests, select a billing entity via the GitHub site, under Settings > Copilot > Features.
Summary
aoai_output_items_page_sizekeyword toevaluate, preserving existing calls with a default of100EvaluationExceptionuser-input shapemax(1, (page_size + 1) // 2)and retain a successfully reduced size for later pages in that runRetry-After, retry headers, and delay calculation to the OpenAI client within the same attempt budgetwith_options(max_retries=0)client; shared-client ownership and unrelated inference, grader creation, and run polling retry behavior are unchangedValidation
python -m pytest tests\unittests\test_aoai_evaluation_pagination.py -q— 25 passedpython -m pytest tests\unittests\test_aoai_evaluation_pagination.py tests\unittests\test_aoai_alignment_missing_rows.py tests\unittests\test_evaluate_aoai.py tests\unittests\test_evaluate.py -q— 246 passed, 7 skippedazpysdk black .— passedazpysdk breaking .— passed; package is currently opted out of breaking-change comparisonazpysdk apistub .— generated the expected public signature, then the wrapper failed during metadata extraction because current main does not containeng\scripts\Extract-APIViewMetadata-Python.ps1; only the intended API signature update is includedazpysdk changelog verify— could not run because the pinned Chronus dependencies could not be installed in this environment due npm authentication/TLS failures; the fragment follows the repository's.chronus/changesformatTests use controlled responses and mocked sleep; no live evaluation qualification is claimed.
Scope
This is SDK Part 1 only. Vienna/ACA integration, ingress changes, adaptive timing thresholds, page-size growth, dataset sampling, and deployment are intentionally unchanged and remain separate follow-up work.