Standardize Output Schema for Evalautors

[m7md7sien]

Description

Standradize Output Scheme for Evalautors

All SDK Contribution checklist:

• The pull request does not introduce [breaking changes]
• CHANGELOG is updated for new features, bug fixes or other significant changes.
• I have read the contribution guidelines.

General Guidelines and Best Practices

• Title of the pull request is clear and informative.
• There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

• Pull request includes test coverage for the included changes.

[Copilot]

Pull request overview

This PR standardizes the output schema across all evaluators in azure-ai-evaluation. Each evaluator now emits a consistent set of keys (<metric>, <metric>_score, <metric>_passed, <metric>_result, <metric>_reason, <metric>_status, <metric>_threshold, <metric>_properties) instead of a mix of bare keys (e.g., fluency), gpt_<metric> keys, and flat token/sample fields. Prompty files are updated to return JSON (with status: completed|skipped, score, reason, properties), and the base prompty evaluator parses this shape uniformly. Tests are updated accordingly; CHANGELOG mentions only the mapping change for document_retrieval/rouge_score.

Changes:

• New uniform output schema (score/passed/result/reason/status/threshold/properties) across all evaluators; token/sample metadata folded into properties.
• Prompty files migrated to json_object response_format and a shared {score, reason, properties, status} contract supporting skipped evaluations.
• EVALUATOR_NAME_METRICS_MAPPINGS collapses document_retrieval and rouge_score to single primary metrics; sub-metrics moved to <metric>_properties.

Reviewed changes

Copilot reviewed 53 out of 53 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
CHANGELOG.md	Adds breaking-changes note (incomplete — only covers mapping change).
assets.json	Bumps assets tag.
_constants.py	Collapses doc_retrieval/rouge metric mappings to single primary metric.
_common/_base_prompty_eval.py	Replaces XML parsing with JSON parsing; adds _get_token_metadata; removes _not_applicable_result.
_common/_base_eval.py	Skips synthesizing _result/_threshold when already present in evaluator output.
_evaluators/_bleu, _gleu, _meteor, _rouge, _f1_score	Updated to new schema with _score/_passed/_status/_properties and pass-fail booleans.
_evaluators/_document_retrieval	Wraps per-metric details under document_retrieval_properties; uses primary NDCG as top-level score.
_evaluators/_groundedness (incl. prompties)	JSON output; _real_call uses shared not-applicable helper.
_evaluators/_relevance, _coherence, _fluency, _similarity, _retrieval (prompties + _relevance.py)	JSON score/reason/status outputs; new unified schema.
_evaluators/_response_completeness (incl. prompty)	JSON output; drops legacy XML parser branch.
_evaluators/_intent_resolution (incl. prompty)	JSON output; new schema.
_evaluators/_task_adherence (incl. prompty)	Replaces flagged boolean with 0/1 score; new schema.
_evaluators/_task_completion (incl. prompty)	Replaces success with numeric score; new schema; adds skipped status.
_evaluators/_task_navigation_efficiency	Adds bare task_navigation_efficiency key alongside _score.
_evaluators/_tool_call_accuracy (incl. prompty)	Fixes self.threshold → self._threshold; raises max_tokens.
_evaluators/_tool_call_success, _tool_input_accuracy, _tool_output_utilization, _tool_selection (incl. prompties)	Migrate to JSON output schema with skipped support and unified keys.
tests/unittests/* and tests/e2etests/*	Updated to new schema; mass-evaluate e2e tests loosened with TODO placeholders.

Comments suppressed due to low confidence (2)

sdk/evaluation/azure-ai-evaluation/tests/e2etests/test_mass_evaluate.py:288

• Same loose assertion / unresolved TODO as in test_evaluate_singleton_inputs: the previous exact len(metrics.keys()) == 88 was replaced with >= 70. This will hide schema regressions where extra unintended columns/metrics appear. Please verify against a live run and restore an exact equality assertion.
  sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_response_completeness/_response_completeness.py:205
• _get_binary_result returns "unknown" for NaN scores, but here <metric>_passed is computed as score_result == "pass", which means an "unknown" outcome is reported as False. Same concern as the base class — consider returning None for <metric>_passed when the result is "unknown" to avoid misleading downstream consumers. This pattern is replicated in _relevance.py, _intent_resolution.py, _task_adherence.py, _task_completion.py, _tool_input_accuracy.py, _tool_selection.py, _tool_call_success.py, and _tool_output_utilization.py.

            score_result = self._get_binary_result(score)

            llm_properties.update(self._get_token_metadata(result if isinstance(result, dict) else {}))

            return {
                self._result_key: score,
                f"{self._result_key}_score": score,
                f"{self._result_key}_passed": score_result == "pass",
                f"{self._result_key}_result": score_result,