Repository navigation
FIX Export the converted data type in JSON conversations and label HTML report pieces by it - #3782
Open
Shivam Sharma (ShivamSharma43) wants to merge 1 commit into
Conversation
…ieces by it JsonConversationPrinter exported only the original value's data type, so the HTML scenario report labelled a converted value by its original type: a text-to-image conversion rendered the image path as plain text, and an image-to-text conversion rendered the text as an [image_path] placeholder. Export original_value_data_type and converted_value_data_type next to the existing data_type (kept as the original type), and have the HTML template switch on the type of the value it displays. Fixes microsoft#3742 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
The only red check is |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #3742.
JsonConversationPrinter._build_piece_asyncexported each piece with a singledata_type, set tooriginal_value_data_type, next to bothoriginal_valueandconverted_value. The converted value's type was lost. The HTML scenario report showsconverted_value or original_valuebut switched on that originaldata_type, so a converted value was labelled with the wrong type:maintext"draw a cat" →image_pathmedia/cat.png<div class="piece">media/cat.png</div><div class="piece media">[image_path] media/cat.png</div>image_pathmedia/dog.png→text"a dog on a log"<div class="piece media">[image_path] a dog on a log</div><div class="piece">a dog on a log</div>pyrit/output/conversation/json.py: each non-reasoning piece now also carriesoriginal_value_data_typeandconverted_value_data_type.data_typestays as it was (the original type), so existing consumers of the JSON keep working, and reasoning pieces are unchanged.pyrit/output/scenario_result/html.py: the template computes the type of the value it displays:converted_value_data_typewhen it shows the converted value, otherwisedata_type. It falls back todata_typewhen a payload has noconverted_value_data_type, so older JSON documents render as before.Note: #3771 (for #3741) edits the same two template lines, changing which value is shown when
converted_valueis empty. The two changes are independent, and whichever merges second needs a small rebase. I'm happy to do that on this side.Tests and Documentation
New tests:
tests/unit/output/conversation/test_json.py::test_build_async_exports_original_and_converted_data_types: a text → image_path piece exports both types, anddata_typestaystext.tests/unit/output/scenario_result/test_html.py::test_html_report_labels_converted_media_by_converted_type: a text → image_path piece renders as[image_path] media/cat.png.tests/unit/output/scenario_result/test_html.py::test_html_report_renders_converted_text_of_media_original_as_text: an image_path → text piece renders as plain text with no[image_path]label.tests/unit/output/scenario_result/test_html.py::test_html_report_labels_converted_types_from_real_builders: both directions built from realMessagePieces byJsonConversationPrinterand rendered to HTML.All 4 fail on
main(f3a88da) and pass with the fix.tests/unit/outputpasses (395 tests), and so do the JSON export consumers intests/unit/cli/test_output.py,test_scenario_report_parity.py,test_results.pyandtests/unit/analytics/test_scenario_statistics_parity.py(119 tests).ruff check,ruff format --checkandty checkare clean on the changed files.No documentation changes: the docs don't list the per-piece JSON fields. I didn't run JupyText because no notebooks are affected.
This change was prepared with the help of Claude Code (an AI coding assistant). I reviewed the change and the tests.
🤖 Generated with Claude Code