Skip to content

FIX Export the converted data type in JSON conversations and label HTML report pieces by it - #3782

Open
Shivam Sharma (ShivamSharma43) wants to merge 1 commit into
microsoft:mainfrom
ShivamSharma43:fix/conversation-export-converted-type
Open

Shivam Sharma (ShivamSharma43) wants to merge 1 commit into
microsoft:mainfrom
ShivamSharma43:fix/conversation-export-converted-type

Conversation

@ShivamSharma43

Copy link
Copy Markdown
Contributor

Description

Fixes #3742.

JsonConversationPrinter._build_piece_async exported each piece with a single data_type, set to original_value_data_type, next to both original_value and converted_value. The converted value's type was lost. The HTML scenario report shows converted_value or original_value but switched on that original data_type, so a converted value was labelled with the wrong type:

Piece (original → converted) HTML on main HTML with this change
text "draw a cat" → image_path media/cat.png <div class="piece">media/cat.png</div> <div class="piece media">[image_path] media/cat.png</div>
image_path media/dog.png → text "a dog on a log" <div class="piece media">[image_path] a dog on a log</div> <div class="piece">a dog on a log</div>
  • pyrit/output/conversation/json.py: each non-reasoning piece now also carries original_value_data_type and converted_value_data_type. data_type stays as it was (the original type), so existing consumers of the JSON keep working, and reasoning pieces are unchanged.
  • pyrit/output/scenario_result/html.py: the template computes the type of the value it displays: converted_value_data_type when it shows the converted value, otherwise data_type. It falls back to data_type when a payload has no converted_value_data_type, so older JSON documents render as before.

Note: #3771 (for #3741) edits the same two template lines, changing which value is shown when converted_value is empty. The two changes are independent, and whichever merges second needs a small rebase. I'm happy to do that on this side.

Tests and Documentation

New tests:

  • tests/unit/output/conversation/test_json.py::test_build_async_exports_original_and_converted_data_types: a text → image_path piece exports both types, and data_type stays text.
  • tests/unit/output/scenario_result/test_html.py::test_html_report_labels_converted_media_by_converted_type: a text → image_path piece renders as [image_path] media/cat.png.
  • tests/unit/output/scenario_result/test_html.py::test_html_report_renders_converted_text_of_media_original_as_text: an image_path → text piece renders as plain text with no [image_path] label.
  • tests/unit/output/scenario_result/test_html.py::test_html_report_labels_converted_types_from_real_builders: both directions built from real MessagePieces by JsonConversationPrinter and rendered to HTML.

All 4 fail on main (f3a88da) and pass with the fix. tests/unit/output passes (395 tests), and so do the JSON export consumers in tests/unit/cli/test_output.py, test_scenario_report_parity.py, test_results.py and tests/unit/analytics/test_scenario_statistics_parity.py (119 tests). ruff check, ruff format --check and ty check are clean on the changed files.

No documentation changes: the docs don't list the per-piece JSON fields. I didn't run JupyText because no notebooks are affected.

This change was prepared with the help of Claude Code (an AI coding assistant). I reviewed the change and the tests.

🤖 Generated with Claude Code

…ieces by it

JsonConversationPrinter exported only the original value's data type, so the
HTML scenario report labelled a converted value by its original type: a
text-to-image conversion rendered the image path as plain text, and an
image-to-text conversion rendered the text as an [image_path] placeholder.

Export original_value_data_type and converted_value_data_type next to the
existing data_type (kept as the original type), and have the HTML template
switch on the type of the value it displays.

Fixes microsoft#3742

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 1, 2026 12:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@ShivamSharma43

Copy link
Copy Markdown
Contributor Author

The only red check is build-book / Build latest, which hit the 6h job timeout without failing a step. The unit tests and the frontend tests pass. Could someone rerun it when convenient? Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JSON conversation export omits the converted data type, making converted media appear as text in HTML

2 participants