Repository navigation
Python: fix(contents): accept RFC 6838 media types and parameters in data URIs - #14663
Chen Yufeiyang (feiiiiii5) wants to merge 2 commits into
Conversation
The data URI parser only allowed letters and hyphens in the media type and treated everything after the first semicolon as the data format, so image/svg+xml was rejected and charset=utf-8;base64 left the base64 payload undecoded. Parse the media type and parameters with the RFC 6838 token character set and keep the optional data format as a token.
There was a problem hiding this comment.
🟡 Changes recommended
The revised parser rejects valid parameter characters and shorthand URIs that omit the media type.
2 open issues
What changed in this PR
This PR updates Python’s data URI parser to accept additional media-type characters and correctly separate parameters from the base64 marker.
Changes:
- Expands the parser’s media-type and parameter character classes.
- Adds regression tests for SVG, vendor media types, and charset parameters before base64.
- Tests
ImageContentconstruction from an SVG data URI.
| File | Description |
|---|---|
python/tests/unit/contents/test_image_content.py |
Adds SVG data URI construction coverage. |
python/tests/unit/contents/test_data_uri.py |
Adds media-type and parameter parsing cases. |
python/semantic_kernel/contents/utils/data_uri.py |
Revises the data URI parsing expression. |
Review effort: Balanced
| # the data format as a token prevents unparsed parameters from being silently | ||
| # treated as part of the format (for example "charset=utf-8;base64"). | ||
| pattern = ( | ||
| r"(((?P<mime_type>[a-zA-Z0-9!#$&^_.+-]+/[a-zA-Z0-9!#$&^_.+-]+)" |
There was a problem hiding this comment.
Fixed in b0e34c5: the optional media type is now separate from the parameters group. Tests cover data:;charset=utf-8,Hello and its base64 form, alongside explicit-media-type controls. The omitted-media-type cases failed before this fix. The complete contents suite passes 407 tests.
| # treated as part of the format (for example "charset=utf-8;base64"). | ||
| pattern = ( | ||
| r"(((?P<mime_type>[a-zA-Z0-9!#$&^_.+-]+/[a-zA-Z0-9!#$&^_.+-]+)" | ||
| r"(?P<parameters>(;[a-zA-Z0-9!#$&^_.+-]+=+[a-zA-Z0-9!#$&^_.+-]+)*))?" |
There was a problem hiding this comment.
Fixed in b0e34c5: parameters use the broader token alphabet separately from the media type alphabet. Tests include *, apostrophe, ~, %20, and %3B in parameter values, and assert the parameter remains parsed rather than being silently treated as data_format. All five failed before the fix; the complete contents suite now passes 407 tests.

Motivation and Context
Valid RFC 2397 data URIs are rejected or silently mis-decoded when the media type contains
+(for exampleimage/svg+xml) or when a parameter precedesbase64(for exampledata:text/plain;charset=utf-8;base64,...).ImageContent(data_uri="data:image/svg+xml;base64,PHN2Zy8+")raisesContentInitializationError: Invalid data uri format.DataUri.from_data_uri("data:text/plain;charset=utf-8;base64,SGVsbG8=")does not raise, but returns the base64 text as bytes instead of decoding it, becausecharset=utf-8;base64is treated as one data-format string. Both are valid data URIs, and the second is a silent data-corruption path.Description
The parser now uses RFC 6838/RFC 2045 token characters for the media type and parameters, and keeps the optional data format as a token. This accepts
image/svg+xml, vendor types such asapplication/vnd.ms-excel, and;charset=utf-8;base64without treating the parameter as part of the base64 marker.Tests added for the media-type
+, a parameter beforebase64, a vendor media type, andImageContentconstruction from an SVG data URI. On the base commit483ffb4, the four new tests fail; on this branch the related contents/prompt/template/text suites are 1,111 passed.ruff checkandruff format --checkare clean.Contribution Checklist
ruff checkandruff format --checkpass on the changed files.