Python: feat(python): Add MCP client OTel spans per GenAI semantic conventions

[TaoChenOSU]

Motivation and Context

Implements MCP client spans per the OTel GenAI Semantic Conventions for MCP, giving users visibility into MCP operations (initialize, tool listing, tool calls, prompt operations) through OpenTelemetry tracing.

Closes #3624
Closes #4697

Description

Adds CLIENT spans for five MCP operations:

Operation	Span Name	Notes
initialize	initialize	Captures MCP session setup; sets mcp.protocol.version from result
tools/list	tools/list	Per-page tool listing
prompts/list	prompts/list	Per-page prompt listing
tools/call	tools/call {tool_name}	Nested under execute_tool when called via FunctionTool; sets error.type="tool_error" for isError results per spec
prompts/get	prompts/get {prompt_name}	Prompt retrieval

Span attributes follow the MCP semantic conventions:

• Required: mcp.method.name
• Conditional: error.type, gen_ai.tool.name, gen_ai.prompt.name
• Recommended: gen_ai.operation.name, mcp.protocol.version, mcp.session.id, network.transport, server.address, server.port

Transport-specific attributes per subclass:

Subclass	network.transport	network.protocol.name	server.address/port
MCPStdioTool	pipe	—	—
MCPStreamableHTTPTool	tcp	http	from URL
MCPWebsocketTool	tcp	websocket	from URL

All span creation is gated behind OBSERVABILITY_SETTINGS.ENABLED.

Files changed:

• observability.py — New OtelAttr entries for MCP attributes, mcp_tracer(), create_mcp_client_span(), set_mcp_span_error() helpers
• _mcp.py — Instrumented all 5 operations, _mcp_base_span_attributes() with transport overrides, extracted _call_tool_with_retries()
• _tools.py — No changes (reverted earlier modifications)
• test_mcp_observability.py — 15 new tests covering all MCP span operations

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the Contribution Guidelines
• All unit tests pass, and I have added new tests where possible
• Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

[github-actions]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/core/agent_framework
_mcp.py	853	65	92%	166, 185, 338, 398–399, 528, 590, 603, 627–628, 647–650, 652–653, 657, 683, 716–718, 720, 773–775, 834–835, 1018, 1059–1060, 1073, 1076, 1085–1086, 1091–1092, 1098, 1146–1147, 1161–1162, 1171–1172, 1177–1178, 1184, 1243, 1246, 1273, 1296–1300, 1344, 1350, 1449, 1456, 1458, 1520, 1835–1836, 2016–2017, 2035
observability.py	853	61	92%	391, 393–394, 397, 400, 403–404, 409–410, 416–417, 423–424, 431, 433–435, 438–440, 445–446, 452–453, 459–460, 467, 644–645, 844, 848–850, 852, 856–857, 861, 899, 901, 912–914, 916–918, 922, 930, 1054–1055, 1290, 1550–1551, 1655, 1779, 1820–1821, 1964, 2155, 2352, 2570, 2572
TOTAL	38132	4420	88%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
7617	34 💤	0 ❌	0 🔥	2m 0s ⏱️

[Copilot]

Pull request overview

Adds Python MCP client tracing that follows the OpenTelemetry GenAI MCP semantic conventions, aiming to consistently emit (or enrich) CLIENT spans for MCP operations and preserve MCP-specific error typing (e.g., error.type="tool_error").

Changes:

• Introduces MCP span helper utilities in observability.py (create/enrich/error helpers + active tool span contextvar).
• Instruments MCP client operations in _mcp.py (initialize, tools/list, prompts/list, tools/call, prompts/get) including transport attributes and retries for tools/call.
• Updates tool invocation tracing in _tools.py to expose the active execute_tool span for MCP enrichment and to preserve MCP error typing; adds a dedicated MCP observability test suite.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
python/packages/core/agent_framework/observability.py	Adds MCP tracing helpers and MCP semantic convention attributes.
python/packages/core/agent_framework/_mcp.py	Instruments MCP operations with CLIENT spans, enrichment logic, transport attrs, and error helpers.
python/packages/core/agent_framework/_tools.py	Publishes the active execute_tool span via contextvar and preserves MCP-specific error.type.
python/packages/core/tests/core/test_mcp_observability.py	Adds tests validating span creation/enrichment, error typing, and transport attributes.

[github-actions]

Automated Code Review

Reviewers: 3 | Confidence: 86%

✓ Security Reliability

The PR is well-structured from a security and reliability standpoint. The contextvar _ACTIVE_TOOL_EXECUTION_SPAN lifecycle is properly managed (set after span creation, reset in finally), URL parsing is defensively wrapped in try/except, span creation is gated behind OBSERVABILITY_SETTINGS.ENABLED, and the _otel_error_type propagation mechanism correctly preserves MCP-specific error types through FunctionTool.invoke() despite capture_exception overwriting. No injection risks, secrets, unsafe deserialization, or resource leaks were identified. There is one minor observability gap: reconnection failure paths in both _call_tool_with_retries and get_prompt do not call set_mcp_span_error, meaning spans created via the new-span path (direct calls without FunctionTool wrapper) will end without error status when reconnection fails, since set_status_on_exception=False.

✓ Test Coverage

The PR adds 19 well-structured tests covering the happy paths for all 5 MCP span operations, span enrichment vs creation, transport attributes, and the observability-disabled gate. The test assertions are meaningful (checking span kind, attributes, status codes) and mocks are appropriate. However, there are notable test coverage gaps: the get_prompt error paths (3 explicit set_mcp_span_error calls) have zero test coverage, and the _call_tool_with_retries non-tool_error error paths (generic exceptions, exhausted retries) are also untested. Since the tools/call tool_error path IS tested, the asymetry with prompts/get error handling is a meaningful gap.

✓ Design Approach

The MCP span coverage looks broadly aligned with the new tests, but there is one design regression in the tools/call enrichment path: when an MCP tool is exposed through a prefixed/local FunctionTool name, the new code overwrites the existing execute_tool span's gen_ai.tool.name with the remote MCP name, so the span no longer identifies the tool the framework actually executed.

---

Automated review by TaoChenOSU's agents