fix: account for inline image payload in context budget

[iuiu-py]

Fixes #10092

What Changed

• Estimate base64 data URI image payloads as context occupancy, including when trusted provider usage is available.
• Propagate MemoryError and OSError(ENOMEM) instead of retrying every fallback provider with the same oversized payload.
• Preserve exception type in agent pipeline logs and user-facing errors, even when str(exception) is empty.

Why

Persisted inline images could grow far beyond the fixed image estimate. The context compressor could not see their real volume, allowing history to exhaust process memory. Once memory exhaustion occurred, broad fallback handling repeated the same oversized request and the final MemoryError appeared as an empty message.

Testing

• Added tests for inline payload estimation, trusted-usage overflow, fatal resource errors bypassing fallback, and exception formatting.
• uv run pytest tests/agent tests/test_error_formatting.py tests/test_tool_loop_agent_runner.py tests/test_process_stage_images.py -q: 347 passed.
• uv run ruff format --check .
• uv run ruff check .

Summary by Sourcery

Prevent oversized persisted inline images from exhausting context memory and ensure fatal resource errors remain visible without triggering unsafe provider fallbacks.

Bug Fixes:

• Account for large inline base64 image payloads when estimating context occupancy, including alongside trusted provider token usage.
• Avoid retrying memory-exhaustion errors through fallback providers.
• Preserve exception types in logs and user-facing errors when exceptions have no message.

Enhancements:

• Centralize exception formatting to provide consistent type-and-detail error messages.

Tests:

• Add coverage for inline image estimation, trusted-usage overflow, resource-exhaustion fallback behavior, and exception formatting.

[sourcery-ai]

Hey - I've found 1 issue

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="astrbot/core/agent/context/token_counter.py" line_range="93" />
<code_context>
+
+    def _estimate_image_tokens(self, url: str) -> int:
+        header, separator, payload = url.partition(",")
+        if separator and header.startswith("data:") and ";base64" in header:
+            # Inline payloads are persisted with history. Their character
+            # volume is a conservative proxy for the memory needed to resend
+            # them and must trigger compression before the process is starved.
+            return IMAGE_TOKEN_ESTIMATE + int(len(payload) * 0.3)
+        return IMAGE_TOKEN_ESTIMATE
</code_context>
<issue_to_address>
**issue (bug_risk):** Valid data URI images whose scheme or base64 marker uses different casing are treated as fixed-size remote images, so their large inline payload is omitted from context occupancy and compression is not triggered.

**Triggers:** When persisted image URLs use a case-insensitive form such as `DATA:image/png;BASE64,...`.

**Suggested fix:** Normalize the URI header to lowercase, or use case-insensitive matching for the `data:` scheme and `base64` marker.

```suggestion
        header = header.lower()
        if separator and header.startswith("data:") and ";base64" in header:
```
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 1 finding to address first, and if the inline-image estimate is wrong, context compression can be delayed and persisted media can drive the process into memory exhaustion; the resulting outage would already have occurred when the change is reverted. The no-fallback handling also changes resource-exhaustion behavior, though reverting restores the prior retry path.

Blocking findings: astrbot/core/agent/context/token_counter.py:93

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

[sourcery-ai]

issue (bug_risk): Valid data URI images whose scheme or base64 marker uses different casing are treated as fixed-size remote images, so their large inline payload is omitted from context occupancy and compression is not triggered.

Triggers: When persisted image URLs use a case-insensitive form such as DATA:image/png;BASE64,....

Suggested fix: Normalize the URI header to lowercase, or use case-insensitive matching for the data: scheme and base64 marker.

Suggested change

	if separator and header.startswith("data:") and ";base64" in header:
	header = header.lower()
	if separator and header.startswith("data:") and ";base64" in header: