Skip to content

POC: Send native images through Claude sessions - #1503

Draft
smsunarto wants to merge 1 commit into
get-bb:mainfrom
smsunarto:bb/investigate-bb-generation-quality-thr_kcvtdzmht5
Draft

POC: Send native images through Claude sessions#1503
smsunarto wants to merge 1 commit into
get-bb:mainfrom
smsunarto:bb/investigate-bb-generation-quality-thr_kcvtdzmht5

Conversation

@smsunarto

Copy link
Copy Markdown
Contributor

Summary

  • send supported local PNG, JPEG, GIF, and WebP attachments as native Claude image content
  • send HTTP(S) image attachments as native URL image content
  • validate local image data and fall back to Read-tool markers for invalid, unsupported, oversized, or over-budget inputs
  • retain one base64 image budget across grouped messages, turns, and SDK stream recovery
  • package Sharp for the host daemon, bb-app, and production bridge bundles

POC status

This is a work-in-progress proof of concept for review. Known follow-up items:

  • requests with more than 20 native images need the stricter Anthropic dimension policy or a native-image count cap
  • animated GIF/WebP validation currently decodes all pages, which can reject valid animations when Sharp reports stacked frame dimensions; Claude uses only the first frame
  • no billed live-Claude smoke test has run

Validation

  • pnpm exec turbo run test typecheck --filter=@bb/agent-runtime --force — 975 tests passed
  • pnpm exec turbo run build --filter=@bb/host-daemon --filter=bb-app --force
  • git diff --check

AGENT GENERATED: by GPT-5.4

Send supported local and remote images as native Claude content blocks, with image validation and retained session budgeting. Keep unsupported or over-budget attachments available through Read-tool markers.

Amp-Thread-ID: https://ampcode.com/threads/T-019ff92d-a8de-71fb-9328-75715f4e9efa
Co-authored-by: Amp <amp@ampcode.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant