Add a vision bridge for text-only main models

[ihubanov]

Some of the best models for long coding sessions are text-only — you pick them for the context window, then hit a wall the moment an image lands in the conversation and the model throws "not multimodal" and wedges the turn. This adds a vision bridge so a text-only main model can work in a multimodal session when the gateway also serves a multimodal model.

What it does

Set a vision model (saved in [vision] or via SETFREE_VISION_MODEL) and SetFree starts a local proxy the CLI talks to instead of the gateway directly. The proxy forwards everything unchanged except image content blocks: each is captioned by the vision model and replaced with text before the request reaches the main model. The text model never sees a raw image block, so it never errors. Captions are cached to disk (append-only NDJSON, keyed by image bytes) so an image is described once ever, not once per turn.

This is the one deliberate exception to SetFree's "step aside, never sit in the request path" rule. It's strictly opt-in — no vision model means no proxy and a launch byte-identical to today. The proxy is localhost, lives only as long as the CLI does, and never touches auth or licensing. The CLI binary is still the one you installed, unmodified.

How

The bridge ports the captioning approach from a forked-CLI implementation of the same idea, adapted to SetFree's "never modify the CLI binary" constraint: since there's no hook to reach inside the request the unmodified binary builds, the proxy sits in the request path instead. It handles Anthropic image-block detection both at the top level of a message's content and nested one level inside tool_result content (the easy-to-miss case that reproduces the not-multimodal error), forces chat_template_kwargs.enable_thinking=false on the captioning call (a vLLM passthrough that keeps a reasoning vision model like Qwen3.5 from burning its token budget on thinking and returning empty), and prewarms captions concurrently before the serial rewrite (each caption is an ~8s round-trip; done serially it looks like a hang).

Lifecycle: the vision path runs the CLI as a child (launcher.Run, new — signal-forwarded) instead of exec-replacing (launcher.Launch), so the parent survives to tear the proxy down when the CLI exits. An idle-timeout (default 1h) is only a backstop for a proxy orphaned by an abnormal parent death; it never culls a live session that's between turns. The non-vision path is unchanged.

New internal/vision/ package; config.Settings gains a [vision] table (API key in the secrets store, never in config.toml); launcher gains Run alongside Launch; app/launch.go starts the proxy between Resolve and Build (adapters untouched). The proxy is the setfree binary itself in a vision-proxy subcommand — no new artifact.

Scope

Covers the Anthropic /v1/messages path — the claude and vscode adapters. Codex uses a different wire format (OpenAI Responses API) and will follow in a separate PR.

Config

[vision]
model = "qwen3.5"        # presence enables the bridge
# base_url and api_key optional, default to the main gateway

Env overrides: SETFREE_VISION_MODEL, SETFREE_VISION_BASE_URL, SETFREE_VISION_API_KEY, SETFREE_VISION_OFF, SETFREE_VISION_CONCURRENCY, SETFREE_VISION_IDLE_TIMEOUT.

Limitation

The proxy can't keep the original image for an on-demand "look closer" re-query — the text model works from the caption. That's the trade for staying out of the binary.

Testing

Unit tests cover block detection (top-level + nested in tool_result), the enable_thinking=false passthrough, caption caching across calls, failure fallback, image-free passthrough, proxy rewrite + header forwarding + invalid-JSON passthrough, and config precedence/round-trip. go build, go vet, gofmt, and go test ./... are all clean, no new dependencies (stdlib only).

Also verified end-to-end against a real gateway: a /v1/messages request with a text-only main model (nvidia/GLM-5.2-NVFP4) and a base64 image, sent through the proxy, was captioned by Qwen3.5-397B-A17B-NVFP4 and returned a correct answer ("Red" for a red square) — no not-multimodal error.

[ihubanov]

First PR here, so GitHub is holding the CI workflow runs until a maintainer approves them. The checks are standard (build/vet/test on ubuntu/macos/windows + gofmt) — I've run them locally against all three platforms and they're green. Happy to adjust anything; just let me know.