Repository navigation
Conversation
A sibling of ollama-worker that runs Apple's on-device 3B FoundationModel via FoundationModels — a local, network-free, key-free LLM. Registers with humd over thrum, advertises model `apple-3b-on-device`, and turns chi:"prompt" into chunk/finish tones. The Rust side (src/) mirrors ollama-worker but, instead of HTTP to Ollama, shells out to a small Swift bridge (bridge/gen) that wraps FoundationModels and streams NDJSON deltas. The only Apple-specific code in the tree is bridge/gen.swift (+ probe.swift); no Apple symbols enter the Rust side, and the bridge is mac-gated. On boot a capability probe keeps the hive present-but-dormant when Apple Intelligence isn't available. Isolated prototype — verified in isolation on macOS 26 (M5 Max): probe reports available, real on-device deltas stream, and the Rust worker compiles + connects to a live humd advertising the model. Not yet co-installed into the mac desktop bundle.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
hives/apple-fm-worker/— a sibling ofollama-workerthat runs Apple's on-device 3B FoundationModel (viaFoundationModels) as a local, network-free, key-free LLM.What it does
humdover thrum and advertises modelapple-3b-on-device, turningchi:"prompt"→chunk/finish— the same contract ollama-worker fulfills for a remote Ollama endpoint.src/) mirrorsollama-workerbut, instead of HTTP, it shells out to a small Swift bridge (bridge/gen) that wrapsFoundationModelsand streams NDJSON deltas.gen --probekeeps the hive present-but-dormant when Apple Intelligence isn't available (off / asset not downloaded / unsupported region), mirroring how ollama-worker degrades when Ollama isn't running.Lock-in is contained
The only Apple-specific code in the whole tree is
bridge/gen.swift(+probe.swift). No Apple symbols enter the Rust side; the bridge is#[cfg(target_os = "macos")]-gated, so non-mac builds of hum never reference it.Verified in isolation (macOS 26, M5 Max 128GB)
gen --probe→{"event":"probe","available":true}genwith a prompt → real on-device deltas +finishhumd, and hands itshelloadvertisingapple-3b-on-deviceNot yet
An end-to-end prompt routed through
humdto the worker and back — that's the last gate before co-installation into the mac desktop bundle. Left as an isolated prototype for that reason.