Precise prompt token counting

[Korolev-Oleg]

Problem

Accurate token counting is required in projects that use large prompts whose size depends on dynamic context.

Without a way to precisely determine the token count of such prompts, runtime errors like exceeds the model’s context window can occur.

Currently, avoiding these errors requires temporary and unreliable workarounds that only estimate the context length.

Solution

In macOS 26.4+, it is possible to call

SystemLanguageModel.default.tokenCount(for: Prompt("..."))

I created a fork where I implemented the Swift function

@_cdecl("FMSystemLanguageModelTokenCount")
public func FMSystemLanguageModelTokenCount(...) -> Int32 {...}

and exposed it through C bindings.

It is connected in core.py as

SystemLanguageModel.token_count(...) -> int: 
    lib.FMSystemLanguageModelTokenCount(...)