straylight

Infrastructure for the sprawl.

  ╷┌─┐┐ ┬┬  ┌─┐┌┐┐┌─┐  ┐─┐┬  o┬─┐┌─┐
  ││─┤└┬┘│  ├─ │││├─   └─┐│  ││ │├─ 
╶─┘┘ ┴ ┴ ┘─┘┴─┘┘└┘┴─┘  ──┘┘─┘┘┘─┘┴─┘

A square of cyberspace directly in front of him flipped sickeningly 
and he found himself in a pale blue graphic that seemed to represent 
a very spacious apartment, low shapes of furniture sketched in hair-fine 
lines of blue neon. A woman stood in front of him, a sort of glowing 
cartoon squiggle of a woman, the face a brown smudge.

"I'm Slide,” the figure said, hands on its hips, “Jaylene. You don't fuck 
with me. Nobody in L.A.” she gestured, a window suddenly snapping into existence 
behind her “fucks with me. You got that?”

What

jaylene-slide is a console cowboy that jacks into OpenAI-compatible inference endpoints (Baseten, Together, Fireworks, etc.), parses their 650-byte-per-token SSE/JSON garbage, and emits clean SIGIL binary frames over ZMQ.

The wire format is ~1.5 bytes per token average. The frames are semantically chunked—tool calls, thinking blocks, code blocks arrive complete or not at all. No more regex racing against the stream. No more partial JSON parsing. No more hoping the UTF-8 didn't split.

Why

Every token from Baseten looks like this:

data: {"id":"chatcmpl-cf31b079a80d4f888a2a09e77d128196","choices":[{"index":0,"delta":{"content":" const","function_call":null,"tool_calls":[],"role":"assistant","refusal":null}}],"created":1770306640,"model":"moonshotai/Kimi-K2.5","service_tier":null,"system_fingerprint":null,"object":"chat.completion.chunk","usage":{"prompt_tokens":22,"completion_tokens":157,"total_tokens":179,"prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0},"completion_tokens_details":{"accepted_prediction_tokens":0,"audio_tokens":0,"reasoning_tokens":0,"rejected_prediction_tokens":0}}}

That's 650 bytes for the token " const".

jaylene-slide emits:

[0x2A]

One byte. Because " const" is hot token #42.

Quick Start

# Set your API key
export JAYLENE_API_KEY="your-baseten-api-key"

# Jack in
nix run github:user/straylight#jaylene-slide -- \
  https://inference.baseten.co/v1/chat/completions \
  --model moonshotai/Kimi-K2.5

# Prompts on stdin, frames on zmq tcp://*:5555

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                         Provider                                │
│  (Baseten, Together, Fireworks, local vLLM, etc.)               │
└─────────────────────────────────┬───────────────────────────────┘
                                  │ SSE/JSON (~650 bytes/token)
                                  ▼
┌─────────────────────────────────────────────────────────────────┐
│                      jaylene-slide                              │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐              │
│  │ Jaylene.Jack│→ │Jaylene.Parse│→ │ Jaylene.Wire│              │
│  │ (HTTP/TLS)  │  │ (Megaparsec)│  │ (Frames)    │              │
│  └─────────────┘  └─────────────┘  └─────────────┘              │
└─────────────────────────────────┬───────────────────────────────┘
                                  │ SIGIL frames (~1.5 bytes/token)
                                  ▼
┌─────────────────────────────────────────────────────────────────┐
│                          ZMQ PUB                                │
│                      tcp://*:5555                               │
└─────────────────────────────────┬───────────────────────────────┘
                                  │
                                  ▼
                            [ clients ]
                       (opencode, your app, etc.)

Packages

Package	Description
jaylene-slide	The ingress adapter binary
jaylene-wire	Wire format types and encoding
jaylene-decode	Client library for consuming frames

Wire Format

SIGIL frames use a distribution-derived encoding:

0xxxxxxx        Hot token (ID 0-126 in lower 7 bits)
10xxxxxx        Extended token (varint follows)
1100xxxx        Stream control:
                  0xC0 = CHUNK_END
                  0xC1 = TOOL_CALL_START
                  0xC2 = TOOL_CALL_END
                  0xC3 = THINK_START
                  0xC4 = THINK_END
                  0xC5 = CODE_BLOCK_START
                  0xC6 = CODE_BLOCK_END
                  0xCF = STREAM_END

Hot tokens are the 127 most frequent tokens for the model. Everything else is escape + varint. Control frames mark semantic boundaries.

---

CLAUDE: Instructions to Make This Operational

Phase 1: Core Compilation (Do This First)

1.1 Verify the skeleton compiles

cd straylight
nix develop  # or cabal build all

The code in this tarball is a sketch. You need to:

1. Fix imports — many modules reference each other but I wrote them stream-of-consciousness. Check:

   • Jaylene.Wire.Frame exports what Jaylene.Chunk.StateMachine imports
   • Jaylene.Parse (was Sigil.Provider.SSE) exports what Main imports
   • Jaylene.HotTable (was Sigil.Tokenizer.HotTable) is wired up

2. Fix the tokenizer — the code assumes a tokenize :: Text -> [Word32] function exists. Options:

   • Use tokenizers hackage package (Rust FFI to HuggingFace tokenizers)
   • Write a pure Haskell BPE impl (slower but no FFI)
   • For testing: use the mock mockTokenize = map (fromIntegral . fromEnum) . T.unpack

3. Fix ZMQ bindings — we use zeromq4-haskell. Make sure:

   import System.ZMQ4.Monadic as ZMQ
   -- or
   import System.ZMQ4 as ZMQ

   The monadic vs non-monadic API differs slightly.

1.2 Get one end-to-end working

Simplest test:

-- Hardcode everything, no CLI parsing
main = do
  let endpoint = "https://inference.baseten.co/v1/chat/completions"
      apiKey = "your-key-here"
      model = "moonshotai/Kimi-K2.5"
  
  -- Just print tokens, no encoding yet
  streamCompletion endpoint apiKey model "Hello" $ \delta ->
    putStrLn $ "DELTA: " <> T.unpack delta

If that prints deltas, the HTTP/SSE path works.

1.3 Test the parser in isolation

cabal repl jaylene-wire
> import Jaylene.Parse
> parseSSE "data: {\"choices\":[{\"delta\":{\"content\":\"hello\"}}]}\n"
Right [SSEData "..."]
> extractDelta "{\"choices\":[{\"delta\":{\"content\":\"hello\"}}]}"
Just "hello"

If that works, Megaparsec is happy.

Phase 2: Hot Table

2.1 Generate a hot table for your model

You need token frequency data. Options:

Option A: Profile real traffic

from collections import Counter
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("moonshotai/Kimi-K2.5")
counter = Counter()

# Feed your actual prompts/completions
for text in your_corpus:
    tokens = tok.encode(text)
    counter.update(tokens)

# Top 127
hot_tokens = [tok_id for tok_id, _ in counter.most_common(127)]

Option B: Use the tokenizer's vocab frequency (approximation)

# Many tokenizers have frequency info, or just use token ID order
# as a proxy (lower IDs often = more frequent in BPE)
hot_tokens = list(range(127))  # crude but works for testing

2.2 Write the hot table file

Format:

[4 bytes] vocab_size (little-endian u32)
[127 * 4 bytes] hot token IDs (little-endian u32 each)
[32 bytes] BLAKE3 hash of the above

import struct
from hashlib import blake2b  # or use blake3 library

def write_hot_table(path, vocab_size, hot_tokens):
    assert len(hot_tokens) == 127
    
    data = struct.pack('<I', vocab_size)
    for tok_id in hot_tokens:
        data += struct.pack('<I', tok_id)
    
    # BLAKE3 hash (use blake3 library, or blake2b as standin)
    h = blake3.blake3(data).digest()
    
    with open(path, 'wb') as f:
        f.write(data)
        f.write(h)

2.3 Build boundary token set

Which tokens are chunk boundaries? At minimum:

• Newline tokens
• Semicolon tokens
• Close brace tokens

boundary_chars = ['\n', ';', '}', ')', ']']
boundary_tokens = set()
for c in boundary_chars:
    boundary_tokens.update(tok.encode(c))

Store as a bitset or serialize similarly to hot table.

Phase 3: ZMQ Integration

3.1 Basic pub/sub test

-- Publisher (in jaylene-slide)
ZMQ.runZMQ $ do
  pub <- ZMQ.socket Pub
  ZMQ.bind pub "tcp://*:5555"
  liftIO $ forever $ do
    ZMQ.send pub [] "test frame"
    threadDelay 1000000

-- Subscriber (separate process)
ZMQ.runZMQ $ do
  sub <- ZMQ.socket Sub
  ZMQ.connect sub "tcp://localhost:5555"
  ZMQ.subscribe sub ""
  forever $ do
    msg <- ZMQ.receive sub
    liftIO $ print msg

3.2 Wire up frame emission

Replace the test string with actual encoded frames:

streamAndEncode conn hotTable prompt $ \frame ->
  ZMQ.send pub [] (frameBytes frame)

Phase 4: Testing

4.1 Run the property tests

cabal test all --test-show-details=direct

The adversarial tests MUST pass. If the parser crashes on garbage input, you have a bug.

4.2 Test against real Baseten traffic

# Terminal 1: run jaylene-slide
nix run .#jaylene-slide -- https://inference.baseten.co/v1/chat/completions -v

# Terminal 2: subscribe to frames
nix run .#jaylene-decode -- tcp://localhost:5555

# Terminal 1: type a prompt
What is 2+2?

# Terminal 2: should see decoded chunks

4.3 Measure compression

# Log raw SSE bytes vs emitted frame bytes
# Should see ~50-100x reduction

Phase 5: Performance (Later)

5.1 Profile

cabal run jaylene-slide -- +RTS -s -RTS ...

Look for:

• GC time (should be <5% with -A64m -I0)
• Allocation rate (should be low, we're reusing buffers)

5.2 Fork blockio-uring for io_uring sockets

This is Phase 2. For now, epoll via GHC's IO manager is fine. The bottleneck is Baseten's token generation rate, not your syscalls.

When ready:

1. Fork well-typed/blockio-uring
2. Add IORING_OP_RECV, IORING_OP_SEND
3. Add socket support to URing type
4. Replace http-client with raw io_uring sockets

File Checklist

After extracting, verify these exist and compile:

• flake.nix — nix build/run/develop
• jaylene-slide/jaylene-slide.cabal
• jaylene-slide/src/Main.hs
• jaylene-slide/src/Jaylene/Jack.hs
• jaylene-slide/src/Jaylene/Parse.hs
• jaylene-slide/src/Jaylene/Wire.hs
• jaylene-slide/src/Jaylene/Emit.hs
• jaylene-slide/src/Jaylene/HotTable.hs
• jaylene-wire/jaylene-wire.cabal
• jaylene-wire/src/Jaylene/Wire/Frame.hs
• jaylene-wire/src/Jaylene/Wire/Encode.hs
• jaylene-wire/src/Jaylene/Wire/Decode.hs
• jaylene-wire/src/Jaylene/Wire/Varint.hs
• jaylene-decode/jaylene-decode.cabal
• test/Spec.hs
• test/SSESpec.hs
• test/StateMachineSpec.hs
• test/FrameSpec.hs
• test/RoundtripSpec.hs
• test/AdversarialSpec.hs

Known Gaps in This Sketch

1. Tokenizer integration — mocked, needs real HF tokenizers FFI or pure impl
2. Sequence matching — checkTransition only checks first token of <think>, needs full sequence matching across token boundaries
3. Buffer overflow — FrameBuilder doesn't bounds-check, add it
4. Reconnection — no retry logic on HTTP disconnect
5. Multiple concurrent streams — single-threaded for now
6. Metrics — no prometheus/statsd export yet

The Goal

When this works:

# Baseten spews 650 bytes/token
# jaylene-slide emits ~1.5 bytes/token
# opencode receives semantically complete chunks
# No more parsing. No more regex. No more prayer.

Console cowboy for the sprawl.