Skip to content

Ulysses attention: repeat KV heads when there are fewer than context ranks - #5779

Draft
parambole wants to merge 1 commit into
parambole/rl-trainer-4600from
parambole/ag-l3-ulysses
Draft

Ulysses attention: repeat KV heads when there are fewer than context ranks#5779
parambole wants to merge 1 commit into
parambole/rl-trainer-4600from
parambole/ag-l3-ulysses

Commits

  1. Commits on Oct 15, 2026