Skip to content

Ulysses attention: repeat KV heads when there are fewer than context ranks - #5779

Draft
parambole wants to merge 1 commit into
parambole/rl-trainer-4600from
parambole/ag-l3-ulysses
Draft

Ulysses attention: repeat KV heads when there are fewer than context ranks#5779
parambole wants to merge 1 commit into
parambole/rl-trainer-4600from
parambole/ag-l3-ulysses

Ulysses attention: repeat KV heads when there are fewer than context …

4dfe6cc
Select commit
Loading
Failed to load commit list.
Google CLA / cla/google succeeded Oct 15, 2026 in 8s

✅ All contributors are covered under a CLA with Google

See https://cla.developers.google.com/ for more info about Google's Contributor License Agreement (CLA).

ℹ️ Googlers: Go here to view more details and manage scans for this pull request.

Details

The following contributors were found for this pull request:

✅ 4dfe6cc Author: @parambole <pa*****le​@google.com>

(Only the first commit for a unique contributor is listed.)

View more details on Google CLA