Issues

• Division by Zero in GPTRewardModel with Empty Batches

  bug
  CarperAI/trlx#609 · Filter by author ohsono opened on Nov 28, 2025

• ValueError: Invalid pattern: '**' can only be an entire path component

  bug
  CarperAI/trlx#607 · Filter by author amrzv opened on Jun 7, 2025

• ImportError: cannot import name 'prepare_model_for_int8_training' from 'peft'

  bug
  CarperAI/trlx#606 · Filter by author amrzv opened on Apr 23, 2025
  1

• Extracting total-loss, PPO-loss, rewards per step, returns per step in RLHF-PPO implementation

  documentation
  CarperAI/trlx#605 · Filter by author rithammajumdarr opened on Feb 17, 2025

• Question about model load

  CarperAI/trlx#604 · Filter by author gray311 opened on Feb 11, 2025

• Does the framework support PPO training for Qwen2?

  feature request
  CarperAI/trlx#603 · Filter by author oldwangggggg opened on Dec 21, 2024

• reward_fn in accelerate_ppo_trainer.py

  CarperAI/trlx#602 · Filter by author Jerrrrykun opened on Dec 11, 2024

• OOM error with PEFT LoRA on Llama2-7B

  bug
  CarperAI/trlx#601 · Filter by author arpaiva opened on Sep 20, 2024
  1

• Load the checkpoint fails

  bug
  CarperAI/trlx#600 · Filter by author AfraAmini opened on Sep 6, 2024

• cannot import name 'flatten_dataclass' from 'trlx.data.ilql_types'

  bug
  CarperAI/trlx#599 · Filter by author AfraAmini opened on Jul 31, 2024

• maybe bug in prepare & load's order

  bug
  CarperAI/trlx#598 · Filter by author daiwk opened on Jul 27, 2024
  1

• Error when running Ray Tune to launch hyperparameter sweep

  bug
  CarperAI/trlx#597 · Filter by author Jing-L97 opened on Jul 26, 2024
  1

• Crash when using save_state with deepspeed: model.state_dict functions incompatible with new deepspeed.

  bug
  CarperAI/trlx#596 · Filter by author JohannesAck opened on Jul 11, 2024

• Data Loader Bug when running t5_summarization_daily_cnn.py

  bug
  CarperAI/trlx#595 · Filter by author yunanyan opened on Jun 4, 2024

• Why train dataloader is not prepared by Accelerator

  bug
  CarperAI/trlx#594 · Filter by author Jiaxin-Wen opened on May 18, 2024

• TRLX Environment customization

  CarperAI/trlx#593 · Filter by author heraldiclily opened on Apr 21, 2024

• Issue of tensors share memory

  bug
  CarperAI/trlx#591 · Filter by author heraldiclily opened on Apr 1, 2024
  2

• [New Feature Request] Add KTO

  feature request
  CarperAI/trlx#590 · Filter by author 1485840691-eng opened on Jan 27, 2024

• RLHF text summarization diverges

  bug
  CarperAI/trlx#589 · Filter by author AlisonWen opened on Jan 14, 2024

• Integration of Self-Play Fine-Tuning (SPIN) Method for Enhancing Large Language Models

  feature request
  CarperAI/trlx#588 · Filter by author SeungyounShin opened on Jan 11, 2024

• Runtime error when running examples (ilql_sentiments_t5.py)

  bug
  CarperAI/trlx#587 · Filter by author youxiho1 opened on Jan 8, 2024
  2

• MPT is not working

  bug
  CarperAI/trlx#585 · Filter by author ouhenio opened on Dec 12, 2023

• when i use trlx ppotrainer train a model llama 13b model, but saved huggingface mode ,but when it inference , it has some strange keys ,and the inference result did not show ,it also have no error , it seems the result disapper

  bug
  CarperAI/trlx#584 · Filter by author ldh127 opened on Dec 3, 2023
  1

• Attention mask when calculating log ratio for PPO

  CarperAI/trlx#582 · Filter by author kmy17518 opened on Nov 23, 2023

• Multi-GPU training errors with peft

  bug
  CarperAI/trlx#581 · Filter by author AliengirlLiv opened on Nov 20, 2023
  1