All pull requests

Pull requests

• feat: Add code_x_glue code_to_code task

  #317·jonleinena opened Dec 11, 2025·

  ·

• [Community Submission] Model: Isaac-AI-OS/isaac-20b, Username: petermantell

  #316·pedroMantAmp opened Oct 15, 2025·

  ·
  1

• Fix UnboundLocalError in APPS task evaluation

  #310·sad-mathematician opened Apr 20, 2025·

  ·

• Updated bigcode-evaluation-harness/leaderboard/README.md

  #305·zoya-hammad opened Mar 13, 2025·

  ·

• Support for remote inference

  #302·pawelknes opened Feb 17, 2025·

  ·

• use tokenizer.chat_template by default for instruction type tasks

  #301·TK-21st opened Jan 25, 2025·

  ·

• Missing comma in MultiPL-E languages

  #299·hrshtv opened Jan 13, 2025·

  ·
  2

• [Pytest] Fix bad import to use relative instead @ module_test

  #298·ggcr opened Jan 11, 2025·

  ·

• Fix the bugs in the ds1000 sample bash script; Fix typos

  #295·gameofby opened Dec 10, 2024·

  ·

• Support multiple datasets from MBPP; Fix missing commas in python list; Fix doc typos;

  #291·gameofby opened Dec 4, 2024·

  ·

• add support for hpu devices

  #281·envsp opened Oct 25, 2024·

  ·
  1

• "," missing in LANGUAGES list

  #280·ArtemisDicoTiar opened Oct 21, 2024·

  ·
  2

• Speedup execute.py: Reuse same manager and dict in

  #277·michaelfeil opened Oct 1, 2024·

  ·

• Basecodes

  #263·Abhineetsoccer opened Aug 14, 2024·

  ·

• Add a new benchmark ENAMEL for evaluating the efficiency of LLM-generated code

  #260·q-rz opened Jul 22, 2024·

  ·

• Fix Max New Tokens in HF's Generation Config

  #257·mostafaelhoushi opened Jul 18, 2024·

  ·
  2

• Fix unnecessary repeated overwrite

  #249·nielstron opened Jun 29, 2024·

  ·

• Add llama3 instruction prompts

  #222·TechxGenus opened Apr 19, 2024·

  ·
  1

• Leaderboard README improvements

  #217·nikita1503 opened Apr 14, 2024·

  ·
  4

• remove pad tokens added by the accelerator.pad_across_processes

  #216·IQ17 opened Apr 13, 2024·

  ·

• Ensure generations get saved in generation_only mode

  #212·Vipitis opened Mar 31, 2024·

  ·
  1

• fix apps evaluate error: local variable 'level' referenced before assignment

  #206·koking0 opened Mar 10, 2024·

  ·
  2

• Update README.md

  #204·AnitaLiu98 opened Mar 2, 2024·

  ·
  4

• Fix loading PAL-GSM few-shot examples

  #196·sxjscience opened Feb 8, 2024·

  ·
  1

• Fix typo in README.md

  #177·ab-10 opened Jan 2, 2024·

  ·