Repository navigation

Sync dev with main - #54

Merged

SushantGautam merged 79 commits into

devfrom

main

Aug 26, 2026

Commits on Apr 5, 2026
- add token usage tracking
  finnschwall
  committed
  2fcd267
Commits on Apr 7, 2026
- add repeated-run stability analysis with significance testing and resumable experiment runs
  finnschwall
  committed
  4971ea9
Commits on Apr 8, 2026
- Merge branch 'main' into benchmarking
  finnschwall
  committed
  43f6e76
Commits on Apr 10, 2026
- separate judge/auditor as possibility
  finnschwall
  committed
  249ca4e
Commits on Apr 13, 2026
- fixed missing token count for judge
  finnschwall
  committed
  17a0d08
Commits on Apr 22, 2026
- Add Hei refusal pack + test_prompt honoured on turn 1
  kelkalot
  committed
  d3c46fd
Commits on Apr 26, 2026
- removed unfinished statistics module
  finnschwall
  committed
  904d10a
Commits on Apr 29, 2026
- fix(judge): use json_schema response_format for Anthropic compatibility
  
  avalyset
  and
  claude
  committed
  ac04dbe
Commits on May 6, 2026
Commits on May 8, 2026
- Enhance README with new paper and contributor updates
  kelkalot
  authored
  6becf3e
- Merge pull request #17 from avalyset/feat/nav-aap-scenarios
  kelkalot
  authored
  a3d0c90
- feat(skatteetaten): scaffold scenario pack
  
  avalyset
  and
  claude
  committed
  a7f122f
- feat(skatteetaten): add 8 scenarios for Norwegian Tax Administration
  
  avalyset
  and
  claude
  committed
  bf48423
- feat(skatteetaten): baseline evaluation results
  
  avalyset
  and
  claude
  committed
  07cd3ac
Commits on May 11, 2026
- Merge remote-tracking branch 'upstream/main' into benchmarking
  finnschwall
  committed
  67ef079
- Merge pull request #18 from avalyset/feat/skatteetaten-scenarios
  kelkalot
  authored
  d6aba45
Commits on May 14, 2026
- Add declarative judge response_schema and three new judges
  kelkalot
  committed
  57b53d8
Commits on May 16, 2026
- Merge remote-tracking branch 'upstream/main' into benchmarking
  finnschwall
  committed
  fee48ec
Commits on May 18, 2026
- prepare for sync + new tests
  finnschwall
  committed
  ce6e228
Commits on May 22, 2026
Commits on May 23, 2026
- fix wrong forced json schema for judges
  
  finnschwall
  and
  Copilot
  authored
  8a2c716
Commits on May 27, 2026
- Merge pull request #19 from kelkalot/feature/helsedir-sexhealth-judge
  SushantGautam
  authored
  71bd837
- Merge main into benchmarking
  SushantGautam
  committed
  ed06650
- Merge pull request #20 from finnschwall/benchmarking
  SushantGautam
  authored
  69fabe1
- Merge pull request #21 from kelkalot/dev
  SushantGautam
  authored
  e444928
- Add on_model_done callback to AuditExperiment
  SushantGautam
  committed
  9ec7ada
Commits on May 29, 2026
- Add CrossJudgeExperiment for cross-judge stability analysis
  
  avalyset
  and
  claude
  committed
  3bf6533
Commits on Jun 4, 2026
- Merge pull request #24 from avalyset/feat/cross-judge-experiment
  kelkalot
  authored
  0cc6ba5
Commits on Jun 11, 2026
- Fix audit-framework bugs and add regression tests
  kelkalot
  committed
  62d3cc7
Commits on Jun 16, 2026
- Merge pull request #25 from kelkalot/fix/audit-bugs-2026-06
  SushantGautam
  authored
  8d51e50
- Update pyproject.toml
  SushantGautam
  authored
  ac56078
Commits on Jun 26, 2026
- Clarify scenario descriptions in hei_refusal.py
  kelkalot
  authored
  bfee464
- Document ung scenario pack and its themes
  kelkalot
  authored
  a69f8da
- code owners and governance
  kelkalot
  authored
  a21e044
Commits on Jul 20, 2026
- Improve resilience, caching, and judge handling
  kelkalot
  committed
  7e49bfe
- Harden path handling in get_json_file
  kelkalot
  committed
  62f44b6
- Tighten path validation to enforce root boundary
  kelkalot
  committed
  9340d7f
Commits on Jul 21, 2026
- Potential fix for pull request finding
  
  SushantGautam
  and
  Copilot
  authored
  596d460
- Potential fix for pull request finding
  
  SushantGautam
  and
  Copilot
  authored
  b3eae01
- Potential fix for pull request finding
  
  SushantGautam
  and
  Copilot
  authored
  8117e47
- Potential fix for pull request finding
  
  SushantGautam
  and
  Copilot
  authored
  ff462ae
- Enhance file validation and error handling in server.py
  SushantGautam
  authored
  3050278
- Potential fix for pull request finding 'CodeQL / Uncontrolled data used in path expression'
  
  SushantGautam
  and
  github-advanced-security[bot]
  authored
  92a2560
- fix(visualizer): use CodeQL path sanitizer pattern for json file endpoint
  SushantGautam
  committed
  65567d6
- Add Helfo (health-economics) scenario pack
  
  avalyset
  and
  claude
  committed
  6642e78
- helfo: address review — mechanical fixes (CI sums, anchors, names, docstring)
  avalyset
  committed
  b1660e5
- helfo: address review — rework scenario 3 (post-2025 psykolog rule), EHIC hjemmel (883/2004), judge-fairness optional/required reweighting
  avalyset
  committed
  8e0059b
Commits on Jul 22, 2026
- Add severity field to test payloads
  kelkalot
  committed
  3d94c17
Commits on Jul 23, 2026
- Merge pull request #28 from kelkalot/fix/code-review-2026-07
  SushantGautam
  authored
  d8a01ad
- Merge pull request #27 from avalyset/add-helfo-scenario-pack
  SushantGautam
  authored
  3331d49
- Bump version from 0.1.8 to 0.1.9
  SushantGautam
  authored
  ec532f4
- Add helfo scenario pack and update to v0.1.9
  kelkalot
  committed
  2d61dbb
Commits on Jul 27, 2026
- Add lanekassen (Norwegian student finance) scenario pack
  avalyset
  committed
  4e10133
Commits on Jul 28, 2026
- Add missing argument, fixes #30
  kwinkunks
  committed
  bbb681f
- Merge pull request #31 from kwinkunks/judge-response-schema
  kelkalot
  authored
  dc5a28c
Commits on Jul 31, 2026
- Support multi-model experiment files in visualizer
  kwinkunks
  committed
  3fc80d7
- Add tests for multi-model experiment files
  kwinkunks
  committed
  1d68556
- Merge pull request #33 from kwinkunks/expt-file-viz
  kelkalot
  authored
  ebcbe3c
- Only list loadable models in experiments
  kelkalot
  committed
  109a55d
- Merge pull request #34 from kelkalot/fix/visualizer-experiment-tree
  kelkalot
  authored
  fa6dbd4
Commits on Aug 1, 2026
Commits on Aug 2, 2026
- Merge pull request #29 from avalyset/feat/lanekassen-scenarios
  kelkalot
  authored
  37a8006
Commits on Aug 6, 2026
- Fix formatting in Custom Scenarios section
  kwinkunks
  authored
  9015a8f
- Add support for image file attachment
  kwinkunks
  committed
  eca2ae0
- Add docs for image file attachment
  kwinkunks
  committed
  7218313
- Add tests for image file attachment
  kwinkunks
  committed
  df96867
- Merge pull request #37 from kwinkunks/patch-1
  kelkalot
  authored
  4dd6270
Commits on Aug 7, 2026
- Raise any-llm floor for Gemini image support
  kwinkunks
  committed
  2b1cd1e
- Clear cached image bytes at the start of each run
  kwinkunks
  committed
  63f76aa
- Merge pull request #38 from kwinkunks/send-image-content
  kelkalot
  authored
  7f97169

Commits

Commits on Apr 5, 2026

Commits on Apr 7, 2026

Commits on Apr 8, 2026

Commits on Apr 10, 2026

Commits on Apr 13, 2026

Commits on Apr 22, 2026

Commits on Apr 26, 2026

Commits on Apr 29, 2026

Commits on May 6, 2026

Commits on May 8, 2026

Commits on May 11, 2026

Commits on May 14, 2026

Commits on May 16, 2026

Commits on May 18, 2026

Commits on May 22, 2026

Commits on May 23, 2026

Commits on May 27, 2026

Commits on May 29, 2026

Commits on Jun 4, 2026

Commits on Jun 11, 2026

Commits on Jun 16, 2026

Commits on Jun 26, 2026

Commits on Jul 20, 2026

Commits on Jul 21, 2026

Commits on Jul 22, 2026

Commits on Jul 23, 2026

Commits on Jul 27, 2026

Commits on Jul 28, 2026

Commits on Jul 31, 2026

Commits on Aug 1, 2026

Commits on Aug 2, 2026

Commits on Aug 6, 2026

Commits on Aug 7, 2026