Readme cost quality chart - #5

Closed

aarora79 wants to merge 67 commits into

aws-samples:mainfrom

aarora79:readme-cost-quality-chart

Commits on Jul 16, 2026

Add report generation scripts and charts
shekharprateek
committed
75f81aa
Rename judge-gpt.json to judge.json across all models
shekharprateek
committed
8d309b1
Rename judge.json to eval.json across all models
shekharprateek
committed
3cbf28c
Update README.md references from judge-gpt.json to eval.json
shekharprateek
committed
9ab0270
Update benchmark README with full directory layout including eval.json and metrics.json
shekharprateek
committed
1a99f23

Commits on Jul 17, 2026

Add non-interactive mode to /swe skill for headless benchmark runs
shekharprateek
committed
9c8cfc3

Commits on Jul 22, 2026

Add headless SWE benchmark scripts (claude -p and direct API)
shekharprateek
committed
237704b
Update Opus 4.8 SSRF artifacts from headless claude -p run
shekharprateek
committed
f0fc613
Conform vLLM client code to repository coding standards
Ubuntu
committed
d4ab34c
Expand CLAUDE.md with full repository coding standards
Ubuntu
committed
4c6e7dd
Conform benchmark scripts to repository coding standards
Ubuntu
committed
291b11d
Fix vLLM YaRN configuration and logging docs
Ubuntu
committed
568f4c8
Add vLLM metrics collector, benchmark client, and HTML dashboard
Ubuntu
committed
6d01bbb
Remove Ollama self-hosting path; consolidate on vLLM
Ubuntu
committed
3403db2
Add SWE benchmark dataset, Pydantic loader, and benchmarks packaging
aarora79
committed
12851cf
Add config-driven headless SWE benchmark runner
aarora79
committed
44bc405
Document three smoke-test methods for a running vLLM endpoint
aarora79
committed
9c9e46f
Document the metrics.json run output in the benchmark README
aarora79
committed
1684920
Add --count flag to run only the first N benchmark tasks
aarora79
committed
d18c6f1
Add configurable task concurrency to SWE headless harness
aarora79
committed
b985d44
Add chart zoom and throughput small-multiples to vLLM dashboard
aarora79
committed
986fa3d
Update mcp-gateway-registry benchmark artifacts from qwen3.6-35b run
aarora79
committed
31b1405

Commits on Jul 23, 2026

Commits on Jul 24, 2026