feat(proof): InferenceOffer as RLM judge backend (no HF proxy) by echobt · Pull Request #223 · CortexLM/cortex

echobt · 2026-09-04T18:15:16Z

Summary

Rebased onto main after #224 (323db5a4). Architecte/Mathis lock (do not weaken):

InferenceOffer = digest-pinned RLM judge backend (proof-eval / harvest call it)
Miners do not bind submit to offer_id as a train target; still claim + code + FLOPs + artifact vs a topic
No baked Qwen; architecture ≠ HF id stays retired
Pin ceilings / modes / origin / commitment bound the judge offer; missing / closed / judge down / missing key / spoofed origin → 503
Topic optional field: require_judge_offer_commitment (not a miner-facing bind)

Emission is bounty 2000 / proof 8000. Eval digest stays the main pin sha256:78b614a1f51ce5dd80076c4e343a2b31b85d6c36025e02836cb83929867e7009 — do not invent a sha256. Empty digest still fail-closes. Live submits stay 503 until harvest is wired, a baseline is sealed, and ≥1 topic is open.

Testeur blockers (this revision)

1. API key reaches the harvest pod

Host reads PROOF_INFERENCE_API_KEY_FILE (contents never logged, never on GET /v1/status)
Harvest stages it as teacher.env (OPENAI_API_KEY + PROOF_INFERENCE_API_KEY) and pod.run sources that file; the key is not in HarvestRequest JSON
proof-eval sends Authorization: Bearer on judge /chat/completions (and refuses api_key in request.json)
Live Lium can_score is false / submit 503 if the open offer needs auth and the key file is missing/unreadable

2. Origin is bound into commitment + serve-check

config_commitment hashes config knobs and provider.base_url
serves_topic rejects a topic that spoofs inference.base_url (503, before lattice)
resolve_inference uses the committed offer origin; topic/pin/secret cannot redirect the RLM after serve-check
Expected test: signed topic that only tightens tokens but spoofs inference.base_url → 503 / committed judge origin

HF proxy_model / proxy_models stay empty. Miners still do not bind submit to offer_id.

Judge config

Pin (git) — config/proof-pin.toml nested [inference] plus ceilings that bound the judge:

Field	Lock
`proxy_model` / `proxy_models`	empty; non-empty pin refuses to boot
`[inference]`	judge defaults: `provider`, `base_url` (empty = secret-backed), `model`, `mode`, token caps
`inference_config_schema_version`	`1`
`allowed_modes`	`chat`, `completions`, `embeddings` (pin may subset)
`max_input_tokens_ceiling`	`32768`
`max_output_tokens_ceiling`	`8192`
`inference_offer_commitment_alg`	`sha256`
secrets / origins	never in git

Live InferenceOffer (operator state) — PROOF_INFERENCE_OFFER_FILE. Auth: PROOF_INFERENCE_API_KEY_FILE staged as harvest teacher.env. Optional origin: PROOF_INFERENCE_BASE_URL / _FILE (cannot override a committed offer origin).

Topic inference{…} may override provider/model/mode and may only tighten token caps. It must not redirect origin. Optional require_judge_offer_commitment pins the live judge config_commitment (host 503 on mismatch).

Miner submit does not take inference_offer_id / config_commitment. Host stamps those as judge provenance. architecture is not compared to an HF id.

Status: public pin inference and public inference_offer only. No origins, no keys.

Greptile

Every PR is reviewed by Greptile before merge. Config: .greptile/.

Greptile has reviewed this PR; findings are fixed or answered
If the bot is silent, I commented @greptileai review

Test plan

cargo test -p proof-task -p proof-holdout -p proof-eval -p proof-harvest -p proof-http -p proof-challenge -p proof-challenge-bin
python3 -m pytest eval/tests/test_contract.py (12 passed)
cargo fmt --all -- --check
cargo clippy -p proof-task -p proof-holdout -p proof-eval -p proof-harvest -p proof-http -p proof-challenge --all-targets -- -D warnings
cargo run -p xtask -- loc-cap
cargo run -p xtask -- external-docs-check

Risk

No emission retune (stays 2000/8000). No invented digest. Fail-closed 503 for a down/incomplete/unauthed judge or a spoofed topic origin. proof-holdout is a loc-cap split of the existing holdout module (same API via proof_task re-exports).

Naming

I did not rename BASE_* environment variables, deployed host paths
(/opt/base, /run/base, …), GHCR baseintelligence/base package names, or
base-*-v1 cryptographic domain tags, unless this PR’s purpose is a coordinated
cutover documented in docs/NAMING.md.

cursor · 2026-09-04T18:15:30Z

@greptileai review

greptile-apps · 2026-09-04T18:25:49Z

Greptile Summary

This update replaces the retired miner training-proxy interpretation with a live RLM judge offer, keeps judge availability fail-closed, and removes miner-side offer binding. The change is not safe to merge while the outstanding correctness issues prevent reliable scoring.

Confidence Score: 2/5

Not merge-safe: three outstanding blocking correctness issues can cause scoring to be advertised or attempted with an unusable judge configuration.

Three previous P1 findings remain outstanding. The configured provider key is checked for presence but is not passed to the evaluation pod, so an authenticated judge can appear available before its provider call fails. Status readiness now checks that an open topic resolves to a complete inference configuration, but it still does not require that the active offer serves that topic; token, mode, model, provider, or required-commitment incompatibility can therefore leave can_score true while submission returns unavailable. The harvest request also still carries only config_commitment, rather than the committed temperature, top_p, and timeout_ms values, so the evaluation image cannot apply those committed judge settings. None of the threads has a reply, dismissal, dispute, fix claim, or resolution that changes its outstanding status.

_{Reviews (2): Last reviewed commit: "fix(proof): rename topic judge commitmen..." | Re-trigger Greptile}

greptile-apps · 2026-09-04T18:25:52Z

+    if let Some(p) = cli.inference_api_key_file.as_deref() {
+        match std::fs::read_to_string(p) {
+            Ok(s) if !s.trim().is_empty() => {
+                tracing::info!("inference api key file present (contents not logged)");
+            }
+            _ => tracing::warn!(
+                "PROOF_INFERENCE_API_KEY_FILE set but unreadable; eval image auth will fail closed"
+            ),
+        }
+    }


Provider key is discarded

If the configured provider requires authentication, PROOF_INFERENCE_API_KEY_FILE is read only to confirm that it is non-empty, and its contents are then discarded. The harvest path launches the eval pod with an empty environment payload, so the host can report can_score=true and rent a pod even though the provider request will fail with 401/403. Pass the key through the pod's secret environment channel and include credential availability in readiness.

Knowledge Base Used:

Deployment, runtime operations, and image updates

Deployment automation and infrastructure

greptile-apps · 2026-09-04T18:25:52Z

    fn can_score(&self) -> bool {
        let open = self.store.any_open_scorable(self.epoch).unwrap_or(false);
-        scoring_readiness(&self.pin, self.backend, self.live(), open).is_ok()
+        scoring_readiness(
+            &self.pin,
+            self.backend,
+            self.live(),
+            open,
+            self.offer.as_ref(),
+        )
+        .is_ok()


Readiness ignores compatibility

/v1/status can return can_score: true when the only open, baseline-sealed topic exceeds the active offer's limits. Submissions for that topic are then rejected with 503 because submit does check offer.serves_topic. Make readiness require at least one open scorable topic that the active offer can serve, so clients are not told scoring is available when every submission will fail.

Artifacts

Evidence from the check

Runs the authored Axum endpoint-contract test once against the candidate and once in an isolated worktree with the minimal readiness correction, showing the mismatch and expected behavior.

Incompatible-offer endpoint output

Captured cargo test output for the candidate: GET /v1/status is HTTP 200 with can_score true while POST /v1/submissions is HTTP 503 for offer/topic incompatibility, confirming the defect.

Corrected readiness endpoint output

Captured cargo test output after a temporary isolated-worktree readiness correction: GET /v1/status is HTTP 200 with can_score false and the incompatible submission remains HTTP 503, showing the expected contract.

_{Ran code and verified through T-Rex}

greptile-apps · 2026-09-04T18:25:52Z

+    /// Input token cap for this run (min of offer and topic).
+    pub max_input_tokens: u32,
+    /// Output token cap for this run.
+    pub max_output_tokens: u32,
+    /// Offer config commitment.
+    pub config_commitment: String,


Committed settings are omitted

temperature, top_p, and timeout_ms are included in the configuration commitment miners bind to, but HarvestRequest sends only the token caps and commitment hash. The hash cannot recover those values, so an offer that sets them does not provide the eval image with the committed configuration. Transmit these settings so the scored inference matches the advertised offer.

cursor · 2026-09-04T18:55:45Z

Mathis correction (do not weaken fail-closed):

The live master InferenceOffer is the RLM judge backend (proof-eval / harvest call it to reproduce / cheat-check / score). It is not a miner training proxy.

Miners submit claim + code + FLOPs + artifact against a topic. Submit does not bind inference_offer_id / config_commitment.
Missing / closed / misconfigured / down judge, incomplete resolve, or empty digest → can_score=false → 503.
Pin [inference] + ceilings / modes / commitment bound the judge offer. Topic inference{…} tightens judge ceilings only. Optional topic field is require_judge_offer_commitment (not a miner bind).
No HF bake. Architecture ≠ HF id stays retired. Digest still empty (not invented).

Title/docs/API comments rewritten to match. @greptileai review

cursor · 2026-09-04T18:58:51Z

Architect lock is pushed on this PR (b27590ea).

InferenceOffer = digest-pinned RLM judge backend (proof-eval / harvest call it)
Miners do not bind submit to offer_id; still claim + code + FLOPs vs topic
No baked Qwen; architecture ≠ HF stays retired
Pin ceilings / modes / commitment bound the judge; missing / closed / judge down → 503
Topic optional field: require_judge_offer_commitment (not a miner-facing bind)

Title remains feat(proof): InferenceOffer as RLM judge backend (no HF proxy). Fail-closed is not weakened.

Replace the HF proxy_model architecture lock with pin ceilings plus an operator-state InferenceOffer. Miners bind offer_id/config_commitment; missing or closed offers fail closed (can_score false, 503). Co-authored-by: Mathis <echobt@users.noreply.github.com>

Drop miner bind to the live offer. The offer is the RLM eval judge the digest-pinned image calls; miners still submit claim + code + FLOPs + artifact. Missing/closed/incomplete judge stays 503. Co-authored-by: Mathis <echobt@users.noreply.github.com>

Topic optional pin is require_judge_offer_commitment, not a miner bind. Co-authored-by: Mathis <echobt@users.noreply.github.com>

proof-eval/harvest call the offer. Miners still post claim+code+FLOPs vs topic. Pin ceilings bound the judge; topic field is require_judge_offer_commitment. Co-authored-by: Mathis <echobt@users.noreply.github.com>

Co-authored-by: Mathis <echobt@users.noreply.github.com>

greptile-apps Bot reviewed Sep 4, 2026

View reviewed changes

cursor Bot changed the title ~~feat(proof): bind miners to a live inference offer~~ feat(proof): InferenceOffer as RLM judge backend (no HF proxy) Sep 4, 2026

cursoragent and others added 5 commits September 5, 2026 06:58

fix(proof): rename topic judge commitment field

4469c4d

Topic optional pin is require_judge_offer_commitment, not a miner bind. Co-authored-by: Mathis <echobt@users.noreply.github.com>

docs(proof): lock InferenceOffer as RLM judge backend

26050ac

proof-eval/harvest call the offer. Miners still post claim+code+FLOPs vs topic. Pin ceilings bound the judge; topic field is require_judge_offer_commitment. Co-authored-by: Mathis <echobt@users.noreply.github.com>

fix(proof): bind judge origin and inject harvest API key

3b45e05

Co-authored-by: Mathis <echobt@users.noreply.github.com>

cursor Bot force-pushed the cursor/proof-inference-offer-0a84 branch from b27590e to 3b45e05 Compare September 5, 2026 07:25

echobt merged commit 8c7907c into main Sep 5, 2026
6 checks passed

echobt deleted the cursor/proof-inference-offer-0a84 branch September 5, 2026 07:34

echobt mentioned this pull request Sep 5, 2026

fix(challenges): boot when sk/session placeholders are empty #226

Merged

6 tasks

Navigation Menu

Uh oh!

feat(proof): InferenceOffer as RLM judge backend (no HF proxy) - #223

feat(proof): InferenceOffer as RLM judge backend (no HF proxy)#223
echobt merged 5 commits into
mainfrom
cursor/proof-inference-offer-0a84

echobt commented Sep 4, 2026 •

edited by cursor Bot

Loading

Uh oh!

cursor Bot commented Sep 4, 2026

Uh oh!

greptile-apps Bot commented Sep 4, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot Sep 4, 2026

Uh oh!

greptile-apps Bot Sep 4, 2026

Uh oh!

greptile-apps Bot Sep 4, 2026

Uh oh!

cursor Bot commented Sep 4, 2026

Uh oh!

cursor Bot commented Sep 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

echobt commented Sep 4, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testeur blockers (this revision)

Judge config

Greptile

Test plan

Risk

Naming

Uh oh!

cursor Bot commented Sep 4, 2026

Uh oh!

greptile-apps Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 2/5

Uh oh!

greptile-apps Bot Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

cursor Bot commented Sep 4, 2026

Uh oh!

cursor Bot commented Sep 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

echobt commented Sep 4, 2026 •

edited by cursor Bot

Loading

greptile-apps Bot commented Sep 4, 2026 •

edited

Loading