Exclude SkillOpt-generated Codex replay sessions from harvest

[kaluli123123]

Summary

• Exclude SkillOpt-generated Codex replay sessions from harvest_codex.
• Reuse the existing agent-session filter and add narrow Codex replay markers.
• Preserve ordinary user Codex sessions with regression coverage.

Root cause

CodexCliBackend.attempt_with_tools() emits internal prompts containing SkillOpt skill, memory, task, judge, and reflection instructions. harvest_codex() did not identify those sessions, so later harvesting could treat SkillOpt's own headless work as user evidence.

Changed behavior

Codex harvesting now skips sessions marked as agent sessions or matching the explicit SkillOpt replay prompt shape. It does not apply the generic short-duration headless heuristic, preserving valid short user sessions.

Test plan

• python3 -m unittest -v tests.test_harvest_codex_replay tests.test_sleep_engine.TestHarvest — 19 passed.
• python3 -m compileall -q skillopt_sleep tests/test_harvest_codex_replay.py — passed.
• git diff --check — passed.
• python3 -m pytest -q — not run successfully because pytest is not installed/available in the environment.

Issue

Fixes #286

Incomplete gates

Full pytest, CI, maintainer review, and merge remain pending. No merge or Issue closure was performed.

[Yifan Yang (Yif-Yang)]

Thank you for tackling the self-harvest feedback loop; preventing generated sessions from becoming user evidence is important.

Reviewed 4fabeeea0821.

skillopt_sleep/harvest_codex.py:35: The current Codex backend's ordinary attempt, judge, and reflect prompts are not recognized here. Capturing the actual calls and feeding their prompts through a synthetic archived session leaves all three in harvest: they start Complete the following task for the user., Score how well the response satisfies the rubric, and You are SkillOpt's optimizer.. Only attempt_with_tools is excluded. Please cover the actual emitted prompts (preferably with an explicit engine-origin marker) and add backend-capture regressions before closing #286.

skillopt_sleep/harvest_codex.py:45: These substring checks also discard ordinary user sessions. A first prompt of Please explain the ## CURRENT SKILL section, followed by an assistant response and perfect, thanks, survives main but produces no harvested session on this head; the same happens with an ordinary ## TASK heading. Please use unambiguous engine provenance or a complete generated prompt shape and add preservation coverage for quoted headings/multi-turn sessions.

[kaluli123123]

Addressed the two findings in 59be37e.

The Codex subprocess boundary now prefixes every engine call with an explicit provenance sentinel, covering ordinary attempt/judge/reflect, tool replay, and benchmark/custom system prompts. Harvesting uses that prefix instead of generic heading substrings; legacy tool replay retains its complete-shape check. Ordinary quoted headings and multi-turn user sessions are preserved.

The regression captures the actual subprocess input from all five paths and feeds it into synthetic archived sessions. Against the previous head, these tests fail with 12 assertions (four engine paths retained and eight user-session cases discarded). With this change, the focused tests pass: 6 tests and 13 subtests.

Validation on Python 3.12: full suite 1,501 passed, 11 skipped; strict documentation build and focused Ruff checks passed. All backend responses were mocked; no live model calls were made. The full-suite run used a minimal PATH because this machine's installed Pi executable changes an unrelated path-resolution test. An initial run also hit an unrelated concurrent staging-pointer race; the final run passed. Upstream CI still needs its normal maintainer approval/execution.