Skip to content

test(cli): pin the run/compare one-line help text, not just a label - #317

Merged
napetrov merged 1 commit into
intel:mainfrom
xaleryb:test/cli-help-line-pin
Oct 31, 2026
Merged

napetrov merged 1 commit into
intel:mainfrom
xaleryb:test/cli-help-line-pin

Conversation

@xaleryb

@xaleryb xaleryb commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #284, which was merged before this could be pushed to its branch.

test_run_and_compare_have_a_one_line_help_in_the_top_level_list pins the changelog's "Changed" claim of #284: run and compare get a one-line help in dokimi --help. Its regex \S.* \[(free|paid)\]$ also matches the command name followed by the cost label, so the test passes when the route table supplies no help text at all.

  • Input: summary_of in dokimi/cli_routes.py returns None.
  • Observed: dokimi --help prints run [free] and the test passes.
  • Expected: the test fails.

The test now asserts the literal lines Run the static benchmark spec and save a snapshot [free] and Compare two saved benchmark snapshots [free]. With the mutant above it fails, while the old test still passes. On main it passes, and so does the full suite. Test-only change, so there is no changelog fragment.

The regex matched the command name followed by a cost label, so the test
passed with no help text at all (summary_of returning None). Assert the
literal lines the route table supplies.

Signed-off-by: Alexander Rybkin <alexander.rybkin@intel.com>
@exai-bot

Copy link
Copy Markdown

✅ OpenCodeReview: Review complete: 0 finding(s) across 1 selected item(s).

Review details
  • Commit reviewed: 9b71831 against merge base 90cc5c3
  • Coverage: 1 files in 1 groups
  • Engine: OpenCodeReview v1.13.3, us.openai.gpt-6.1-sol
  • Usage: 16k tokens (7k from cache), 5s
  • Tool calls: 2 (1 code_search, 1 file_read)
  • dokimi/tests/test_cli_routes.py: 1 file

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.72%. Comparing base (90cc5c3) to head (9b71831).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #317   +/-   ##
=======================================
  Coverage   93.72%   93.72%           
=======================================
  Files         279      279           
  Lines       36178    36178           
  Branches     6134     6134           
=======================================
+ Hits        33907    33909    +2     
+ Misses       1463     1462    -1     
+ Partials      808      807    -1     

see 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@napetrov
napetrov merged commit 8b2115c into intel:main Oct 31, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants