Skip to content

test: prune redundant e2e matrix tests / test: 精简重复的端到端矩阵测试 - #3560

Merged
functionstackx merged 1 commit into
mainfrom
test/audit-e2e-matrix-redundancy
Sep 29, 2026
Merged

functionstackx merged 1 commit into
mainfrom
test/audit-e2e-matrix-redundancy

Conversation

@functionstackx

Copy link
Copy Markdown
Collaborator

Description

Apply the OpenClaw test-audit skill to one focused matrix-generation batch in inferencex-e2e/. Remove five redundant test functions and consolidate their contracts into existing behavioral tests.

Only inferencex-e2e/infx/tests/matrix/test_generate_sweep_configs.py changes: +10/-56 lines, net -46 test lines; zero production or tooling changes.

Coverage ownership

Removed test Retained proof
test_exp_name_format test_sweep_expands_each_sequence_length_across_concurrencies now checks explicit experiment names for every 1k1k and 8k1k row.
test_max_model_len_calculation The same expansion test now checks independent expected context lengths, 2304 and 9472, for every row.
test_runner_node_filter_without_runner_type test_runner_filter_keeps_each_commands_label_and_duplicate_policy exercises the full-sweep adapter with no explicit runner type and asserts exact runner ordering and duplicate behavior. Explicit-runner and no-match tests remain.
test_multinode_conc_as_list Its exact [2150] assertion moves into test_multinode_entry_structure, which uses the same arguments and fixtures.
test_single_node_flag_skips_multinode test_single_node_only_from_mixed uses the same multi-node fixture alongside valid single-node work, verifies nonempty output, and excludes every multi-node row.

The audit covered the production adapters, shared expansion and row builders, callers, sibling tests, CI routing, and relevant history. No production seam can be removed by this batch. Schema, malformed-input, topology, parallelism, provenance, installed-package, and executable workflow-shell tests remain; other test owners are outside this PR.

Validation

  • Baseline: 441 matrix tests passed.

  • Final: 996 tests passed (436 matrix + 560 workflow tests):

    uv run --project inferencex-e2e --locked --all-extras \
      --with pytest --with pytest-xdist python -m pytest \
      -c inferencex-e2e/pyproject.toml \
      inferencex-e2e/infx/tests/matrix inferencex-e2e/infx/tests/workflows -q -n 4
  • Three temporary production mutations were each caught by retained tests: incorrect experiment name, context padding changed from 256 to 255, and bypassed topology filtering. All mutations were reverted before the final run; the production diff is empty.

  • Repository Ruff lint and format checks passed (106 files already formatted); git diff --check passed. Ruff excludes the test directory by repository policy.

  • Independent read-only review found no P0–P2 findings and separately passed all 436 matrix tests.

  • The Codex autoreview helper was unavailable in this session (reviewer_unavailable / engine_failed); the independent review above is a fallback, not a successful Codex run.

  • The first workflow run was launched outside the repository and had 20 checkout-root lookup failures. Running from the repository root passed all 560 workflow tests.

  • The full repository CPU suite has not been run locally; GitHub CI owns that broader gate. No GPU, Slurm, benchmark sweep, or evaluation dispatch was performed.

AI model disclosure

  • GPT 6 Astra Fast: audit, implementation, mutation checks, local validation, and PR preparation. This is the identifier exposed by the runtime.
  • Independent review agent: read-only coverage review and separate matrix test run. Its exact model/version was not exposed and could not be verified.

Type of Change

  • Other: test-only consolidation and deletion of redundant test bodies.

Checklist

  • AI model disclosure is complete.
  • Changes were tested locally as listed above.
  • Documentation: not applicable; no user-facing behavior or procedure changes.
  • Performance changelog: not applicable; no benchmark, recipe, configuration, or production changes.
  • Sweep reuse: not applicable; no sweep or merge requested. This PR is intentionally left unmerged.
中文

改动说明

按 OpenClaw 的 test-audit 技能,对 inferencex-e2e/ 中矩阵生成测试做一次小范围清理。删除五个重复测试函数,把仍有价值的断言合并到已有行为测试中。

仅修改 inferencex-e2e/infx/tests/matrix/test_generate_sweep_configs.py:新增 10 行、删除 56 行,测试代码净减 46 行;生产代码和工具代码均无改动。

  • 将实验名称和最大上下文长度断言合并到序列长度与并发度展开测试,检查每条生成记录,并保留独立给定的期望值。
  • 将多节点并发列表 [2150] 的断言移入使用相同输入的条目测试。
  • 删除无显式 runner 类型的重复筛选测试;保留的测试覆盖相同适配器,并检查 runner 的精确顺序和重复项规则。
  • 删除单独验证多节点排除的测试;保留的混合输入测试同时检查多节点记录被排除、合法单节点记录未被全部丢弃。

审查了生产适配器、共享展开逻辑、条目构造器、调用方、相邻测试、CI 路由和相关历史。没有可随本次测试清理删除的生产接口。配置校验、异常输入、拓扑、并行参数、来源追踪、安装包行为和实际执行工作流 shell 的测试均保留;其他模块不纳入本 PR。

验证

  • 修改前 441 个矩阵测试通过。
  • 最终 996 个测试通过:436 个矩阵测试、560 个工作流测试。命令见上方。
  • 临时引入的错误实验名称、上下文余量从 256 改为 255、绕过拓扑筛选,均被保留测试捕获。随后恢复全部临时改动,重新运行测试;生产代码差异为空。
  • 仓库 Ruff 检查、格式检查及 git diff --check 通过。仓库 Ruff 配置排除了测试目录。
  • 独立只读审查无 P0–P2 问题,并单独运行通过全部 436 个矩阵测试。
  • 本次会话中的 Codex autoreview 工具不可用,返回 reviewer_unavailable / engine_failed。独立审查是替代审查,不代表 Codex 审查成功。
  • 首次从仓库外运行工作流测试时,20 个测试因无法定位仓库根目录失败;从仓库根目录重跑后,560 个测试全部通过。
  • 未在本地运行整个仓库的 CPU 测试集,完整范围交由 GitHub CI 验证。未启动 GPU、Slurm、性能 sweep 或评测任务。

AI 模型使用说明

  • GPT 6 Astra Fast:负责审计、修改、临时变异验证、本地测试和 PR 编写。名称来自运行环境提供的模型标识。
  • 独立审查 agent:负责只读覆盖率审查及独立矩阵测试;运行环境未提供其精确模型名称或版本,无法核实。

本次仅整理测试,不涉及文档、性能变更记录或 sweep 复用。本 PR 按请求保持未合并状态。

合并端到端矩阵生成的重复测试,保留实验名称、上下文长度、并发列表及筛选行为的覆盖。
@functionstackx
functionstackx requested a review from a team September 29, 2026 01:27

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, straightforward test consolidation with no production code changes. Reviewed: the exp-name/max-model-len merge (now checked per-row for both seq-len pairs across all concurrencies, still asserting exact expected values); the runner_node_filter-without-explicit-runner_type removal (test_runner_filter_keeps_each_commands_label_and_duplicate_policy uses the same runner_type=None default and asserts exact runner ordering/duplicates, parametrized over node_names including the empty case); and the multinode single-node-flag-skip removal (test_single_node_only_from_mixed uses a mixed single+multinode config and asserts no "prefill"-bearing entries survive, which is a stronger check than the deleted test's plain empty-result assertion). All claimed replacement tests exist in the file and genuinely exercise the same code paths.

Extended reasoning...

Single-file, test-only diff (+10/-56) in inferencex-e2e/infx/tests/matrix/test_generate_sweep_configs.py; no production code, no security-sensitive surface touched. Traced each of the three deleted tests (test_exp_name_format, test_max_model_len_calculation, test_runner_node_filter_without_runner_type, test_multinode_conc_as_list, test_single_node_flag_skips_multinode) against the fixtures and surviving tests to confirm the PR description's coverage-ownership table is accurate rather than aspirational. Found the merged/retained assertions actually reach the same code paths with equal or better precision (e.g., exact ordering assertions instead of a bare length check), so no coverage regression.

@functionstackx
functionstackx merged commit 15ee6da into main Sep 29, 2026
5 checks passed
@functionstackx
functionstackx deleted the test/audit-e2e-matrix-redundancy branch September 29, 2026 02:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant