Skip to content

CI never runs the repo's own QA validation (validate-opencode.sh, shellcheck on scripts, zizmor on action.yml) #13

Description

@dceoy

Problem

AGENTS.md declares metadata/documentation checks the primary test suite, but the parts of that suite that guard this repo's actual deliverables never run in CI:

  1. .agents/skills/local-qa/scripts/validate-opencode.sh (agent frontmatter validation + /review-pr agent-reference validation) is only invoked from qa.sh locally. No CI job executes it.
  2. The reusable github-actions-lint-and-scan workflow is invoked with search-path: .github/workflows (.github/workflows/ci.yml:28), so actionlint/zizmor only cover workflow files. action.yml — the shipped artifact — is never scanned in CI, and .agents/skills/local-qa/scripts/*.sh are never shellchecked in CI.

Evidence

  • .github/workflows/ci.yml contains only the reusable lint/scan job (scoped to .github/workflows), yaml-lint, the major-version-tag job, and dependabot-auto-merge. Nothing runs validate-opencode.sh or shellcheck over the repo's scripts.
  • qa.sh (lines 13, 16–18, 21) runs shellcheck, zizmor, actionlint, checkov, and validate-opencode.sh — but only on a developer's machine.
  • dependabot-auto-merge (ci.yml:48, unconditional: true) merges dependency PRs gated only on these partial checks.
  • Verified locally: ./.agents/skills/local-qa/scripts/validate-opencode.sh → OK; shellcheck .agents/skills/local-qa/scripts/*.sh → clean; zizmor .github/workflows action.yml → one low-confidence github-env finding on the GITHUB_PATH append in action.yml:91 (the standard installer pattern, same as upstream), which needs an inline # zizmor: ignore[github-env] or config suppression before gating on it.

Impact

Regressions in the bundled toolkit merge silently — e.g. renaming a file under .opencode/agents/ without updating .opencode/commands/review-pr.md, or breaking agent frontmatter that opencode needs — because the only check that would catch them never runs on PRs. Shell/security defects in action.yml composite steps are likewise invisible to CI, including on auto-merged Dependabot PRs.

Suggested fix

Add a small job to .github/workflows/ci.yml for push/PR events:

repo-qa:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0  # v7.0.0
      with:
        persist-credentials: false
    - run: ./.agents/skills/local-qa/scripts/validate-opencode.sh
    - run: git ls-files -z -- '*.sh' | xargs -0 shellcheck
    - run: pipx run zizmor action.yml   # after suppressing the known github-env finding

Validation

On a branch, rename .opencode/agents/code-reviewer.md (without touching review-pr.md) and confirm the new CI job fails with referenced agent 'code-reviewer' has no file under .opencode/agents/; revert and confirm it passes.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions