Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/actions/setup-openshell/action.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: Setup OpenShell
description: Install the pinned OpenShell CLI and wait for the gateway to become ready.
name: Set up local OpenShell for CI
description: Install the pinned OpenShell CLI and wait for the local CI gateway; workspace and provider setup belong to the caller.

runs:
using: composite
Expand Down
44 changes: 44 additions & 0 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# GitHub workflows

This directory contains repository CI and reusable GitHub Actions workflows
for defined operations, such as PR review. Each reusable workflow fixes the
job permissions, trusted checkout, concurrency, and setup for its task bundle.
It is the cross-repository interface supplied by `harness-openshell`.

[`pr-review-reusable.yml`](pr-review-reusable.yml) currently provides PR review.
It checks out trusted workflow code from the caller's default branch, treats the
pull-request diff as data, obtains a repository-scoped GitHub App token, and
invokes the `github-pr-reviewer` task bundle. Its OpenShell REST policy permits
the sandboxed agent to read the selected PR and post inline comments to it.
The token's repository permissions and the policy's PR-specific HTTP methods
and paths are separate restrictions. Consumers should pin both the workflow
reference and its `harness-ref` input to the same immutable commit SHA.

## Gateway setup: local CI and managed deployment

The reviewer invokes [`setup-openshell`](../actions/setup-openshell/action.yml)
to install the pinned OpenShell CLI and wait for the local CI gateway. The
trusted [`scripts/pr-review.sh`](../../scripts/pr-review.sh) wrapper stages the
diff, creates the temporary workspace and providers, configures inference, and
renders the PR-specific policy before invoking the `harness` CLI. The CLI
composes the task and manages its sandbox lifecycle.

The CLI already supports a direct managed-gateway connection. Moving this
review job to the intended managed StackRox deployment still requires platform
ownership of workspace membership, provider credentials and their refresh or
expiry, matching inference routes, and CI network access. A pre-provisioned
provider name does not by itself keep a short-lived GitHub token usable.
See [managed reviewer requirements](../../docs/ci.md#managed-reviewer-transition).

Once that contract is established, replace the job's local setup and temporary
provider bootstrap with the managed connection. Preserve the task's allowed
operations and equivalent OpenShell policy and provider boundaries.

Comments may be posted during agent execution. Artifacts retain diagnostics;
cleanup or cancellation does not undo GitHub operations that already succeeded.

Add another reusable workflow only when the capability has a distinct trigger,
permission, or trust contract. Keep review and merge separate, and keep
repository-specific task behavior in [workloads/](../../workloads/) rather than
growing a single workflow with general-purpose image, policy, provider, or
command inputs.
423 changes: 208 additions & 215 deletions README.md

Large diffs are not rendered by default.

106 changes: 81 additions & 25 deletions docs/ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ main/tag pushes publish images and update the shared registry cache.

## HyperShell validation

HyperShell validation runs locally from the Red Hat network because its OIDC
issuer is VPN-only. The Harness workflow connects directly through the
HyperShell is the managed OpenShell environment used by these validation
examples. Validation runs locally from the Red Hat network because its OIDC
issuer is VPN-only. The `harness` CLI connects directly through the
OpenShell Go SDK. It does not persist a gateway registration or use a gateway
administrator account at runtime.

Expand Down Expand Up @@ -43,7 +44,8 @@ unset GOOGLE_VERTEX_AI_TOKEN

The selected Vertex project must grant this service account prediction access;
it need not be the project that owns the account. A successful local run proves
the OpenCode/Gemini runtime path, not the Claude reviewer or review quality.
the OpenCode/Gemini runtime path; review publication and quality need their own
validation.
The smoke validates the final response marker, propagates agent and cleanup
failures, and attempts cleanup on SIGINT/SIGTERM. Forced termination (SIGKILL or
runner loss) cannot execute shell cleanup. Credential-free orchestration tests
Expand All @@ -66,6 +68,11 @@ The App installation must grant `Contents: read` and `Pull requests: read and
write` repository permissions, and include the repository being reviewed. The
workflow requests only those permissions when minting the installation token.

The token is repository-scoped; OpenShell's rendered REST policy further
restricts sandbox requests to the selected PR's allowed read and inline-comment
endpoints. The Actions job's `contents: read` permission is a separate token
boundary from these explicitly requested GitHub App permissions.

The workflow accepts the App Client ID as
`openshell-github-app-client-id`; callers explicitly forward only
`VERTEX_AI_SERVICE_ACCOUNT_KEY` and `OPENSHELL_GITHUB_APP_PRIVATE_KEY`. The
Expand All @@ -75,8 +82,8 @@ sandbox environment variables, payloads, agent arguments, or artifacts.
Installation tokens expire after one hour and are revoked by the token action
after the job.

The Harness repository's `ai-review.yml` is a thin caller of the pinned GitHub
review workflow, so `pull_request_target` runs use the same path as consuming
The `harness-openshell` repository's `ai-review.yml` is a thin caller of the
pinned GitHub review workflow, so `pull_request_target` runs use the same path as consuming
repositories. Changes to that caller are exercised after they reach the default
branch; before then, use `actionlint` and the local `scripts/pr-review.sh`
commands below. Normal reviews remain `pull_request_target` runs from the
Expand All @@ -90,20 +97,31 @@ an artifact link. Seven-day artifacts hold input revisions, diff/hash, execution
metadata, raw output/diagnostics, and `review.txt`. Reviews are advisory inline
comments only; they do not approve, request changes, or merge.

The reviewer runs Claude Code through `inference.local` and Google Vertex AI.
The workflow keeps `REVIEW_MODEL` and `REVIEW_CLI_MODEL` explicit; the currently
validated default is `claude-haiku-4-5@20251001` / `haiku`. Vertex identifies
Sonnet 4.5 as `claude-sonnet-4-5@20250929`; switch both values together only
after the CI service account can invoke that model.

Only trusted default-branch code runs on the host. The pinned sandbox receives
the PR diff and a PR-scoped GitHub token; OpenShell permits only inline comment
POSTs to that exact PR. Label/head/base are rechecked before execution and
publication. Diffs over 256 KiB are rejected; execution and diagnostic output
are bounded. The
completion check rejects errors, tool calls, empty or truncated responses—not
incorrect findings. Artifacts remain unvalidated model output. Cleanup covers
success, failure, and normal cancellation, but cannot guarantee runner-loss cleanup.
The active reviewer runs OpenCode with Gemini 2.5 Pro through `inference.local`
and Google Vertex AI. The model is selected in
[`scripts/pr-review.sh`](../scripts/pr-review.sh) and
[`opencode-harness.yaml`](../workloads/github-pr-reviewer/workflow/opencode-harness.yaml).
Keep those selections aligned and verify model access with the CI identity
when changing them.

The host uses trusted caller default-branch inputs and the pinned
`harness-openshell` revision. The sandbox receives the PR diff and attaches the
`github-review` provider's masked proxy interface, not the raw token. Its REST
policy allows selected PR reads and inline-comment POSTs to that exact PR.
The instructions request at most three comments; the policy does not enforce
comment count or review quality.

The wrapper rechecks label/head/base before launching the agent and after the
run. Comments can be posted during execution, so the final host check is not
a gate before each comment. Diffs over 256 KiB are rejected; execution and
diagnostic output are bounded. The
[completion check](../scripts/review/README.md) validates the OpenCode event
stream, including text and a terminal stop event, with recognized exceptions
for unresolvable comment positions and shell parser failures. It does not
validate finding correctness.
Artifacts remain unvalidated model output. Cleanup covers success, failure,
and normal cancellation, but cannot guarantee runner-loss cleanup or undo
comments that have already been posted.

Locally, use `gh` authentication, `jq`, GNU `timeout` (Homebrew `coreutils` on
macOS), and the Vertex token/project variables above. Use a new absolute artifact
Expand All @@ -117,8 +135,49 @@ bash scripts/pr-review.sh prepare
bash scripts/pr-review.sh run
```

Unit tests use fake commands, not Vertex. Structured findings and publication
are deferred.
Unit tests use fake commands, not Vertex. The agent can already publish inline
comments directly through the allowed API endpoint. A structured findings
format and a separate publication stage remain deferred.

## Current reviewer setup

The [reusable workflow](../.github/workflows/pr-review-reusable.yml) invokes
[`setup-openshell`](../.github/actions/setup-openshell/action.yml), which installs
the pinned OpenShell CLI and waits for gateway readiness. This CI path uses a
local gateway. The trusted [`scripts/pr-review.sh`](../scripts/pr-review.sh)
wrapper creates a temporary workspace, registers `github-review` and
`vertex-review`, configures inference, renders the PR-specific policy, and
invokes the `harness` CLI. The wrapper removes the temporary providers and
workspace during cleanup.

The CLI verifies provider references and manages sandbox execution. It does
not perform the wrapper's provider provisioning or handle provider credentials.

## Managed reviewer transition

Direct managed-gateway connectivity is already implemented in the CLI. The
reusable reviewer still uses the local setup above; moving it requires an
agreed managed integration contract:

- **Runtime access:** a CI identity with the required workspace membership,
plus network access to the gateway and OIDC issuer. The HyperShell issuer
described here is reachable only from the Red Hat network/VPN.
- **Workspace isolation:** an explicit choice of shared or dedicated workspace
and the task's provider names and allowed operations.
- **Provider credential lifecycle:** platform ownership of GitHub App token
minting or refresh, repository and permission selection, and credential
replacement or expiry. Pre-provisioning a provider name does not keep an
expired installation token usable.
- **Inference and policy:** a matching provider/model route and equivalent
policy enforcement, so ordinary task runs can use existing references.

Once these requirements are met, replace the reviewer's local gateway and
temporary provider setup with managed authentication and platform-owned
resources. Preserve the task inputs and allowed GitHub operations. This
transition does not require adding provider management to the `harness` CLI.

The following bootstrap examples describe the managed validation environment;
they are not evidence that the reusable reviewer has completed this transition.

## One-time platform bootstrap

Expand Down Expand Up @@ -165,7 +224,7 @@ ordinary applies only read the matching provider and route; they neither need
workspace-admin permission nor receive the Vertex credential in the sandbox.
If a workflow selects a different provider, model, or route, the compatibility
reconciliation performs an admin-only upsert in that workspace. Treat that as
isolated-workspace setup, not a shared-workspace runtime operation; Harness does
isolated-workspace setup, not a shared-workspace runtime operation; the CLI does
not restore the previous route after the run.

Validate from the VPN with:
Expand All @@ -191,9 +250,6 @@ used by the workflow. The client secret remains in
plan, or command output. Administrator credentials remain outside repository CI
and ordinary validation.

This document is also used to exercise the label-driven artifact-only review
workflow on a small documentation-only change.

## Workflow contract

The reusable portion is the `target` block in `test/hypershell-workflow.yaml`.
Expand Down
23 changes: 19 additions & 4 deletions docs/workflow-format.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,17 @@
# Workflow format

This repository accepts one document shape: a version 1 OpenShell workflow.
The CLI command already identifies the document type, so the format does not
The `harness` CLI accepts one document shape: a version 1 harness workflow
document that composes native OpenShell inputs. This differs from a GitHub
Actions workflow, which supplies CI triggers and trusted setup. The CLI command
already identifies the document type, so the format does not
use Kubernetes-style `kind`, `apiVersion`, `metadata`, or `spec` wrappers.

## Minimal shape

This illustrates the schema. Supply a usable image, existing providers, and
the task's policy and payloads before applying it; see the
[task bundles](../workloads/) for concrete inputs.

```yaml
version: 1
name: pr-review
Expand Down Expand Up @@ -71,7 +77,16 @@ run. The current SDK adapter transfers files through OpenShell's authenticated
SSH tunnel; it does not invoke the OpenShell CLI or copy gateway credentials.

String values may contain `${VAR}` references resolved from the calling
process environment. Harness does not load `.env` files implicitly.
process environment. The `harness` CLI does not load `.env` files implicitly.

## External actions and downloaded outputs

An agent may perform operations such as posting an inline PR comment during
execution when provider permissions and OpenShell's REST policy allow them.
The `outputs` field describes downloaded files, not external actions or an
authorization record. Cleanup removes the sandbox; it does not undo a posted
comment or completed merge. A failed or cancelled run may already have
performed permitted operations.

## Security contract

Expand All @@ -85,7 +100,7 @@ Workflow, policy, and payload declarations are trusted host-side inputs. Do not
run an untrusted PR-supplied workflow with a credentialed host context; the
trusted PR-review workflow checks out its workflow from the default branch and
stages the PR diff as data. Interpolated values are redacted from display
projections, but Harness does not attempt to detect credentials embedded as
projections, but the CLI does not attempt to detect credentials embedded as
literal YAML values.

Inference route reconciliation currently writes a changed route and therefore
Expand Down
14 changes: 14 additions & 0 deletions images/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Sandbox images

This directory contains reusable OpenShell sandbox image build contexts. An
image supplies a runtime toolchain; it does not own provider credentials,
workflow policy, skills, prompts, or task permissions.

A task bundle selects an image, and the `harness` CLI passes that image reference
to OpenShell when it creates the sandbox. Local and managed gateways use the
same contract. Add a separate image only when a task needs a materially
different system toolchain; otherwise reuse an existing image and keep
task-specific behavior in `workloads/`.

StackRox image profiles and build instructions are documented in
[`stackrox/README.md`](stackrox/README.md).
2 changes: 1 addition & 1 deletion scripts/pr-review.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ write_summary() {
[[ -d "$REVIEW_DIR" ]] || return
{
printf '## AI review: %s\n\nPR #%s; head: %s\n\n' "$state" "$REVIEW_PR" "$head"
printf 'Artifact-only model output; not an approval or a validated finding list.\n'
printf 'Advisory review; the agent may have posted inline comments. Artifacts are not an approval or a validated finding list.\n'
if [[ -n "${GITHUB_RUN_ID:-}" ]]; then
printf '\n[Review artifacts](%s/%s/actions/runs/%s#artifacts)\n' "${GITHUB_SERVER_URL:-https://github.com}" "$REVIEW_REPOSITORY" "$GITHUB_RUN_ID"
fi
Expand Down
6 changes: 4 additions & 2 deletions scripts/review/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,10 @@ Current component:

- `validate-agent-output.sh REVIEW_DIR` validates the bounded OpenCode event
stream emitted by the PR reviewer. It rejects malformed event-looking lines,
requires text and a terminal stop event, and permits only the narrow
comment-position tool failure that the PR reviewer can safely tolerate.
requires text and a terminal stop event, and recognizes narrow exceptions
for comment-position and shell parser tool failures. It checks event-stream
completion, not finding correctness or whether every external action was
appropriate. Comments can already have been posted when this check runs.

This validator is intentionally scoped to the PR-review workflow until a second
workflow demonstrates a stable event and publication contract. It is not a
Expand Down
38 changes: 35 additions & 3 deletions workflow/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,32 @@
# Workflow runner

This directory contains the Go implementation of the Harness OpenShell workflow
runner:
This directory contains the Go implementation of the `harness` CLI, the small
composition and sandbox lifecycle component of `harness-openshell`. A harness
workflow document declares one run; a GitHub Actions workflow supplies the CI
trigger and trusted setup around that run.

The runner:

- loads and validates a versioned workflow document;
- resolves a local or managed OpenShell target;
- verifies referenced providers and reconciles the declared inference route;
- creates a sandbox with the selected image, policy, provider attachments, and
command;
- uploads source and payloads, observes execution, downloads outputs, and
deletes the sandbox by default.

It does not start an OpenShell gateway, create workspaces or providers, handle
provider credentials, or choose task-specific permissions. Trusted setup or
platform administration owns those resources, while task bundles and their CI
adapters own task behavior and trust decisions.

The sandboxed agent can perform external operations through OpenShell's
credential-backed network proxy when its token permissions and task policy
allow them. Downloaded output files are separate from those operations.
Deleting a sandbox does not reverse a posted comment or completed merge, and
an execution failure does not prove that no external operation occurred.

## Layout

- `main.go` is the `harness` CLI entrypoint.
- `cmd/` contains the CLI commands.
Expand All @@ -15,7 +40,14 @@ repository tooling remain one Go module. Build the CLI with:
go build -o harness ./workflow
```

Workload bundles remain under `workloads/`; they are inputs to this runner, not
Task bundles remain under `workloads/`; they are inputs to this runner, not
Go packages. Repository-level integration tests remain under `test/` because
they exercise shell workflows and gateway lifecycle behavior rather than the
runner packages themselves.

The same binary can connect to a developer's selected local gateway or directly
to a managed gateway. The current reusable reviewer still uses
[`setup-openshell`](../.github/actions/setup-openshell/action.yml) and the
[trusted review wrapper](../scripts/pr-review.sh) for its local CI path.
Moving that integration requires managed access and provider-lifecycle setup
around the CLI; see [the transition requirements](../docs/ci.md#managed-reviewer-transition).
Loading
Loading