Skip to content

fix: avoid unsupported XSStrike flags in pentester prompt#343

Open
mason5052 wants to merge 2 commits into
vxcontrol:mainfrom
mason5052:codex/issue-335-xsstrike-args
Open

fix: avoid unsupported XSStrike flags in pentester prompt#343
mason5052 wants to merge 2 commits into
vxcontrol:mainfrom
mason5052:codex/issue-335-xsstrike-args

Conversation

@mason5052

@mason5052 mason5052 commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Add a narrow Pentester prompt guardrail for XSStrike command construction and cover it with a template-rendering regression test.

Problem

Issue #335 reports PentAGI executing xsstrike with unsupported -c and -o /dev/null arguments. Investigation did not find a hardcoded XSStrike command or -c -o /dev/null pattern in repo prompts, schemas, config, examples, or docs. The issue evidence shows the installed XSStrike help output rejects those flags, so the likely failure mode is model-generated CLI flag reuse during pentest command composition.

Solution

Update backend/pkg/templates/prompts/pentester.tmpl with a small CLI argument protocol that tells the Pentester agent to verify tool-specific flags, avoid copying flags across tools, and specifically avoid xsstrike -c or xsstrike -o unless the installed xsstrike --help documents them.

Add TestPentesterPromptXSStrikeArgumentGuidance to ensure the rendered Pentester prompt keeps this guardrail in place.

User Impact

Users running XSS testing flows should be less likely to hit the reported XSStrike argument error. This does not change runtime tool execution, Docker image contents, schemas, frontend behavior, or unrelated tools.

Test Plan

  • go test ./pkg/templates - passed in a Go-enabled verification environment.
  • git diff --check - passed.
  • rg -n -i "xsstrike|xss strike|xssstrike" backend frontend examples README.md --glob '!backend/pkg/tools/testdata/sploitus_result_nginx.json' - only the new guardrail and regression test mention XSStrike.
  • rg -n -- "xsstrike -c|xsstrike -o|-o /dev/null|unrecognized arguments" backend frontend examples README.md - only the new negative guidance/test references xsstrike -c and xsstrike -o; no stale executable example remains.
  • docker image ls vxcontrol/kali-linux / xsstrike --help - not run locally because Docker is not installed in this environment; XSStrike CLI behavior was verified from the Issue [Bug]: xsstrike: error: unrecognized arguments: -c -o /dev/null #335 evidence.

Refs #335

Update: guardrail hardening

  • The CLI argument protocol now explicitly forbids xsstrike -o /dev/null (the exact flag combination reported in [Bug]: xsstrike: error: unrecognized arguments: -c -o /dev/null #335) and documents the correct alternative for saving, reducing, or discarding output (shell redirection such as > /dev/null), addressing the root cause: the model substituting an unsupported output flag.
  • TestPentesterPromptXSStrikeArgumentGuidance was extended to also assert the xsstrike -o /dev/null negative guidance and the shell-redirection alternative.

Validation limitation: this is a prompt-level guardrail; it reduces but cannot guarantee model behavior. Runtime reproduction of #335 against a live PentAGI + LLM was not performed in this environment, so efficacy is validated structurally (the prompt now names the exact failing flags and the supported alternative) and via the template regression test, not by live execution.

Copilot AI review requested due to automatic review settings June 6, 2026 23:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds explicit CLI-flag verification guidance to the pentester system prompt (focused on XSStrike), and introduces a regression test to ensure this guidance remains present in the rendered template.

Changes:

  • Added a <cli_argument_protocol> section to the pentester prompt to discourage copying/inventing unsupported flags (with XSStrike-specific examples).
  • Added a template rendering test that asserts the new guidance (and key XSStrike strings) is included in the rendered prompt.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
backend/pkg/templates/templates_test.go Adds a regression test ensuring the pentester prompt includes XSStrike CLI-argument guidance.
backend/pkg/templates/prompts/pentester.tmpl Introduces <cli_argument_protocol> guidance discouraging unsupported flag usage and recommending --help verification.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants