fix: avoid unsupported XSStrike flags in pentester prompt#343
Open
mason5052 wants to merge 2 commits into
Open
fix: avoid unsupported XSStrike flags in pentester prompt#343mason5052 wants to merge 2 commits into
mason5052 wants to merge 2 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds explicit CLI-flag verification guidance to the pentester system prompt (focused on XSStrike), and introduces a regression test to ensure this guidance remains present in the rendered template.
Changes:
- Added a
<cli_argument_protocol>section to the pentester prompt to discourage copying/inventing unsupported flags (with XSStrike-specific examples). - Added a template rendering test that asserts the new guidance (and key XSStrike strings) is included in the rendered prompt.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| backend/pkg/templates/templates_test.go | Adds a regression test ensuring the pentester prompt includes XSStrike CLI-argument guidance. |
| backend/pkg/templates/prompts/pentester.tmpl | Introduces <cli_argument_protocol> guidance discouraging unsupported flag usage and recommending --help verification. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a narrow Pentester prompt guardrail for XSStrike command construction and cover it with a template-rendering regression test.
Problem
Issue #335 reports PentAGI executing
xsstrikewith unsupported-cand-o /dev/nullarguments. Investigation did not find a hardcoded XSStrike command or-c -o /dev/nullpattern in repo prompts, schemas, config, examples, or docs. The issue evidence shows the installed XSStrike help output rejects those flags, so the likely failure mode is model-generated CLI flag reuse during pentest command composition.Solution
Update
backend/pkg/templates/prompts/pentester.tmplwith a small CLI argument protocol that tells the Pentester agent to verify tool-specific flags, avoid copying flags across tools, and specifically avoidxsstrike -corxsstrike -ounless the installedxsstrike --helpdocuments them.Add
TestPentesterPromptXSStrikeArgumentGuidanceto ensure the rendered Pentester prompt keeps this guardrail in place.User Impact
Users running XSS testing flows should be less likely to hit the reported XSStrike argument error. This does not change runtime tool execution, Docker image contents, schemas, frontend behavior, or unrelated tools.
Test Plan
go test ./pkg/templates- passed in a Go-enabled verification environment.git diff --check- passed.rg -n -i "xsstrike|xss strike|xssstrike" backend frontend examples README.md --glob '!backend/pkg/tools/testdata/sploitus_result_nginx.json'- only the new guardrail and regression test mention XSStrike.rg -n -- "xsstrike -c|xsstrike -o|-o /dev/null|unrecognized arguments" backend frontend examples README.md- only the new negative guidance/test referencesxsstrike -candxsstrike -o; no stale executable example remains.docker image ls vxcontrol/kali-linux/xsstrike --help- not run locally because Docker is not installed in this environment; XSStrike CLI behavior was verified from the Issue [Bug]: xsstrike: error: unrecognized arguments: -c -o /dev/null #335 evidence.Refs #335
Update: guardrail hardening
xsstrike -o /dev/null(the exact flag combination reported in [Bug]: xsstrike: error: unrecognized arguments: -c -o /dev/null #335) and documents the correct alternative for saving, reducing, or discarding output (shell redirection such as> /dev/null), addressing the root cause: the model substituting an unsupported output flag.TestPentesterPromptXSStrikeArgumentGuidancewas extended to also assert thexsstrike -o /dev/nullnegative guidance and the shell-redirection alternative.Validation limitation: this is a prompt-level guardrail; it reduces but cannot guarantee model behavior. Runtime reproduction of #335 against a live PentAGI + LLM was not performed in this environment, so efficacy is validated structurally (the prompt now names the exact failing flags and the supported alternative) and via the template regression test, not by live execution.