PromptGuard is a full-stack AI safety system designed to analyze user prompts and prevent unsafe or unethical requests before they reach generative AI models.
It combines rule-based filtering, machine learning risk scoring, and explainable decisions in a clean, production-style architecture.
Recent incidents involving generative AI misuse highlight the need for pre-generation safety layers.
PromptGuard focuses on:
- Preventing harmful prompts (explicit, abusive, exploitative)
- Providing transparent explanations for decisions
- Suggesting safe rewrites instead of hard blocking when possible
- User enters a prompt
- Rule-based checks detect explicit violations
- ML model computes a risk score
- Decision engine classifies the prompt as:
- ✅ SAFE
- ✏️ REWRITE
- ❌ BLOCK
- User receives:
- Decision
- Risk score
- Explanation
- Suggested rewrite (if applicable)
- React (Vite)
- Tailwind CSS v4
- FastAPI
- Hugging Face Transformers
- Rule-based + ML hybrid safety logic
cd backend
python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt
uvicorn app:app --reload
### Frontend
```bash
cd frontend/promptguard-ui
npm install
npm run dev