Skip to content

JVikramV/PromptGuard

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 

Repository files navigation

🛡️ PromptGuard

PromptGuard is a full-stack AI safety system designed to analyze user prompts and prevent unsafe or unethical requests before they reach generative AI models.

It combines rule-based filtering, machine learning risk scoring, and explainable decisions in a clean, production-style architecture.


🚀 Why PromptGuard?

Recent incidents involving generative AI misuse highlight the need for pre-generation safety layers.

PromptGuard focuses on:

  • Preventing harmful prompts (explicit, abusive, exploitative)
  • Providing transparent explanations for decisions
  • Suggesting safe rewrites instead of hard blocking when possible

🧠 How It Works

  1. User enters a prompt
  2. Rule-based checks detect explicit violations
  3. ML model computes a risk score
  4. Decision engine classifies the prompt as:
    • ✅ SAFE
    • ✏️ REWRITE
    • ❌ BLOCK
  5. User receives:
    • Decision
    • Risk score
    • Explanation
    • Suggested rewrite (if applicable)

🏗️ Tech Stack

Frontend

  • React (Vite)
  • Tailwind CSS v4

Backend

  • FastAPI
  • Hugging Face Transformers
  • Rule-based + ML hybrid safety logic

📸 Screenshots

🧠 Prompt Analysis Interface

Prompt Analysis UI

❌ Blocked Prompt Example

Blocked Prompt

✏️ Rewrite Suggestion Example

Rewrite Prompt

✅ Safe Prompt Example

Safe Prompt

⚙️ Run Locally

Backend

cd backend
python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt
uvicorn app:app --reload

### Frontend

```bash
cd frontend/promptguard-ui
npm install
npm run dev


About

AI safety layer to analyze and prevent unsafe prompts

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages