AI-Powered Market Research & Autonomous Web Intelligence Engine
Built for the WeMakeDevs × Bright Data — Into the Scrape-Verse Hackathon
Market intelligence today is manual, fragmented, and slow. Teams spend days searching Google, checking competitor pricing pages, analyzing keyword rankings, and hunting for customer leads across multiple websites.
Growth Intelligence solves this by turning a single natural-language business question into an autonomous, evidence-first intelligence pipeline powered by Bright Data:
Every metric, opportunity, and recommendation generated by the platform is backed by real, verifiable web evidence linked directly to its source URL.
┌──────────────────────────────────────────────┐
│ Natural Language Prompt │
│ "Analyze digital certificate market in India"│
└──────────────────────┬───────────────────────┘
│
▼
┌──────────────────────────────────────────────┐
│ Orchestrator Agent (NVIDIA NIM) │
│ Decomposes intent into 3-4 search queries │
└──────────────────────┬───────────────────────┘
│
▼
┌──────────────────────────────────────────────┐
│ Web Discovery (Bright Data SERP API) │
│ Fetches live Google organic search rankings│
└──────────────────────┬───────────────────────┘
│
▼
┌──────────────────────────────────────────────┐
│ URL Normalization & Classification │
│ Tags URLs: Competitor | Pricing | SEO | Lead│
└──────────────────────┬───────────────────────┘
│
▼
┌──────────────────────────────────────────────┐
│ Bright Data Scraper Studio Collectors │
│ • Competitor & Catalog: c_mt5uz67r1w2doly4u1│
│ • Target Audience/Lead: c_mt5vsb1h1zodlpw0xw│
└──────────────────────┬───────────────────────┘
│
▼
┌───────────────────────────────────────┴───────────────────────────────────────┐
│ │
▼ ▼ ▼
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Competitor │ │ Pricing │ │ SEO Radar │
│ Agent │ │ Agent │ │ Agent │
└──────┬───────┘ └──────┬───────┘ └──────┬───────┘
│ │ │
└──────────────────────────────────────┼──────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────┐
│ Insight & Synthesis Agent │
│ Generates factor-weighted growth ROI actions │
└──────────────────────┬───────────────────────┘
│
▼
┌──────────────────────────────────────────────┐
│ Evidence-First Executive React Dashboard │
└──────────────────────────────────────────────┘
- Converts high-level business goals into precise, high-intent Google search keywords.
- Queries Google SERP in real time via the Bright Data SERP API (
/requestendpoint withglcountry targeting). - Captures organic rank, snippet, domain, and title provenance for every discovered URL.
- Combines heuristic path/domain matching with LLM intent classification.
- Automatically routes URLs to specialized downstream extraction pipelines (Competitors, Pricing, SEO, Target Audience).
- Competitor Intelligence Collector (
c_mt5uz67r1w2doly4u1): Extracts company name, tagline, core services, target audience, products, tech stack, and collaborators. - Target Audience & Organization Collector (
c_mt5vsb1h1zodlpw0xw): Extracts organization profiles, credential demand signals, public contact URLs, and verifiable use cases.
- 🏢 Competitor Agent: Tracks positioning, product catalog matrix, and market share.
- 💰 Pricing Agent: Monitors pricing plans, billing intervals, and detects aggressive price drops/hikes.
- 📈 SEO Radar Agent: Identifies high-intent topic vacuums from SERP observations and generates complete landing page blueprints.
- 👥 Target Audience Agent: Discovers qualified business leads and institutional demand with contact links.
- 💡 Insight & Opportunity Agent: Synthesizes cross-agent findings into high-ROI actionable strategies with confidence scores and factor weights.
- Operates entirely on live, dynamic web data extracted through Bright Data and stored in a persistent SQLite database.
- Full provenance transparency: every insight links back to the exact URL and timestamp of collection.
| Layer | Technologies |
|---|---|
| Web Data & Scraping | Bright Data SERP API, Bright Data Scraper Studio, Bright Data Unlocker |
| AI / LLM | NVIDIA NIM (meta/muse-glimmer-30b), OpenAI SDK |
| Backend API | FastAPI (Python 3.11), SQLAlchemy Async, SQLite (aiosqlite), Pydantic v2 |
| Frontend UI | React 18, Vite, TypeScript, Tailwind CSS, Lucide Icons |
| State & Networking | React Context API, HTTPX, Axios, WebSocket/Polling real-time sync |
- Node.js (v18+)
- Python (3.11+)
- Bright Data API Token (configured in
server/.env)
cd server
python -m venv venv
venv\Scripts\activate # Windows
# source venv/bin/activate # Linux/Mac
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8000API documentation available at http://localhost:8000/docs.
cd app
npm install
npm run devWeb dashboard opens at http://localhost:5173.
- Navigate to the New Research page or open the global Command Palette (
Ctrl + K/Cmd + K). - Enter your research question (e.g., "Digital certificate and ID card generator companies in India").
- The platform executes the multi-agent pipeline:
- Formulating search plan with NVIDIA NIM
- Discovering organic competitors via Bright Data SERP API
- Classifying candidate URLs
- Triggering Scraper Studio collectors
- Synthesizing pricing shifts, SEO radar gaps, and customer opportunities
- Review the completed report with interactive breakdowns for Discovered Websites, Competitors, Pricing, SEO Blueprints, and Actionable Opportunities.
Developed for the WeMakeDevs × Bright Data — Into the Scrape-Verse Hackathon.
Special thanks to Bright Data for their web scraping infrastructure, SERP API, and Scraper Studio platform.