Perception AI

A sentiment analysis platform that pulls live social data and surfaces public perception trends, built to explore how people actually talk about topics in real time rather than relying on static datasets.

Overview

Perception AI ingests posts from Bluesky, runs them through sentiment analysis models, and presents the results through a live dashboard with search history and user accounts.

Tech Stack

• Frontend: Next.js
• Backend: Flask, deployed on Hugging Face Spaces
• Data processing: PySpark
• NLP: Hugging Face sentiment models
• Auth & storage: Firebase (Google Auth, Firestore)

Features

• Live sentiment analysis on Bluesky posts for a given search query
• Google sign-in via Firebase Auth
• Per-user search history, synced in real time with Firestore onSnapshot
• Frontend components wired to live API data end-to-end (no mock/static data)

Architecture Notes

• Bluesky authentication uses a gunicorn-safe lazy singleton pattern to avoid re-authenticating per worker process.
• Search history documents are scoped with user-prefixed IDs to prevent cross-user data collisions in Firestore (this was a real bug caught and fixed — see below).
• Backend was migrated from Render to Hugging Face Spaces for hosting the Flask + PySpark + model inference stack.

System Design

Data flow:

1. User submits a search query from the Next.js frontend.
2. Flask API authenticates against Bluesky (via the lazy singleton client) and pulls matching posts.
3. Raw posts are passed through a PySpark pipeline for cleaning/batching before sentiment scoring.
4. Hugging Face sentiment model scores each post; aggregated results (e.g. positive/negative/neutral distribution) are returned to the frontend.
5. Query + results are written to Firestore under the user's UID-prefixed document path.
6. Firestore onSnapshot listeners push the new entry to the search history panel in real time — no polling, no manual refresh.

Why PySpark: batching and parallelizing sentiment inference across a post set rather than scoring one at a time — matters more as query result sizes grow.

Why Hugging Face Spaces over Render: Spaces gives free/cheaper GPU-backed inference for the sentiment models, which matters since PySpark + model inference is the heaviest part of the stack.

Security model: Firestore rules enforce that a document under users/{uid}/history/* can only be read/written by the matching authenticated {uid} — this is the fix for the cross-user leak, enforced server-side rather than trusted to client logic.

Tech Stats

Fill in with real numbers where you have them:

Metric	Value
Avg. API response time (search → results)	2 mins
Posts processed per query (avg / max)	400
Sentiment model	cardiffnlp/twitter-roberta-base-sentiment-latest

Running Locally

# Frontend
cd frontend
npm install
npm run dev

# Backend
cd backend
pip install -r requirements.txt
flask run

You'll need Firebase project credentials and a Hugging Face API token (or local model weights) configured via environment variables.

Status

Actively maintained. Backend hosted on Hugging Face Spaces; frontend on Vercel.