This project uses PostgreSQL (rather than PocketBase) for simpler operations, reliability, and deployment.
A full-stack news aggregation platform that pulls articles from 4 Aotearoa New Zealand news sources, stores them in PostgreSQL, and displays them in a modern React interface.
See the quickstart guide for full setup and installation instructions.
For the containerized stack, run scripts/start-compose.ps1. For Azure, follow Azure full-stack setup (GitHub Actions or scripts/deploy-azure.ps1).
- 📜 4 Active Scrapers extracting 80 articles (20 from each source)
- ✨ RSS + HTML Parsing for reliable article extraction
- 🚘 Auto-discovery system for easy scraper registration
- 🔣 Template system for adding new sources quickly
- 🚅 React Frontend with modern UI
- 📔 ASP.NET Core Backend with REST API
- 💾 PostgreSQL for persistent relational storage
| Source | Type | Articles | Status |
|---|---|---|---|
| Stuff NZ | RSS | 20+ | 📂 Working |
| RNZ | RSS | 20+ | 📂 Working |
| 1News NZ | RSS | 20+ | 📂 Working |
| NZ Herald | HTML (Semantic) | 20+ | 📂 Working |
- Frontend: React + Vite (Azure:
newsagg-uiApp Service + ACR) - Backend: ASP.NET Core 10 Web API
- Scraper: Python 3.13+ with BeautifulSoup & feedparser
- Database: PostgreSQL 17
newsagg/
├── frontend/ # React frontend
│ └── src/ # React components & styles
├── backend/ # .NET Web API
│ └── NewsAggregator.Api/
│ ├── Controllers/ # API endpoints
│ ├── Models/ # Data models
│ └── Services/ # Business logic
├── scraper/ # Python web scraper
│ ├── scrapers/ # 4 individual scrapers
│ ├── scrapers/base_scraper.py # Base template class
│ ├── services/ # API communication
│ └── debug/ # Test & debug utilities
└── docs/ # Documentation
├── Quickstart.md # Setup guide
├── NewsSources.md # Add new scrapers
└── APIDoc.md # API reference