Skip to content

README.md

Latest commit

 

History

History
181 lines (142 loc) · 6.17 KB

README.md

File metadata and controls

181 lines (142 loc) · 6.17 KB

Daily News Word

Daily News Word is a modern, responsive news aggregation platform that fetches, processes, and displays news articles from multiple global RSS sources. It provides real-time news with filters, pagination, caching, and automated fetching. Users can browse news by country and category, toggle between light/dark theme, and read full articles from original sources.

Features

  • Global News Aggregation: Collects news from trusted sources including BBC, NYT, The Guardian, The Hindu, Times of India, DW, Al Jazeera, and more.
  • Category & Country Filters: Filter articles by country and category (Politics, Business, Technology, Sports, Science, Health, Entertainment, etc.).
  • Responsive Frontend: Mobile-friendly interface built with Tailwind CSS with support for light/dark themes.
  • Pagination: Browse articles page by page using desktop and mobile navigation controls.
  • Caching: Frontend sessionStorage cache reduces API calls and improves performance. Backend Redis caching ensures rate limiting and faster responses.
  • Automated Fetching & Cron Jobs:
    • Daily reset at 2 AM to refresh all news.
    • Incremental fetch every 3 hours for new articles.
    • Safe retry mechanisms with concurrency limits.
  • PostgreSQL Database: Stores news with indexes for efficient queries by country, category, and publication date.
  • Summarization (Future Feature): Extracts article crux from long articles.
  • Rate Limiting: Distributed rate limiting using Redis to prevent API overuse.
  • RSS Parsers & Modular Design: Each news source has a dedicated parser that normalizes articles into a consistent format (title, crux, image_url, source, source_country, url, published_at). Parsers are modular, so adding new feeds is as simple as creating a parser file following the existing structure.
  • Example & Extensibility: For instance, the NYT parser uses rss-parser to map media:content to image_url and contentEncoded to crux. The system supports multiple global sources, and new feeds can be integrated seamlessly by adding corresponding parsers.

Screenshots

Light Mode

Light Mode

Dark Mode

Dark Mode

Tech Stack

  • Frontend: HTML, Tailwind CSS, JavaScript
  • Backend: Node.js, Express.js
  • Database: PostgreSQL
  • Caching: Redis, sessionStorage
  • RSS Parsing: rss-parser
  • Scheduling: node-cron
  • Environment Management: dotenv

Project Structure

Daily-News-Word/  
│    
├── public/  
│   ├── index.html        # Frontend HTML template  
│   └── scripts.js        # Frontend logic (rendering, pagination, caching, theme)  
│  
├── parsers/              # RSS feed parsers  
│   ├── nyt.js            # Parser for The New York Times  
│   └── bbc.js            # Parser for BBC News  
│  
├── cronJobs.js           # Scheduled jobs (daily reset + incremental fetching)  
├── newsFetcher.js        # Fetches and stores articles into PostgreSQL  
├── summarizer.js         # (Future) Article summarization  
├── server.js             # Express server and API routes  
├── redisClient.js        # Redis client connection helper  
│  
├── SQLSchema.sql         # Database schema for the news table  
├── resetSQL.sql          # Script to reset the database table  
├── redisSetup.sh         # Helper script for Redis Docker setup and management  
│  
├── screenshots/          # Screenshots for README or documentation  
│   ├── lightMode.png  
│   └── darkMode.png  
│  
├── .env                  # Environment variables (not committed to Git)  
├── .gitignore            # Git ignore rules for sensitive and generated files  
├── LICENSE               # MIT License for the project  
├── package.json          # Node.js dependencies and scripts  
└── README.md             # Project documentation

Setup & Installation

1. Clone the repository

git clone https://github.com/HattoriMan/DailyNewsWord.git
cd Daily-News-Word

2. Install dependencies

npm install

3. Configure environment variables

Create a .env file in the project root:

DB_USER=your_db_user
DB_HOST=localhost
DB_NAME=your_db_name
DB_PASSWORD=your_db_password
DB_PORT=5432
PORT=3000
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=your_redis_container_password

4. Initialize the PostgreSQL database

psql -U your_db_user -d your_db_name -f SQLSchema.sql

5. Run Redis (Docker required)

NOTE: Ensure Docker is installed and running before executing the command.

docker run -d \
--name dnw-redis \
-p 127.0.0.1:6379:6379 \
-v redis-data:/data \
redis:latest \
redis-server \
--requirepass your_redis_container_password \
--appendonly yes \
--maxmemory 256mb \
--maxmemory-policy allkeys-lru

Check status:

Check out redisSetup.sh to find relevant commands to run Redis, check status, stop container and remove it.

6. Start the backend server

node server.js

7. Start cron jobs (optional but recommended)

node cronJobs.js

8. Open the frontend

Visit http://localhost:3000 in your browser.

To reinitialize table

psql -U your_db_user -d your_db_name -f resetDB.sql

Usage

  1. Select a Country and Category to filter news.

  2. Click Fetch News to retrieve articles.

  3. Use Next / Previous buttons to navigate pages.

  4. Toggle Light/Dark Mode using the top-right button.

  5. Click Read Full Story to open the original article.

Future Features

  • Article Summarization: Summarize long articles automatically using summarizer.js.

  • Deployment & Production Setup: Prepare the platform for production deployment with proper environment configuration, process management, and server setup.

Acknowledgements

Thanks to all RSS news agencies that make this project possible:

BBC, NYT, The Guardian, The Hindu, Times of India, Deutsche Welle, Al Jazeera, France 24, ABC, CBC, Indian Express, LiveMint, and many more.

License

This project is licensed under the MIT License.