Skip to content

KoushikAD1234/Real-time-LOG-monitoring-platform

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Log-Watcher-Web-App

Real time Log watcher web based application with websocket and Redis-adapter.

Directory structure

  1. Client
  2. Server : server.js , socket.js 2a. services : tailService.js
  3. log : contain the main app.log file which needs to be tracked.
  4. log generator : generator.js : responsible for generating content in app.log file in every n sex interval

Setting up the Project

To start with frontend:

  1. npx create-react-app client-logger.
  2. npm i socket.io-client // need to google once before running this command.
  3. To start the project: npm start

To start with backend:

  1. npm i
  2. npm i cors
  3. npm i express
  4. npm i socket.io (Need to google)
  5. npm i redis (Need to google)
  6. npm i @socket.io/redis-adapter (Need to google)
  7. To run the project : node server.js / nodemon server.js (npm i nodemon)

To run the redis server over docker, run this command: docker run -p 6379:6379 redis

Tradeoffs and Explanation of concepts:

  1. We use a Socket.IO adapter with Redis mainly for scaling the WebSocket system across multiple server instances. Without it, code works only if there is one Node.js server.
  2. Redis acts as a central message broker.
  3. Instead of local event broadcasting, use Redis pub/sub to synchronize events across servers.
  4. Redis is good for this because it is: -> very fast -> in-memory -> supports pub/sub -> simple to integrate
  5. "In production we would use Redis adapter for horizontal scaling."

About

A distributed real-time log monitoring system that ingests high-volume logs, processes them using stream pipelines, and provides searchable, actionable insights with low latency.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors