A scalable and high-performance full-stack system designed for massive CSV file processing. Capable of handling millions of rows, this solution leverages NestJS, BullMQ, Redis, and PostgreSQL on the backend, combined with a React Native mobile client for real-time monitoring via WebSocket.
- Handles multiple concurrent CSV uploads
- Background queue-based processing with horizontal scalability
- Real-time progress tracking via WebSocket
- Optimized mobile rendering for large datasets (infinite scroll with FlatList tuning)
This monorepo contains two projects:
api
: Backend API built with NestJS, PostgreSQL, Prisma, Redis, and BullMQ.mobile
: Mobile app built with React Native, using Expo, and WebSocket support for real-time progress tracking.
From the root folder, navigate to the api/
project:
cd api
Then follow the steps in its README.md to:
-
Install dependencies
-
Set up the
.env
file -
Start Redis and PostgreSQL using Docker
-
Run database migrations
-
Start the server
From the root folder, navigate to the mobile/
project:
cd mobile
Then follow the steps in its README.md to:
-
Install dependencies
-
Configure the .env file with the API base URL
-
Start the Expo development server
-
Run the app on an emulator or physical device
-
Upload a CSV file and watch the real-time processing progress
Short demo of the system in action:
A step-by-step video tutorial explaining the project setup and architecture will be available soon.
Created by Gustavo Silveira
LinkedIn