|
1 | | -# Instagram Event Scraper API |
| 1 | +# Wat2Do |
2 | 2 |
|
3 | | -This project has been restructured to follow a modern Django-based architecture. |
| 3 | +<p align="center"> |
| 4 | + <img src="frontend/public/wat2do-logo.svg" alt="Wat2Do Logo" width="180"/> |
| 5 | +</p> |
4 | 6 |
|
5 | | -## Project Structure |
| 7 | +<p align="center"> |
| 8 | + <a href="https://wat2do.ca" target="_blank"> |
| 9 | + <img src="https://img.shields.io/badge/Live%20Site-wat2do.ca-blue?style=flat-square" alt="Live Site"/> |
| 10 | + </a> |
| 11 | + <a href="https://github.com/ericahan22/bug-free-octo-spork/actions"> |
| 12 | + <img src="https://img.shields.io/github/actions/workflow/status/ericahan22/bug-free-octo-spork/update-events-data.yml?branch=main&style=flat-square" alt="GitHub Actions Status"/> |
| 13 | + </a> |
| 14 | + <img src="https://img.shields.io/badge/Backend-Django%204.2.7-green?style=flat-square" alt="Django Version"/> |
| 15 | + <img src="https://img.shields.io/badge/Frontend-React%2019.1.0-blue?style=flat-square" alt="React Version"/> |
| 16 | +</p> |
6 | 17 |
|
7 | | -``` |
8 | | -my_project/ |
9 | | -│ |
10 | | -├── docker-compose.yml |
11 | | -├── requirements.txt |
12 | | -│ |
13 | | -├── backend/ |
14 | | -│ ├── Dockerfile |
15 | | -│ ├── manage.py |
16 | | -│ ├── requirements.txt |
17 | | -│ |
18 | | -│ ├── scraping/ # Placeholder for future scraping logic |
19 | | -│ │ └── __init__.py |
20 | | -│ |
21 | | -│ ├── my_django_project/ # Django project folder (settings, etc.) |
22 | | -│ │ ├── __init__.py |
23 | | -│ │ ├── settings.py |
24 | | -│ │ ├── urls.py |
25 | | -│ │ ├── wsgi.py |
26 | | -│ │ └── asgi.py |
27 | | -│ |
28 | | -│ ├── app/ # API app (no templates/static) |
29 | | -│ │ ├── __init__.py |
30 | | -│ │ ├── apps.py |
31 | | -│ │ ├── urls.py |
32 | | -│ │ └── views.py |
33 | | -│ |
34 | | -│ └── scripts/ # Scraping scripts and data files |
35 | | -│ ├── __init__.py |
36 | | -│ ├── ai_client.py |
37 | | -│ ├── instagram_feed.py |
38 | | -│ ├── wusa_club_directory_scraper.py |
39 | | -│ ├── club_info.csv |
40 | | -│ └── event_info.csv |
41 | | -│ |
42 | | -└── frontend/ # React app (handled independently) |
43 | | - └── ... |
44 | | -``` |
| 18 | +<a href="https://wat2do.ca" target="_blank">Wat2Do.ca</a> is a web app to help you discover club events at the University of |
| 19 | +Waterloo, scraped directly from Instagram. |
45 | 20 |
|
46 | | -## Migration Summary |
| 21 | +## ✨ Features |
47 | 22 |
|
48 | | -### Changes Made: |
49 | | -1. **Restructured folders**: Moved scraping files from `/scraping/` to `/backend/scripts/` |
50 | | -2. **Migrated from Flask to Django**: Converted Flask API to Django REST Framework |
51 | | -3. **Updated requirements.txt**: Replaced Flask dependencies with Django and DRF |
52 | | -4. **Created Docker setup**: Added Dockerfile for backend containerization |
53 | | -5. **Created empty frontend folder**: Ready for React development |
54 | | -6. **Created empty docker-compose.yml**: Ready for multi-container setup |
| 23 | +- **Browse, search, and filter events:** See upcoming and past events from campus clubs |
| 24 | +- **Club directory:** Explore all clubs with links to their website/Instagram |
| 25 | +- **Email newsletter:** Subscribe to get the latest events in your inbox, once daily |
55 | 26 |
|
56 | | -### API Endpoints (Django REST Framework): |
57 | | -- `GET /api/` - Home endpoint with API info |
58 | | -- `GET /api/health/` - Health check |
59 | | -- `GET /api/events/` - Get all events from event_info.csv |
60 | | -- `GET /api/clubs/` - Get all clubs from club_info.csv |
61 | | -- `GET /api/events/search/?club_name=<name>` - Search events by club name |
| 27 | +**How It Works:** |
| 28 | +Events are automatically collected from club Instagram posts, using the OpenAI API to extract details from captions and images. Data is refreshed daily. |
62 | 29 |
|
63 | | -## Setup Instructions |
| 30 | +## 🛠️ Project Structure |
64 | 31 |
|
65 | | -### Frontend (React) |
66 | | -```bash |
67 | | -cd frontend |
68 | | -npm i |
69 | | -npm run dev |
70 | 32 | ``` |
| 33 | +. |
| 34 | +├── backend/ |
| 35 | +│ ├── manage.py |
| 36 | +│ ├── requirements.txt |
| 37 | +│ ├── apps/ # Django apps (events, clubs, newsletter, etc.) |
| 38 | +│ ├── scraping/ # Instagram scraping scripts |
| 39 | +│ └── config/ # Settings and URLs |
| 40 | +├── frontend/ |
| 41 | +│ ├── src/ |
| 42 | +│ │ ├── app/ |
| 43 | +│ │ │ └── App.tsx |
| 44 | +│ │ ├── features/ |
| 45 | +│ │ └── shared/ |
| 46 | +│ │ ├── components/ |
| 47 | +│ │ └── hooks/ |
| 48 | +│ ├── package.json |
| 49 | +│ └── public/ |
| 50 | +``` |
| 51 | + |
| 52 | +### Environment Setup |
71 | 53 |
|
72 | | -### Backend (Django) |
| 54 | +#### Backend |
73 | 55 | ```bash |
74 | 56 | cd backend |
| 57 | +python -m venv .venv |
75 | 58 | source .venv/bin/activate |
76 | 59 | pip install -r requirements.txt |
| 60 | +export USE_SQLITE=1 |
77 | 61 | python manage.py migrate |
78 | | -python manage.py runserver |
| 62 | +python manage.py runserver 8000 |
79 | 63 | ``` |
80 | 64 |
|
81 | | -### Docker |
| 65 | +#### Frontend |
82 | 66 | ```bash |
83 | | -cd backend |
84 | | -docker build -t instagram-scraper-api . |
85 | | -docker run -p 3000:3000 instagram-scraper-api |
| 67 | +cd frontend |
| 68 | +npm install |
| 69 | +npm run build |
| 70 | +npm run dev |
86 | 71 | ``` |
87 | 72 |
|
88 | | -### Database Migrations |
89 | | -```bash |
90 | | -# 1. Modify model file |
91 | | -# 2. Generate migration |
92 | | -python manage.py makemigrations events |
93 | | -# 2.5 Double review the generated migration file before applying |
94 | | -# 2.6 Run a dry run of the generated migration |
95 | | -python manage.py migrate --dry-run |
96 | | -# 3. Apply migration |
97 | | -python manage.py migrate events |
98 | | -``` |
| 73 | +## 🏗️ Build & Test |
| 74 | + |
| 75 | +### Backend |
| 76 | +- `python manage.py check` — Django config check |
| 77 | +- `python manage.py test` — Run Django tests |
| 78 | +- `python manage.py migrate` — Apply migrations |
| 79 | + |
| 80 | +### Frontend |
| 81 | +- `npm run build` — Production build |
| 82 | +- `npm run lint` — ESLint check |
| 83 | +- `npm run dev` — Dev server |
| 84 | +- `npm run preview` — Preview production build |
| 85 | + |
| 86 | +## 🌐 API Endpoints |
| 87 | + |
| 88 | +| Description | URL/Command | |
| 89 | +|---------------------|----------------------------------------------------------| |
| 90 | +| API info | `curl http://localhost:8000/` | |
| 91 | +| Health check | `curl http://localhost:8000/health/` | |
| 92 | +| Get all events | `curl http://localhost:8000/events/` | |
| 93 | +| Get all clubs | `curl http://localhost:8000/clubs/` | |
| 94 | +| Find similar events | `curl "http://localhost:8000/test-similarity/?text=..."` | |
| 95 | + |
| 96 | +--- |
| 97 | + |
| 98 | +## 🤝 Support |
| 99 | + |
| 100 | +If you have questions or feedback, please reach out at <a href="https://wat2do.ca/contact" target="_blank">wat2do.ca/contact</a> or add a <a href="https://github.com/ericahan22/bug-free-octo-spork/issues" target="_blank">GitHub issue</a>. |
99 | 101 |
|
100 | | -## Dependencies |
101 | | - |
102 | | -### Backend Requirements: |
103 | | -- Django 4.2.7 |
104 | | -- Django REST Framework 3.14.0 |
105 | | -- Django CORS Headers 4.3.1 |
106 | | -- Pandas 2.0.3 |
107 | | -- NumPy 1.24.3 |
108 | | -- Gunicorn 21.2.0 |
109 | | -- Plus scraping utilities (instaloader, requests, beautifulsoup4, openai) |
110 | | - |
111 | | -## Notes |
112 | | - |
113 | | -- The Django server runs on port 3000 |
114 | | -- CSV files are now located in `/backend/scripts/` |
115 | | -- API endpoints are prefixed with `/api/` |
116 | | -- CORS is enabled for frontend integration |
117 | | -- Database is SQLite (can be changed to PostgreSQL for production) |
118 | | - |
119 | | -## API Commands |
120 | | - |
121 | | -| Goal/Description | cURL Command | |
122 | | -|------------------|--------------| |
123 | | -| Get API info | `curl http://localhost:3000/api/` | |
124 | | -| Health check | `curl http://localhost:3000/health/` | |
125 | | -| Get all events | `curl http://localhost:3000/api/events/` | |
126 | | -| Get all clubs | `curl http://localhost:3000/api/clubs/` | |
127 | | -| Find similar events | `curl "http://localhost:3000/api/test-similarity/?text=Your%20Search%20Text"` | |
| 102 | +Enjoy discovering events! |
0 commit comments