Skip to content

Commit 888c41a

Browse files
committed
Implement Firebase Storage rules and add functions for PDF upload processing
1 parent 209ba4f commit 888c41a

File tree

9 files changed

+1265
-19
lines changed

9 files changed

+1265
-19
lines changed

functions/README.md

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
# Firebase Cloud Functions for Grade Prediction
2+
3+
This directory contains the Firebase Cloud Functions for the grade prediction application. These functions handle PDF processing, OCR, grade prediction, and data storage.
4+
5+
## Overview
6+
7+
The application uses Firebase Cloud Functions to:
8+
9+
1. Upload and process PDF documents (syllabus and transcript)
10+
2. Extract text from PDFs using OCR
11+
3. Predict final grades using both ChatGPT and Linear Regression models
12+
4. Store predictions and analyses in Firestore
13+
14+
## Function Categories
15+
16+
### Authentication Functions
17+
18+
- `create_user_profile`: Create a user profile after registration
19+
- `delete_user_data`: Delete a user's data when account is deleted
20+
- `get_user_profile`: Get a user's profile data
21+
- `update_user_profile`: Update a user's profile data
22+
23+
### OCR Functions
24+
25+
- `extract_text_from_pdf`: Extract text from a PDF file
26+
- `process_syllabus`: Process syllabus PDF to extract course information
27+
- `process_transcript`: Process transcript PDF to extract grade information
28+
- `process_uploaded_pdf`: Triggered when a PDF is uploaded to Firebase Storage
29+
30+
### OpenAI API Functions
31+
32+
- `analyze_grades`: Analyze extracted grade data using OpenAI API
33+
- `predict_final_grade`: Predict final grade based on current grades and weights
34+
- `extract_assignments`: Extract upcoming assignments and exams from syllabus
35+
36+
### Storage Functions
37+
38+
- `get_upload_url`: Generate a signed URL for uploading a PDF to Firebase Storage
39+
40+
### ML Prediction Functions
41+
42+
- `predict_with_linear_regression`: Predict final grade using Linear Regression model
43+
- `add_training_data`: Add new training data for the linear regression model
44+
45+
### Combined Prediction Functions
46+
47+
- `get_combined_prediction`: Get combined prediction from both ChatGPT and Linear Regression models
48+
- `get_latest_predictions`: Get the latest predictions for a user
49+
50+
### Document Processing Functions
51+
52+
- `upload_and_process_document`: Upload and process a document (syllabus or transcript)
53+
- `get_document_status`: Get the status of a document upload and processing
54+
- `get_user_documents`: Get all documents uploaded by a user
55+
56+
## Workflow
57+
58+
1. User uploads a syllabus and transcript PDF
59+
2. PDFs are stored in Firebase Storage
60+
3. OCR extracts text from PDFs
61+
4. Extracted text is processed to get structured data
62+
5. Structured data is used to predict final grades
63+
6. Predictions are stored in Firestore
64+
65+
## Database Structure
66+
67+
### Firestore Collections
68+
69+
- `/users/{userId}`: User profile data
70+
- `/users/{userId}/documents/{documentType}`: Extracted text from PDFs
71+
- `/users/{userId}/syllabi/{syllabusId}`: Structured syllabus data
72+
- `/users/{userId}/transcripts/{transcriptId}`: Structured transcript data
73+
- `/users/{userId}/predictions/{predictionId}`: ChatGPT predictions
74+
- `/users/{userId}/ml_predictions/{predictionId}`: Linear Regression predictions
75+
- `/users/{userId}/combined_predictions/{predictionId}`: Combined predictions
76+
- `/users/{userId}/analyses/{analysisId}`: Grade analyses
77+
- `/users/{userId}/assignments/{assignmentsId}`: Extracted assignments
78+
- `/users/{userId}/document_uploads/{documentId}`: Document upload metadata
79+
- `/training_data/students`: Training data for Linear Regression model
80+
81+
### Firebase Storage
82+
83+
- `/users/{userId}/syllabus/{filename}.pdf`: Syllabus PDFs
84+
- `/users/{userId}/transcript/{filename}.pdf`: Transcript PDFs
85+
86+
## Environment Variables
87+
88+
- `OPENAI_API_KEY`: OpenAI API key for ChatGPT predictions
89+
90+
## Dependencies
91+
92+
- `firebase_functions`: Firebase Cloud Functions SDK
93+
- `firebase_admin`: Firebase Admin SDK
94+
- `google-cloud-firestore`: Google Cloud Firestore SDK
95+
- `openai`: OpenAI API SDK
96+
- `numpy`: Numerical computing library
97+
- `scikit-learn`: Machine learning library
98+
- `PyMuPDF`: PDF processing library
99+
- `python-dotenv`: Environment variable management

functions/combined_predict.py

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
from firebase_functions import https_fn
2+
from firebase_admin import firestore
3+
import google.cloud.firestore
4+
import json
5+
import os
6+
7+
@https_fn.on_call()
8+
def get_combined_prediction(req: https_fn.CallableRequest) -> dict:
9+
"""
10+
Get combined prediction from both ChatGPT and Linear Regression models.
11+
"""
12+
if not req.auth:
13+
raise https_fn.HttpsError(
14+
code=https_fn.FunctionsErrorCode.UNAUTHENTICATED,
15+
message="User must be authenticated"
16+
)
17+
18+
user_id = req.auth.uid
19+
20+
# Get prediction IDs from request
21+
chatgpt_prediction_id = req.data.get("chatgptPredictionId")
22+
ml_prediction_id = req.data.get("mlPredictionId")
23+
24+
if not chatgpt_prediction_id or not ml_prediction_id:
25+
raise https_fn.HttpsError(
26+
code=https_fn.FunctionsErrorCode.INVALID_ARGUMENT,
27+
message="Both ChatGPT and ML prediction IDs are required"
28+
)
29+
30+
# Get predictions from Firestore
31+
try:
32+
db = firestore.client()
33+
34+
# Get ChatGPT prediction
35+
chatgpt_prediction_ref = db.collection("users").document(user_id).collection("predictions").document(chatgpt_prediction_id)
36+
chatgpt_prediction_doc = chatgpt_prediction_ref.get()
37+
38+
if not chatgpt_prediction_doc.exists:
39+
raise https_fn.HttpsError(
40+
code=https_fn.FunctionsErrorCode.NOT_FOUND,
41+
message="ChatGPT prediction not found"
42+
)
43+
44+
chatgpt_prediction = chatgpt_prediction_doc.to_dict()
45+
46+
# Get ML prediction
47+
ml_prediction_ref = db.collection("users").document(user_id).collection("ml_predictions").document(ml_prediction_id)
48+
ml_prediction_doc = ml_prediction_ref.get()
49+
50+
if not ml_prediction_doc.exists:
51+
raise https_fn.HttpsError(
52+
code=https_fn.FunctionsErrorCode.NOT_FOUND,
53+
message="ML prediction not found"
54+
)
55+
56+
ml_prediction = ml_prediction_doc.to_dict()
57+
58+
# Combine predictions
59+
chatgpt_grade = chatgpt_prediction["prediction"]["grade"]
60+
ml_grade = ml_prediction["prediction"]["grade"]
61+
62+
# Simple average of both predictions
63+
combined_grade = (float(chatgpt_grade) + float(ml_grade)) / 2
64+
65+
# Create combined prediction
66+
combined_prediction = {
67+
"grade": combined_grade,
68+
"chatgpt_grade": chatgpt_grade,
69+
"ml_grade": ml_grade,
70+
"reasoning": chatgpt_prediction["prediction"]["reasoning"],
71+
"confidence": "medium" # Default confidence
72+
}
73+
74+
# Determine confidence based on agreement between models
75+
grade_difference = abs(float(chatgpt_grade) - float(ml_grade))
76+
if grade_difference < 5:
77+
combined_prediction["confidence"] = "high"
78+
elif grade_difference > 15:
79+
combined_prediction["confidence"] = "low"
80+
81+
# Store combined prediction in Firestore
82+
combined_ref = db.collection("users").document(user_id).collection("combined_predictions").document()
83+
84+
combined_data = {
85+
"chatgptPrediction": chatgpt_prediction,
86+
"mlPrediction": ml_prediction,
87+
"combinedPrediction": combined_prediction,
88+
"createdAt": firestore.SERVER_TIMESTAMP
89+
}
90+
91+
combined_ref.set(combined_data)
92+
93+
return {
94+
"success": True,
95+
"predictionId": combined_ref.id,
96+
"prediction": combined_prediction
97+
}
98+
99+
except Exception as e:
100+
raise https_fn.HttpsError(
101+
code=https_fn.FunctionsErrorCode.INTERNAL,
102+
message=f"Error getting combined prediction: {str(e)}"
103+
)
104+
105+
@https_fn.on_call()
106+
def get_latest_predictions(req: https_fn.CallableRequest) -> dict:
107+
"""
108+
Get the latest predictions for a user.
109+
"""
110+
if not req.auth:
111+
raise https_fn.HttpsError(
112+
code=https_fn.FunctionsErrorCode.UNAUTHENTICATED,
113+
message="User must be authenticated"
114+
)
115+
116+
user_id = req.auth.uid
117+
118+
# Get predictions from Firestore
119+
try:
120+
db = firestore.client()
121+
122+
# Get latest ChatGPT prediction
123+
chatgpt_predictions = db.collection("users").document(user_id).collection("predictions").order_by("createdAt", direction=firestore.Query.DESCENDING).limit(1).stream()
124+
chatgpt_prediction = None
125+
for doc in chatgpt_predictions:
126+
chatgpt_prediction = doc.to_dict()
127+
chatgpt_prediction["id"] = doc.id
128+
break
129+
130+
# Get latest ML prediction
131+
ml_predictions = db.collection("users").document(user_id).collection("ml_predictions").order_by("createdAt", direction=firestore.Query.DESCENDING).limit(1).stream()
132+
ml_prediction = None
133+
for doc in ml_predictions:
134+
ml_prediction = doc.to_dict()
135+
ml_prediction["id"] = doc.id
136+
break
137+
138+
# Get latest combined prediction
139+
combined_predictions = db.collection("users").document(user_id).collection("combined_predictions").order_by("createdAt", direction=firestore.Query.DESCENDING).limit(1).stream()
140+
combined_prediction = None
141+
for doc in combined_predictions:
142+
combined_prediction = doc.to_dict()
143+
combined_prediction["id"] = doc.id
144+
break
145+
146+
return {
147+
"success": True,
148+
"chatgptPrediction": chatgpt_prediction,
149+
"mlPrediction": ml_prediction,
150+
"combinedPrediction": combined_prediction
151+
}
152+
153+
except Exception as e:
154+
raise https_fn.HttpsError(
155+
code=https_fn.FunctionsErrorCode.INTERNAL,
156+
message=f"Error getting latest predictions: {str(e)}"
157+
)

0 commit comments

Comments
 (0)