Skip to content

Commit da14206

Browse files
richieb21YashK2005
authored and
lucyqqi
committed
[LLSC-82] Intake Form DB Migrations + CRUD operations (#45)
## Notion ticket link <!-- Please replace with your ticket's URL --> [Intake Form Crud](https://www.notion.so/uwblueprintexecs/Task-Board-db95cd7b93f245f78ee85e3a8a6a316d) <!-- Give a quick summary of the implementation details, provide design justifications if necessary --> ## Implementation description Added CRUD endpoints, migrations and schemas for the intake form. <!-- What should the reviewer do to verify your changes? Describe expected results and include screenshots when appropriate --> ## Steps to test 1. Run locally using docker compose 2. Verify that the migrations run and that the tables show up in psql 3. Create a sample user to create an intake form submissions using Postman <!-- Draw attention to the substantial parts of your PR or anything you'd like a second opinion on --> ## What should reviewers focus on? * Make sure the migrations and indexes make sense * That the authentication logic and protection is appropriate * That the data being stored is correct and encapsulates everything needed ## Checklist - [ ] My PR name is descriptive and in imperative tense - [ ] My commit messages are descriptive and in imperative tense. My commits are atomic and trivial commits are squashed or fixup'd into non-trivial commits - [ ] I have run the appropriate linter(s) - [ ] I have requested a review from the PL, as well as other devs who have background knowledge on this PR or who will be building on top of this PR --------- Co-authored-by: YashK2005 <[email protected]>
1 parent 5bb3ff6 commit da14206

31 files changed

+4444
-575
lines changed

‎.github/workflows/backend-ci.yml‎

Lines changed: 214 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,214 @@
1+
name: Backend CI
2+
3+
on:
4+
push:
5+
branches: [ main, develop ]
6+
paths:
7+
- 'backend/**'
8+
- '.github/workflows/backend-ci.yml'
9+
pull_request:
10+
branches: [ main, develop ]
11+
paths:
12+
- 'backend/**'
13+
- '.github/workflows/backend-ci.yml'
14+
15+
jobs:
16+
test:
17+
runs-on: ubuntu-latest
18+
19+
strategy:
20+
matrix:
21+
python-version: [3.12]
22+
23+
services:
24+
postgres:
25+
image: postgres:15
26+
env:
27+
POSTGRES_PASSWORD: testpassword
28+
POSTGRES_USER: testuser
29+
POSTGRES_DB: llsc_test
30+
options: >-
31+
--health-cmd pg_isready
32+
--health-interval 10s
33+
--health-timeout 5s
34+
--health-retries 5
35+
ports:
36+
- 5432:5432
37+
38+
steps:
39+
- uses: actions/checkout@v4
40+
41+
- name: Set up Python ${{ matrix.python-version }}
42+
uses: actions/setup-python@v5
43+
with:
44+
python-version: ${{ matrix.python-version }}
45+
46+
- name: Install PDM
47+
run: |
48+
pip install pdm
49+
50+
- name: Cache PDM dependencies
51+
uses: actions/cache@v4
52+
with:
53+
path: ~/.cache/pdm
54+
key: ${{ runner.os }}-pdm-${{ hashFiles('backend/pdm.lock') }}
55+
restore-keys: |
56+
${{ runner.os }}-pdm-
57+
58+
- name: Install dependencies
59+
working-directory: ./backend
60+
run: |
61+
pdm sync --group test --group lint --group dev
62+
63+
- name: Set up environment variables
64+
working-directory: ./backend
65+
run: |
66+
echo "POSTGRES_DATABASE_URL=postgresql://testuser:testpassword@localhost:5432/llsc_test" >> .env
67+
echo "SECRET_KEY=test-secret-key-for-ci" >> .env
68+
echo "ENVIRONMENT=test" >> .env
69+
70+
- name: Run database migrations
71+
working-directory: ./backend
72+
run: |
73+
pdm run alembic upgrade head
74+
75+
- name: Run linting
76+
working-directory: ./backend
77+
run: |
78+
pdm run ruff check .
79+
pdm run ruff format --check .
80+
81+
# TODO: Re-enable mypy when type annotations are improved
82+
# - name: Run type checking
83+
# working-directory: ./backend
84+
# run: |
85+
# pdm run mypy app/ --ignore-missing-imports
86+
87+
- name: Run unit tests
88+
working-directory: ./backend
89+
run: |
90+
pdm run python -m pytest tests/unit/ -v --cov=app --cov-report=xml --cov-report=term-missing
91+
92+
- name: Run integration tests
93+
working-directory: ./backend
94+
run: |
95+
pdm run python -m pytest tests/functional/ -v
96+
97+
- name: Upload coverage to Codecov
98+
uses: codecov/codecov-action@v4
99+
with:
100+
file: ./backend/coverage.xml
101+
directory: ./backend
102+
flags: backend
103+
name: backend-coverage
104+
105+
- name: Run security scan
106+
working-directory: ./backend
107+
run: |
108+
pdm run bandit -r app/ -f json -o security-report.json || true
109+
110+
- name: Upload security report
111+
uses: actions/upload-artifact@v4
112+
if: always()
113+
with:
114+
name: security-report
115+
path: backend/security-report.json
116+
117+
e2e-tests:
118+
runs-on: ubuntu-latest
119+
needs: test
120+
121+
services:
122+
postgres:
123+
image: postgres:15
124+
env:
125+
POSTGRES_PASSWORD: testpassword
126+
POSTGRES_USER: testuser
127+
POSTGRES_DB: llsc_test
128+
options: >-
129+
--health-cmd pg_isready
130+
--health-interval 10s
131+
--health-timeout 5s
132+
--health-retries 5
133+
ports:
134+
- 5432:5432
135+
136+
steps:
137+
- uses: actions/checkout@v4
138+
139+
- name: Set up Python 3.12
140+
uses: actions/setup-python@v5
141+
with:
142+
python-version: 3.12
143+
144+
- name: Install PDM
145+
run: |
146+
pip install pdm
147+
148+
- name: Install dependencies
149+
working-directory: ./backend
150+
run: |
151+
pdm sync --group test --group lint --group dev
152+
153+
- name: Set up environment variables
154+
working-directory: ./backend
155+
run: |
156+
echo "POSTGRES_DATABASE_URL=postgresql://testuser:testpassword@localhost:5432/llsc_test" >> .env
157+
echo "SECRET_KEY=test-secret-key-for-ci" >> .env
158+
echo "ENVIRONMENT=test" >> .env
159+
echo "TEST_SCRIPT_BACKEND_URL=http://localhost:8000" >> .env
160+
echo "[email protected]" >> .env
161+
echo "TEST_SCRIPT_PASSWORD=testpassword" >> .env
162+
163+
- name: Run database migrations
164+
working-directory: ./backend
165+
run: |
166+
pdm run alembic upgrade head
167+
168+
- name: Start backend server
169+
working-directory: ./backend
170+
run: |
171+
pdm run uvicorn app.server:app --host 0.0.0.0 --port 8000 &
172+
sleep 10 # Wait for server to start
173+
174+
- name: Run E2E tests
175+
working-directory: ./
176+
run: |
177+
pdm run python -m pytest e2e-tests/ -v --tb=short
178+
179+
docker-build:
180+
runs-on: ubuntu-latest
181+
needs: test
182+
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
183+
184+
steps:
185+
- uses: actions/checkout@v4
186+
187+
- name: Set up Docker Buildx
188+
uses: docker/setup-buildx-action@v3
189+
190+
- name: Build Docker image
191+
working-directory: ./backend
192+
run: |
193+
docker build -t llsc-backend:latest .
194+
195+
- name: Test Docker image
196+
run: |
197+
docker run --rm llsc-backend:latest python --version
198+
199+
notify:
200+
runs-on: ubuntu-latest
201+
needs: [test, e2e-tests]
202+
if: always()
203+
204+
steps:
205+
- name: Notify on success
206+
if: needs.test.result == 'success' && needs.e2e-tests.result == 'success'
207+
run: |
208+
echo "✅ All tests passed! Backend is ready for deployment."
209+
210+
- name: Notify on failure
211+
if: needs.test.result == 'failure' || needs.e2e-tests.result == 'failure'
212+
run: |
213+
echo "❌ Tests failed! Please check the logs."
214+
exit 1

‎backend/FORMS_SCHEMA_README.md‎

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
# Forms Schema Documentation
2+
3+
This document describes the database schema for the forms system that was implemented.
4+
5+
## Tables Created
6+
7+
### 1. Core User Data
8+
9+
#### `user_data`
10+
Stores single-valued user fields:
11+
- `id` - UUID primary key
12+
- `date_of_birth` - Date field
13+
- `email` - String field
14+
- `phone` - String field
15+
16+
### 2. Multi-valued Reference Tables
17+
18+
#### `treatments`
19+
Stores available treatment options:
20+
- `id` - Integer primary key
21+
- `name` - Unique treatment name
22+
23+
Pre-populated with:
24+
- Chemotherapy, Immunotherapy, Radiation Therapy, Surgery, Targeted Therapy, Hormone Therapy, Stem Cell Transplant, CAR-T Cell Therapy, Clinical Trial, Palliative Care
25+
26+
#### `experiences`
27+
Stores cancer-related experiences:
28+
- `id` - Integer primary key
29+
- `name` - Unique experience name
30+
31+
Pre-populated with:
32+
- PTSD, Relapse, Anxiety, Depression, Fatigue, Neuropathy, Hair Loss, Nausea, Loss of Appetite, Sleep Problems, Cognitive Changes, Financial Stress, Relationship Changes, Body Image Issues, Survivorship Concerns
33+
34+
#### `qualities`
35+
Stores ranking/matching qualities:
36+
- `id` - Integer primary key
37+
- `slug` - Unique identifier (e.g., 'same_age')
38+
- `label` - Human-readable description
39+
40+
Pre-populated with matching criteria like same age, diagnosis, treatment, location, etc.
41+
42+
### 3. Bridge Tables (Many-to-Many)
43+
44+
#### `user_treatments`
45+
Links users to their treatments
46+
47+
#### `user_experiences`
48+
Links users to their experiences
49+
50+
### 4. Ranking System
51+
52+
#### `ranking_preferences`
53+
Stores user ranking preferences:
54+
- `user_id` - Reference to users table
55+
- `quality_id` - Reference to qualities table
56+
- `rank` - Integer ranking (1 = most important)
57+
58+
### 5. Form System
59+
60+
#### `forms`
61+
Form definitions and versioning:
62+
- `id` - UUID primary key
63+
- `name` - Form name
64+
- `version` - Version number
65+
- `type` - Enum (intake, ranking, secondary, become_volunteer, become_participant)
66+
67+
#### `form_submissions`
68+
Raw form submission data:
69+
- `id` - UUID primary key
70+
- `form_id` - Reference to forms table
71+
- `user_id` - Reference to users table
72+
- `submitted_at` - Timestamp
73+
- `answers` - JSONB field with raw form data
74+
75+
## Usage Examples
76+
77+
### Accessing Multi-valued Fields
78+
Thanks to SQLAlchemy relationships, you can access multi-valued fields as lists:
79+
80+
```python
81+
from app.models import UserData
82+
83+
# Get a user
84+
user = session.query(UserData).first()
85+
86+
# Access treatments as a list
87+
user_treatments = user.treatments # Returns list of Treatment objects
88+
treatment_names = [t.name for t in user.treatments]
89+
90+
# Access experiences as a list
91+
user_experiences = user.experiences # Returns list of Experience objects
92+
```
93+
94+
### Creating Form Submissions
95+
```python
96+
from app.models import FormSubmission
97+
import json
98+
99+
submission = FormSubmission(
100+
form_id=form_uuid,
101+
user_id=user_uuid,
102+
answers={
103+
"date_of_birth": "1990-01-01",
104+
"treatments": ["Chemotherapy", "Radiation Therapy"],
105+
"experiences": ["Anxiety", "Fatigue"],
106+
# ... other form fields
107+
}
108+
)
109+
session.add(submission)
110+
session.commit()
111+
```
112+
113+
### Setting Ranking Preferences
114+
```python
115+
from app.models import RankingPreference
116+
117+
# User ranks "same_diagnosis" as most important (rank 1)
118+
pref = RankingPreference(
119+
user_id=user_uuid,
120+
quality_id=2, # same_diagnosis quality
121+
rank=1
122+
)
123+
session.add(pref)
124+
session.commit()
125+
```
126+
127+
## Form Processing Workflow
128+
129+
1. **Form Submission**: Raw data stored in `form_submissions.answers` as JSON
130+
2. **Data Parsing**: Custom parser extracts structured data from JSON
131+
3. **Database Population**: Parsed data populates `user_data` and relationship tables
132+
4. **Versioning**: Multiple submissions create new form versions while preserving history
133+
134+
This design allows for:
135+
- Flexible form structures without schema changes
136+
- Historical tracking of all submissions
137+
- Structured querying of user data for matching algorithms
138+
- Easy addition of new treatment/experience/quality options

‎backend/app/models/Experience.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
from sqlalchemy import Column, Integer, String
2+
from sqlalchemy.orm import relationship
3+
4+
from .Base import Base
5+
6+
7+
class Experience(Base):
8+
__tablename__ = "experiences"
9+
id = Column(Integer, primary_key=True)
10+
name = Column(String, unique=True, nullable=False) # 'PTSD', 'Relapse', etc.
11+
12+
# Back reference for many-to-many relationship
13+
users = relationship("UserData", secondary="user_experiences", back_populates="experiences")

‎backend/app/models/Form.py‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
import uuid
2+
3+
from sqlalchemy import Column, Enum, Integer, String
4+
from sqlalchemy.dialects.postgresql import UUID
5+
6+
from .Base import Base
7+
8+
9+
class Form(Base):
10+
__tablename__ = "forms"
11+
12+
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
13+
name = Column(String, nullable=False) # 'Intake - Participant Caregiver'
14+
version = Column(Integer, default=1, nullable=False)
15+
type = Column(
16+
Enum("intake", "ranking", "secondary", "become_volunteer", "become_participant", name="form_type"),
17+
nullable=False,
18+
)

0 commit comments

Comments
 (0)