Skip to content

Update backup metadata schema to support all OkPy message types#4

Merged
phrdang merged 5 commits intomainfrom
update-schema
Apr 2, 2025
Merged

Update backup metadata schema to support all OkPy message types#4
phrdang merged 5 commits intomainfrom
update-schema

Conversation

@phrdang
Copy link
Copy Markdown
Member

@phrdang phrdang commented Mar 19, 2025

  • Rename backups_metadata table to backup_metadata
  • Update schema of backup_metadata to store paths/locations to every OkPy message type (except "email" since that's redundant)
  • Create new table okpy_messages which contains human-readable information about OkPy message types
  • Refactor code related to storing backup data to be split across multiple files:
    • db.py: SQL commands and hardcoded SQL table values
    • models.py: classes representing a backup and an OkPy message
    • storage.py: imports from db.py and models.py and stores the actual logic of writing the files and inserting rows into the database given a .json dump of OkPy API responses
  • Add deidentification of student emails

phrdang added 2 commits March 19, 2025 09:56
- Rename `backups_metadata` to `backup_metadata` table
- Create `okpy_messages` table
- Store all kinds of backups (except kind "email")
- `store` command works on test dataset, will cleanup code next
@phrdang phrdang self-assigned this Mar 19, 2025
phrdang added 3 commits March 19, 2025 10:21
- Convert contents to string (sometimes it can be `bool` like for the "submit" backup type
- Only create backup file and add to db if response was 200 OK
@phrdang phrdang merged commit 243cf24 into main Apr 2, 2025
2 of 4 checks passed
@phrdang phrdang deleted the update-schema branch April 2, 2025 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant