Implement database schema (#381)

# Purpose
Closes #321

# Warning: Breaking Changes
- All GS members will need to install PostgreSQL by following the
instructions in the `POSTGRESQL_SETUP.md` to run the backend
- Dropping support for using Windows as the development environment. Use
WSL instead if you are on Windows. See in the instructions in the
`README.md`. Make sure to install Ubuntu 22 as the distro of choice as
Ubuntu 24 has issues with installing Python packages in editable mode.

# New Changes
- Implement database schema for non auth related tables
- Switch the pytest workflow runner to use only Ubuntu
- Remove Docker setup instructions from the README as it is was unused
and there doesn't make sense to spend the effort to maintain it.
- Add PostgreSQL setup instructions
- Add database configuration files
- Create tests for database tables
- Create `MainCommands` get endpoint under the
`/api/v1/mcc/main-commands/` path
- Add some data for the `MainCommands`
- Make schemas, tables and data for `MainCommands` generate
automatically. The database must be created manually
- Fix minor README issues

# Testing
- Create unit tests for all tables
- Test the MainCommands get endpoint manually:

![Screenshot from 2025-04-13
13-02-59](https://github.com/user-attachments/assets/bfe74a98-e359-47c5-9921-98651902865c)

- Validate that the MainCommands exist in the database:


![image](https://github.com/user-attachments/assets/8b888b06-d8c1-4878-a498-cc84f3b74d2a)

# Outstanding Changes
- Add auth related tables once we determine how to setup auth
- Improve the table tests to test data validation and more complex
relations.
- Improve the configuration files. This will be handled by #335 
- Add loguru to SQLModel logger. This will be handled by #380 
- Pull `MainCommands` and `MainTelemetry` from a configuration file and
have it synced with the OBC
- Add data validation for `MainCommands` table. This will be handled by
#356
- Determine what `subtype` for packets means. This is an CSDC
requirement but we can easily add this to the database later.
- Make `Commands.params` match with `MainCommands.params`
- Make `Telemetry.value` match with `MainTelemetry.params`
- Split up the README into separate READMEs for GS and Firmware where
instructions and data doesn't overlap
- Add test for `MainCommand` get endpoint

Author: Yarik-Popov

.github/workflows/pytest.yml

@@ -8,11 +8,10 @@ on:
 
 jobs:
   pytest:
-    runs-on: ${{ matrix.os }}
+    runs-on: ubuntu-latest
 
     strategy:
       matrix:
-        os: [ubuntu-latest, windows-latest]
         python-version: ['3.11']
 
     steps:
@@ -25,6 +24,8 @@ jobs:
 
     - name: Install dependencies
       run: |
+        sudo apt update
+        sudo apt install -y postgresql
         python -m pip install --upgrade pip
         pip install -r requirements.txt
         pip install -e .

.gitignore

@@ -64,6 +64,7 @@ compile_commands.json
 *.sw?
 
 .env
+*.db
 
 /build*
 

README.md

@@ -41,49 +41,8 @@ Download Code Composer Studio (CCS): https://www.ti.com/tool/CCSTUDIO. This will
 
 Download UniFlash here: https://www.ti.com/tool/UNIFLASH#downloads. This will be used for flashing the RM46.
 
-#### Docker Development Environment
-
-It's **highly recommended** that you set up your development environment using Docker and VSCode, especially if you're new to software development. If you follow the instructions in this section, you can skip the **Windows/MacOS/Linux** sections. If you know what you're doing, feel free to set up your dev environment however you like using the instructions in the **Windows/MacOS/Linux** sections for reference. However, there may be a lack of support from other leads/members who don't use the same setup.
-
-##### Docker Desktop Installation & Configuration
-
-1. Install Docker Desktop App from [this link](https://www.docker.com/products/docker-desktop/)
-   - You can choose to sign-up/create an account but it's not required. You can also skip the "Tell-us about what you do" section.
-2. Open Docker Desktop and click on `Dev Environments` from the side-panel
-   - Click on create on `Create +` in the top-right corner.
-3. Setting up the repo
-   - Name the Environment as desired
-   - For the `Choose source` option, select `Local directory` and then select the `OBC-firmware` repository folder that you cloned earlier.
-   - Click `Continue`
-   - Once the container is created, you should be able to open the container in VSCode. If you have VSCode, you can press `Open in VSCode`. If you don't have VSCode, you can get it here: https://code.visualstudio.com/download
-
-##### Installing Dependencies
-
-Once you open the docker instance, open a terminal in VSCode and run the following commands. The dollar sign in your terminal should be prefaced by something like this: `root ➜ /com.docker.devenvironments.code (main ✗)`.
-
-This command opens a terminal in VSCode: `` Ctrl + Shift + `  ``
-
-Enter these commands in your terminal:
-```sh
-sudo apt-get update
-sudo apt-get install -y python3-pip build-essential cmake gcc-multilib g++-multilib curl
-pip3 install -r requirements.txt
-curl -fsSL https://deno.land/install.sh | sh # Deno is required for pre-commit
-pre-commit install
-```
-
-##### Testing The Container Build
-
-To test whether your Dev environment has been set up correctly run the commands in the **Building** section. The OBC firmware and test builds should pass. All tests should succeed.
-
-**Note**: The docker container uses pre-configured git (one added to the original OS path by the user). So you should be able to pull and push to the OBC repo as necessary.
-
-**Tip**: Use the `git config --list` command on the VsCode terminal to confirm your git info.
-
 #### **Windows**
 
-Skip this section if you set up a Docker development environment.
-
 1. Download WSL2: https://learn.microsoft.com/en-us/windows/wsl/install
 
 2. In WSL2, run the following:
@@ -117,11 +76,14 @@ Skip this section if you set up a Docker development environment.
    - Once your PATH is set up and pre-commit is installed you can use `pre-commit run --all-files` to format all of your files before committing
      **Note:** pre-commit is used to format your code whenever you make a commit.
 
-You'll be using WSL2 primarily for building the firmware and running tests.
+You'll be using WSL2 for all development.
 
-#### **MacOS**
+5. Setup the PostgreSQL database
 
-Skip this section if you set up a Docker development environment.
+This setup is only required for GS members. Please follow the instructions located in [POSTGRESQL_SETUP.md](gs/POSTGRESQL_SETUP.md)
+
+
+#### **MacOS**
 
 1. Install required build tools (CMake, Make, gcc)
 
@@ -131,7 +93,7 @@ brew install make
 brew install gcc
 ```
 
-2. Install Python 3.11 and setup Python virtual environment (Only required for Backend devs)
+2. Install Python 3.11 and setup Python virtual environment (Only required for GS devs)
 
 Run the following commands in the OBC-firmware directory:
 
@@ -150,10 +112,12 @@ curl -fsSL https://deno.land/install.sh | sh # Deno is required for pre-commit
 pip install -r requirements.txt # You may want to create a Python virtual env before this if you haven't already
 pre-commit install
 ```
+4. Setup the PostgreSQL database
 
-#### **Linux**
+This setup is only required for GS members. Please follow the instructions located in [POSTGRESQL_SETUP.md](gs/POSTGRESQL_SETUP.md)
 
-Skip this section if you set up a Docker development environment.
+
+#### **Linux**
 
 1. Install required build tools (CMake, Make, gcc)
 
@@ -162,7 +126,7 @@ sudo apt-get update
 sudo apt-get install build-essential gcc-multilib g++-multilib curl
 ```
 
-2. Install Python 3.11 and setup Python virtual environment (Only required for Backend devs)
+2. Install Python 3.11 and setup Python virtual environment (Only required for GS devs)
 
 Run the following commands in the OBC-firmware directory:
 
@@ -182,6 +146,10 @@ pip install -r requirements.txt # You may want to create a Python virtual env be
 pre-commit install
 ```
 
+4. Setup the PostgreSQL database
+
+This setup is only required for GS members. Please follow the instructions located in [POSTGRESQL_SETUP.md](gs/POSTGRESQL_SETUP.md)
+
 ### Building
 
 #### **OBC Firmware**
@@ -296,6 +264,12 @@ We use Code Composer Studio for debugging the firmware. **TODO**: Write a tutori
 
 To run the frontend, you will need Deno 2 installed which was installed in the pre-commit setup instructions above.
 
+#### **Setting up the Frontend **
+```sh
+cd gs/frontend
+deno install --frozen
+```
+
 #### **Running the ARO Frontend**
 
 ```sh

gs/POSTGRESQL_SETUP.md

@@ -0,0 +1,78 @@
+# PostgreSQL Setup
+
+## Table of Contents
+
+- [Installation](#installation)
+- [Database Setup](#database-setup)
+- [Environment Variable Setup](#environment-variable-setup)
+- [Sources](#sources)
+
+**[Back to top](#table-of-contents)**
+
+## Installation
+
+Run the following command dependant on your setup.
+If you have Windows, install WSL. See the instructions in the top level [README.md](../../README.md)
+
+### Linux/WSL
+
+```sh
+sudo apt update
+sudo apt install postgresql-16
+```
+
+### MacOS
+
+```sh
+brew install postgresql@16
+brew services start postgresql
+```
+
+**[Back to top](#table-of-contents)**
+
+## Database Setup
+
+Run `sudo -u postgres psql --version` to check that it was installed properly. It should print that major version is 16.
+
+Below are the commands to setup PostgreSQL for the ground station
+```sh
+sudo -u postgres psql # Connect to the PostgreSQL CLI
+CREATE USER username WITH PASSWORD 'password' SUPERUSER;  # Change username to your user on the machine and password to a strong password
+# Log out of this session by pressing Ctrl + D
+```
+Log back in by running `psql` if username you created above matches the current user, you don't need to log in.
+Otherwise, you will need to run `psql -U username` and then enter the password created above to proceed.
+```sh
+CREATE DATABASE gs; # gs database
+\c gs; # Connect to the gs database
+```
+
+**[Back to top](#table-of-contents)**
+
+## Environment Variable Setup
+
+Find the `gs/backend/config/template.env` and create a copy of it in the same directory with the name `.env`
+Inside your newly created `.env` file, put your username and password created above inside the quotes for `GS_DATABASE_USER` and `GS_DATABASE_PASSWORD` respectively.
+
+In the end your `.env` file, should look like this:
+```sh
+GS_DATABASE_USER="username"      # replace username with your created username
+GS_DATABASE_PASSWORD="password"  # replace password with your created password above
+GS_DATABASE_LOCATION="localhost" # Change this if using a remote server
+GS_DATABASE_PORT="5432"          # Default PostgreSQL port. Change if needed
+GS_DATABASE_NAME="gs"            # Name of the database. NOTE: Make sure to create this manually before running the backend of the first time
+```
+
+Note: You can choose to remove the first to lines of the `.env` that start with \# as those lines are comments.
+
+Now, you can start the backend as by running `fastapi dev gs/backend/main.py` from the top level directory to run in development mode.
+
+**[Back to top](#table-of-contents)**
+
+## Sources
+
+- [How to Install and Setup PostgreSQL on Ubuntu 20.04 | Step-by-Step](https://www.cherryservers.com/blog/how-to-install-and-setup-postgresql-server-on-ubuntu-20-04)
+- [Install and configure PostgreSQL](https://documentation.ubuntu.com/server/how-to/databases/install-postgresql/index.html)
+- [How To Install PostgreSQL on Ubuntu 20.04 \[Quickstart\]](https://www.digitalocean.com/community/tutorials/how-to-install-postgresql-on-ubuntu-20-04-quickstart)
+
+**[Back to top](#table-of-contents)**

gs/backend/api/backend_setup.py

@@ -7,6 +7,7 @@
 from gs.backend.api.v1.aro.endpoints.user import aro_user_router
 from gs.backend.api.v1.mcc.endpoints.aro_requests import aro_requests_router
 from gs.backend.api.v1.mcc.endpoints.commands import commands_router
+from gs.backend.api.v1.mcc.endpoints.main_commands import main_commands_router
 from gs.backend.api.v1.mcc.endpoints.telemetry import telemetry_router
 
 
@@ -24,6 +25,7 @@ def setup_routes(app: FastAPI) -> None:
     app.include_router(commands_router, prefix=f"{mcc_prefix}/commands")
     app.include_router(telemetry_router, prefix=f"{mcc_prefix}/telemetry")
     app.include_router(aro_requests_router, prefix=f"{mcc_prefix}/requests")
+    app.include_router(main_commands_router, prefix=f"{mcc_prefix}/main-commands")
 
 
 def setup_middlewares(app: FastAPI) -> None:

gs/backend/api/lifespan.py

@@ -3,8 +3,15 @@
 
 from fastapi import FastAPI
 
+from gs.backend.data.database.engine import get_db_session, setup_database
+from gs.backend.data.resources.utils import add_main_commands
+
 
 @asynccontextmanager
 async def lifespan(_: FastAPI) -> AsyncGenerator[None, None]:
     """Lifecycle event for the FastAPI app."""
+    # Must all the get_db_session each time when pass it into a separate function.
+    # Otherwise, will get transaction is inactive error
+    setup_database(get_db_session())
+    add_main_commands(get_db_session())
     yield

gs/backend/api/v1/mcc/endpoints/main_commands.py

@@ -0,0 +1,18 @@
+from fastapi import APIRouter, Depends
+from sqlmodel import Session, select
+
+from gs.backend.api.v1.mcc.models.responses import MainCommandsResponse
+from gs.backend.data.database.engine import get_db_session
+from gs.backend.data.tables.main_tables import MainCommand
+
+main_commands_router = APIRouter(tags=["MCC", "Main Commands"])
+
+
+@main_commands_router.get("/")
+async def get_main_commands(db_session: Session = Depends(get_db_session)) -> MainCommandsResponse:
+    """
+    @brief Gets the main commands that are available for the MCC
+    """
+    main_commands_query = select(MainCommand)
+    items = list(db_session.exec(main_commands_query).all())
+    return MainCommandsResponse(data=items)

gs/backend/api/v1/mcc/models/responses.py

@@ -0,0 +1,11 @@
+from pydantic import BaseModel
+
+from gs.backend.data.tables.main_tables import MainCommand
+
+
+class MainCommandsResponse(BaseModel):
+    """
+    The main commands response model.
+    """
+
+    data: list[MainCommand]

gs/backend/config/__init__.py

@@ -0,0 +1,19 @@
+# TODO:(335) Improve loading the configuration
+from os import environ
+from typing import Final
+
+from dotenv import load_dotenv
+
+load_dotenv()
+
+
+# TODO: Make these throw an exception if they are None
+GS_DATABASE_USER = environ.get("GS_DATABASE_USER")
+GS_DATABASE_PASSWORD = environ.get("GS_DATABASE_PASSWORD")
+GS_DATABASE_LOCATION = environ.get("GS_DATABASE_LOCATION")
+GS_DATABASE_PORT = environ.get("GS_DATABASE_PORT")
+GS_DATABASE_NAME = environ.get("GS_DATABASE_NAME")
+
+DATABASE_CONNECTION_STRING: Final[
+    str
+] = f"postgresql+psycopg2://{GS_DATABASE_USER}:{GS_DATABASE_PASSWORD}@{GS_DATABASE_LOCATION}:{GS_DATABASE_PORT}/{GS_DATABASE_NAME}"