Asynced the Database #659 and #649 #660

juliansalvador727 · 2026-01-21T03:30:49Z

Purpose

Closes #659 and #649
Converts the entire backend database layer from synchronous to asynchronous operations to improve concurrent request handling and FastAPI compatibility.

New Changes

Converted all database operations to async/await pattern
Updated database engine to use AsyncEngine and AsyncSession from SQLAlchemy
Changed database connection string to use postgresql+asyncpg:// driver
Added asyncpg as a dependency for async PostgreSQL support
Converted all database wrapper classes and methods to async
Updated all API endpoints that interact with the database to async
Modified migration script to use asyncio.run() for async execution

Modified Files (22 total):

Core Database Layer:

gs/backend/data/database/engine.py
gs/backend/config/config.py

Abstract & Main Wrappers:

gs/backend/data/data_wrappers/abstract_wrapper.py
gs/backend/data/data_wrappers/wrappers.py

MCC Wrappers (8 files):

gs/backend/data/data_wrappers/mcc_wrappers/commands_wrapper.py
gs/backend/data/data_wrappers/mcc_wrappers/main_command_wrapper.py
gs/backend/data/data_wrappers/mcc_wrappers/comms_session_wrapper.py
gs/backend/data/data_wrappers/mcc_wrappers/main_telemetry_wrapper.py
gs/backend/data/data_wrappers/mcc_wrappers/packet_commands_wrapper.py
gs/backend/data/data_wrappers/mcc_wrappers/packet_telemetry_wrapper.py
gs/backend/data/data_wrappers/mcc_wrappers/packet_wrapper.py
gs/backend/data/data_wrappers/mcc_wrappers/telemetry_wrapper.py

ARO Wrappers (4 files):

gs/backend/data/data_wrappers/aro_wrapper/aro_user_login_wrapper.py
gs/backend/data/data_wrappers/aro_wrapper/aro_user_auth_token_wrapper.py
gs/backend/data/data_wrappers/aro_wrapper/aro_user_data_wrapper.py
gs/backend/data/data_wrappers/aro_wrapper/aro_request_wrapper.py

Resource Utils & Migration:

gs/backend/data/resources/utils.py
gs/backend/migrate.py

API Layer (4 files):

gs/backend/api/lifespan.py
gs/backend/api/v1/aro/endpoints/user.py
gs/backend/api/v1/mcc/endpoints/commands.py
gs/backend/api/v1/mcc/endpoints/main_commands.py

Testing

I have tested this PR by running the MCC website (Backend starts successfully with async database operations)
I have tested this PR by running the ARO website (All user endpoints function correctly with async operations)
I have unit-tested this PR. (Backend database tests need to be written/updated for async compatibility)
I have tested this PR on a board if the code will run on a board. (N/A - backend only changes)

Testing Notes:

FastAPI server starts successfully with fastapi dev gs/backend/main.py
All database operations are now non-blocking
Migration script works correctly with python3 gs/backend/migrate.py
All modules load without errors (verified via pytest coverage report)

Outstanding Changes

Unit tests for async database operations need to be written or updated

github-actions · 2026-01-21T03:31:22Z

Pull reviewers stats

Stats of the last 120 days for UWOrbital:

User	Total reviews	Time to review	Total comments
Adityya-K 🥇	36 ▀▀▀▀	3d 5h 53m ▀	106 ▀▀▀
camspec 🥈	34 ▀▀▀	3d 3h 23m ▀	191 ▀▀▀▀▀
proprogrammer504 🥉	13 ▀	7d 1h 30m ▀▀	44 ▀
Syzygicality	8 ▀	8h 20m	17
kepler452b123	5	1d 11h 18m	2
joannalauu	3	4d 21h 20m ▀	19
c4bae	1	13d 1h 35m ▀▀▀▀	2
panthpatel2016	1	4h 30m	6

^{⚡️ Pull request stats}

Syzygicality

Please fix pytests before Kevin and I can make a proper code review! Most failures are due to incorrect typehinting in the pytests and use of .exec() on asyncsession, which should instead be .execute()

juliansalvador727 · 2026-01-21T21:20:33Z

Summary of Changes

requirements.txt - added asyncpg==0.29.0 for SQLAlchemy's async PostgreSQL support
gs/backend/data/resources/utils.py - changed all exec(query) -> execute(query) and added .scalars() call
gs/backend/data/data_wrappers/abstract_wrapper.py - added attr-defined and unused-coroutine to suppress false positive mypy errors. also added a # type: ignore[unused-coroutine] comment on mcc and aro wrappers to session.delete() calls to show this.
I don't know if the changes to interfaces/obc_gs_interface/ax25/__init__.py are necessary
they were done in order to pass python_test/test_ax25.py::test_I_frame_encode_decode which is out of the scope of Refactor DB connection to be asynchronous #659 and Refactor backend endpoints (not db) to ensure full async #649.

proprogrammer504

slightly terrified, it looks good mostly. there are some inconsistencies with usage of async and await. additionally, there is a change to the ax25 protocol that i do not understand. this may be a dangerous change. finally, i want another evaluation on whether or not the things which we asynced actually NEED to be asynced. i feel that if something doesnt require it, it should be left as is as it is safer to have less moving parts.

tldr, maybe dont touch ax25, fix inconsistencies.

proprogrammer504 · 2026-01-21T23:16:28Z

gs/backend/config/config.py

 DATABASE_CONNECTION_STRING: Final[
    str
-] = f"postgresql+psycopg2://{GS_DATABASE_USER}:{GS_DATABASE_PASSWORD}@{GS_DATABASE_LOCATION}:{GS_DATABASE_PORT}/{GS_DATABASE_NAME}"
+] = f"postgresql+asyncpg://{GS_DATABASE_USER}:{GS_DATABASE_PASSWORD}@{GS_DATABASE_LOCATION}:{GS_DATABASE_PORT}/{GS_DATABASE_NAME}"


does this allow for the db to be asynchronous? i thought it already was asynchronous

the new one is the async driver for psql connection

proprogrammer504 · 2026-01-21T23:17:05Z

gs/backend/data/data_wrappers/abstract_wrapper.py

-        with get_db_session() as session:
+        async with get_db_session() as session:
            obj = self.model(**data)
            session.add(obj)


i mean since we're going nuts and awaiting everything, does this need await as well?

yep, though looking at it, we could just add session: AsyncSession = Depends(get_db_session) into the function args of the abstract wrapper

proprogrammer504 · 2026-01-21T23:17:38Z

gs/backend/data/data_wrappers/abstract_wrapper.py

-            session.commit()
-            return obj
+            # Preserve object state before deleting
+            obj_copy = self.model(**{c.name: getattr(obj, c.name) for c in obj.__table__.columns})  # type: ignore[attr-defined]


why not use session.delete(obj)?

yeah cursor bugged out here thinking that db deletion deletes the object from the memory, though it doesnt. pls return obj, not obj_copy

proprogrammer504 · 2026-01-21T23:19:05Z

gs/backend/data/data_wrappers/aro_wrapper/aro_user_auth_token_wrapper.py

        if auth_token:
-            session.delete(auth_token)
-            session.commit()
+            session.delete(auth_token)  # type: ignore[unused-coroutine]


yea, this seems more logical. also the await seems inconsistent. sometimes session uses await yet other times it does not.

in terms of db connection, awaits should only be used on IO or expensive processes, so usually commit and refresh session methods

proprogrammer504 · 2026-01-21T23:20:38Z

gs/backend/data/data_wrappers/mcc_wrappers/packet_commands_wrapper.py

            raise ValueError("Packet command not found.")
-        session.delete(command)
-        session.commit()
+        session.delete(command)  # type: ignore[unused-coroutine]


i just realized that there r a bunch of the type: ignore unused coroutine comments. what r those about?

linting error escape most likely

proprogrammer504 · 2026-01-21T23:22:05Z

gs/backend/data/database/engine.py

+from contextlib import asynccontextmanager
+
+from sqlalchemy import text
+from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession, create_async_engine


do we understand the side effects of using async engine and async session instead of the regular ones?

proprogrammer504 · 2026-01-21T23:22:58Z

gs/backend/data/resources/utils.py

-    result = session.exec(query).first()
-    if not result:
+    result = await session.execute(query)
+    if not result.scalars().first():


what does this check for exactly? would this prevent main commands from being added in some rare cases?

if the db has already been populated with the data, then this conditional ensures that they wont insert it again, so we dont have duplicated datasets in the tables.

proprogrammer504 · 2026-01-21T23:24:49Z

interfaces/obc_gs_interface/ax25/__init__.py

-        if (data[-1] == 0 and len(data) > RS_ENCODED_DATA_SIZE + AX25_NON_INFO_BYTES) or (
-            data[-1] == 0 and len(data) < RS_ENCODED_DATA_SIZE
-        ):
+        # Only remove the byte if we're certain it's padding (exactly 1 byte more than expected for I frames)


note that this change could be dangerous, it also is unrelated to making stuff asynchronous. my concern is that this is a part of the ax25 protocol, changing anything here could have unforeseen consequences especially when we dont understand how it works.

i agree. this change is out of the scope of the task

proprogrammer504 · 2026-01-21T23:25:36Z

python_test/test_aro_data_wrapper.py

@@ -20,60 +20,97 @@
 from gs.backend.data.enums.aro_requests import ARORequestStatus


note that these endpoints are outdated

what should be tested and asynced (if not already) are eddies abstracted data wrappers. also correction, they are not endpoints they are data wrappers.

yeah, this is alr a task -> #635

Syzygicality

some of my own changes for now, will check up on pytests later

Syzygicality · 2026-01-21T23:54:31Z

gs/backend/migrate.py


-if __name__ == "__main__":
+
+async def main() -> None:


wrap this whole script in one async with get_db_session() as session: instead of multiple

Syzygicality · 2026-01-23T04:12:49Z

gs/backend/data/data_wrappers/abstract_wrapper.py

+            obj = await session.get(self.model, obj_id)
            if not obj:
                raise ValueError(f"{self.model.__name__} with ID {obj_id} not found.")
-            session.delete(obj)


revert this change to return obj, not make a obj copy

juliansalvador727 added 2 commits January 20, 2026 21:44

its asyncing time

09ffbf3

asynced

d5cd1e5

Syzygicality requested changes Jan 21, 2026

View reviewed changes

juliansalvador727 added 2 commits January 21, 2026 15:56

passing most pytests

a75690f

added async to requirements txt and changed exec -> execute

f48e262

juliansalvador727 self-assigned this Jan 21, 2026

proprogrammer504 requested changes Jan 21, 2026

View reviewed changes

Syzygicality requested changes Jan 23, 2026

View reviewed changes

		@@ -20,60 +20,97 @@
		from gs.backend.data.enums.aro_requests import ARORequestStatus

Asynced the Database #659 and #649 #660

Are you sure you want to change the base?

Asynced the Database #659 and #649 #660

Uh oh!

Conversation

juliansalvador727 commented Jan 21, 2026

Purpose

New Changes

Modified Files (22 total):

Testing

Outstanding Changes

Uh oh!

github-actions bot commented Jan 21, 2026

Pull reviewers stats

Uh oh!

Syzygicality left a comment

Choose a reason for hiding this comment

Uh oh!

juliansalvador727 commented Jan 21, 2026

Summary of Changes

Uh oh!

proprogrammer504 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Syzygicality left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants