refactor: major database overhaul #65
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Introduced BackupModel to manage backups with relationships to PadModel. - Created PadModel to represent pads with relationships to UserModel and BackupModel. - Added UserModel to handle user data and relationships with PadModel. - Each model includes fields for UUID, timestamps, and relevant data types, utilizing SQLAlchemy for ORM functionality.
…schema - Added BaseModel to centralize common fields and schema configuration for all models. - Refactored BackupModel, PadModel, and UserModel to inherit from BaseModel, enhancing code reusability. - Updated relationships and added indexing for improved query performance. - Implemented to_dict methods in models for easier data serialization.
- Added a new repository module for database operations, including UserRepository, PadRepository, and BackupRepository. - Each repository provides methods for creating, retrieving, updating, and deleting records related to users, pads, and backups. - Enhanced modularity and organization of database interactions within the application.
- Introduced UserService, PadService, and BackupService to handle business logic related to users, pads, and backups. - Each service includes methods for creating, retrieving, updating, and deleting records, enhancing modularity and organization of the application. - Added an __init__.py file to facilitate service module imports.
- Added a new database module to manage database connections and session handling using SQLAlchemy with async capabilities. - Introduced init_db function to initialize the database schema and tables. - Created repository and service dependency functions for user, pad, and backup management. - Updated requirements.txt to include psycopg2-binary for PostgreSQL support.
- Changed VARCHAR to String for username and email fields in UserModel to align with SQLAlchemy best practices. - This update enhances code consistency and readability within the database model.
- Refactored BackupModel, PadModel, and UserModel to utilize SCHEMA_NAME for schema configuration, enhancing consistency across models. - Removed the get_schema method from BaseModel to streamline schema handling. - Updated ForeignKey references to align with the new schema approach, improving clarity and maintainability.
- Replaced custom UUIDType with SQLAlchemy's built-in UUID type for source_id and owner_id fields in BackupModel and PadModel, respectively. - This change enhances code clarity and aligns with SQLAlchemy best practices for UUID handling.
…ization - Introduced SCHEMA_NAME in base_model.py for consistent schema handling across models. - Updated init_db function to use CreateSchema for schema creation, improving clarity and maintainability. - Adjusted imports in models to reflect the new schema configuration, enhancing code organization.
- Modified UserModel to use SQLAlchemy's UUID type for the primary key, aligning with Keycloak's UUID requirements. - Updated UserRepository's create method to accept a user_id parameter, allowing for explicit user ID assignment during user creation.
- Updated UserModel to include new fields: email_verified, name, given_name, family_name, and roles, allowing for more comprehensive user data management. - Modified UserRepository's create method to accept these new fields, facilitating their inclusion during user creation. - Introduced a new user router to handle user creation and retrieval endpoints, improving API functionality and user management capabilities.
- Introduced functions for decoding JWT tokens and retrieving current user information, enhancing authentication flow. - Implemented user creation logic for new users based on token data, improving user management capabilities. - Added an admin role check to enforce access control, ensuring only authorized users can access certain resources.
- Introduced TemplatePadModel to represent the template pads table in the app schema, enhancing the database structure. - Defined columns for name, display_name, and data, ensuring comprehensive data management for template pads. - Added an index on display_name for improved query performance.
…d management - Introduced TemplatePadRepository for database operations related to template pads, including create, read, update, and delete functionalities. - Added TemplatePadService to encapsulate business logic for template pad management, ensuring data validation and error handling. - Created a new router for template pad endpoints, providing API access for creating, retrieving, updating, and deleting template pads, with admin access control. - Updated existing modules to integrate the new template pad features, enhancing overall application functionality.
- Added docstrings to the create_user, get_all_users, get_user_info, get_user_count, and get_user functions to clarify their purpose and access restrictions (admin only). - This enhancement improves code documentation and aids in understanding the functionality of user management endpoints.
- Added python-multipart to requirements.txt to support file uploads in the application. - Ensured psycopg2-binary remains included for PostgreSQL database interactions, maintaining necessary dependencies for backend functionality.
- Added a new function to load templates from JSON files into the database, ensuring templates are created if they do not already exist. - Updated the lifespan of the FastAPI application to include the loading of templates during startup. - Refactored the TemplatePadRepository to update and delete templates using their name instead of ID, improving usability. - Introduced a new router for template pad endpoints, expanding API capabilities for template management. - Added a default template JSON file to provide a starting point for users, enhancing user experience.
- Introduced UserSession class to unify user session handling, integrating authentication data with user information. - Updated AuthDependency to utilize UserSession, enhancing session validation and user data retrieval. - Replaced SessionData references with UserSession across various routers for consistent user session management. - Added new methods to UserSession for accessing user attributes and caching user data from the database. - Removed deprecated canvas router and introduced a new pad router for managing user pads and backups, improving API structure and functionality.
- Simplified UserSession initialization by decoding the JWT token directly, enhancing security and clarity. - Removed unused to_dict method from UserSession to reduce code complexity. - Updated AuthDependency to eliminate unnecessary user service dependency, streamlining session validation. - Corrected API endpoint in hooks.ts from '/api/user/me' to '/api/users/me' for consistency with routing. - Cleaned up imports in user_router.py to maintain code organization.
…eness - Updated import statement in main.py to reflect the new workspace_router file structure. - Modified UserModel to enforce unique email addresses for users, improving data integrity. - Introduced a new workspace_router.py file to manage workspace-related endpoints, replacing the deprecated workspace.py, streamlining the API structure.
- Refactored pad router to streamline user pad management, including saving and retrieving canvas data. - Updated user session handling to use user.id instead of user_id for consistency. - Changed API endpoints in frontend hooks to align with new pad routing structure, enhancing clarity and usability. - Removed deprecated default canvas data retrieval function, simplifying the codebase.
- Modified WorkspaceState interface to include 'name' and 'id' properties, enhancing workspace identification. - Updated ActionButton and Terminal components to utilize 'name' instead of 'workspace_id' for URL generation, improving consistency across the application. - Ensured all relevant references to workspace identification are aligned with the new state structure, streamlining the codebase.
- Deleted db.py to streamline database management and reduce redundancy. - Updated main.py to directly import init_db from the database module, simplifying the database initialization process. - Introduced auth_router.py to handle authentication routes, enhancing modularity and organization of the codebase. - Ensured dotenv loading is handled in the database module for consistent environment variable access.
…ment - Added a new dependency function to provide a CoderAPI instance for use in routers. - Updated auth_router to utilize CoderAPI in the callback endpoint, enhancing authentication flow. - Modified workspace_router to incorporate CoderAPI in workspace state retrieval and management functions, improving workspace operations. - Streamlined dependency management across routers for better modularity and code organization.
- Replaced dotenv loading in CoderAPI with centralized configuration from config.py, enhancing consistency and maintainability. - Updated CoderAPI initialization to use environment variables directly from the config module. - Adjusted error messages to reflect the new configuration approach, improving clarity for required variables. - Streamlined imports across various modules to utilize the new configuration structure, promoting better organization.
- Replaced direct references to OIDC_CONFIG with environment variables for OIDC_SERVER_URL, OIDC_REALM, OIDC_CLIENT_ID, and OIDC_CLIENT_SECRET, enhancing configuration management. - Improved code clarity and maintainability by centralizing authentication URL generation in the config module.
- Integrated PyJWKClient for secure JWT verification, improving token validation by using signing keys from JWKS. - Updated UserSession initialization to decode tokens with verification, enhancing security and error handling. - Added a caching mechanism for the JWKS client to optimize performance. - Cleaned up token expiration checks to ensure accurate validation and error reporting.
- Introduced a Redis connection pool to optimize Redis client management, improving performance and resource utilization. - Updated session management functions to utilize the new Redis connection pool, ensuring efficient access to session data. - Implemented a new method in BackupService to retrieve backups for a user's first pad using a join operation, addressing the N+1 query problem. - Enhanced pad router to create backups conditionally based on time intervals, improving backup efficiency and management. - Streamlined user router to utilize the new Redis client retrieval method, promoting better code organization and maintainability.
- Updated API endpoint paths in pad, template pad, and user routers to remove trailing slashes for consistency. - Enhanced datetime handling in BackupService to include timezone information, improving accuracy in backup creation timing. - Adjusted frontend API calls to align with the updated endpoint structure, ensuring seamless integration across the application.
- Changed CORS middleware to allow all origins by updating the allow_origins parameter to ["*"], enhancing flexibility for cross-origin requests.
- Implemented a new method in UserService to synchronize user data with information from the authentication token, creating or updating the user as necessary. - Updated the user router to utilize this new synchronization method, enhancing user data management and ensuring consistency between the database and authentication token data.
- Added Alembic configuration and migration scripts to facilitate database schema changes. - Implemented a run_migrations function to execute migrations during application startup. - Created migration scripts to transfer data from the old public schema to the new pad_ws schema, ensuring data integrity and consistency. - Updated requirements.txt to include Alembic as a dependency for migration management.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR completely rewrites the database logic and interaction from pad.ws to the postgresql instance.