added some basic copilot instructions

Author: bigabig

.github/copilot-instructions.md

@@ -0,0 +1,26 @@
+# Project Overview
+
+This project is a web application that allows users to manage, search, annotate, analyze and interprete reseach materials.
+
+## Folder Structure
+
+- `/frontend`: Contains the source code for the frontend.
+- `/backend`: Contains the source code for the backend.
+- `/docker`: Contains docker configurations.
+- `/tools`: Contains various scripts.
+
+## Core Concepts
+
+We use the following terms throughout the project:
+
+- **User**: An individual who uses the application.
+- **Project**: A collection of source documents, annotations, codes, tags, and memos related to a specific research endeavor.
+- **Source Document**: Any document (text, image, audio, video) data that is being analyzed. Short: "sdoc".
+- **Metadata**: Information about a source document, such as title, author, date created, etc.
+- **Tag**: A label assigned to a source document for categorization.
+- **Code**: A category used for annotating source documents.
+- **Annotation**: A segment of a source document that has been assigned a code.
+  - **Span Annotation**: An annotation that applies to a specific span of text within a source document.
+  - **Sentence Annotation**: An annotation that applies to an entire sentence within a source document.
+  - **Bbox Annotation**: An annotation that applies to a bounding box within an image document.
+- **Memo**: A note or comment added to a source document or annotation.

.github/instructions/README.md

@@ -0,0 +1,19 @@
+# Copilot Instructions
+
+We can add context-dependent instructions automatically to our prompts! These instructions are currently stored in:
+
+- `.github/copilot-instructions.md`: Generic instructions always added to prompts
+- `.github/instructions`: Context dependent instructions, added to the prompts depending on the file location
+
+Additionally, we can specify reusable prompts or tasks that can be executed:
+
+- `.github/prompts`
+- `/backend/.github/prompts`
+- `/frontend/.github/prompts`
+
+For more information, see:
+
+- Prompting best practices: https://docs.github.com/en/copilot/get-started/best-practices
+- Prompt engineering: https://docs.github.com/en/copilot/concepts/prompting/prompt-engineering
+- Custom instructions: https://docs.github.com/en/copilot/how-tos/configure-custom-instructions/add-repository-instructions#creating-repository-wide-custom-instructions-1
+- Example custom instructions: https://docs.github.com/en/copilot/tutorials/customization-library/custom-instructions

.github/instructions/backend.instructions.md

@@ -0,0 +1,70 @@
+---
+applyTo: "backend/**/*.py"
+---
+
+## Backend
+
+The backend is built using Python with FastAPI as the web framework.
+It provides RESTful API endpoints for the frontend to interact with.
+It follows a modular architecture, separating concerns into different folders and files.
+
+### Libraries and Frameworks
+
+Dependencies are listed in pyproject.toml and managed with uv.
+
+- FastAPI for building the API (REST endpoints)
+- SQLAlchemy for database interactions
+- Alembic for database migrations
+- Pydantic for data validation
+- Omegaconf for configuration management
+- Loguru for logging
+
+### Databases
+
+The backend works with multiple databases:
+
+- PostgreSQL as the primary relational database
+- Weaviate for vector storage and similarity search
+- Elasticsearch for full-text search capabilities
+- Filesystem storage for storing uploaded files
+
+### Folder Structure
+
+- `/backend/configs`: Configuration files for different environments
+- `/backend/src`: Main application code
+  - `/common`: Utility functions and helpers
+  - `/core`: Core application logic. All core concepts are implemented here.
+  - `/migrations`: Alembic database migrations
+  - `/modules`: Application logic organized by feature. Modules use core services and systems to implement features.
+  - `/repos`: Connections to external services, e.g., databases, file storage, etc.
+  - `/systems`: Reusable systems that can be plugged into application logic
+- `/backend/tests`: Test cases for the backend code
+
+### File Naming Conventions
+
+The purpose of the file is reflected in its name.
+We use the following suffixes to indicate file types:
+- `*_endpoint.py`: API endpoint definitions
+
+- `*_dto.py`: Pydantic schemas (Data Transfer Objects) for request and response validation of payloads
+- `*_orm.py`: SQL Database models and schemas
+- `*_crud.py`: CRUD operations for database models (SQL)
+
+- `*_collection.py`: Weaviate collection schemas
+- `*_embedding_crud.py`: CRUD operations for embeddings stored in Weaviate
+
+- `*_elastic_index.py`: Elasticsearch index schemas
+- `*_elastic_crud.py`: CRUD operations for Elasticsearch indices
+
+- `*_system.py`: Reusable systems that encapsulate specific functionality
+- `*_service.py`: Business logic and service layer implementations
+- `*_repo.py`: Data access layer and repository implementations
+- `*_utils.py`: Utility functions and helpers
+
+### Coding Instructions
+
+When writing backend code, follow these guidelines:
+
+- Use type hints for function signatures and variable declarations.
+- Write docstrings for all public functions and classes.
+- Use loguru for logging sensible events and errors.

.github/instructions/backend_tests.instructions.md

@@ -0,0 +1,40 @@
+---
+applyTo: "backend/test/**/*.py"
+---
+
+When writing Python tests:
+
+## Test Structure Essentials
+
+- Use pytest as the primary testing framework
+- Follow AAA pattern: Arrange, Act, Assert
+- Write descriptive test names that explain the behavior being tested
+- Keep tests focused on one specific behavior
+
+## Key Testing Practices
+
+- Test edge cases and error conditions, not just happy paths
+
+## Example Test Pattern
+
+```python
+import pytest
+from unittest.mock import Mock, patch
+
+class TestUserService:
+    @pytest.fixture
+    def user_service(self):
+        return UserService()
+
+    @pytest.mark.parametrize("invalid_email", ["", "invalid", "@test.com"])
+    def test_should_reject_invalid_emails(self, user_service, invalid_email):
+        with pytest.raises(ValueError, match="Invalid email"):
+            user_service.create_user({"email": invalid_email})
+
+    @patch('src.user_service.email_validator')
+    def test_should_handle_validation_failure(self, mock_validator, user_service):
+        mock_validator.validate.side_effect = ConnectionError()
+
+        with pytest.raises(ConnectionError):
+            user_service.create_user({"email": "test@example.com"})
+```

.github/instructions/frontend.instructions.md

@@ -0,0 +1,46 @@
+---
+applyTo: "frontend/**/*.ts, frontend/**/*.tsx"
+---
+
+## Frontend
+
+The frontend is built using Typescript with React as the web framework.
+It interacts with the backend through RESTful API endpoints.
+It follows a component-based architecture, organizing UI elements into reusable components.
+
+### Libraries and Frameworks
+
+Dependencies are listed in package.json and managed with npm.
+
+- React for building user interfaces
+- React Router for client-side routing
+- Tanstack Query for data fetching and server state management
+- Redux Toolkit for global state management
+- React Hook Form for form state management
+- React MUI for UI components, styling, theming, and icons
+
+### Folder Structure
+
+- `/frontend/bin`: Various scripts
+- `/frontend/public`: Public assets like logos
+- `/frontend/src`: Main application code
+  - `/api`: Generated API client and Hooks for API interactions
+  - `/auth`: Authentication and authorization logic
+  - `/components`: Reusable React components
+  - `/layouts`: Layout components for different page structures
+  - `/plugins`: Configurations for third-party plugins and integrations (MUI, Tanstack Query, Redux Toolkit, etc.)
+  - `/router`: React Router route definitions
+  - `/store`: Redux Toolkit store configuration
+  - `/utils`: Utility functions and helpers
+  - `/views`: Page components representing different views/screens
+
+### File Naming Conventions
+
+TODO
+
+### Coding Instructions
+
+When writing frontend code, follow these guidelines:
+
+- Use type hints for function signatures and variable declarations.
+- Write docstrings for all public functions and classes.

.github/prompts/codereview.prompt.md

@@ -0,0 +1,31 @@
+---
+agent: ask
+---
+When reviewing code, focus on:
+
+## Security Critical Issues
+- Check for hardcoded secrets, API keys, or credentials
+- Look for SQL injection and XSS vulnerabilities
+- Verify proper input validation and sanitization
+- Review authentication and authorization logic
+
+## Performance Red Flags
+- Identify N+1 database query problems
+- Spot inefficient loops and algorithmic issues
+- Check for memory leaks and resource cleanup
+- Review caching opportunities for expensive operations
+
+## Code Quality Essentials
+- Functions should be focused and appropriately sized
+- Use clear, descriptive naming conventions
+- Ensure proper error handling throughout
+
+## Review Style
+- Be specific and actionable in feedback
+- Explain the "why" behind recommendations
+- Acknowledge good patterns when you see them
+- Ask clarifying questions when code intent is unclear
+
+Always prioritize security vulnerabilities and performance issues that could impact users.
+
+Always suggest changes to improve readability.

backend/.github/prompts/add-config.prompt.md

@@ -0,0 +1,37 @@
+## Backend Configuration Details
+
+The backend configuration is managed using Omegaconf, allowing for flexible and hierarchical settings management:
+
+Backend:
+
+- Configuration files are stored in the `/backend/configs` directory.
+- Different configuration files are used for various environments (e.g., development, production).
+- Some settings can be overridden by environment variables (e.g., database credentials, API keys).
+- Such environment variables are defined in `/backend/.env` file that is not committed to version control for security reasons.
+- Instead, `/backend/.env.example` file is provided to show the required variables.
+
+Docker:
+
+- Docker Compose files (`/docker/`) need to forward relevant environment variables to the backend services.
+- Docker Compose files read environment variables from the `/docker/.env` file, which is not committed to version control.
+- Again, `/docker/.env.example` is provided to illustrate the necessary variables.
+
+Scripts:
+
+- For easy setup, we provide the `/bin/setup-envs.sh` file which copies the example files to `.env` and initializes environment variables.
+
+## Guidelines for Adding New Configuration Options
+
+When adding new configuration options, consider if they should be configurable via environment variables, especially for sensitive data or settings that may vary between deployments.
+
+If no:
+
+1. Update the configuration files in `/backend/configs`.
+
+If yes:
+
+1. Add the configuration files in `/backend/configs`.
+2. Add corresponding environment variable entries in `/backend/.env.example` and `/backend/.env`.
+3. Add corresponding entries in `/docker/.env.example` and `/docker/.env`.
+4. Update the `/bin/setup-envs.sh` script if necessary to handle new environment variables.
+5. Update Docker Compose files in `/docker/` to forward the new environment variables to the backend services.

frontend/.github/prompts/form.prompt.md

@@ -0,0 +1,18 @@
+---
+agent: agent
+---
+
+Your goal is to generate a new React form component.
+
+Ask for the form name and fields if not provided.
+
+Requirements for the form:
+
+- Use `react-hook-form` for form state management:
+  - Always define TypeScript types for your form data
+  - Prefer _uncontrolled_ components using register
+  - Use `defaultValues` to prevent unnecessary rerenders
+- Use `yup` for validation:
+  - Create reusable validation schemas in separate files
+  - Use TypeScript types to ensure type safety
+  - Customize UX-friendly validation rules