Cursor files refinery-updater (#119)

* Cursor files refinery-updater

* submodules merge

Author: lumburovskalina

.cursor/rules/api-models.mdc

@@ -0,0 +1,107 @@
+---
+description: Rules for Pydantic models and request/response validation
+globs: ["app.py"]
+alwaysApply: true
+---
+
+# API Models Guidelines
+
+Pydantic models validate request bodies and ensure type safety. Models are defined directly in `app.py` or imported when needed.
+
+## Model Definition
+
+**Basic structure:**
+```python
+from pydantic import BaseModel
+
+class UpgradeToAWS(BaseModel):
+    only_existing: bool
+    remove_from_minio: bool
+    force_overwrite: bool
+```
+
+## Naming Conventions
+
+- Use descriptive names that reflect the operation: `UpgradeToAWS`, `MigrationRequest`
+- No need for `Body` suffix in this service (simpler naming)
+
+## Usage in Routes
+
+```python
+from pydantic import BaseModel
+
+class UpgradeToAWS(BaseModel):
+    only_existing: bool
+    remove_from_minio: bool
+    force_overwrite: bool
+
+@app.post("/migrate_minio_to_aws")
+def migrate_minio_to_aws(request: UpgradeToAWS) -> int:
+    from upgrade_logic.pre_redesign.upgrade_from_minio_to_aws import upgrade
+    upgrade(
+        request.only_existing, 
+        request.remove_from_minio, 
+        request.force_overwrite
+    )
+    return 0
+```
+
+## Field Types
+
+**Basic types:**
+```python
+from pydantic import BaseModel
+from typing import Optional, List
+
+class MigrationRequest(BaseModel):
+    service_name: str
+    target_version: str
+    dry_run: bool = False  # Default value
+    options: Optional[List[str]] = None  # Optional field
+```
+
+## Field Validation
+
+```python
+from pydantic import BaseModel, Field, field_validator
+
+class UpgradeToAWS(BaseModel):
+    only_existing: bool = Field(description="Only migrate existing organization buckets")
+    remove_from_minio: bool = Field(description="Remove files from MinIO after migration")
+    force_overwrite: bool = Field(default=False, description="Overwrite existing files")
+    
+    @field_validator('only_existing')
+    @classmethod
+    def validate_only_existing(cls, v):
+        if not isinstance(v, bool):
+            raise ValueError('only_existing must be a boolean')
+        return v
+```
+
+## Best Practices
+
+1. **Simple models**: Keep models simple and focused on the request data
+2. **Type hints**: Use proper type hints for all fields
+3. **Defaults**: Provide sensible defaults for optional fields
+4. **Descriptive names**: Use clear, descriptive model names
+5. **Validation**: Add validators when input validation is needed
+6. **Documentation**: Use `Field(description=...)` for API documentation
+7. **Location**: Define models in `app.py` near their usage, or create separate file if many models
+
+## Example: Complete Model with Validation
+
+```python
+from pydantic import BaseModel, Field, field_validator
+
+class VersionUpdateRequest(BaseModel):
+    service: str = Field(description="Service name to update")
+    target_version: Optional[str] = Field(None, description="Target version (defaults to latest)")
+    force: bool = Field(False, description="Force update even if already at version")
+    
+    @field_validator('service')
+    @classmethod
+    def validate_service(cls, v):
+        if not v or not v.strip():
+            raise ValueError('Service name cannot be empty')
+        return v.strip().upper()
+```

.cursor/rules/exceptions.mdc

@@ -0,0 +1,112 @@
+---
+description: Rules for exception handling and custom exceptions
+globs: ["**/*.py"]
+alwaysApply: true
+---
+
+# Exceptions Guidelines
+
+## Exception Locations
+
+**Submodule exceptions:**
+```python
+from submodules.model.exceptions import EntityNotFoundException, EntityAlreadyExistsException
+```
+
+**Standard Python exceptions:**
+- `ValueError`: Invalid input values
+- `Exception`: Generic exceptions (use sparingly)
+
+## Usage Patterns
+
+**Raising exceptions:**
+```python
+# Validation
+if not version:
+    raise ValueError("Version is required")
+
+# Version format validation
+def is_newer(v1: str, v2: str) -> bool:
+    a = [__get_int_from_string(x) for x in v1.split(".")]
+    b = [__get_int_from_string(x) for x in v2.split(".")]
+    if len(a) != len(b) and len(a) != 3:
+        raise Exception("invalid version format")
+    return __is_newer_int(a, b)
+```
+
+**Handling in routes:**
+```python
+@app.post("/endpoint")
+def endpoint():
+    session_token = general.get_ctx_token()
+    try:
+        result = util.some_operation()
+        return responses.JSONResponse(
+            status_code=status.HTTP_200_OK,
+            content=jsonable_encoder(result),
+        )
+    except ValueError as e:
+        return responses.PlainTextResponse(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            content=f"Invalid input: {str(e)}",
+        )
+    except Exception as e:
+        print(f"Error: {e}", flush=True)
+        return responses.PlainTextResponse(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            content=f"Error: {str(e)}",
+        )
+    finally:
+        general.remove_and_refresh_session(session_token)
+```
+
+**Handling in utility functions:**
+```python
+def __last_tag(repo_link: str) -> str:
+    try:
+        g = git.cmd.Git()
+        blob = g.ls_remote(repo_link, sort="-v:refname", tags=True)
+        if len(blob) == 0:
+            return "0.0.0"
+        tag = blob.split("\n")[0].split("/")[-1]
+        if tag[0] == "v":
+            return tag[1:]
+        return tag
+    except Exception:
+        # Return safe default instead of raising
+        return "0.0.0"
+```
+
+**Handling in upgrade functions:**
+```python
+def gateway_1_14_0_add_cognition_strategy_complexity() -> bool:
+    cognition_url = os.getenv("COGNITION_GATEWAY")
+    if not cognition_url:
+        print("No cognition gateway url found. Skipping...")
+        return False  # Return False instead of raising
+    
+    try:
+        response = requests.post(f"{cognition_url}/api/v1/strategies/internal/...")
+        if response.status_code != 200:
+            print(f"Failed to update. Status code: {response.status_code}")
+            return False
+        return True
+    except Exception as e:
+        print(f"Error calling external service: {e}", flush=True)
+        return False
+```
+
+## HTTP Status Code Mapping
+
+- `400`: `ValueError` - Invalid input parameters
+- `500`: `Exception` - Unexpected errors
+
+## Best Practices
+
+1. **Return False for failures**: Upgrade functions should return `False` instead of raising exceptions
+2. **Safe defaults**: Return safe default values (e.g., `"0.0.0"` for version) when operations fail
+3. **Log errors**: Use `print()` with `flush=True` to log errors and important information
+4. **Handle external calls**: Always handle exceptions from external service calls gracefully
+5. **Validate inputs**: Use `ValueError` for invalid input validation
+6. **Don't swallow silently**: Log errors even when returning safe defaults
+7. **Session cleanup**: Always use `finally` blocks to ensure session cleanup even on exceptions

.cursor/rules/fastapi-routes.mdc

@@ -0,0 +1,167 @@
+---
+description: Rules for FastAPI route definitions and HTTP handling
+globs: ["app.py"]
+alwaysApply: true
+---
+
+# FastAPI Routes Guidelines
+
+Routes handle HTTP request/response logic. Keep routes thin - validate input, call utility functions, format responses.
+
+## Route Structure
+
+Routes are defined directly in `app.py`:
+
+```python
+from fastapi import FastAPI, responses, status
+from fastapi.encoders import jsonable_encoder
+from submodules.model.business_objects import general
+import util
+
+app = FastAPI(title="refinery-updater")
+
+@app.post("/update_to_newest")
+def update_to_newest() -> responses.PlainTextResponse:
+    session_token = general.get_ctx_token()
+    if util.update_to_newest():
+        msg = "updated to current version"
+    else:
+        msg = "nothing to update"
+    general.remove_and_refresh_session(session_token)
+    return responses.PlainTextResponse(status_code=status.HTTP_200_OK, content=msg)
+```
+
+## Response Patterns
+
+**Plain text responses:**
+```python
+@app.post("/update_to_newest")
+def update_to_newest() -> responses.PlainTextResponse:
+    # ... logic ...
+    return responses.PlainTextResponse(
+        status_code=status.HTTP_200_OK, 
+        content="updated to current version"
+    )
+```
+
+**JSON responses:**
+```python
+@app.get("/version_overview")
+def version_overview() -> responses.JSONResponse:
+    session_token = general.get_ctx_token()
+    return_values = util.version_overview()
+    general.remove_and_refresh_session(session_token)
+    return responses.JSONResponse(
+        status_code=status.HTTP_200_OK,
+        content=jsonable_encoder(return_values),
+    )
+```
+
+**Conditional response types:**
+```python
+@app.get("/has_updates")
+def has_updates(as_html_response: bool = False) -> responses.JSONResponse:
+    session_token = general.get_ctx_token()
+    return_value = util.has_updates()
+    general.remove_and_refresh_session(session_token)
+    if as_html_response:
+        return responses.HTMLResponse(content=str(return_value))
+    return responses.JSONResponse(
+        status_code=status.HTTP_200_OK,
+        content=return_value,
+    )
+```
+
+## Session Management
+
+**Always manage database sessions:**
+```python
+@app.post("/endpoint")
+def endpoint():
+    session_token = general.get_ctx_token()
+    try:
+        # ... database operations ...
+        return responses.JSONResponse(...)
+    finally:
+        general.remove_and_refresh_session(session_token)
+```
+
+## Request Body Models
+
+Use Pydantic models for request validation:
+
+```python
+from pydantic import BaseModel
+
+class UpgradeToAWS(BaseModel):
+    only_existing: bool
+    remove_from_minio: bool
+    force_overwrite: bool
+
+@app.post("/migrate_minio_to_aws")
+def migrate_minio_to_aws(request: UpgradeToAWS) -> int:
+    from upgrade_logic.pre_redesign.upgrade_from_minio_to_aws import upgrade
+    upgrade(request.only_existing, request.remove_from_minio, request.force_overwrite)
+    return 0
+```
+
+## Query Parameters
+
+```python
+@app.get("/has_updates")
+def has_updates(as_html_response: bool = False) -> responses.JSONResponse:
+    # Query parameter with default value
+    pass
+```
+
+## Health Check Pattern
+
+```python
+@app.get("/healthcheck")
+def healthcheck() -> responses.PlainTextResponse:
+    text = ""
+    status_code = status.HTTP_200_OK
+    database_test = general.test_database_connection()
+    if not database_test.get("success"):
+        error_name = database_test.get("error")
+        text += f"database_error:{error_name}:"
+        status_code = status.HTTP_500_INTERNAL_SERVER_ERROR
+    if not text:
+        text = "OK"
+    return responses.PlainTextResponse(text, status_code=status_code)
+```
+
+## Error Handling
+
+Handle errors gracefully and return appropriate status codes:
+
+```python
+@app.post("/endpoint")
+def endpoint():
+    session_token = general.get_ctx_token()
+    try:
+        result = util.some_operation()
+        return responses.JSONResponse(
+            status_code=status.HTTP_200_OK,
+            content=jsonable_encoder(result),
+        )
+    except Exception as e:
+        print(f"Error: {e}", flush=True)
+        return responses.PlainTextResponse(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            content=f"Error: {str(e)}",
+        )
+    finally:
+        general.remove_and_refresh_session(session_token)
+```
+
+## Best Practices
+
+1. **Keep routes thin**: Delegate business logic to `util.py` or `upgrade_logic/`
+2. **Session management**: Always use `get_ctx_token()` and `remove_and_refresh_session()`
+3. **Type hints**: Use return type annotations (`-> responses.PlainTextResponse`)
+4. **Status codes**: Use appropriate HTTP status codes from `status` module
+5. **JSON encoding**: Use `jsonable_encoder()` for complex objects
+6. **Error handling**: Handle exceptions and return appropriate error responses
+7. **Route naming**: Use snake_case for route paths: `/update_to_newest`, `/version_overview`
+8. **Health checks**: Include health check endpoints that test critical dependencies