Enhance implementation of MovieVerse AI components and configurations & sync code with private codebase

DEPLOYMENT.md

@@ -176,17 +176,27 @@ flowchart LR
 
 ### Option 2: Kubernetes (EKS/GKE/AKS)
 
-1. **Apply base resources:**
+Use the consolidated Kubernetes stack in `kubernetes/` for both the core microservices and rollout patterns.
+
+1. **Apply microservices base resources:**
+   ```bash
+   kubectl apply -f kubernetes/base/
+   kubectl apply -f kubernetes/infra/
+   kubectl apply -f kubernetes/services/
+   kubectl apply -f kubernetes/edge/
+   ```
+
+2. **Apply legacy blue-green base resources (optional):**
    ```bash
    kubectl apply -f kubernetes/base/
    ```
 
-2. **Deploy using Blue-Green strategy:**
+3. **Deploy using Blue-Green strategy:**
    ```bash
    kubectl apply -f kubernetes/blue-green/
    ```
 
-3. **Or deploy using Canary strategy:**
+4. **Or deploy using Canary strategy:**
    ```bash
    kubectl apply -f kubernetes/canary/
    ```

MovieVerse-AI/Dockerfile

@@ -0,0 +1,22 @@
+FROM python:3.11-slim
+
+WORKDIR /opt/movieverse-ai
+
+ENV PYTHONDONTWRITEBYTECODE=1 \
+    PYTHONUNBUFFERED=1
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    && rm -rf /var/lib/apt/lists/*
+
+COPY MovieVerse-AI/requirements.txt ./requirements.txt
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY MovieVerse-AI/movieverse_ai ./movieverse_ai
+COPY MovieVerse-AI/feature_repo ./feature_repo
+
+RUN mkdir -p /opt/movieverse-ai/models /opt/movieverse-ai/index
+
+EXPOSE 9000
+
+CMD ["uvicorn", "movieverse_ai.api.main:app", "--host", "0.0.0.0", "--port", "9000"]

MovieVerse-AI/Dockerfile.airflow

@@ -0,0 +1,15 @@
+FROM apache/airflow:2.8.3
+
+USER root
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    && rm -rf /var/lib/apt/lists/*
+USER airflow
+
+COPY MovieVerse-AI/requirements.txt /tmp/requirements.txt
+RUN pip install --no-cache-dir -r /tmp/requirements.txt
+
+COPY MovieVerse-AI/movieverse_ai /opt/airflow/movieverse_ai
+COPY MovieVerse-AI/feature_repo /opt/airflow/feature_repo
+
+ENV PYTHONPATH=/opt/airflow

MovieVerse-AI/README.md

@@ -0,0 +1,183 @@
+# MovieVerse AI Platform
+
+MovieVerse-AI is a production-grade machine learning platform that powers recommendations, similarity search, sentiment analysis, and ranking for MovieVerse. It includes data ingestion, feature store, model training, registry, inference APIs, and monitoring.
+
+## Capabilities
+
+- Personalized recommendations (collaborative + content-aware)
+- Movie similarity search using vector embeddings
+- Review sentiment classification
+- Ranking model for feed/search ordering
+- Feature store with offline/online serving (Feast + Redis)
+- MLflow model registry and experiment tracking
+- Kafka-driven event ingestion
+- Drift monitoring with Evidently
+
+## Stack
+
+- Python 3.11
+- FastAPI + Uvicorn for online inference
+- LightFM for recommender models
+- SentenceTransformers + FAISS for embeddings
+- scikit-learn for sentiment and ranking models
+- Feast for feature storage
+- MLflow for registry and experiments
+- Kafka for event streaming
+- PostgreSQL + MySQL + MongoDB data sources
+- Redis online store
+
+## Architecture
+
+Online inference flow:
+
+```mermaid
+flowchart LR
+  Backend[MovieVerse Backend] -->|HTTP| API[MovieVerse AI API]
+  API --> Reco[Recommender Service]
+  API --> Similar[Similarity + Embeddings]
+  API --> Ranker[Ranking Service]
+  API --> Sentiment[Sentiment Service]
+  API --> Summarize[Summarization Service]
+  API --> Vision[Vision Service]
+  Reco --> Redis[(Redis Online Store)]
+  Similar --> FAISS[(FAISS Index)]
+  API --> Feast[Feature Store]
+  Feast --> Redis
+  Feast --> Postgres[(PostgreSQL Feature Repo)]
+  API --> MLflow[(MLflow Registry)]
+```
+
+Training + feature pipeline flow:
+
+```mermaid
+flowchart TD
+  Kafka[(Kafka Events)] --> Ingest[Event Ingestion]
+  MySQL[(MySQL Movies)] --> Ingest
+  Postgres[(PostgreSQL Ratings/Reviews)] --> Ingest
+  Mongo[(Mongo Metadata)] --> Ingest
+  Ingest --> Features[Build Feature Tables]
+  Features --> Feast[Feast Feature Store]
+  Feast --> Materialize[Materialize Online Features]
+  Materialize --> Redis[(Redis Online Store)]
+  Features --> Train[Train Models]
+  Train --> MLflow[(MLflow Registry)]
+  MLflow --> Sync[Sync Latest Models]
+  Sync --> API[MovieVerse AI API]
+```
+
+## Directory Layout
+
+```
+MovieVerse-AI/
+├── movieverse_ai/           # Core package (API, models, pipelines)
+├── feature_repo/            # Feast feature store definitions
+├── airflow/dags/            # Training + materialization workflows
+├── sql/                     # Database schema for AI data
+├── k8s/                     # Kubernetes deployment assets
+├── docker-compose.ai.yml    # Local AI stack runtime
+├── docker-compose.airflow.yml # Airflow scheduler/runtime
+├── requirements.txt         # Production dependencies
+└── Dockerfile               # Inference service image
+```
+
+## Services
+
+- `movieverse_ai.api.main`: Online inference API
+- `movieverse_ai.pipelines.ingest_events`: Kafka ingestion into Postgres
+- `movieverse_ai.pipelines.train_recommender`: Recommendation training
+- `movieverse_ai.pipelines.train_sentiment`: Review sentiment training
+- `movieverse_ai.pipelines.build_ranking_features`: Ranking feature generation
+- `movieverse_ai.pipelines.train_ranker`: Ranking model training
+- `movieverse_ai.pipelines.build_embeddings`: Embedding + FAISS index
+- `movieverse_ai.pipelines.build_feature_tables`: User/movie feature tables for Feast
+- `movieverse_ai.pipelines.materialize_features`: Feast materialization
+- `movieverse_ai.pipelines.sync_models`: Pull latest artifacts from MLflow
+- `movieverse_ai.monitoring.drift_monitor`: Drift reports
+
+## Local Run
+
+1) Start the AI stack (Postgres, Redis, Kafka, MLflow, MinIO):
+
+```bash
+docker compose -f MovieVerse-AI/docker-compose.ai.yml up -d
+```
+
+The AI API is exposed at `http://localhost:9100` when running via Docker Compose.
+
+2) Initialize the AI database schema:
+
+```bash
+psql postgresql://movieverse:movieverse@localhost:5433/movieverse_ai \
+  -f MovieVerse-AI/sql/postgres_init.sql
+```
+
+3) (Optional) Start Airflow for scheduled pipelines:
+
+```bash
+psql postgresql://movieverse:movieverse@localhost:5433/postgres -c "CREATE DATABASE airflow"
+docker compose -f MovieVerse-AI/docker-compose.airflow.yml up -d
+```
+
+Airflow connects to the AI stack over the `movieverse-ai` Docker network, so start the AI stack first.
+
+4) Train models and build embeddings:
+
+```bash
+cd MovieVerse-AI
+python -m movieverse_ai.pipelines.train_recommender
+python -m movieverse_ai.pipelines.train_sentiment
+python -m movieverse_ai.pipelines.build_ranking_features
+python -m movieverse_ai.pipelines.train_ranker
+python -m movieverse_ai.pipelines.build_embeddings
+python -m movieverse_ai.pipelines.build_feature_tables
+```
+
+5) Start the inference API:
+
+```bash
+cd MovieVerse-AI
+uvicorn movieverse_ai.api.main:app --host 0.0.0.0 --port 9000
+```
+
+## Environment Variables
+
+All configuration is driven by `MOVIEVERSE_AI_` prefixed environment variables. Defaults are set for the local AI stack.
+
+Key variables:
+
+- `MOVIEVERSE_AI_POSTGRES_DSN`
+- `MOVIEVERSE_AI_MYSQL_DSN`
+- `MOVIEVERSE_AI_MONGO_URI`
+- `MOVIEVERSE_AI_REDIS_URL`
+- `MOVIEVERSE_AI_KAFKA_BOOTSTRAP_SERVERS`
+- `MOVIEVERSE_AI_MLFLOW_TRACKING_URI`
+- `MOVIEVERSE_AI_S3_ENDPOINT_URL`
+
+## Production Deployment
+
+- Kubernetes manifests live in `MovieVerse-AI/k8s/` and deploy the inference API.
+- Use managed AWS services (RDS, MSK, OpenSearch, ElastiCache) and point the AI stack to them via environment variables.
+- MLflow artifacts are stored in S3/MinIO; align this with the same bucket used in your infra stack.
+
+## Data Requirements
+
+The AI pipelines expect these tables:
+
+- `ratings` (user_id, movie_id, rating, created_at)
+- `reviews` (user_id, movie_id, rating, review_text, created_at)
+- `ranking_features` (movie_id, popularity, avg_rating, rating_count, recency_days, label)
+- `user_features` and `movie_features` for Feast
+
+Movie metadata is sourced from MySQL `movies` (movie_id, title, overview, genres, release_date).
+
+## API Endpoints
+
+- `GET /healthz`
+- `GET /metrics`
+- `POST /recommendations`
+- `POST /similar`
+- `POST /sentiment`
+- `POST /rank`
+- `POST /summarize`
+- `POST /genres/classify`
+- `POST /vision/labels`

MovieVerse-AI/airflow/dags/movieverse_ai_pipeline.py

@@ -0,0 +1,59 @@
+from datetime import datetime, timedelta
+
+from airflow import DAG
+from airflow.operators.bash import BashOperator
+
+DEFAULT_ARGS = {
+    "owner": "movieverse-ai",
+    "depends_on_past": False,
+    "retries": 1,
+    "retry_delay": timedelta(minutes=10),
+}
+
+with DAG(
+    dag_id="movieverse_ai_pipeline",
+    default_args=DEFAULT_ARGS,
+    description="MovieVerse AI training and feature materialization",
+    schedule_interval="0 2 * * *",
+    start_date=datetime(2024, 1, 1),
+    catchup=False,
+    max_active_runs=1,
+) as dag:
+    train_recommender = BashOperator(
+        task_id="train_recommender",
+        bash_command="python -m movieverse_ai.pipelines.train_recommender",
+    )
+
+    train_sentiment = BashOperator(
+        task_id="train_sentiment",
+        bash_command="python -m movieverse_ai.pipelines.train_sentiment",
+    )
+
+    train_ranker = BashOperator(
+        task_id="train_ranker",
+        bash_command="python -m movieverse_ai.pipelines.train_ranker",
+    )
+
+    build_ranking_features = BashOperator(
+        task_id="build_ranking_features",
+        bash_command="python -m movieverse_ai.pipelines.build_ranking_features",
+    )
+
+    build_embeddings = BashOperator(
+        task_id="build_embeddings",
+        bash_command="python -m movieverse_ai.pipelines.build_embeddings",
+    )
+
+    materialize_features = BashOperator(
+        task_id="materialize_features",
+        bash_command="python -m movieverse_ai.pipelines.materialize_features",
+    )
+
+    build_feature_tables = BashOperator(
+        task_id="build_feature_tables",
+        bash_command="python -m movieverse_ai.pipelines.build_feature_tables",
+    )
+
+    (train_recommender, train_sentiment) >> build_embeddings
+    build_ranking_features >> train_ranker
+    (build_embeddings, train_ranker) >> build_feature_tables >> materialize_features