DST Dashboard Backend and Frontend V1

[mamoutou-diarra]

Context

This PR adds the DST Dashboard app to 10ksim. The dashboard is composed of a frontend (React) and a backend (FastAPI/uvicorn + MongoDB), both residing in dst_dashboard/. The app exposes endpoints to create/update/delete experiments, panels, and datasets. Datasources (VictoriaLogs, Prometheus) are provided via config.yaml. All write endpoints are JWT-protected. Tokens are generated from an admin page (accessible from the lab homepage, Authentik SSO required) that also provides an experiment CRUD UI for pasting JSON. Data is fetched using DataPuller and scrape_utils, and transform functions convert datasets into ECharts-ready panel specs.

Main changes

Backend

• Added experiment, dataset, and panel processors to fetch data, apply transformations, build ECharts-ready panels, and store them on MongoDB
• Added transform functions for boxplot, timeseries, and utilities like top-N filtering, groupBy, firstN, regex_match
• Implemented a MongoDB storage layer, using GridFS to store large experiment datasets/panels in chunks
• Added CRUD endpoints for experiments, datasets, and panels, exposed via Swagger, with writes gated behind JWT
• Added an admin page for token generation and experiment CRUD (paste JSON)
• Parallelized dataset and panel processing across a thread pool
• Added unit tests for panel processors
• Added script and example for adding new ewperiment json data (They can be added also by curl)

Frontend

• Used React frontend, served via nginx
• Used vac logo as main logo
• Used logos logo as favicon
• Lazy-loads panels as they're needed instead of fetching everything on every page load, reducing bandwidth use and page-load time

Deployment

• Added k8s manifests for the backend, frontend, and MongoDB (dev/prod)

App URLs

• Public DST Dashboard: https://dashboard.lab.vac.dev
• Admin Page (token generation, Add/Update/Delete Experiments): https://api.dashboard.lab.vac.dev/admin/token (require Authentication via keycloak)

[AlbertoSoutullo]

Can we split this in back and front?

[mamoutou-diarra]

Can we split this in back and front?

Ok, no problem