DashAI 0.2.0 (#272)

* Add DeleteSessionConfirmationModal for session deletion confirmation

* Enhance get_all_generative_sessions to return sessions ordered by creation date

* Refactor handleAddSession to add it first in the list

* Fix loading state management by resetting isLoadingMessage on process retrieval

* Fix output processing in GenerativeJob to use config directly

* Remove unneeded Stable Diffusion model options from schema

* Fix loading message state management to check process status before hiding

* Remove unused HistoryIcon import from SessionHistoryModal

* Translate Spanish text to English in SessionHistoryModal and improve scrollbar styling

* Add endpoint to retrieve generated images by path

* Refactor image output processing in ImageGenerationTask and update ImageMessage component to use API endpoint for image retrieval

* Update FullscreenImageChat to use API for image retrieval instead of base64 data

* Add DeepSeekModel for text generation using llama.cpp library

* Refactor Gemma and Qwen models to improve output handling by adjusting stop sequences and echo settings

* Enhance LLM models with new schema fields for improved text generation parameters

* Refactor comments and update parameter history logging in generative session endpoints

* Fix SessionBar overflow handling

* feat: new nodes

* Remove debug print statements from get_generative_image endpoint

* Add error handling and logging for generative process in GenerativeJob; update GenerativeChat to display failure notifications

* Refactor image generation components: rename ImageGenerationTask and ImageGenerationModel

* Adjust layout properties in SessionBar and SessionList for improved responsiveness and visual consistency

* Refactor generative components: update metadata types and handle message rendering

* Enhance generative process handling: refactor input management, update database models, and integrate new ChatInput component for improved user interaction

* Add config dependency to upload_generative_process and pass LOCAL_PATH to input preparation

* Fix key prop in MessageContent component for unique identification of message boxes

* Refactor message rendering with unique key generation

* Pass cardinality prop to MessageContent component in ChatBubble

* Implement input processing methods in generative task classes and update upload process to handle input from database

* Add MediaInput and TextInput components for enhanced message input handling

* Add ControlNetTask class for image generation with input processing methods

* Update generate method signature to accept tuple input for enhanced flexibility

* Add ControlNet models

* Add ControlNet models and tasks to the container

* Add delay before enqueing process to obtain input processed from server

* Add diffusers and opencv-python to requirements.txt

* Rename FluxBasicModel to FluxV1Model and update imports accordingly

* Remove unnecessary else clause for Hugging Face API key validation in FluxV1Model

* Remove debug log for groupedSessions in SessionList component

* Remove debug print statements from upload_generative_process and ControlNetTask

* Set a minimum width for the SessionBar component

* Refactor imports in GenerativeChat component and reset task and messages state on effect trigger

* Fix message type determination logic in MessageContent component

* Reorder state updates in handleSessionClick for clarity

* Remove unnecessary else clause for Hugging Face API key validation

* Rename StableDiffusionControlNetModel to StableDiffusionControlNetCannyModel and update imports

* Add device parameter to get_depth_map and update related logic in SimpleControlNetModel

* Reset task and messages state in useEffect for taskName changes

* Refactor MediaInput component styles and adjust maxRows for text input

* Add overflow hidden style to main container in Generative component

* Update get_depth_map function to include dtype parameter for improved precision handling

* Refactor get_depth_map function to remove dtype parameter and simplify autocast logic

* Replace Annotated import from typing to typing_extensions

* Remove console log statements for session updates and parameter changes

* Remove console log statement from session deletion handler

* Created staging branch to save a stable branch

* Fix image path generation in process_output method to ensure correct saving location in db

* Add Tuple import to typing and update type hints in model and task files for retro-compatobility with python 3.8

* Update model schema documentation for DeepSeek, Gemma, and Qwen models

* Update comment in ParamsBar component to improve clarity

* Fix history handling LLMGenerationTask

* fix: results table and model parameters

* Add IMAGES_PATH to DefaultSettings and update build_config_dict to include it

* Refactor image path handling in generative tasks to use IMAGES_PATH instead of LOCAL_PATH

* Add llama-cpp-python dependency to requirements.txt

* Add torchvision dependency to requirements.txt

* Update torchvision version to 0.19.1 in requirements.txt

* Fix context window size retrieval from gemma model

* Update requirements to include diffusers, torchvision and llama-cpp-python

* Add CUDA support for torch and torchvision in requirements

* Fix Python version comment in requirements.txt and add starlette dependency

* Remove task endpoint and its associated router from API

* Refactor code for improved readability and consistency across multiple files

* Fix formatting of error message in get_generative_session function

* Improve formatting and readability of descriptions in schema fields across multiple model files

* Refactor list comprehensions for improved readability in multiple files

* Ensure images_path directory exists before saving images in ControlNetTask

* Remove display_name from component dict in register_component method

* Remove DISPLAY_NAME attributes from ControlNetTask, LLMGenerationTask, and TextToImageGenerationTask; update SelectTaskMenu and TaskBox components to use 'name' instead of 'display_name' for task identification.

* Update requirements.txt

* Reorder libraries and extra url in requirements.in

* Remove pywin32 from requirements.txt

* fix: pipeline page

* feat: validate pipeline

* feat: validate nodes

* Update SelectModelMenu to display selected task name in model selection prompt

* Add onChange handler to Autocomplete for task selection

* Refactor SelectTaskMenu and TaskBox for improved layout and responsiveness

* Refactor LLM generation components to use TextToTextGenerationTask and remove deprecated LLMGenerationModel

* fix: component registry and dnd

* Refactor Qwen model to use model_name for repository ID and update schema definition

* Update outputs_cardinality to "n" in ControlNetTask and TextToImageGenerationTask metadata

* Remove huggingface_key from SimpleSchema and its usage in SimpleControlNetModel initialization

* Add unit tests for SimpleControlNetModel and StableDiffusionV2Model initialization and generation

* Add generative tutorial and related images to documentation

* Add tests for TextToTextGenerationTask and TextToImageGenerationTask, including metadata validation and input/output processing

* Add API tests for session creation and retrieval, including invalid scenarios

* Add 404 error handling for non-existent generative processes

* Add tests for generative process API, including session creation and retrieval

* Move generative models test files to separate folder

* Update non-existent session test to use numeric ID and improve error message

* Refactor ParamsBar and SelectTaskMenu components for improved layout and responsiveness; adjust spacing in FormSchemaRenderFields and update Generative component's width settings.

* fix: nodes types

* Fix feature extractor initialization to include device parameter for depth estimation.

* Applied pre commit to refactor multiple files

* Remove test_generate function from test_simple_contronet_model.py

* Add protobuf to requirements.in and requirements.txt

* Add ControlNet task tests and improve temp path cleanup

* Add creation of images path in create_app function

* Refactor generative process handling: Introduce ProcessData model, update task methods to use ProcessData for input/output

* Fix history handling in generative tasks

* Remove unused simpleHash function and simplify key generation in MessageContent component

* Update type hints in generative task classes to use Tuple for consistency

* Rename endpoint function and update table name for generative session parameter history

* Update requirements.txt: Adjust Python version to modify protobuf version

* Update image processing to use float16 for improved performance

* Update model reference in StableDiffusionControlNetCannyModel to use medium version

* Fix typo in stable diffusion control net model

* Update model reference in StableDiffusionControlNetCannyModel to use stable-diffusion-3.5-large

* Update FluxV1Model to use float16 for improved performance

* Add conditional rendering for NewSessionButton and display message when no session is selected

* Display "Untitled Session" for sessions without a name in GenerativeChat and SessionBox components

* Refactor styling in ParamsBar component for consistent margin and padding

* Refactor CustomLayout to allow customizable padding for child components

* Refactor SelectTaskMenu and TaskBox components for improved layout and styling

* Refactor generative components style to use theme colors for consistency

* Refactor SelectTaskMenu and TaskBox components for improved task display and layout

* Moved history button to parameters list component

* Pass stepIndex prop to SessionBar component for improved new session button handling

* Refactor SearchBar and SelectTaskMenu components for improved search functionality and UI consistency

* Refactor SelectModelMenu component for improved layout and styling of form elements

* Fix delete_generative_session removing associated processes and parameter history

* Refactor layout of Generative component for improved responsiveness and styling; update ParamsBar and SessionBar components for better UI consistency

* Refactor component styles for improved consistency and layout; adjust padding and margins in Generative, MainGenerativeBox, SelectModelMenu, TaskBox, and Generative pages

* fix: node exploration

* feat: retrieve model node

* Improve text column handling in DistilBertTransformer.
Add torch with cuda118 in requirements.

* fix package version

* Remove StableDiffusionControlNetCannyModel and its references from the codebase

* Remove FluxV1Model and its references from the codebase

* Add display_name attribute to component registry and update tests

* Add DISPLAY_NAME attribute to generative tasks

* Update task name reference to use display_name in SelectTaskMenu; remove unused imports in TaskBox

* Refactor history handling in prepare_for_task method for clarity; update test case to match new format

* Add utility function to check GPU offloading support for Llama models

* Remove hidden "Add image" label from MediaInput component

* Refactor code using pre commit

* Refactor generative models schema to support multiple GPUs

* Add display_name to session data and update related components for improved session management

* Update type hints for negative_prompt field in StableDiffusionSchema to use Optional

* Add validation for model parameters in upload_generative_session endpoint

* Add schema utility functions for Yup validation

* Fix setFieldValue calls to ensure validation is triggered on form updates

* Refactor ParamsBar and SelectModelMenu components to integrate schema preprocessing and validation schema setup

* Refactor schema preprocessing to improve handling of nullable types and update comments for clarity

* Remove DeepSeek model and schema implementation from the codebase

* Remove DeepSeekModel references from initial_components and models initialization

* Add device configuration to Gemma and Qwen models for inference

* Remove use_gpu parameter from QwenModel initialization

* fix: front

* fix: nodes descriptions

* Remove debug console.log for selectedSessionId in SessionBar component

* Fix session click handler to use correct task name in SessionList component

* Fix generative session endpoint adding model_name to the response and fix tests for process and session API

* Replace SimpleControlNetModel with StableDiffusionXLV1ControlNet and update related tests

* Add Generative button to Home page for AI model interaction

* Refactor ParamsBar and SelectModelMenu components to improve state management and validation schema handling

* Add back navigation and update model selection UI in SelectModelMenu

* Enhance SelectModelMenu and related components to include display names and improve session handling

* Rename 'n_ctx' to 'context_window' in Gemma and Qwen models for clarity

* Update alignment in SelectTaskMenu to stretch for better layout consistency

* feat: delete node button

* Update dtype in depth map estimation and model initialization to float32 for improved precision

* Update torch_dtype in StableDiffusionXLV1ControlNet to float32 for improved precision

* Update torch_dtype handling in depth map estimation for device compatibility

* Refactor depth map estimation to use float16 for improved performance

* Remove old requirements files and update installation instructions to use pip-tools for better dependency management.

* Refactor GPU availability check in llama_utils.py to ensure correct function is apply depending in llamacpp version

* Fix: initial values in ParamsBar Form bug

* Add back navigation and update model selection UI in SelectModelMenu

* Refactor prepare_for_task method to include history in structured format for improved input preparation

* Refactor text generation methods in GemmaModel and QwenModel to use create_chat_completion for improved message handling

* Fix: disable send button when input is empty in TextInput component

* Enhance TextMessage component to render message using Markdown for improved formatting

* Enhance error handling in GenerativeJob and improve failure notification in GenerativeChat

* Refactor: Run formatter

* Refactor: Clean up imports and improve logging message formatting in llama_utils.py and update __is_gpu_available_for_llama_cpp_v02

* Enhance QwenSchema: Add new model version "Qwen/Qwen3-4B-GGUF" and fix filename format in QwenModel

* Fix generate method signature to accept a list of dictionaries for prompts

* feat: add height control to plots and a fullscreen button

* Fix fullscreen icon

* Revert requirements changes from 6eff065, keeping Distilbert updates

* fix: retrieve model node

* fix: yarn

* Enhance CSV and Excel dataloaders with additional schema fields and parameter handling; update JSON dataloader description; add xlrd to requirements.

* Developed comprehensive tutorials on how to upload plugins to PyPI and detailed the plugin development process, including structure and requirements.

* merge develop

* Fix precommits

* Fix dataloader tests and some warnings

* fix: await job

* refactor: pipelines module

* fix: enqueueSnackbar and yarn

* fix: pre-commit

* fix: yarn

* fix: yarn build

* fix: yarn

* fix: experiments results width

* fix: node name help bar

* Upgraded plotly version. Clean install of yarn lock

* Updated yarn version

* feat: configurable node

* Added dvclive to gitignore. Fixed tests with old syntax

* Fixed requirements to make docs work

* Test para CI/CD de docs

* Test para CI/CD de docs

* Trying to fix requirements and docs

* Trying to fix requirements+rewrote the entire docs action from scratch

* Docs fixed. Trying to fix requirements for CI/CD

* Removed pip-compile generated requirements.txt file

* Ran out of memory, removed some .fit and stable diffusion generation

* Ran out of memory, removed stable difussion tests completly

* Fixed SHAP test, removed more memory intensive tests

* fix: refactor import statements in ConfigureExplorersStep and NodeSidebar components to avoid circular dependency

* Updated the version of dashai to 0.2.0 previous to production merge

---------

Co-authored-by: Irozuku <emiliodu@gmail.com>
Co-authored-by: Creylay <careyesla@gmail.com>
Co-authored-by: Carla Hayler <carla.hayler@ug.uchile.cl>
Co-authored-by: Felipe Cardenas <felipeygna.fc@gmail.com>

Author: 5 people

(old)requirements-dev.txt

@@ -10,6 +10,8 @@ jobs:
         working-directory: DashAI/front
     steps:
       - uses: actions/checkout@v3
+      - name: Enable Corepack and use Yarn 3
+        run: corepack enable
       - name: Use Node.js
         uses: actions/setup-node@v3
         with:

(old)requirements.txt

@@ -4,12 +4,12 @@ on:
     branches:
       - production
       - develop
-# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
+
 permissions:
   contents: read
   pages: write
   id-token: write
-# Allow one concurrent deployment
+
 concurrency:
   group: "pages"
   cancel-in-progress: true
@@ -20,28 +20,52 @@ jobs:
     environment:
       name: github-pages
       url: ${{ steps.deployment.outputs.page_url }}
+
     steps:
       - name: Checkout
         uses: actions/checkout@v3
 
       - name: Set up Python
         uses: actions/setup-python@v4
         with:
-          python-version: '3.10'
+          python-version: "3.10"
+          cache: "pip"
+          cache-dependency-path: |
+            requirements.txt
+            requirements-dev.txt
+            docs/requirements.txt
+
+      - name: Install system build tools (gcc, g++, cmake, ninja)
+      # Compilará llama-cpp-python sin dramas
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y build-essential cmake ninja-build
 
       - name: Build docs
-        uses: ammaraskar/sphinx-action@6.1.3
-        with:
-          docs-folder: "docs/"
-          pre-build-command: "pip install -e ."
+        env:
+          # Asegura que scikit-build encuentre el compilador
+          CC: gcc
+          CXX: g++
+          # Evita ruidos de pip
+          PIP_DISABLE_PIP_VERSION_CHECK: "1"
+        run: |
+          python -m pip install -U pip setuptools wheel scikit-build-core
+          # Dependencias SOLO de docs
+          pip install -r docs/requirements.txt
+          # Instalación normal del proyecto (con todas las deps)
+          pip install -e .
+          # (opcional) limpia stubs viejos por si quedaron versionados
+          rm -rf docs/source/api/generated || true
+          # Construir HTML directamente (sin action)
+          sphinx-build -b html docs/source docs/build/html
 
       - name: Setup Pages
         uses: actions/configure-pages@v2
 
-      - name: Upload artifactfi
+      - name: Upload artifact
         uses: actions/upload-pages-artifact@v3
         with:
-          path: 'docs/build/html'
+          path: "docs/build/html"
           retention-days: 1
 
       - name: Deploy to GitHub Pages

.github/workflows/build-test.yaml

@@ -1,4 +1,4 @@
-Byte-coVscode mpiled / optimized / DLL files
+# Vscode compiled / optimized / DLL files
 *.py[cod]
 *$py.class
 
@@ -136,7 +136,7 @@ venv.bak/
 
 # mkdocs documentation
 /site
-
+docs/source/api/generated
 # mypy
 .mypy_cache/
 .dmypy.json
@@ -172,9 +172,13 @@ user_runs
 user_models
 user_explainers
 __pycache__
+dvclive
 
 # ruff cache
 .ruff_cache
 
 # MacOS
 *.DS_Store
+
+db.sqlite
+trained_models/

.github/workflows/docs.yaml

@@ -8,7 +8,10 @@
     explainers,
     explorations,
     explorers,
+    generative_process,
+    generative_session,
     jobs,
+    pipelines,
     plugins,
     predict,
     runs,
@@ -25,4 +28,7 @@
 api_router_v1.include_router(jobs.router, prefix="/job")
 api_router_v1.include_router(runs.router, prefix="/run")
 api_router_v1.include_router(predict.router, prefix="/predict")
+api_router_v1.include_router(generative_session.router, prefix="/generative-session")
+api_router_v1.include_router(generative_process.router, prefix="/generative-process")
+api_router_v1.include_router(pipelines.router, prefix="/pipelines")
 api_router_v1.include_router(plugins.router, prefix="/plugin")