Starting (#2)

* Windows is a pain

* Windows is a pain 2

* Hmm at least its working now. Time to add some UI

* Some working UI with warmup .. i need to change that and make it better.

* A bit better. More work needed. Time to eat.

* A bit better. More work needed.

* Again progress. Stable version.

* Further progress. Now fonts.

* Further progress

* UI is good enough lets focus on api.

* Working api and UI and added queue for tasks.

* Small cleanup before next round.

* Few upgrades to the UI and API. Working on defaults.

* Corrected default issue and locked dependencies.

* Working initial build.

* Fix the current build while working on whl

Author: TheMasterOfDisasters

Dockerfile

@@ -2,6 +2,7 @@
 # Stage 1: Build environment with dependencies and models
 # ============================================================
 FROM python:3.10-slim AS builder
+#FROM pytorch/pytorch:2.8.0-cuda12.9-cudnn9-devel AS builder
 
 WORKDIR /app
 
@@ -19,8 +20,12 @@ RUN pip install --upgrade pip setuptools wheel
 RUN pip install -r requirements.txt
 # RUN python -m unidic download
 
+# Install matching prebuilt wheel
+#RUN pip install https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.3.17/flash_attn-2.7.4+cu129torch2.8-cp311-cp311-linux_x86_64.whl
+
+
 # Install Flash Attention for faster attention mechanisms
-RUN pip install flash-attn --no-build-isolation
+# RUN pip install flash-attn --no-build-isolation
 
 # Now bring in the actual application code
 COPY app/ .
@@ -38,6 +43,7 @@ RUN python init_downloads.py && \
 # Stage 2: Final runtime image
 # ============================================================
 FROM python:3.10-slim
+#FROM pytorch/pytorch:2.8.0-cuda12.9-cudnn9-devel
 
 WORKDIR /app
 
@@ -46,6 +52,7 @@ RUN apt-get update && apt-get install -y \
     && rm -rf /var/lib/apt/lists/*
 
 COPY --from=builder /usr/local /usr/local
+#COPY --from=builder /opt/conda /opt/conda
 COPY --from=builder /app /app
 COPY --from=builder /root/.cache/huggingface /root/.cache/huggingface
 # COPY --from=builder /root/nltk_data /root/nltk_data