Merge pull request #44 from HYLODE/dev

Dev

Author: docsteveharris

.github/workflows/main.yml

@@ -64,8 +64,10 @@ jobs:
           set -o allexport; source .env; set +o allexport
           export HYUI_UID=9998
           docker compose up -d --build
-          docker compose exec api pytest tests/unit
-          docker compose exec apps pytest tests/unit
+          docker compose exec api pytest tests/smoke
+          docker compose exec api pytest tests/mock
+          docker compose exec api pytest -m api tests/endpoints
+          docker compose exec apps pytest -m app tests/endpoints
 
   test_e2e:
     # The type of runner that the job will run on

README.md

@@ -1,6 +1,6 @@
 [![CI](https://github.com/HYLODE/HyUi/actions/workflows/main.yml/badge.svg?branch=main)](https://github.com/HYLODE/HyUi/actions/workflows/main.yml)
 
-# `hyui`
+# Hyperlocal Bed Demand Forecasting (User Interface)
 
 User interface for the HYLODE project Much of the project structure is based
 on the[`govcookiecutter` template project][govcookiecutter] but is adapted
@@ -14,178 +14,8 @@ code including the backend is in `./src/api/app1`, and the application itself is
 kept in `./src/apps/app1`. An annotated figure of the directory structure is shown
 below.
 
-## Quick start
+The documentation is available [here](docs/index.md).
 
-### Deployment not development
-
-From the commandline of the GAE
-
-```sh
-git clone https://github.com/HYLODE/HyUi.git
-cd HyUi
-cp .env.example .env
-# Now hand edit the .env file with usernames/passwords
-# Set ENV=prod (rather than dev)
-pytest # OPTIONAL
-docker-compose up -d --build && docker-composes logs -f
-```
-
-Go to [](http://my-host-name:8094/docs) for the API
-Go to [](http://my-host-name:8095) for the dashboard landing page
-
-### Development (local)
-
-### Development (GAE)
-
-
-
-## First run
-
-### Installation
-
-You will need to do this twice:
-
-- on your local development machine
-- on the UCLH generic application environment
-
-Regardless, open the terminal and **git clone**.
-
-```sh
-git clonehttps://github.com/HYLODE/HyUi.git
-cd HyUi
-```
-
-### Development (Local)
-
-We assume that the majority of the development work will be on your own machine. You will therefore need to set-up your own dev environment. I have been using [conda miniforge](https://github.com/conda-forge/miniforge) because this has the best compatibility with the ARM processor on the Mac. I think it should also work on other machines but I have not tested this.
-
-From within the HyUi directory
-
-```sh
-conda env create --file=./dev/steve/environment-short.yml
-conda activate hyuiv4
-```
-
-Then confirm your set-up is OK
-
-```sh
-pytest src/tests/smoke
-```
-
-#### Local development without docker
-
-Backend (all routes)
-
-```sh
-cd ./src
-uvicorn api.main:app --reload --workers 1 --host 0.0.0.0 --port 8094
-```
-
-then navigate to [http://localhost:8094/docs]() to view the API documentation
-
-... `app/main.py` hosts the various routes for the different apps
-
-
-Frontend (per app)
-
-```sh
-cd ./src
-python apps/app1/index.py
-```
-
-
-#### Local development with docker
-
-
-### Development (Hospital)
-
-There are two tasks that _must_ happen within the hospital environment: (1) preparing realistic mock data (2) deployment. The installation steps differ because here we do not have **sudo** (root) access (admin privileges). This means work must be performed using a combination of the default command line tools and docker.
-
-### Preparing mock (synthetic) data
-
-We will use the tooling provided by the [Synthetic Data Lab](https://sdv.dev) from a JupyterLab in a browser on a hospital machine. You will need access to the GAE to run this.
-
-#### Scenario 1 (data via SQL)
-
-Ensure that your database credentials are stored in the `./.secrets` file.
-From the GAE commandline, navigate to the `synth` directory (`cd ./synth`), then use the Makefile commands as follows
-
-1. `make mock1build` to build a docker image with JupyterLab and sdv pre-installed.
-2. `make mock2run` to spin up JupyterLab (e.g. Working from GAE07 this will be at [](http://UCLVLDDPRAGAE07:8091) but the URL server will depend on the GAE).
-3. `make mock3copyin` This will copy the example notebook `synth_test_data.ipynb` into the `./synth/portal` directory that is attached to the JupyterNotebook. Now open the example notebook `synth_test_data.ipynb` using JupyterLab and work through the steps. Create your SQL query and save as `query_live.sql` file must return a *SELECT* statement. Save just the synthesised mock data to `mock.h5`, and the query (`query_live.sql`). Be **careful** and ensure that you specify 'fakers' for all direct identifiers. We recommend the four eyes approach wherein a second person reviews your work before an export.
-4. `make mock4copyout` to copy just the query and the synthetic data. Do not copy the notebook out of the `portal` directory unless you are sure you have stripped all personally identifiable data (i.e. clear all cells before saving).
-5. `make mock5stop` to stop the JupyterLab instance and clear up
-
-#### Scenario 2 (data via an http `get` request)
-
-This is similar to the steps above but does not depend on the query or database credentials. You are likely to need to use the Python requests library to get the data that will be used by [sdv](https://sdv.dev).
-
-**YOU MUST NOW PREPARE YOUR DATA MODEL IN `./src/api/models.py`**
-This is a quality control step that ensures the data you handle is of the correct type.
-let's generalise the naming so that query is matched to results which has rows
-and results is a pydantic / sqlmodel class
-by hand, specify as per ... https://sqlmodel.tiangolo.com/tutorial/create-db-and-table/
-
-
-A simple Pandas dataframe with two string columns and a timestamp.
-
-```python
->>> df.types
-firstname                          object
-lastname                           object
-admission_datetime                 datetime64[ns]
-```
-
-The equivalent SQLModel. Note that `firstname` is optional but that `lastname` and `admission_datetime` are not.
-
-```python
-from sqlmodel import SQLModel
-from datetime import datetime
-
-class ResultsBase(SQLModel):
-    """
-    Generic results class to hold data returned from
-    the SQL query or the API
-    """
-    firstname: Optional[str]
-    lastname: str
-    admission_datetime: datetime
-
-```
-
-You can also use the [`@validator`](https://pydantic-docs.helpmanual.io/usage/validators/) decorator function to add additional validation.
-### Deployment
-
-Set the environment variable to *prod*, then run *docker-compose*.
-
-```sh
-export ENV=prod
-pytest
-docker-compose up -d --build && docker-compose logs -f
-```
-
-You will need create a local `./.secrets` file with database credentials so preferably set the *ENV* to `prod` there.
-
-
-
-
-
-## Frontend vs Backend
-
-### Backend
-This is a Python FastApi server that is exposed on port 8094 when running `docker-compose up -d` from the project root, or `uvicorn main:app` when running from `src/api/` locally.
-
-### Frontend
-This is a Plotly Dash app served on port 8095.
-
-### Orchestrating front and back end
-
-**IMPORTANT**: ensure you have created a ./.secrets file with at least the same info as the ./.secrets.example version
-
-```bash
-docker-compose down
-docker-compose up -d --build && docker-compose logs -f
-```
 
 
 ## Project structure

docker-compose.yml

@@ -87,12 +87,13 @@ services:
       dockerfile: Dockerfile.playwright
       args:
         <<: *build-args-common
-    command: ./wait-for http://apps:8095 -- pytest --noconftest -m smoke hyui/src/tests/e2e
+    command: ./wait-for http://apps:8095 -- pytest -m e2e src/tests/playwright
     volumes:
-      - ./:/hyui
+      - ./src:/src
     environment:
       <<: *proxy-common
       <<: *hyui-common-env
+      Docker: "True"
     env_file:
       - .env
     # depends on does not 'wait'

docs/conf.py

@@ -47,7 +47,7 @@
 
 # General information about the project.
 project = "HyUI"
-author = "HYLODE"
+author = "The HYLODE Team"
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the built