|
| 1 | +# FlowerPower Web Application |
| 2 | + |
| 3 | +This is a Sanic web application for FlowerPower. |
| 4 | + |
| 5 | +## Running the application |
| 6 | + |
| 7 | +1. Install the necessary dependencies: |
| 8 | + ```bash |
| 9 | + pip install flowerpower[webserver] |
| 10 | + ``` |
| 11 | +2. **Project Discovery:** |
| 12 | + - The web application looks for FlowerPower projects in a directory named `fp_projects` located at the root of this repository. |
| 13 | + - Each subdirectory within `fp_projects` is considered a potential project if it contains a `conf/project.yml` file. |
| 14 | + - If the `fp_projects` directory does not exist at the repository root, the application will fall back to searching for projects in the current working directory (`.` when you run `python flowerpower_webapp/main.py`). This allows you to run the web app from within a single project's directory. |
| 15 | + - To get started, you can create the `fp_projects` directory in the repository root: |
| 16 | + ```bash |
| 17 | + mkdir fp_projects |
| 18 | + ``` |
| 19 | + - Then, place your FlowerPower project folders inside `fp_projects`. For example: |
| 20 | + ``` |
| 21 | + fp_projects/ |
| 22 | + ├── my_project_alpha/ |
| 23 | + │ ├── conf/ |
| 24 | + │ │ └── project.yml |
| 25 | + │ └── ... (other project files) |
| 26 | + └── my_project_beta/ |
| 27 | + ├── conf/ |
| 28 | + │ └── project.yml |
| 29 | + └── ... (other project files) |
| 30 | + ``` |
| 31 | + - If no projects are found in `fp_projects` (when it's the designated root), the application will automatically create a couple of sample projects (`sample_project_1`, `sample_project_2`) for demonstration purposes when you first run it. |
| 32 | +
|
| 33 | +3. Run the application: |
| 34 | + ```bash |
| 35 | + python flowerpower_webapp/main.py |
| 36 | + ``` |
| 37 | +
|
| 38 | +The application will be available at `http://localhost:8000`. You can navigate to `/projects` to see the list of discovered projects. |
| 39 | +
|
| 40 | +### Creating a New Project |
| 41 | +
|
| 42 | +You can create a new FlowerPower project directly from the web interface: |
| 43 | +
|
| 44 | +1. Navigate to the main page or the project listing page (`/projects`). |
| 45 | +2. Click on the "Create New Project" link/button. This will take you to the `/projects/new` page. |
| 46 | +3. Enter a unique name for your new project in the input field. The project name should consist of letters, numbers, underscores, hyphens, or dots. |
| 47 | +4. Click the "Create Project" button. |
| 48 | +5. If successful, the application will create: |
| 49 | + * A new directory for your project under the `fp_projects` directory (e.g., `fp_projects/your_project_name/`). |
| 50 | + * Inside the project directory: |
| 51 | + * `conf/project.yml` (a default project configuration file). |
| 52 | + * `conf/pipelines/` (a directory for your pipeline configuration files). |
| 53 | + * `pipelines/` (a directory for your pipeline code modules). |
| 54 | +6. You will see a success message, and the newly created project will appear on the `/projects` listing page. |
| 55 | +7. If there's an error (e.g., project name already exists, invalid characters), an error message will be displayed. |
| 56 | + |
| 57 | +### Viewing and Editing Project Configuration |
| 58 | + |
| 59 | +Once a project is created or discovered, you can view and edit its `project.yml` configuration: |
| 60 | + |
| 61 | +1. Navigate to the project listing page (`/projects`). |
| 62 | +2. Click on the name of the project you wish to view or edit. This will take you to the project's detail page (e.g., `/projects/your_project_name`). |
| 63 | +3. The content of the project's `conf/project.yml` file will be displayed in a text area. |
| 64 | +4. **Editing:** |
| 65 | + * You can directly modify the YAML content in the text area. |
| 66 | + * After making your changes, click the "Save Configuration" button. |
| 67 | +5. **Validation:** |
| 68 | + * The application will first check if the entered text is valid YAML. |
| 69 | + * Then, it will validate if the YAML content conforms to the `ProjectConfig` schema (e.g., checking for required fields, correct data types). |
| 70 | +6. **Outcome:** |
| 71 | + * If the configuration is valid and saved successfully, a success message will be displayed, and the `project.yml` file on the server will be updated. |
| 72 | + * If there are errors (e.g., invalid YAML format, schema validation failure, file system issues), an error message will be displayed, and the file will not be saved. |
| 73 | +7. **File Not Found:** |
| 74 | + * If a project directory exists but its `conf/project.yml` file is missing, the page will display an error message. |
| 75 | + * A default `project.yml` template for that project (based on its name) will be loaded into the text area. You can then modify and save this template to create the `project.yml` file. |
| 76 | + |
| 77 | +### Listing Project Pipelines |
| 78 | + |
| 79 | +You can view a list of all available pipelines within a specific FlowerPower project: |
| 80 | + |
| 81 | +1. Navigate to the project's detail page (where you view/edit `project.yml`). |
| 82 | +2. Click on the "View Pipelines" link. This will take you to the pipeline listing page for that project (e.g., `/projects/your_project_name/pipelines`). |
| 83 | +3. The page will display a list of all pipelines discovered in the project. |
| 84 | + * FlowerPower discovers pipelines by looking for Python files (e.g., `my_pipeline.py`) in the project's `pipelines/` directory that have a corresponding configuration file (e.g., `my_pipeline.yml`) in the `conf/pipelines/` directory. |
| 85 | +4. Each listed pipeline name will be a link to a (currently placeholder) page for that specific pipeline's details. |
| 86 | +5. If no pipelines are found in the project, a "No pipelines found for this project." message will be displayed. |
| 87 | +6. If there's an error (e.g., the project's `conf/project.yml` is missing or malformed, which might be needed by the `PipelineManager`), an error message will be shown. |
| 88 | +7. From this page, you can also navigate to: |
| 89 | + * "Add New Pipeline". |
| 90 | + * Back to the project's configuration page. |
| 91 | + * Back to the list of all projects. |
| 92 | + * Back to the home page. |
| 93 | + |
| 94 | +### Adding a New Pipeline to a Project |
| 95 | + |
| 96 | +You can create the basic file structure for a new pipeline within a project: |
| 97 | + |
| 98 | +1. Navigate to the pipeline listing page for the desired project (e.g., `/projects/your_project_name/pipelines`). |
| 99 | +2. Click on the "Add New Pipeline" button/link. This will take you to the form for adding a new pipeline (e.g., `/projects/your_project_name/pipelines/new`). |
| 100 | +3. **Enter Pipeline Name:** |
| 101 | + * In the "Pipeline Name" input field, enter a name for your new pipeline. |
| 102 | + * The name must be a valid Python identifier (e.g., `my_new_data_pipeline`). This generally means it should consist of letters, numbers, and underscores, and cannot start with a number. |
| 103 | +4. Click the "Create Pipeline" button. |
| 104 | +5. **Outcome:** |
| 105 | + * **Success:** If the name is valid and the pipeline doesn't already exist, the application will: |
| 106 | + * Create a new Python file: `fp_projects/your_project_name/pipelines/your_pipeline_name.py`. |
| 107 | + * Create a new YAML configuration file: `fp_projects/your_project_name/conf/pipelines/your_pipeline_name.yml`. |
| 108 | + * A success message will be displayed, and the input field for the pipeline name will be cleared. |
| 109 | + * The newly created pipeline will then appear on the pipeline list page for that project. |
| 110 | + * **Error:** |
| 111 | + * If the pipeline name is invalid (e.g., contains spaces, starts with a number), an error message will be displayed. |
| 112 | + * If a pipeline with that name already exists in the project, an error message will indicate this. |
| 113 | + * If there are other issues (e.g., problems with the `PipelineManager` or file system permissions), a general error message will be shown. |
| 114 | +6. After creating a pipeline, you will need to manually edit its `.py` and `.yml` files to define its functionality and configuration. The web application currently only creates the boilerplate files. |
| 115 | +
|
| 116 | +### Viewing and Editing Pipeline Configuration |
| 117 | +
|
| 118 | +For each pipeline within a project, you can view and edit its specific YAML configuration file (e.g., `your_pipeline_name.yml`): |
| 119 | +
|
| 120 | +1. Navigate to the pipeline listing page for the desired project (e.g., `/projects/your_project_name/pipelines`). |
| 121 | +2. Click on the name of the pipeline whose configuration you wish to view or edit. This will take you to that pipeline's detail page (e.g., `/projects/your_project_name/pipelines/your_pipeline_name`). |
| 122 | +3. The content of the pipeline's `.yml` file (located in `fp_projects/your_project_name/conf/pipelines/`) will be displayed in a text area. |
| 123 | +4. **Editing:** |
| 124 | + * You can directly modify the YAML content in the text area. |
| 125 | + * After making your changes, click the "Save Pipeline Configuration" button. |
| 126 | +5. **Validation:** |
| 127 | + * The application will first check if the entered text is valid YAML. |
| 128 | + * Then, it will validate if the YAML content conforms to the `PipelineConfig` schema (e.g., checking for expected sections like `params`, `run`, etc.). |
| 129 | +6. **Outcome:** |
| 130 | + * If the configuration is valid and saved successfully, a success message will be displayed, and the pipeline's `.yml` file on the server will be updated. |
| 131 | + * If there are errors (e.g., invalid YAML format, schema validation failure, file system issues), an error message will be displayed, and the file will not be saved. |
| 132 | +7. **File Not Found:** |
| 133 | + * If a pipeline's `.yml` configuration file is missing (e.g., if only the `.py` file exists or after a new pipeline is created via the UI but before its config is saved for the first time): |
| 134 | + * The page will display an error message indicating the file was not found. |
| 135 | + * A default `PipelineConfig` template for that pipeline (based on its name) will be loaded into the text area. |
| 136 | + * You can then modify and save this template to create the pipeline's `.yml` configuration file. |
| 137 | + |
| 138 | +### Visualizing Pipeline DAG (Directed Acyclic Graph) |
| 139 | + |
| 140 | +You can view a visual representation of a pipeline's structure (its DAG): |
| 141 | +
|
| 142 | +1. Navigate to the pipeline's detail/configuration page (e.g., `/projects/your_project_name/pipelines/your_pipeline_name`). |
| 143 | +2. Click on the "View DAG" link. This will take you to the DAG visualization page for that pipeline (e.g., `/projects/your_project_name/pipelines/your_pipeline_name/dag`). |
| 144 | +3. **DAG Display:** |
| 145 | + * If successful and `graphviz` is installed on the server, an SVG image of the DAG will be displayed. |
| 146 | + * If `graphviz` is not installed, or if there are errors in the pipeline's Python code or configuration that prevent DAG generation, an error message will be displayed. The page might show a placeholder or a textual representation if SVG generation fails. |
| 147 | +4. **Dependencies:** |
| 148 | + * DAG visualization as an SVG image requires **Graphviz** to be installed on the server where the FlowerPower web application is running. If Graphviz is not found, the application will attempt to gracefully handle this by showing an error message. |
| 149 | +5. **Troubleshooting:** |
| 150 | + * If you see an error message instead of a DAG, check the following: |
| 151 | + * Ensure Graphviz is installed and in the system's PATH. |
| 152 | + * Verify that the pipeline's Python module (`.py` file) and its YAML configuration (`.yml` file) are correctly defined and free of syntax errors. |
| 153 | + * Check the application logs for more detailed error messages. |
| 154 | +6. From the DAG page, you can navigate back to the pipeline's configuration, the pipeline list for the project, or the home page. |
0 commit comments