Merge pull request #1 from rabea-al/rabea/lock_requirements

Pytorch Component library maintenance

Author: MFA-X-AI

.github/workflows/run-workflow-tests.yml

@@ -5,21 +5,35 @@ on:
     branches: [ main ]
   pull_request:
     branches: "*"
+  workflow_dispatch:
 
 jobs:
   build-and-run:
     runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.9", "3.10", "3.11"]
     env:
       TEST_XIRCUITS: |
         examples/PytorchTrainModel.xircuits
         examples/PytorchPredictFromModel.xircuits
         examples/PytorchTrainCustomNN.xircuits
-
     steps:
     - name: Checkout Repository
       uses: actions/checkout@v3
 
-    - name: Install xircuits
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v4
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Create virtual environment
+      run: |
+        python -m venv venv
+        echo "${{ github.workspace }}/venv/bin" >> $GITHUB_PATH
+
+    - name: Install xircuits in virtual environment
       run: pip install xircuits
 
     - name: Set Environment Variables
@@ -28,20 +42,24 @@ jobs:
         echo "LIBRARY_NAME=$LIBRARY_NAME" >> $GITHUB_ENV
         COMPONENT_LIBRARY_PATH="xai_components/${LIBRARY_NAME}"
         echo "COMPONENT_LIBRARY_PATH=$COMPONENT_LIBRARY_PATH" >> $GITHUB_ENV
+        if [ "${{ github.event_name }}" == "pull_request" ]; then
+          echo "BRANCH_NAME=${{ github.head_ref }}" >> $GITHUB_ENV
+        else
+          echo "BRANCH_NAME=${GITHUB_REF#refs/heads/}" >> $GITHUB_ENV
+        fi
 
     - name: List Xircuits
-      run: xircuits list 
+      run: xircuits list
 
     - name: Clone Repository
       run: |
-        # as of 1.9, xai_pytorch still lives in the base xircuits installation
         rm -rf ${{ env.COMPONENT_LIBRARY_PATH }}
-        if [ "${{ github.ref }}" == "refs/heads/main" ]; then
-          git clone https://github.com/${{ github.repository }} ${{ env.COMPONENT_LIBRARY_PATH }}
+        if [ "${{ github.event_name }}" == "pull_request" ]; then
+          REPO_URL="${{ github.event.pull_request.head.repo.clone_url }}"
         else
-          PR_BRANCH=${{ github.event.pull_request.head.ref }}
-          git clone -b $PR_BRANCH https://github.com/${{ github.repository }} ${{ env.COMPONENT_LIBRARY_PATH }}
+          REPO_URL="https://github.com/${{ github.repository }}"
         fi
+        git clone -b ${{ env.BRANCH_NAME }} $REPO_URL ${{ env.COMPONENT_LIBRARY_PATH }}
 
     - name: Install Component Library
       run: |
@@ -51,12 +69,15 @@ jobs:
         else
           echo "requirements.txt not found."
         fi
-
+          
     - name: Test .xircuits Workflows
       run: |
         export PYTHONPATH="${GITHUB_WORKSPACE}:${PYTHONPATH}"
         LOG_FILE="${GITHUB_WORKSPACE}/workflow_logs.txt"
         TEST_FILES=$(echo "$TEST_XIRCUITS" | tr '\n' ' ')
+        echo "Repository: $LIBRARY_NAME" > $LOG_FILE
+        echo "Branch: $BRANCH_NAME" >> $LOG_FILE
+        echo -e "Testing Files:\n$TEST_FILES" >> $LOG_FILE
         IFS=' ' read -r -a FILE_ARRAY <<< "$TEST_FILES"
         FAIL=0
         if [ ${#FILE_ARRAY[@]} -eq 0 ]; then
@@ -66,7 +87,7 @@ jobs:
             FULL_PATH="${COMPONENT_LIBRARY_PATH}/${file}"
             if [ -f "$FULL_PATH" ]; then
               WORKFLOW_LOG_FILE="${FULL_PATH%.*}_workflow_log.txt"
-              echo "\n\nProcessing $FULL_PATH..." | tee -a $LOG_FILE
+              echo -e "\n\nProcessing $FULL_PATH..." | tee -a $LOG_FILE
               xircuits compile $FULL_PATH "${FULL_PATH%.*}.py" 2>&1 | tee -a $LOG_FILE
               python "${FULL_PATH%.*}.py" 2>&1 | tee -a $WORKFLOW_LOG_FILE
               LAST_LINE=$(tail -n 1 "$WORKFLOW_LOG_FILE")
@@ -92,7 +113,7 @@ jobs:
 
     - name: Upload log file
       if: always()
-      uses: actions/upload-artifact@v3
+      uses: actions/upload-artifact@v4
       with:
-        name: workflow-logs
+        name: ${{ env.LIBRARY_NAME }}-validation-workflow-${{ matrix.python-version }}
         path: ${{ github.workspace }}/workflow_logs.txt

README.md

@@ -1,43 +1,109 @@
-# Xircuits Comprehensive PyTorch Libraries
+
+
+<p align="center">
+  <a href="https://github.com/XpressAI/xircuits/tree/master/xai_components#xircuits-component-library-list">Component Libraries</a> •
+  <a href="https://github.com/XpressAI/xircuits/tree/master/project-templates#xircuits-project-templates-list">Project Templates</a>
+  <br>
+  <a href="https://xircuits.io/">Docs</a> •
+  <a href="https://xircuits.io/docs/Installation">Install</a> •
+  <a href="https://xircuits.io/docs/category/tutorials">Tutorials</a> •
+  <a href="https://xircuits.io/docs/category/developer-guide">Developer Guides</a> •
+  <a href="https://github.com/XpressAI/xircuits/blob/master/CONTRIBUTING.md">Contribute</a> •
+  <a href="https://www.xpress.ai/blog/">Blog</a> •
+  <a href="https://discord.com/invite/vgEg2ZtxCw">Discord</a>
+</p>
+
+
+
+
+
+
+<p align="center"><i>Xircuits Component Library for integrating PyTorch! Build and deploy machine learning models with ease.</i></p>
+
+## Xircuits Comprehensive PyTorch Libraries
 
 These libraries collectively provide extensive functionalities for integrating PyTorch into Xircuits workflows, encompassing both general PyTorch operations and neural network-specific tasks.
 
-### Installation
+## Table of Contents
+
+- [Prerequisites](#prerequisites)
+- [Installation](#installation)
+- [Getting Started with PyTorch XAI Components](#getting-started-with-pytorch-xai-components)
+- [Try The Examples](#try-the-examples)
+- [Components Overview](#components-overview)
+- [Contributing](#contributing)
+
+## Prerequisites
+
+Before using this library, you’ll need the following:
+
+1. **Python 3.8** or higher
+2. **Xircuits**
+3. The following Python packages:
+   - **torch**
+   - **torchvision**
+   - **pillow**
+
+
+## Installation
+
+To use this component library, ensure that you have an existing [Xircuits setup](https://xircuits.io/docs/main/Installation). You can then install the PyTorch library using the [component library interface](https://xircuits.io/docs/component-library/installation#installation-using-the-xircuits-library-interface), or through the CLI using:
+
 
-To use this component library in Xircuits, simply run the Xircuits install command in your working directory.
 ```bash
 xircuits install pytorch
 ```
 
-Alternatively, you may also clone this repository into your `xai_components` directory and install it manually via
+Alternatively, you may clone this repository into your `xai_components` directory and install it manually:
+
+```bash
+# To clone the PyTorch component library into your Xircuits project directory
+git clone https://github.com/XpressAI/xai-pytorch.git xai_components/xai_pytorch
 ```
-pip install -r requirements.txt
+```bash
+# Install required dependencies
+pip install -r xai_components/xai_pytorch/requirements.txt
 ```
 
-## Xircuits PyTorch Component Library
-Core components essential for developing and deploying machine learning models using PyTorch in Xircuits, covering tasks from data preparation to model evaluation.
-
-### Components
-- **LoadTorchVisionDataset**: Loads Torch Vision datasets like MNIST and CIFAR.
-- **TorchDataLoader**: Creates DataLoader instances for efficient batch processing.
-- **ExampleTorchModelConfig**: Sets up an example Torch model with configurable parameters.
-- **TrainTorchModel**: Trains a Torch model with a given DataLoader and configuration.
-- **TestTorchModel**: Tests a trained Torch model for performance evaluation.
-- **SaveTorchModelState**: Saves the state of a trained Torch model.
-- **LoadTorchModelState**: Loads a saved Torch model's state.
-- **TorchModelPredict**: Performs predictions using a trained Torch model.
-- **Image2TorchTensor**: Converts images to Torch tensors for model input.
-- **TorchModelPredictFromTensor**: Predicts using a Torch model and tensor input.
-
-## Xircuits PyTorch Neural Network Component Library
-
-Designed for building and configuring neural network models in Xircuits. It offers flexibility in customizing architectures for various machine learning and deep learning tasks.
-
-### Components
-- **TorchModel**: Builds custom Torch neural network models.
-- **TorchAddLinearLayer**: Adds linear layers to a model.
-- **TorchAddConv1DLayer** and **TorchAddConv2DLayer**: Integrates 1D and 2D convolutional layers.
-- **TorchAddTransformerEncoderLayer** and **TorchAddTransformerDecoderLayer**: Adds Transformer Encoder and Decoder layers.
-- **TorchLSTM**: Incorporates LSTM layers for sequence data.
-- **TorchAddReluLayer**: Adds ReLU activation layers.
-- **TorchAddDropoutLayer**: Includes dropout layers for regularization.
+## Getting Started With PyTorch XAI Components
+
+After successfully installing the necessary libraries and components, you are ready to explore the PyTorch XAI Components Library. This library enables you to develop comprehensive machine learning workflows within Xircuits. Refer to the provided documentation and examples to master the integration, customization, and utilization of deep learning components using PyTorch XAI.
+
+## Try The Examples
+
+We have provided an example workflow to help you get started with the PyTorch XAI Components Library. Give it a try and see how you can create a custom deep learning workflow for your projects.
+
+### PytorchTrainModel
+
+This example demonstrates the initial setup of a machine learning workflow in Xircuits using the PyTorch XAI Components Library. It involves training a custom neural network model on your dataset. This step is crucial as the output model will be used in subsequent examples.
+
+### PytorchPredictFromModel
+
+Following the training process, this example shows how to use the trained model to make predictions. It leverages the model trained by the `PytorchTrainModel` to demonstrate the prediction process on new data, showcasing how to integrate trained models into practical applications.
+
+### PytorchTrainCustomNN
+
+This example extends the capabilities of the PyTorch XAI Components by illustrating how to train a more complex, custom neural network. It utilizes advanced features of PyTorch to tailor a neural network to specific needs beyond standard models.
+
+**Note:** It is important to run the `PytorchTrainModel` example first, as the other examples depend on the output from this initial training step.
+
+
+## Components Overview
+
+The PyTorch XAI Components Library offers an array of components tailored for various stages of the deep learning lifecycle. Explore these components and refer to their documentation to enhance your ability to develop sophisticated deep learning workflows.
+
+## Contributing
+
+We welcome contributions to the **Pytorch XAI Components Library**! If you would like to contribute:
+
+1. Fork the repository.
+2. Create a new branch for your feature or bug fix.
+3. Open a pull request with a detailed description of your changes.
+
+Please feel free to suggest new components, improvements, or optimizations. If you encounter any issues or have ideas for enhancements, you can open an issue in the repository.
+
+---
+
+### License
+
+This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for more information.