Merge pull request #32 from Sage-Bionetworks-Workflows/bwmac/orca-293/airflow-constraints

CI testing

Author: BWMac

.gitignore

@@ -60,3 +60,7 @@ MANIFEST
 
 # Metaflow files
 .metaflow
+
+# Virtual Environment Directories
+/venv
+/py-orca-venv-*

CONTRIBUTING.md

@@ -1,6 +1,6 @@
 # Contributing
 
-Welcome to `orca` contributor's guide.
+Welcome to `py-orca` contributor's guide.
 
 This document focuses on getting any potential contributor familiarized with
 the development processes, but [other kinds of contributions] are also appreciated.
@@ -16,7 +16,7 @@ behavior guidelines.
 
 ## Issue Reports
 
-If you experience bugs or general issues with `orca`, please have a look
+If you experience bugs or general issues with `py-orca`, please have a look
 on the [issue tracker].
 If you don't see anything useful there, please feel free to fire an issue report.
 
@@ -34,17 +34,17 @@ you help us to identify the root cause of the issue.
 
 ## Documentation Improvements
 
-You can help improve `orca` docs by making them more readable and coherent, or
+You can help improve py-`orca` docs by making them more readable and coherent, or
 by adding missing information and correcting mistakes.
 
-`orca` documentation uses [Sphinx] as its main documentation compiler.
+`py-orca` documentation uses [Sphinx] as its main documentation compiler.
 This means that the docs are kept in the same repository as the project code, and
 that any documentation update is done in the same way was a code contribution.
 The documentation is written using [CommonMark] with [MyST] extensions.
 
 :::{tip}
    Please notice that the [GitHub web interface] provides a quick way of
-   propose changes in `orca`'s files. While this mechanism can
+   propose changes in `py-orca`'s files. While this mechanism can
    be tricky for normal code contributions, it works perfectly fine for
    contributing to the docs, and can be quite handy.
 
@@ -87,7 +87,7 @@ Before you work on any non-trivial code contribution it's best to first create
 a report in the [issue tracker] to start a discussion on the subject.
 This often provides additional considerations and avoids unnecessary work.
 
-### Clone the repository
+### Development Environment Setup
 
 1. Create an user account on GitHub if you do not already have one.
 
@@ -98,22 +98,25 @@ This often provides additional considerations and avoids unnecessary work.
 
    ```console
    git clone git@github.com:Sage-Bionetworks-Workflows/py-orca.git
-   cd orca
+   cd py-orca
    ```
 
-4. Install `pipx` to easily run Python CLI tools like `tox` and `pipenv`.
-
 4. Create an isolated virtual environment containing package dependencies,
    including those needed for development (*e.g.* testing, documentation) by running:
 
    ```console
-   pipx run tox -e pipenv
+   ./dev_setup.sh <your_supported_python_version>
+   source py-orca-venv-<your_supported_python_version>/bin/activate
    ```
 
+   **Note:**
+   Ensure that you have a supported Python version installed on your local machine before running `dev_setup.sh`. `py-orca` currently supports Python version 3.10 and 3.11. You can [install](https://github.com/pyenv/pyenv?tab=readme-ov-file#installation) `pyenv`for easy Python version management.
+
+
 5. Install [pre-commit] hooks:
 
    ```
-   pipenv run pre-commit install
+   pre-commit install
    ```
 
    `orca` comes with a lot of hooks configured to automatically help the

README.md

@@ -24,21 +24,6 @@ Refer to [`.env.example`](.env.example) for the format of their values as well a
 
 Once your environment is set, you can create a virtual environment, install the Python dependencies, and run the demonstration script (after downloading it) as follows. Note that you will need to update the `s3_prefix` parameter so that it points to an S3 bucket that is accessible to your Tower workspace.
 
-### Using pipenv
-```bash
-# Create and activate a Python virtual environment (tested with Python 3.11)
-pipenv shell
-
-# Install Python dependencies
-pipenv install
-
-# If development dependencies are required to be installed
-pipenv install -d
-
-# Run the script using an example dataset
-python3 demo.py run --dataset_id 'syn51514585' --s3_prefix 's3://orca-service-test-project-tower-bucket/outputs'
-```
-
 ### Manually creating a virtual environment
 ```bash
 # Create and activate a Python virtual environment (tested with Python 3.10)

dev_setup.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+
+# Usage: ./dev_setup.sh <python_version>
+
+# Check if Python version argument is provided
+if [ -z "$1" ]; then
+  echo "Please provide the Python version as an argument."
+  exit 1
+fi
+# Check if Python version is supported
+if [ "$1" != "3.10" ] && [ "$1" != "3.11" ]; then
+  echo "Unsupported Python version. Please use 3.10 or 3.11."
+  exit 1
+fi
+# Set up and activate a Python 3.11 virtual environment
+python$1 -m venv py-orca-venv-$1
+source py-orca-venv-$1/bin/activate
+# Upgrade pip to latest version
+pip install --upgrade pip
+# Install airflow with constraints
+pip install -r requirements-airflow.txt
+# Install py-orca
+pip install -e '.[all,testing,dev]'

requirements-airflow.txt

@@ -0,0 +1,2 @@
+-c https://raw.githubusercontent.com/apache/airflow/constraints-2.7.2/constraints-3.10.txt
+apache-airflow==2.7.2

tox.ini

@@ -19,6 +19,8 @@ passenv =
     HOME
     *_CONNECTION_URI
     SETUPTOOLS_*
+deps =
+    -r {toxinidir}/requirements-airflow.txt
 extras =
     testing
     all
@@ -92,16 +94,3 @@ deps = twine
 commands =
     python -m twine check dist/*
     python -m twine upload {posargs:--repository {env:TWINE_REPOSITORY:testpypi}} dist/*
-
-
-[testenv:pipenv]
-description =
-    Refresh the Pipfile.lock file based on the latest dependencies in setup.cfg
-skip_install = True
-changedir = {toxinidir}
-setenv =
-    PIPENV_IGNORE_VIRTUALENVS = 1
-deps = pipenv
-commands =
-    pipenv lock --dev
-    pipenv install --dev