diff --git a/.github/ISSUE_TEMPLATE/CONTRIBUTING.md b/.github/CONTRIBUTING.md similarity index 100% rename from .github/ISSUE_TEMPLATE/CONTRIBUTING.md rename to .github/CONTRIBUTING.md diff --git a/.github/workflows/development.yaml b/.github/workflows/development.yaml index af38318..f8c587e 100644 --- a/.github/workflows/development.yaml +++ b/.github/workflows/development.yaml @@ -193,6 +193,15 @@ jobs: asset_path: ${{env.PHARUS_SDIST_PATH}} asset_name: pip-pharus-${{env.PHARUS_VERSION}}.tar.gz asset_content_type: application/gzip + - name: Upload deploy docker environment + uses: actions/upload-release-asset@v1 + env: + GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} + with: + upload_url: ${{steps.create_gh_release.outputs.upload_url}} + asset_path: docker-compose-deploy.yaml + asset_name: docker-compose-deploy.yaml + asset_content_type: application/yaml publish-images: if: github.event_name == 'push' needs: publish-release @@ -287,7 +296,7 @@ jobs: GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} with: upload_url: ${{needs.publish-release.outputs.release_upload_url}} - asset_path: docs-static-pharus-${PHARUS_VERSION}.zip - asset_name: docs-static-pharus-${PHARUS_VERSION}.zip + asset_path: "docs-static-pharus-${{env.PHARUS_VERSION}}.zip" + asset_name: "docs-static-pharus-${{env.PHARUS_VERSION}}.zip" asset_content_type: application/zip # fail_on_unmatched_files: true diff --git a/CHANGELOG.md b/CHANGELOG.md index 04a2ee2..e81c395 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,19 +2,19 @@ Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) convention. -## [0.1.0b1] - 2021-03-11 +## [0.1.0b2] - 2021-03-12 ### Fixed - Fixed behavior where using list_table with a nonexistent schema_name creates it instead of returning an error message (#65) PR #63 ### Changed -- Contribution policy to follow directly the general DataJoint Contribution Guideline. (#91) PR #94 +- Contribution policy to follow directly the general DataJoint Contribution Guideline. (#91) PR #94, #95 ### Added -- Issue templates for bug reports and enhancement requests. PR #94 -- Docker environment for documentation build. (#92) PR #94 -- Add Sphinx-based documentation source and fix parsing issues. (#92) PR #94 -- GitHub Actions automation that publishes on release new docs to release and GitHub Pages. (#92) PR #94 +- Issue templates for bug reports and enhancement requests. PR #94, #95 +- Docker environment for documentation build. (#92) PR #94, #95 +- Add Sphinx-based documentation source and fix parsing issues. (#92) PR #94, #95 +- GitHub Actions automation that publishes on release new docs to release and GitHub Pages. (#92) PR #94, #95 ## [0.1.0b0] - 2021-02-26 @@ -50,6 +50,6 @@ Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and - Support for DataJoint attribute types: `varchar`, `int`, `float`, `datetime`, `date`, `time`, `decimal`, `uuid`. - Check dependency utility to determine child table references. -[0.1.0b1]: https://github.com/datajoint/pharus/compare/0.1.0b0...0.1.0b1 +[0.1.0b2]: https://github.com/datajoint/pharus/compare/0.1.0b0...0.1.0b2 [0.1.0b0]: https://github.com/datajoint/pharus/compare/0.1.0a5...0.1.0b0 [0.1.0a5]: https://github.com/datajoint/pharus/releases/tag/0.1.0a5 \ No newline at end of file diff --git a/README.rst b/README.rst index 9c1df67..ce8548b 100644 --- a/README.rst +++ b/README.rst @@ -5,100 +5,49 @@ User Documentation The Pharus project is still early in its life and the maintainers are currently actively developing with a priority of addressing first critical issues directly related to the deliveries of `Alpha `_ and `Beta `_ milestones. Please be advised that while working through our milestones, we may restructure/refactor the codebase without warning until we issue our `Official Release `_ currently planned as ``0.1.0`` on ``2021-03-31``. -``pharus`` is a generic REST API server backend for DataJoint pipelines built on top of ``flask``, ``datajoint``, and ``pyjwt``. +``pharus`` is a generic REST API server backend for `DataJoint `_ pipelines built on top of ``flask``, ``datajoint``, and ``pyjwt``. - `Documentation `_ -- `Package `_ +- `PyPi Package `_ +- `Docker Image `_ +- `Release `_ - `Source `_ -Requirements for Preferred Setup --------------------------------- +Installation +------------ + +If you have not done so already, please install the following dependencies. - `Docker `_ - `Docker Compose `_ -Run Locally w/ Docker ---------------------- - -- Copy a ``*-docker-compose.yaml`` file corresponding to your usage to ``docker-compose.yaml``. This file is untracked so feel free to modify as necessary. Idea is to commit anything generic but system/setup dependent should go on 'your' version i.e. local UID/GID, etc. -- Check the first comment which will provide the best instruction on how to start the service; yes, it is a bit long. Note: Any of the keyword arguments prepended to the ``docker-compose`` command can be safely moved into a dedicated ``.env`` and read automatically if they are not evaluated i.e. ``$(...)``. Below is a brief description of the non-evaluated environment variables: - - .. code-block:: bash - - PY_VER=3.8 # Python version: 3.6|3.7|3.8 - IMAGE=djtest # Image type: djbase|djtest|djlab|djlabhub - DISTRO=alpine # Distribution: alpine|debian - AS_SCRIPT= # If 'TRUE', will not keep container alive but run tests and exit - -.. note:: - - Deployment options currently being considered are `Docker Compose `_ and `Kubernetes `_. - -Run Locally w/ Python ---------------------- - -- Set environment variables for port assignment (``PHARUS_PORT``, defaults to 5000) and API route prefix (``PHARUS_PREFIX`` e.g. ``/api``, defaults to empty string). -- For development, use CLI command ``pharus``. This method supports hot-reloading so probably best coupled with ``pip install -e ...``. -- For production, use ``gunicorn --bind 0.0.0.0:${PHARUS_PORT} pharus.server:app``. - -Run Tests for Development w/ Pytest and Flake8 ----------------------------------------------- - -- Set ``pharus`` testing environment variables: - - .. code-block:: bash - - PKG_DIR=/opt/conda/lib/python3.8/site-packages/pharus # path to pharus installation - TEST_DB_SERVER=example.com:3306 # testing db server address - TEST_DB_USER=root # testing db server user (needs DDL privilege) - TEST_DB_PASS=unsecure # testing db server password - -- For syntax tests, run ``flake8 ${PKG_DIR} --count --select=E9,F63,F7,F82 --show-source --statistics`` -- For pytest integration tests, run ``pytest -sv --cov-report term-missing --cov=${PKG_DIR} /main/tests`` -- For style tests, run ``flake8 ${PKG_DIR} --count --max-complexity=20 --max-line-length=95 --statistics`` +Prerequisites +------------- -Creating Sphinx Documentation from Scratch ------------------------------------------- +Download the ``docker-compose-deploy.yaml`` docker environment from the source located `here `_. -Recommend the follow to be ran within the ``pharus`` container in ``docs`` Docker Compose environment. +Running the API server +---------------------- -- Run the following commands and complete the prompts as requested. +To start the API server, use the command: .. code-block:: bash - mkdir docs - cd docs - sphinx-quickstart - -- In ``docs/conf.py`` add to the beginning: - - .. code-block:: python + PHARUS_VERSION=0.1.0b2 docker-compose -f docker-compose-deploy.yaml up -d - import os - import sys - sys.path.insert(0, os.path.abspath('..')) - -- In ``docs/conf.py:extensions`` append ``['sphinx.ext.autodoc', 'sphinxcontrib.httpdomain']``. See ``requirements_docs.txt`` and ``docker-compose-docs.yaml`` for details on documentation dependencies. -- Run the following to automatically generate the API docs: +To stop the API server, use the command: .. code-block:: bash - sphinx-apidoc -o . .. ../tests/* ../setup.py - -- Add ``modules`` within the ``toctree`` directive in ``index.rst``. -- Build the docs by running: - - .. code-block:: bash - - make html + PHARUS_VERSION=0.1.0b2 docker-compose -f docker-compose-deploy.yaml down References ---------- +- DataJoint + + - https://datajoint.io + - DataJoint LabBook (a companion frontend) - https://github.com/datajoint/datajoint-labbook - -- Python Tutorial for Flask, Swagger, and Automated docs - - - https://realpython.com/flask-connexion-rest-api/#reader-comments diff --git a/docker-compose-deploy.yaml b/docker-compose-deploy.yaml index 9ebe958..1e740e9 100644 --- a/docker-compose-deploy.yaml +++ b/docker-compose-deploy.yaml @@ -1,5 +1,5 @@ -# PHARUS_VERSION=0.1.0b1 docker-compose -f docker-compose-deploy.yaml pull -# PHARUS_VERSION=0.1.0b1 docker-compose -f docker-compose-deploy.yaml up -d +# PHARUS_VERSION=0.1.0b2 docker-compose -f docker-compose-deploy.yaml pull +# PHARUS_VERSION=0.1.0b2 docker-compose -f docker-compose-deploy.yaml up -d # # Intended for production deployment. # Note: You must run both commands above for minimal outage diff --git a/docs/dev_notes.rst b/docs/dev_notes.rst index c229146..53e345a 100644 --- a/docs/dev_notes.rst +++ b/docs/dev_notes.rst @@ -1,4 +1,98 @@ General Notes ============= -Dev docs TBD. +Requirements for Preferred Setup +-------------------------------- + +- `Docker `_ +- `Docker Compose `_ + +Run Locally w/ Docker +--------------------- + +- Copy the docker environment (i.e. ``*-docker-compose.yaml``) of your choice corresponding to your usage to ``docker-compose.yaml``. This file is untracked so feel free to modify as necessary. Idea is to commit anything generic but system/setup dependent should go on 'your' version i.e. local UID/GID, etc. +- Check the first header comment which will provide the best instruction on how to start the service; yes, it is a bit long. Note: Any of the keyword arguments prepended to the ``docker-compose`` command can be safely moved into a dedicated ``.env`` and read automatically if they are not evaluated i.e. ``$(...)``. Below is a brief description of the non-evaluated environment variables: + + .. code-block:: bash + + PY_VER=3.8 # Python version: 3.6|3.7|3.8 + IMAGE=djtest # Image type: djbase|djtest|djlab|djlabhub + DISTRO=alpine # Distribution: alpine|debian + AS_SCRIPT= # If 'TRUE', will not keep container alive but run tests and exit + +.. note:: + + Deployment options currently being considered are `Docker Compose `_ and `Kubernetes `_. + +Run Locally w/ Python +--------------------- + +- Set environment variables for port assignment (``PHARUS_PORT``, defaults to 5000) and API route prefix (``PHARUS_PREFIX`` e.g. ``/api``, defaults to empty string). +- For development, use CLI command ``pharus``. This method supports hot-reloading so probably best coupled with ``pip install -e ...``. +- For production, use ``gunicorn --bind 0.0.0.0:${PHARUS_PORT} pharus.server:app``. + +Run Tests for Development w/ Pytest and Flake8 +---------------------------------------------- + +- Set ``pharus`` testing environment variables: + + .. code-block:: bash + + PKG_DIR=/opt/conda/lib/python3.8/site-packages/pharus # path to pharus installation + TEST_DB_SERVER=example.com:3306 # testing db server address + TEST_DB_USER=root # testing db server user (needs DDL privilege) + TEST_DB_PASS=unsecure # testing db server password + +- For syntax tests, run ``flake8 ${PKG_DIR} --count --select=E9,F63,F7,F82 --show-source --statistics`` +- For pytest integration tests, run ``pytest -sv --cov-report term-missing --cov=${PKG_DIR} /main/tests`` +- For style tests, run ``flake8 ${PKG_DIR} --count --max-complexity=20 --max-line-length=95 --statistics`` + +Creating Sphinx Documentation from Scratch +------------------------------------------ + +Recommend the follow to be ran within the ``pharus`` container in ``docs`` Docker Compose environment. + +- Run the following commands and complete the prompts as requested. + + .. code-block:: bash + + mkdir docs + cd docs + sphinx-quickstart + +- In ``docs/conf.py`` add to the beginning: + + .. code-block:: python + + import os + import sys + sys.path.insert(0, os.path.abspath('..')) + +- In ``docs/conf.py:extensions`` append ``['sphinx.ext.autodoc', 'sphinxcontrib.httpdomain']``. See ``requirements_docs.txt`` and ``docker-compose-docs.yaml`` for details on documentation dependencies. +- Run the following to automatically generate the API docs: + + .. code-block:: bash + + sphinx-apidoc -o . .. ../tests/* ../setup.py + +- Add ``modules`` within the ``toctree`` directive in ``index.rst``. +- Build the docs by running: + + .. code-block:: bash + + make html + +References +---------- + +- DataJoint + + - https://datajoint.io + +- DataJoint LabBook (a companion frontend) + + - https://github.com/datajoint/datajoint-labbook + +- Python Tutorial for Flask, Swagger, and Automated docs + + - https://realpython.com/flask-connexion-rest-api/#reader-comments diff --git a/pharus/version.py b/pharus/version.py index c36d94e..a89786e 100644 --- a/pharus/version.py +++ b/pharus/version.py @@ -1,2 +1,2 @@ """Package metadata.""" -__version__ = '0.1.0b1' +__version__ = '0.1.0b2'