Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
File renamed without changes.
13 changes: 11 additions & 2 deletions .github/workflows/development.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,15 @@ jobs:
asset_path: ${{env.PHARUS_SDIST_PATH}}
asset_name: pip-pharus-${{env.PHARUS_VERSION}}.tar.gz
asset_content_type: application/gzip
- name: Upload deploy docker environment
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
with:
upload_url: ${{steps.create_gh_release.outputs.upload_url}}
asset_path: docker-compose-deploy.yaml
asset_name: docker-compose-deploy.yaml
asset_content_type: application/yaml
publish-images:
if: github.event_name == 'push'
needs: publish-release
Expand Down Expand Up @@ -287,7 +296,7 @@ jobs:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
with:
upload_url: ${{needs.publish-release.outputs.release_upload_url}}
asset_path: docs-static-pharus-${PHARUS_VERSION}.zip
asset_name: docs-static-pharus-${PHARUS_VERSION}.zip
asset_path: "docs-static-pharus-${{env.PHARUS_VERSION}}.zip"
asset_name: "docs-static-pharus-${{env.PHARUS_VERSION}}.zip"
asset_content_type: application/zip
# fail_on_unmatched_files: true
14 changes: 7 additions & 7 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,19 @@

Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) convention.

## [0.1.0b1] - 2021-03-11
## [0.1.0b2] - 2021-03-12

### Fixed
- Fixed behavior where using list_table with a nonexistent schema_name creates it instead of returning an error message (#65) PR #63

### Changed
- Contribution policy to follow directly the general DataJoint Contribution Guideline. (#91) PR #94
- Contribution policy to follow directly the general DataJoint Contribution Guideline. (#91) PR #94, #95

### Added
- Issue templates for bug reports and enhancement requests. PR #94
- Docker environment for documentation build. (#92) PR #94
- Add Sphinx-based documentation source and fix parsing issues. (#92) PR #94
- GitHub Actions automation that publishes on release new docs to release and GitHub Pages. (#92) PR #94
- Issue templates for bug reports and enhancement requests. PR #94, #95
- Docker environment for documentation build. (#92) PR #94, #95
- Add Sphinx-based documentation source and fix parsing issues. (#92) PR #94, #95
- GitHub Actions automation that publishes on release new docs to release and GitHub Pages. (#92) PR #94, #95

## [0.1.0b0] - 2021-02-26

Expand Down Expand Up @@ -50,6 +50,6 @@ Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and
- Support for DataJoint attribute types: `varchar`, `int`, `float`, `datetime`, `date`, `time`, `decimal`, `uuid`.
- Check dependency utility to determine child table references.

[0.1.0b1]: https://github.com/datajoint/pharus/compare/0.1.0b0...0.1.0b1
[0.1.0b2]: https://github.com/datajoint/pharus/compare/0.1.0b0...0.1.0b2
[0.1.0b0]: https://github.com/datajoint/pharus/compare/0.1.0a5...0.1.0b0
[0.1.0a5]: https://github.com/datajoint/pharus/releases/tag/0.1.0a5
93 changes: 21 additions & 72 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,100 +5,49 @@ User Documentation

The Pharus project is still early in its life and the maintainers are currently actively developing with a priority of addressing first critical issues directly related to the deliveries of `Alpha <https://github.com/datajoint/pharus/milestone/1>`_ and `Beta <https://github.com/datajoint/pharus/milestone/2>`_ milestones. Please be advised that while working through our milestones, we may restructure/refactor the codebase without warning until we issue our `Official Release <https://github.com/datajoint/pharus/milestone/3>`_ currently planned as ``0.1.0`` on ``2021-03-31``.

``pharus`` is a generic REST API server backend for DataJoint pipelines built on top of ``flask``, ``datajoint``, and ``pyjwt``.
``pharus`` is a generic REST API server backend for `DataJoint <https://datajoint.io>`_ pipelines built on top of ``flask``, ``datajoint``, and ``pyjwt``.

- `Documentation <https://datajoint.github.io/pharus>`_
- `Package <https://pypi.org/project/pharus/>`_
- `PyPi Package <https://pypi.org/project/pharus/>`_
- `Docker Image <https://hub.docker.com/r/datajoint/pharus>`_
- `Release <https://github.com/datajoint/pharus/releases/latest>`_
- `Source <https://github.com/datajoint/pharus>`_

Requirements for Preferred Setup
--------------------------------
Installation
------------

If you have not done so already, please install the following dependencies.

- `Docker <https://docs.docker.com/get-docker/>`_
- `Docker Compose <https://docs.docker.com/compose/install/>`_

Run Locally w/ Docker
---------------------

- Copy a ``*-docker-compose.yaml`` file corresponding to your usage to ``docker-compose.yaml``. This file is untracked so feel free to modify as necessary. Idea is to commit anything generic but system/setup dependent should go on 'your' version i.e. local UID/GID, etc.
- Check the first comment which will provide the best instruction on how to start the service; yes, it is a bit long. Note: Any of the keyword arguments prepended to the ``docker-compose`` command can be safely moved into a dedicated ``.env`` and read automatically if they are not evaluated i.e. ``$(...)``. Below is a brief description of the non-evaluated environment variables:

.. code-block:: bash
PY_VER=3.8 # Python version: 3.6|3.7|3.8
IMAGE=djtest # Image type: djbase|djtest|djlab|djlabhub
DISTRO=alpine # Distribution: alpine|debian
AS_SCRIPT= # If 'TRUE', will not keep container alive but run tests and exit
.. note::

Deployment options currently being considered are `Docker Compose <https://docs.docker.com/compose/install/>`_ and `Kubernetes <https://kubernetes.io/docs/tutorials/kubernetes-basics/>`_.

Run Locally w/ Python
---------------------

- Set environment variables for port assignment (``PHARUS_PORT``, defaults to 5000) and API route prefix (``PHARUS_PREFIX`` e.g. ``/api``, defaults to empty string).
- For development, use CLI command ``pharus``. This method supports hot-reloading so probably best coupled with ``pip install -e ...``.
- For production, use ``gunicorn --bind 0.0.0.0:${PHARUS_PORT} pharus.server:app``.

Run Tests for Development w/ Pytest and Flake8
----------------------------------------------

- Set ``pharus`` testing environment variables:

.. code-block:: bash
PKG_DIR=/opt/conda/lib/python3.8/site-packages/pharus # path to pharus installation
TEST_DB_SERVER=example.com:3306 # testing db server address
TEST_DB_USER=root # testing db server user (needs DDL privilege)
TEST_DB_PASS=unsecure # testing db server password
- For syntax tests, run ``flake8 ${PKG_DIR} --count --select=E9,F63,F7,F82 --show-source --statistics``
- For pytest integration tests, run ``pytest -sv --cov-report term-missing --cov=${PKG_DIR} /main/tests``
- For style tests, run ``flake8 ${PKG_DIR} --count --max-complexity=20 --max-line-length=95 --statistics``
Prerequisites
-------------

Creating Sphinx Documentation from Scratch
------------------------------------------
Download the ``docker-compose-deploy.yaml`` docker environment from the source located `here <https://github.com/datajoint/pharus/releases/latest/download/docker-compose-deploy.yaml>`_.

Recommend the follow to be ran within the ``pharus`` container in ``docs`` Docker Compose environment.
Running the API server
----------------------

- Run the following commands and complete the prompts as requested.
To start the API server, use the command:

.. code-block:: bash
mkdir docs
cd docs
sphinx-quickstart
- In ``docs/conf.py`` add to the beginning:

.. code-block:: python
PHARUS_VERSION=0.1.0b2 docker-compose -f docker-compose-deploy.yaml up -d
import os
import sys
sys.path.insert(0, os.path.abspath('..'))
- In ``docs/conf.py:extensions`` append ``['sphinx.ext.autodoc', 'sphinxcontrib.httpdomain']``. See ``requirements_docs.txt`` and ``docker-compose-docs.yaml`` for details on documentation dependencies.
- Run the following to automatically generate the API docs:
To stop the API server, use the command:

.. code-block:: bash
sphinx-apidoc -o . .. ../tests/* ../setup.py
- Add ``modules`` within the ``toctree`` directive in ``index.rst``.
- Build the docs by running:

.. code-block:: bash
make html
PHARUS_VERSION=0.1.0b2 docker-compose -f docker-compose-deploy.yaml down
References
----------

- DataJoint

- https://datajoint.io

- DataJoint LabBook (a companion frontend)

- https://github.com/datajoint/datajoint-labbook

- Python Tutorial for Flask, Swagger, and Automated docs

- https://realpython.com/flask-connexion-rest-api/#reader-comments
4 changes: 2 additions & 2 deletions docker-compose-deploy.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# PHARUS_VERSION=0.1.0b1 docker-compose -f docker-compose-deploy.yaml pull
# PHARUS_VERSION=0.1.0b1 docker-compose -f docker-compose-deploy.yaml up -d
# PHARUS_VERSION=0.1.0b2 docker-compose -f docker-compose-deploy.yaml pull
# PHARUS_VERSION=0.1.0b2 docker-compose -f docker-compose-deploy.yaml up -d
#
# Intended for production deployment.
# Note: You must run both commands above for minimal outage
Expand Down
96 changes: 95 additions & 1 deletion docs/dev_notes.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,98 @@
General Notes
=============

Dev docs TBD.
Requirements for Preferred Setup
--------------------------------

- `Docker <https://docs.docker.com/get-docker/>`_
- `Docker Compose <https://docs.docker.com/compose/install/>`_

Run Locally w/ Docker
---------------------

- Copy the docker environment (i.e. ``*-docker-compose.yaml``) of your choice corresponding to your usage to ``docker-compose.yaml``. This file is untracked so feel free to modify as necessary. Idea is to commit anything generic but system/setup dependent should go on 'your' version i.e. local UID/GID, etc.
- Check the first header comment which will provide the best instruction on how to start the service; yes, it is a bit long. Note: Any of the keyword arguments prepended to the ``docker-compose`` command can be safely moved into a dedicated ``.env`` and read automatically if they are not evaluated i.e. ``$(...)``. Below is a brief description of the non-evaluated environment variables:

.. code-block:: bash
PY_VER=3.8 # Python version: 3.6|3.7|3.8
IMAGE=djtest # Image type: djbase|djtest|djlab|djlabhub
DISTRO=alpine # Distribution: alpine|debian
AS_SCRIPT= # If 'TRUE', will not keep container alive but run tests and exit
.. note::

Deployment options currently being considered are `Docker Compose <https://docs.docker.com/compose/install/>`_ and `Kubernetes <https://kubernetes.io/docs/tutorials/kubernetes-basics/>`_.

Run Locally w/ Python
---------------------

- Set environment variables for port assignment (``PHARUS_PORT``, defaults to 5000) and API route prefix (``PHARUS_PREFIX`` e.g. ``/api``, defaults to empty string).
- For development, use CLI command ``pharus``. This method supports hot-reloading so probably best coupled with ``pip install -e ...``.
- For production, use ``gunicorn --bind 0.0.0.0:${PHARUS_PORT} pharus.server:app``.

Run Tests for Development w/ Pytest and Flake8
----------------------------------------------

- Set ``pharus`` testing environment variables:

.. code-block:: bash
PKG_DIR=/opt/conda/lib/python3.8/site-packages/pharus # path to pharus installation
TEST_DB_SERVER=example.com:3306 # testing db server address
TEST_DB_USER=root # testing db server user (needs DDL privilege)
TEST_DB_PASS=unsecure # testing db server password
- For syntax tests, run ``flake8 ${PKG_DIR} --count --select=E9,F63,F7,F82 --show-source --statistics``
- For pytest integration tests, run ``pytest -sv --cov-report term-missing --cov=${PKG_DIR} /main/tests``
- For style tests, run ``flake8 ${PKG_DIR} --count --max-complexity=20 --max-line-length=95 --statistics``

Creating Sphinx Documentation from Scratch
------------------------------------------

Recommend the follow to be ran within the ``pharus`` container in ``docs`` Docker Compose environment.

- Run the following commands and complete the prompts as requested.

.. code-block:: bash
mkdir docs
cd docs
sphinx-quickstart
- In ``docs/conf.py`` add to the beginning:

.. code-block:: python
import os
import sys
sys.path.insert(0, os.path.abspath('..'))
- In ``docs/conf.py:extensions`` append ``['sphinx.ext.autodoc', 'sphinxcontrib.httpdomain']``. See ``requirements_docs.txt`` and ``docker-compose-docs.yaml`` for details on documentation dependencies.
- Run the following to automatically generate the API docs:

.. code-block:: bash
sphinx-apidoc -o . .. ../tests/* ../setup.py
- Add ``modules`` within the ``toctree`` directive in ``index.rst``.
- Build the docs by running:

.. code-block:: bash
make html
References
----------

- DataJoint

- https://datajoint.io

- DataJoint LabBook (a companion frontend)

- https://github.com/datajoint/datajoint-labbook

- Python Tutorial for Flask, Swagger, and Automated docs

- https://realpython.com/flask-connexion-rest-api/#reader-comments
2 changes: 1 addition & 1 deletion pharus/version.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
"""Package metadata."""
__version__ = '0.1.0b1'
__version__ = '0.1.0b2'