Contributor docs update

Author: sujaypatil96

CODE_OF_CONDUCT.md

@@ -55,7 +55,7 @@ further defined and clarified by project maintainers.
 ## Enforcement
 
 Instances of abusive, harassing, or otherwise unacceptable behavior may be
-reported by contacting the project team at [INSERT EMAIL ADDRESS]. All
+reported by contacting the project team at <milen.nikolov@sagebase.org>. All
 complaints will be reviewed and investigated and will result in a response that
 is deemed necessary and appropriate to the circumstances. The project team is
 obligated to maintain confidentiality with regard to the reporter of an incident.

CONTRIBUTION.md

@@ -1,59 +1,101 @@
 # Contributing
 
-When contributing to this repository, please first discuss the change you wish to make via issue,
-email, or any other method with the owners of this repository before making a change. 
+When contributing to this repository, please first discuss the change you wish to make via issue, email, or any other method with the owners of this repository before making a change.
 
-Please note we have a code of conduct, please follow it in all your interactions with the project.
+Please note we have a [code of conduct](CODE_OF_CONDUCT.md), please follow it in all your interactions with the project.
 
-## Getting started
+## How to contribute
 
-### General Contribution Instructions
+### Reporting bugs or feature requests
 
-1. Fork the repository.
-2. Clone the forked repository.
-3. Push all your changes to the dev branch of the forked repository.
-4. Create pull requests to the origin repository.
+You can use the [`Issues`](https://github.com/Sage-Bionetworks/schematic/issues) tab to **create bug and feature requests**. Providing enough details to the developers to verify and troubleshoot your issue is paramount:
+- **Provide a clear and descriptive title as well as a concise summary** of the issue to identify the problem.
+- **Describe the exact steps which reproduce the problem** in as many details as possible.
+- **Describe the behavior you observed after following the steps** and point out what exactly is the problem with that behavior.
+- **Explain which behavior you expected to see** instead and why.
+- **Provide screenshots of the expected or actual behaviour** where applicable.
 
-### Setup Project for Development and Testing
+### General contribution instructions
+
+1. Follow the [Github docs](https://help.github.com/articles/fork-a-repo/) to make a copy (a fork) of the repository to your own Github account.
+2. [Clone the forked repository](https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/cloning-a-repository-from-github/cloning-a-repository) to your local machine so you can begin making changes.
+3. Make sure this repository is set as the [upstream remote repository](https://docs.github.com/en/github/collaborating-with-pull-requests/working-with-forks/configuring-a-remote-for-a-fork) so you are able to fetch the latest commits.
+4. Push all your changes to the `develop` branch of the forked repository.
+
+*Note*: Make sure you have you have the latest version of the `develop` branch on your local machine.
+
+```
+git checkout develop
+git pull upstream develop
+```
+
+5. Create pull requests to the upstream repository.
+
+### The development lifecycle
+
+1. Pull the latest content from the `develop` branch of this central repository (not your fork).
+2. Create a branch off the `develop` branch. Name the branch appropriately, either briefly summarizing the bug (ex., `spatil/add-restapi-layer`) or feature or simply use the issue number in the name (ex., `spatil/issue-414-fix`).
+3. After completing work and testing locally, push the code to the appropriate branch on your fork.
+4. In Github, create a pull request from the bug/feature branch of your fork to the `develop` branch of the central repository.
+
+> A Sage Bionetworks engineer must review and accept your pull request. A code review (which happens with both the contributor and the reviewer present) is required for contributing.
+
+### Development environment setup
+
+1. Install [package dependencies](https://sage-schematic.readthedocs.io/en/develop/README.html#installation-requirements-and-pre-requisites)
+2. Clone the `schematic` package repository
+
+```
+git clone https://github.com/Sage-Bionetworks/schematic.git
+```
 
-1. Install [package dependencies](https://sage-schematic.readthedocs.io/en/develop/README.html#installation-requirements-and-pre-requisites).
-2. Clone the `schematic` package repository: `git clone https://github.com/Sage-Bionetworks/schematic.git`
 3. [Create and activate](https://sage-schematic.readthedocs.io/en/develop/README.html#virtual-environment-setup) a virtual environment.
 4. Run the following commands to build schematic and install the package along with all of its dependencies:
-   ```python
-   cd schematic  # change directory to schematic
-   git checkout develop  # switch to develop branch of schematic
-   poetry build # build source and wheel archives
-   pip install dist/schematicpy-0.1.11-py3-none-any.whl  # install wheel file
-   ```
+
+```
+cd schematic  # change directory to schematic
+git checkout develop  # switch to develop branch of schematic
+poetry build # build source and wheel archives
+pip install dist/schematicpy-x.y.z-py3-none-any.whl  # install wheel file
+```
+
+*Note*: Use the appropriate version number (based on the version of the codebase you are pulling) while installing the wheel file above.
+
 5. [Obtain](https://sage-schematic.readthedocs.io/en/develop/README.html#obtain-google-credentials-file-s) appropriate Google credentials file(s).
 6. [Obtain and Fill in](https://sage-schematic.readthedocs.io/en/develop/README.html#fill-in-configuration-file-s) the `config.yml` file and the `.synapseConfig` file as well as described in the `Fill in Configuration File(s)` part of the documentation.
 7. [Run](https://docs.pytest.org/en/stable/usage.html) the test suite.
 
-Note: To ensure that all tests run successfully, contact your DCC liason and request to be added to the `schematic-dev` [team](https://www.synapse.org/#!Team:3419888) on Synapse.
+*Note*: To ensure that all tests run successfully, contact your DCC liason and request to be added to the `schematic-dev` [team](https://www.synapse.org/#!Team:3419888) on Synapse.
+
+8. To test new changes made to any of the modules within `schematic`, do the following:
 
-## Pull Request Process
+```
+# make changes to any files or modules
+pip uninstall schematicpy  # uninstall package
+poetry build
+pip install dist/schematicpy-x.y.z-py3-none-any.whl  # install wheel file
+```
 
-1. Ensure any install or build dependencies are removed before the end of the layer when doing a 
-   build.
-2. If needed, update the README.md with details of changes to the interface, this includes new environment 
-   variables, exposed ports, useful file locations and container parameters.
-3. You may merge the Pull Request in once you have the sign-off of at least one other developer, or if you 
-   do not have permission to do that, you may request the second reviewer to merge it for you.
-4. When merging into dev into master follow release procedures (TODO: releas process to be determined)
+## Testing
 
-## Updating Synapse Test Resources
+All code added to the client must have tests. The Python client uses pytest to run tests. The test code is located in the [tests](https://github.com/Sage-Bionetworks/schematic/tree/develop-docs-update/tests) subdirectory.
+
+You can run the test suite in the following way:
+
+```
+pytest -vs tests/
+```
+
+### Updating Synapse test resources
 
 1. Duplicate the entity being updated (or folder if applicable).
 2. Edit the duplicates (_e.g._ annotations, contents, name).
-3. Update the test suite in your branch to use these duplicates, including the expected values in the test assertions. 
-4. Open a PR as per the usual process (see above). 
-5. Once the PR is merged, leave the original copies on Synapse to maintain support for feature branches that were forked from `develop` before your update. 
-   - If the old copies are problematic and need to be removed immediately (_e.g._ contain sensitive data), proceed with the deletion and alert the other contributors that they need to merge the latest `develop` branch into their feature branches for their tests to work. 
+3. Update the test suite in your branch to use these duplicates, including the expected values in the test assertions.
+4. Open a PR as per the usual process (see above).
+5. Once the PR is merged, leave the original copies on Synapse to maintain support for feature branches that were forked from `develop` before your update.
+   - If the old copies are problematic and need to be removed immediately (_e.g._ contain sensitive data), proceed with the deletion and alert the other contributors that they need to merge the latest `develop` branch into their feature branches for their tests to work.
 
 ## Code style
 
-* Please consult this code style guide prior contributing code to the project:
-http://google.github.io/styleguide/pyguide.html
-
+* Please consult the [Google Python style guide](http://google.github.io/styleguide/pyguide.html) prior to contributing code to this project.
 * Be consistent and follow existing code conventions and spirit.

README.md

@@ -3,7 +3,7 @@
 
 ## Introduction
 
-SCHEMATIC is an acronym for _Schema Engine for Manifest Ingress and Curation_. The Python based infrastructure provides a _novel_ schema-based, data ingress ecosystem, that is meant to streamline the process of dataset annotation, metadata validation and submission to an asset store for various data contributors.
+SCHEMATIC is an acronym for _Schema Engine for Manifest Ingress and Curation_. The Python based infrastructure provides a _novel_ schema-based, metadata ingress ecosystem, that is meant to streamline the process of biomedical dataset annotation, metadata validation and submission to a data repository for various data contributors.
 
 ## Installation Requirements and Pre-requisites
 
@@ -61,4 +61,5 @@ Active contributors and maintainers:
 - [Milen Nikolov](https://github.com/milen-sage)
 - [Sujay Patil](https://github.com/sujaypatil96)
 - [Bruno Grande](https://github.com/BrunoGrandePhD)
+- [Robert Allaway](https://github.com/allaway)
 - [Xengie Doan](https://github.com/xdoan)

docs/README.rst

@@ -34,6 +34,13 @@ on ``macOS``, you can consider using `miniconda <https://docs.conda.io/en/latest
 
 -  `poetry <https://github.com/python-poetry/poetry>`__
 
+If you are using a ``Windows`` machine, typical bash programs will not work 
+on `cmd` in the same way as they work in the Linux/MacOS terminals. To circumvent this, 
+it is recommended that you set up  
+*Bash on Windows* (`WSL <https://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/>`__),  
+`Cygwin <https://cygwin.com/index.html>`__ or `Git Bash <https://gitforwindows.org/>`__ so you can easily execute the 
+command line utilities that are described later in these docs.
+
 **Note**: Make sure you are a registered and certified user on
 `synapse.org <https://www.synapse.org/>`__, and also have all the
 right permissions to download credentials files in the following steps.
@@ -52,7 +59,9 @@ credentials files.
 
 .. code:: bash
 
-    source .venv/bin/activate # activate the `venv` virtual environment
+    # activate the `.venv` virtual environment
+    source .venv/bin/activate   # on Linux and MacOS
+    .\.venv\Scripts\activate    # on Windows
 
 3.2. Installing
 ~~~~~~~~~~~~~~~~~
@@ -67,13 +76,14 @@ credentials files.
 There are two main configuration files that need to be edited –
 `config.yml <https://github.com/Sage-Bionetworks/schematic/blob/develop/config.yml>`__
 and
-`.synapseConfig <https://github.com/Sage-Bionetworks/synapsePythonClient/blob/v2.2.2-rc/synapseclient/.synapseConfig>`__.
+`.synapseConfig <https://raw.githubusercontent.com/Sage-Bionetworks/synapsePythonClient/v2.3.0-rc/synapseclient/.synapseConfig>`__.
 
 Download a copy of the ``.synapseConfig`` file, open the file in the
 editor of your choice and edit the
-`authtoken <https://github.com/Sage-Bionetworks/synapsePythonClient/blob/ba42e2d35673d19ce2b5bbd089f4fc1e99aca178/synapseclient/.synapseConfig#L9>`__
+`username <https://github.com/Sage-Bionetworks/synapsePythonClient/blob/v2.3.0-rc/synapseclient/.synapseConfig#L8>`__ and
+`authtoken <https://github.com/Sage-Bionetworks/synapsePythonClient/blob/v2.3.0-rc/synapseclient/.synapseConfig#L9>`__
 attribute under the
-`[authentication] <https://github.com/Sage-Bionetworks/synapsePythonClient/blob/ba42e2d35673d19ce2b5bbd089f4fc1e99aca178/synapseclient/.synapseConfig#L7>`__
+`[authentication] <https://github.com/Sage-Bionetworks/synapsePythonClient/blob/v2.3.0-rc/synapseclient/.synapseConfig#L7>`__
 section.
 
  Description of config.yml attributes
@@ -102,7 +112,7 @@ section.
             location: "data/schema_org_schemas/example.jsonld" # path to JSON-LD data model
             file_type: "local" # only type "local" is supported currently
             validation_schema: "~/path/to/validation_schema.json" # path to custom JSON Validation Schema JSON file
-            log_location: "~/path/to/log_folder/" # auto-generated JSON Validation Schemas can be logged
+            log_location: "~/path/to/log_folder/validation_schema.json" # auto-generated JSON Validation Schemas can be logged
         
 
 Note: Paths can be specified relative to the `config.yml` file or as absolute paths.
@@ -125,7 +135,10 @@ refer to the ``Credentials`` section in the
 document.
 
 Use the ``schematic_service_account_creds.json`` file for the service
-account mode of authentication (*for Google services/APIs*).
+account mode of authentication (*for Google services/APIs*). Service accounts 
+are special Google accounts that can be used by applications to access Google APIs 
+programmatically via OAuth2.0, with the advantage being that they do not require 
+human authorization.
 
 Note: The ``Selection Options`` dropdown which allows the user to select
 multiple values in a cell during manifest annotation `does not
@@ -199,4 +212,5 @@ Active contributors and maintainers:
 -  `Milen Nikolov <https://github.com/milen-sage>`__
 -  `Sujay Patil <https://github.com/sujaypatil96>`__
 -  `Bruno Grande <https://github.com/BrunoGrandePhD>`__
+-  `Robert Allaway <https://github.com/allaway>`__
 -  `Xengie Doan <https://github.com/xdoan>`__

docs/md/details.md

@@ -1,5 +1,6 @@
 # Credentials
 
+## OAuth2.0
 The `credentials.json` file is required when you are using [`OAuth2`](https://developers.google.com/identity/protocols/oauth2) to authenticate with the Google APIs.
 
 Steps involved in the `OAuth2` [authorization flow](https://github.com/Sage-Bionetworks/schematic/blob/develop/schematic/utils/google_api_utils.py#L18):
@@ -8,4 +9,10 @@ Steps involved in the `OAuth2` [authorization flow](https://github.com/Sage-Bion
 - If the `token.pickle` file does not exist, the app will load the credentials for you from the `credentials.json` file and create a `token.pickle` in your project root directory when you access any of the schematic services (e.g., through the Command Line Interface) and the authorization flow completes for the first time. For this:
   - A URL will be prompted to you when you access a _schematic_ CLI utility. Copy the URL from the console and open it in your browser.
   - If you are not already logged into your Google account, you will be prompted to log in. If you are logged into multiple Google accounts, you will be asked to select one account to use for the authorization.
-  - You might see a "Google hasn’t verified this app" warning from Chrome on your browser. To resolve this, click on "Advanced" and then "Quickstart (unsafe)". Select a Google account and authorize the _Quickstart_ script to access Google Drive/Sheets under your account.
+  - You might see a "Google hasn’t verified this app" warning from Chrome on your browser. To resolve this, click on "Advanced" and then "Quickstart (unsafe)". Select a Google account and authorize the _Quickstart_ script to access Google Drive/Sheets under your account.
+
+## Service Account
+- The Google OAuth 2.0 system supports server-to-server interactions such as those between a web application and a Google service. For this scenario you need a service account, which is an account that belongs to your application instead of to an individual end user. Your application calls Google APIs on behalf of the service account, so users aren't directly involved.
+- Typically, an application uses a service account when the application uses Google APIs to work with its own data rather than a user's data.
+- The `service_account_creds.json` file is a key file that schematic will need access to in order to use the service account mode of authentication.
+-