[SYNPY-1589] Implement "Evaluation" OOP model

docs/reference/experimental/async/evaluation.md

@@ -0,0 +1,21 @@
+# Evaluation
+
+Contained within this file are experimental interfaces for working with the Synapse Python
+Client. Unless otherwise noted these interfaces are subject to change at any time. Use
+at your own risk.
+
+## API reference
+
+::: synapseclient.models.Evaluation
+    options:
+        inherited_members: true
+        members:
+            - store_async
+            - get_async
+            - delete_async
+            - get_all_evaluations_async
+            - get_available_evaluations_async
+            - get_evaluations_by_project_async
+            - get_acl_async
+            - update_acl_async
+            - get_permissions_async

docs/reference/experimental/sync/evaluation.md

@@ -0,0 +1,21 @@
+# Evaluation
+
+Contained within this file are experimental interfaces for working with the Synapse Python
+Client. Unless otherwise noted these interfaces are subject to change at any time. Use
+at your own risk.
+
+## API reference
+
+::: synapseclient.models.Evaluation
+    options:
+        inherited_members: true
+        members:
+            - store
+            - get
+            - delete
+            - get_all_evaluations
+            - get_available_evaluations
+            - get_evaluations_by_project
+            - get_acl
+            - update_acl
+            - get_permissions

docs/tutorials/python/evaluation.md

@@ -0,0 +1,61 @@
+# Evaluations
+An Evaluation is essentially a container that organizes and manages the submission, assessment, and scoring of data, models, or other research artifacts.
+It allows teams to set up challenges where participants contribute their work, and those contributions can be systematically reviewed or scored.
+
+This tutorial will walk you through the basics of working with Evaluations using the Synapse Python client.
+
+## Tutorial Purpose
+In this tutorial you will:
+
+1. Create and update an Evaluation on Synapse
+1. Update the ACL (Access Control List) of an Evaluation on Synapse
+1. Retrieve and delete all Evaluations from a given Project
+
+## Prerequisites
+* You have completed the [Project](./project.md) tutorial, or have an existing Project on Synapse to work from
+* You have a Synapse user or team ID to share the evaluation with
+
+## 1. Create and update an Evaluation on Synapse
+
+In this first part, we'll be showing you how to interact with an Evaluation object as well as introducing you to its two core functionalities `store()` and `get()`.
+
+```python
+{!docs/tutorials/python/tutorial_scripts/evaluation.py!lines=5-46}
+```
+
+## 2. Update the ACL of an Evaluation on Synapse
+
+Like Synapse entities, Evaluations have ACLs that can be used to control who has access to your evaluations and what level of access they have. Updating the ACL of an Evaluation object is slightly different from updating other Evaluation components, because the ACL is not an attribute of the Evaluation object. Let's see an example of how this looks:
+
+```python
+{!docs/tutorials/python/tutorial_scripts/evaluation.py!lines=54-64}
+```
+
+You can also remove principals from an ACL by simply feeding `update_acl` an empty list for the `access_type` argument, like so:
+
+```python
+{!docs/tutorials/python/tutorial_scripts/evaluation.py!lines=66-67}
+```
+
+## 3. Retrieve and delete all Evaluations from a given Project
+
+Now we will show how you can retrieve lists of Evaluation objects, rather than retrieving them one-by-one with `get()`. This is a powerful tool if you want to perform the same action on all the evaluations in a given project, for example, like what we're about to do here:
+
+```python
+{!docs/tutorials/python/tutorial_scripts/evaluation.py!lines=69-75}
+```
+
+## Source code for this tutorial
+
+<details class="quote">
+  <summary>Click to show me</summary>
+
+```python
+{!docs/tutorials/python/tutorial_scripts/evaluation.py!}
+```
+</details>
+
+## References
+- [Evaluation][synapseclient.models.Evaluation]
+- [Project][synapseclient.models.Project]
+- [syn.login][synapseclient.Synapse.login]

docs/tutorials/python/tutorial_scripts/evaluation.py

@@ -0,0 +1,75 @@
+"""
+Here is where you'll find the code for the Evaluation tutorial.
+"""
+
+from synapseclient import Synapse
+from synapseclient.models import Evaluation, Project
+
+syn = Synapse()
+syn.login()
+
+# REQUIRED: Set this to the Synapse user ID or team ID you want to grant permissions to
+# Do NOT leave this as None - the script will not work properly
+PRINCIPAL_ID = None  # Replace with actual user/team ID
+
+# Retrieve the Project where your Evaluation will be stored
+project = Project(name="My uniquely named project about Alzheimer's Disease").get()
+project_id = project.id
+
+print(f"Working within Project: {project_id}")
+
+# Create a new Evaluation object
+evaluation = Evaluation(
+    name="My Challenge Evaluation for Study ABC - Round 1",
+    description="Evaluation for my data challenge",
+    content_source=project_id,
+    submission_instructions_message="Submit CSV files only",
+    submission_receipt_message="Thank you for your submission!",
+)
+
+# Create the Evaluation on Synapse
+evaluation.store()
+
+print("Evaluation has been created with the following name and description:")
+print(evaluation.name)
+print(evaluation.description)
+
+# Update the Evaluation object's name and description
+evaluation.name = "My Challenge Evaluation for Study XYZ - Round 1"
+evaluation.description = "Updated description for my evaluation"
+
+# Update the Evaluation on Synapse
+evaluation.store()
+
+print("Evaluation has been updated with the following name and description:")
+print(evaluation.name)
+print(evaluation.description)
+
+# Confirm what's in Synapse matches the evaluation stored
+from_synapse = Evaluation(id=evaluation.id).get()
+
+print("The following evaluation has been retrieved from Synapse:")
+print(from_synapse)
+
+# Update the Evaluation's ACL on Synapse by adding a new user
+assert (
+    PRINCIPAL_ID is not None
+), "PRINCIPAL_ID must be set to the Synapse user ID or team ID you want to grant permissions to."
+
+evaluation.update_acl(principal_id=PRINCIPAL_ID, access_type=["READ", "SUBMIT"])
+
+# Get the Evaluation's ACL to confirm the update
+acl = evaluation.get_acl()
+print("The following ACL has been retrieved from Synapse:")
+print(acl)
+
+# Now let's remove the user we just added from the Evaluation's ACL
+evaluation.update_acl(principal_id=PRINCIPAL_ID, access_type=[])
+
+# Finally let's retrieve all Evaluations stored within this project, including the one we just created
+evaluations_list = Evaluation.get_evaluations_by_project(project_id)
+
+# Let's delete the evaluation we created for this tutorial, and any other evaluations in this project (uncomment below to enable deletion)
+# for evaluation_to_delete in evaluations_list:
+#     print(f"Deleting evaluation: {evaluation_to_delete.name}")
+#     evaluation_to_delete.delete()

mkdocs.yml

@@ -27,6 +27,7 @@ nav:
           - Project: tutorials/python/project.md
           - Folder: tutorials/python/folder.md
           - File: tutorials/python/file.md
+          - Evaluation: tutorials/python/evaluation.md
           - Annotation: tutorials/python/annotation.md
           # - Versions: tutorials/python/versions.md
           # - Activity/Provenance: tutorials/python/activity.md
@@ -85,6 +86,7 @@ nav:
           - Project: reference/experimental/sync/project.md
           - Folder: reference/experimental/sync/folder.md
           - File: reference/experimental/sync/file.md
+          - Evaluation: reference/experimental/sync/evaluation.md
           - Table: reference/experimental/sync/table.md
           - VirtualTable: reference/experimental/sync/virtualtable.md
           - Dataset: reference/experimental/sync/dataset.md
@@ -104,6 +106,7 @@ nav:
               - Project: reference/experimental/async/project.md
               - Folder: reference/experimental/async/folder.md
               - File: reference/experimental/async/file.md
+              - Evaluation: reference/experimental/async/evaluation.md
               - Table: reference/experimental/async/table.md
               - VirtualTable: reference/experimental/async/virtualtable.md
               - Dataset: reference/experimental/async/dataset.md

synapseclient/api/__init__.py

@@ -63,6 +63,17 @@
     update_activity,
     update_entity_acl,
 )
+from .evaluation_services import (
+    create_or_update_evaluation,
+    delete_evaluation,
+    get_all_evaluations,
+    get_available_evaluations,
+    get_evaluation,
+    get_evaluation_acl,
+    get_evaluation_permissions,
+    get_evaluations_by_project,
+    update_evaluation_acl,
+)
 from .file_services import (
     AddPartResponse,
     get_file_handle,
@@ -261,4 +272,14 @@
     "is_user_certified",
     # table_services
     "create_table_snapshot",
+    # evaluation_services
+    "create_or_update_evaluation",
+    "get_evaluation",
+    "get_evaluations_by_project",
+    "get_all_evaluations",
+    "get_available_evaluations",
+    "delete_evaluation",
+    "get_evaluation_acl",
+    "update_evaluation_acl",
+    "get_evaluation_permissions",
 ]