Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
a061fb4
tdd initial tests
jaymedina Sep 29, 2025
d85a9d7
initial intro of dataclasses
jaymedina Sep 29, 2025
50498b7
expose api services for submission object
jaymedina Sep 30, 2025
94cedc6
style and update docstring
jaymedina Sep 30, 2025
cd19e0c
add submission and submissionstatus models
jaymedina Oct 10, 2025
99cba20
add submission status retrieval and update methods; remove empty subm…
jaymedina Oct 10, 2025
9ca2448
pipe query params directly into restAPI httpx requests
jaymedina Oct 14, 2025
f0d7ecc
new dataclass object submission_bundle
jaymedina Oct 14, 2025
91b947b
move submission services functions to evaluation_services.py
jaymedina Nov 5, 2025
a6b46d9
renaming imports, to_synapse_request, request body refactor
jaymedina Nov 10, 2025
b68bd8e
patching up store method signature
jaymedina Nov 10, 2025
5a44f07
update docs
jaymedina Nov 10, 2025
65664d1
new suite of tests
jaymedina Nov 10, 2025
d214673
submissionstatus rework as a mutable object
jaymedina Nov 10, 2025
2781797
bug fix for Statuses: updated to_synapse_request to follow same patte…
jaymedina Nov 11, 2025
3230b1f
replace != with is not for full object comparison (not just keys)
jaymedina Nov 11, 2025
2d43e5d
expose the is_private arg for to_submission_status_annotations ONLY F…
jaymedina Nov 11, 2025
d4d63ba
fixed submission status/submission annotations
jaymedina Nov 12, 2025
8b9f9c9
add support for legacy annotations
jaymedina Nov 13, 2025
6aa914a
remove debug prints
jaymedina Nov 13, 2025
2d67fcc
get_all_submission_statuses now returns a list of substat objects
jaymedina Nov 13, 2025
c368e49
docstring updates
jaymedina Nov 13, 2025
fbd43f6
update submissionbundle docstrings, add more examples
jaymedina Nov 14, 2025
d5fac7e
initial sync test for status. moved evaluation_id retrieval to fill_f…
jaymedina Nov 14, 2025
47797c0
update submissionBundle submissionstatus with evaluation_id
jaymedina Nov 14, 2025
e8e5361
patch sync substatus integ tests. style.
jaymedina Nov 14, 2025
58ffe42
fix submissionStatus integ tests and has_changed attribute
jaymedina Nov 17, 2025
f34667d
new substatus async integ tests. can_cancel can now be modified by an…
jaymedina Nov 20, 2025
5e762f0
new test class for submission cancel functionality
jaymedina Nov 20, 2025
1bd7f4b
substatus async unit tests
jaymedina Nov 20, 2025
4c8904d
remove compare=false for some attributes. update sync unit tests
jaymedina Nov 20, 2025
a223320
add submissionBundle integration tests
jaymedina Nov 20, 2025
f059751
add submissionBundle unit tests
jaymedina Nov 20, 2025
3cb1915
remove unnecessary imports and add style
jaymedina Nov 20, 2025
1fb0fbf
get_evaluation_submissions returns generator object
jaymedina Nov 21, 2025
281ed3d
get_user_submissions returns generator object
jaymedina Nov 21, 2025
1168cae
submissionBundle methods return generators
jaymedina Nov 21, 2025
a9977b2
address final todo: implement docker_tag, and note in docs that versi…
jaymedina Nov 21, 2025
689fcb1
add async page in api references
jaymedina Sep 30, 2025
c54609b
add initial submission, status, bundle docs
jaymedina Nov 21, 2025
e0a79cc
updated tutorial purpose. add api reference to navbar
jaymedina Nov 24, 2025
35efed7
new tutorial scripts
jaymedina Nov 24, 2025
e07206d
remove try/excepts. add line references. add resources and source cod…
jaymedina Dec 1, 2025
e193753
add reference to File model
jaymedina Dec 1, 2025
b710da0
deprecate old submission model
jaymedina Dec 1, 2025
7917011
style
jaymedina Dec 1, 2025
2021fb7
Merge branch 'synpy-1590-submission-model-main' into synpy-1590-submi…
jaymedina Dec 22, 2025
4718119
add submission status and bundle to references
jaymedina Dec 22, 2025
8510672
no need to import Submission for submission_organizer tutorial
jaymedina Dec 22, 2025
727e302
status should be CLOSED not CANCELLED. add Submission import back (it…
jaymedina Dec 22, 2025
15e450c
set tutorial global vars to None
jaymedina Dec 23, 2025
7e57d72
fix line references
jaymedina Dec 23, 2025
1982ba6
remove limits
jaymedina Dec 23, 2025
2bd0304
add 2 more examples for Submission docstring
jaymedina Dec 23, 2025
8cdb88c
use the same evaluation_id in all examples
jaymedina Dec 23, 2025
985b727
elaborate on submissions, statuses, bundles, and their relationship t…
jaymedina Dec 23, 2025
fb6709c
remove AcessControllable from being inherited
jaymedina Dec 23, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/reference/experimental/async/submission.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Submission

Contained within this file are experimental interfaces for working with the Synapse Python
Client. Unless otherwise noted these interfaces are subject to change at any time. Use
at your own risk.

## API Reference

::: synapseclient.models.Submission
options:
inherited_members: true
members:
- store_async
- get_async
- delete_async
- cancel_async
- get_evaluation_submissions_async
- get_user_submissions_async
- get_submission_count_async
14 changes: 14 additions & 0 deletions docs/reference/experimental/async/submission_bundle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Submission Bundle

Contained within this file are experimental interfaces for working with the Synapse Python
Client. Unless otherwise noted these interfaces are subject to change at any time. Use
at your own risk.

## API Reference

::: synapseclient.models.SubmissionBundle
options:
inherited_members: true
members:
- get_evaluation_submission_bundles_async
- get_user_submission_bundles_async
16 changes: 16 additions & 0 deletions docs/reference/experimental/async/submission_status.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Submission Status

Contained within this file are experimental interfaces for working with the Synapse Python
Client. Unless otherwise noted these interfaces are subject to change at any time. Use
at your own risk.

## API Reference

::: synapseclient.models.SubmissionStatus
options:
inherited_members: true
members:
- get_async
- store_async
- get_all_submission_statuses_async
- batch_update_submission_statuses_async
19 changes: 19 additions & 0 deletions docs/reference/experimental/sync/submission.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Submission

Contained within this file are experimental interfaces for working with the Synapse Python
Client. Unless otherwise noted these interfaces are subject to change at any time. Use
at your own risk.

## API Reference

::: synapseclient.models.Submission
options:
inherited_members: true
members:
- store
- get
- delete
- cancel
- get_evaluation_submissions
- get_user_submissions
- get_submission_count
14 changes: 14 additions & 0 deletions docs/reference/experimental/sync/submission_bundle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Submission Bundle

Contained within this file are experimental interfaces for working with the Synapse Python
Client. Unless otherwise noted these interfaces are subject to change at any time. Use
at your own risk.

## API Reference

::: synapseclient.models.SubmissionBundle
options:
inherited_members: true
members:
- get_evaluation_submission_bundles
- get_user_submission_bundles
16 changes: 16 additions & 0 deletions docs/reference/experimental/sync/submission_status.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Submission Status

Contained within this file are experimental interfaces for working with the Synapse Python
Client. Unless otherwise noted these interfaces are subject to change at any time. Use
at your own risk.

## API Reference

::: synapseclient.models.SubmissionStatus
options:
inherited_members: true
members:
- get
- store
- get_all_submission_statuses
- batch_update_submission_statuses
157 changes: 157 additions & 0 deletions docs/tutorials/python/submission.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
# Submissions, SubmissionStatuses, SubmissionBundles
## What are Submissions?

In Synapse, a Submission is your entry to a challenge or evaluation queue. When you participate in a computational challenge or collaborative project, you submit your work (such as predictions, models, or analysis results) as a Submission to be evaluated and scored.

## Key Concepts

Before working with Submissions, it's helpful to understand how they fit into Synapse:

- **Entity**: Your actual work stored in Synapse (like a File containing predictions or a Docker image with your model)
- **Evaluation**: A queue that accepts and organizes submissions for a specific challenge or project
- **Submission**: The object associated with submitting your Entity to an Evaluation queue, creating a record that can be tracked and scored

## How Submissions Work

When you submit an Entity to an Evaluation:
- The Submission creates an immutable record linking your Entity to that Evaluation
- The Evaluation owns this Submission record (not you as the submitter)
- Organizers can add scores and feedback through a SubmissionStatus object
- You can track your Submissions and view their statuses

## Related Objects

The Python client provides three object types for working with Submissions:

- **Submission**: Represents your entry in an Evaluation queue
- **SubmissionStatus**: Tracks scoring information and feedback for a Submission
- **SubmissionBundle**: Combines a Submission and its SubmissionStatus for convenient access

## What You'll Learn

This tutorial covers two perspectives:

1. **Participating in challenges**: Making and tracking your submissions
1. **Organizing challenges**: Scoring and managing submissions from participants

## Tutorial Purpose

In this tutorial:

As a participant of a Synapse challenge, you will

1. Make a submission to an existing evaluation queue on Synapse
1. Fetch your existing submission
1. Count your submissions
1. Fetch all of your submissions from an existing evaluation queue on Synapse
1. Check the status of your submission
1. Cancel your submission

As an organizer of a Synapse challenge, you will

1. Annotate a submission to score it
1. Batch-update submission statuses
1. Fetch the submission bundle for a given submission
1. Allow cancellation of submissions
1. Delete submissions

## Prerequisites
* You have completed the [Evaluation](./evaluation.md) tutorial, or have an existing Evaluation on Synapse to work from
* You have an existing entity with which to make a submission (can be a [File](./file.md) or Docker Repository)
* You have the correct permissions on the Evaluation queue for your desired tutorial section (participant or organizer)

## 1. Participating in a Synapse challenge

### 1. Make a submission to an existing evaluation queue on Synapse

```python
{!docs/tutorials/python/tutorial_scripts/submission_participant.py!lines=32-54}
```

### 2. Fetch your existing submission

```python
{!docs/tutorials/python/tutorial_scripts/submission_participant.py!lines=56-71}
```

### 3. Count your submissions

```python
{!docs/tutorials/python/tutorial_scripts/submission_participant.py!lines=72-88}
```

### 4. Fetch all of your submissions from an existing evaluation queue on Synapse

```python
{!docs/tutorials/python/tutorial_scripts/submission_participant.py!lines=90-101}
```

### 5. Check the status of your submission

```python
{!docs/tutorials/python/tutorial_scripts/submission_participant.py!lines=103-125}
```

### 6. Cancel your submission

```python
{!docs/tutorials/python/tutorial_scripts/submission_participant.py!lines=126-143}
```

## 2. Organizing a Synapse challenge

### 1. Annotate a submission to score it

```python
{!docs/tutorials/python/tutorial_scripts/submission_organizer.py!lines=33-60}
```

### 2. Batch-update submission statuses

```python
{!docs/tutorials/python/tutorial_scripts/submission_organizer.py!lines=62-99}
```

### 3. Fetch the submission bundle for a given submission

```python
{!docs/tutorials/python/tutorial_scripts/submission_organizer.py!lines=101-136}
```

### 4. Allow cancellation of submissions

```python
{!docs/tutorials/python/tutorial_scripts/submission_organizer.py!lines=138-177}
```

### 5. Delete submissions

```python
{!docs/tutorials/python/tutorial_scripts/submission_organizer.py!lines=179-209}
```

## Source code for this tutorial

<details class="quote">
<summary>Click to show me (source code for Participant)</summary>

```python
{!docs/tutorials/python/tutorial_scripts/submission_participant.py!}
```
</details>

<details class="quote">
<summary>Click to show me (source code for Organizer)</summary>

```python
{!docs/tutorials/python/tutorial_scripts/submission_organizer.py!}
```
</details>

## References
- [Evaluation][synapseclient.models.Evaluation]
- [File][synapseclient.models.File]
- [Submission][synapseclient.models.Submission]
- [SubmissionStatus][synapseclient.models.SubmissionStatus]
- [SubmissionBundle][synapseclient.models.SubmissionBundle]
- [syn.login][synapseclient.Synapse.login]
Loading
Loading