[SYNPY-1589] Implement core functionality of Evaluation OOP model #1245

jaymedina · 2025-09-11T14:41:58Z

See #1244

synapseclient/models/evaluation.py

…luation

…ration tests.

…ividually. test improvements.

synapseclient/api/evaluation_services.py

…he class object itself rather than feeding it into the method. new to_synapse_request

synapseclient/models/evaluation.py

…able field. style

synapseclient/api/evaluation_services.py

synapseclient/core/utils.py

synapseclient/models/evaluation.py

…ctive client

linglp

Hi @jaymedina ! Thanks for your hard work. I looked through the PR too and added some comments. To summarize:

Just like @BryanFauble's earlier comment about pagination, I wonder if it would make sense to implement pagination for all functions that return a Pagination<Evaluation> response based on the documentation. That includes the following function:

get_evaluations_by_project_async (see doc)
get_all_evaluations_async (see doc)
get_available_evaluations_async (see doc)

I also suggest adding two dataclasses:

UserEvaluationPermissions: link
AccessControlList: link

AccessControlList would wrap results from GET /evaluation/{evalId}/acl, and UserEvaluationPermissions would handle GET /evaluation/{evalId}/permissions.

synapseclient/api/evaluation_services.py

synapseclient/models/evaluation.py

synapseclient/api/evaluation_services.py

synapseclient/models/evaluation.py

…ixture properly

synapseclient/models/evaluation.py

synapseclient/api/evaluation_services.py

synapseclient/core/utils.py

thomasyu888

🔥 I'll defer to @BryanFauble or @andrewelamb for final review when they a chance. I let them know that we're focussed on everything Curator related but this is coming together nicely!

…parse params with urlencode

BryanFauble

Great work here! I didn't see anything else that popped out at me. Thanks for implementing this.

) * Initialize protocol class + Evaluation dataclass * exposing Evaluation API services. dataclass updates. syntax updates. * dataclass methods are added which wrap API service functions * fixed missing imports and syntax failures * Evaluations can be imported like from synapseclient.models import Evaluation * correct ACL inputs and output types. first pass at fixing async integration tests. * fix import issues * fix type errors. api module and its functions can now be imported individually. test improvements. * add etag to prevent issues from OCC when updating an evaluation * some formatting * refactor: update and create will rely on the attributes changing in the class object itself rather than feeding it into the method. new to_synapse_request * remove principal_id param. cannot be used. adding examples to docstrings * patch all async tests * style * remove unnecessary imports * complete refactor of store/update. store is now used for updating too * add example to Evaluation dataclass * new evaluation protocol class. moved out to protocols subfolder * add synchronous integration tests * new unit tests. updated integration tests. content_source is an immutable field. style * show available access types * remove todo * get logger instance * no need for request_type param in store_evaluation_async. removing CASE 2 for handling ID of new evals. remove integration test * docstring example formatting Co-authored-by: BryanFauble <[email protected]> * update examples in docstrings * the logger called in merge_dataclass_entities is now retrieved from active client * remove _async suffix from evaluation_services functions * style * updated docstrings * store -> create_or_update, updated tests * style * refactor update_acl dataclass method for user qol * give users option to remove principals from ACL * convert sync test methods to async so it calls schedule_for_cleanup fixture properly * style * if/elif -> if/else * iteratively grab the attributes instead * request_type is now a CREATE/UPDATE enum * feed query params directly into client httpx response call so it can parse params with urlencode * style * grab the logger outside of the for-loop --------- Co-authored-by: BryanFauble <[email protected]>

jaymedina added 4 commits September 11, 2025 10:39

Initialize protocol class + Evaluation dataclass

ff145d7

exposing Evaluation API services. dataclass updates. syntax updates.

d3f6aed

dataclass methods are added which wrap API service functions

e6f6ac7

fixed missing imports and syntax failures

8d9d8b3

jaymedina commented Sep 15, 2025

View reviewed changes

synapseclient/models/evaluation.py Show resolved Hide resolved

jaymedina added 6 commits September 15, 2025 11:59

Evaluations can be imported like from synapseclient.models import Eva…

3195cd5

…luation

correct ACL inputs and output types. first pass at fixing async integ…

63ed998

…ration tests.

fix import issues

821fbd3

fix type errors. api module and its functions can now be imported ind…

cfa4436

…ividually. test improvements.

add etag to prevent issues from OCC when updating an evaluation

6d14de3

some formatting

6e92d8e

jaymedina commented Sep 16, 2025

View reviewed changes

synapseclient/api/evaluation_services.py Outdated Show resolved Hide resolved

refactor: update and create will rely on the attributes changing in t…

c05c628

…he class object itself rather than feeding it into the method. new to_synapse_request

jaymedina commented Sep 16, 2025

View reviewed changes

synapseclient/models/evaluation.py Outdated Show resolved Hide resolved

jaymedina added 9 commits September 17, 2025 12:54

remove principal_id param. cannot be used. adding examples to docstrings

62fde96

patch all async tests

f743c68

style

b7bb1a6

remove unnecessary imports

87b07bb

complete refactor of store/update. store is now used for updating too

ecfc2d2

add example to Evaluation dataclass

fc6190f

new evaluation protocol class. moved out to protocols subfolder

4211317

add synchronous integration tests

f6c23cb

new unit tests. updated integration tests. content_source is an immut…

54032e9

…able field. style

jaymedina marked this pull request as ready for review September 22, 2025 16:38

jaymedina requested a review from a team as a code owner September 22, 2025 16:39

show available access types

7cdf25e

BryanFauble reviewed Sep 23, 2025

View reviewed changes

synapseclient/api/evaluation_services.py Outdated Show resolved Hide resolved

BryanFauble reviewed Sep 23, 2025

View reviewed changes

synapseclient/api/evaluation_services.py Outdated Show resolved Hide resolved

BryanFauble reviewed Sep 23, 2025

View reviewed changes

synapseclient/core/utils.py Outdated Show resolved Hide resolved

BryanFauble reviewed Sep 23, 2025

View reviewed changes

synapseclient/models/evaluation.py Outdated Show resolved Hide resolved

jaymedina added 3 commits September 25, 2025 10:38

the logger called in merge_dataclass_entities is now retrieved from a…

4a33c9f

…ctive client

remove _async suffix from evaluation_services functions

808c040

style

5e604ef

linglp reviewed Sep 29, 2025

View reviewed changes

updated docstrings

af6d6bc

BryanFauble mentioned this pull request Sep 30, 2025

[SYNPY-1589] Implement documentation of Evaluation OOP model #1247

Merged

jaymedina added 6 commits October 1, 2025 10:34

store -> create_or_update, updated tests

39995be

style

e5ad067

refactor update_acl dataclass method for user qol

0e220de

give users option to remove principals from ACL

17177f4

convert sync test methods to async so it calls schedule_for_cleanup f…

8fd7db2

…ixture properly

style

1d577a3

andrewelamb reviewed Oct 6, 2025

View reviewed changes

synapseclient/models/evaluation.py Show resolved Hide resolved

andrewelamb reviewed Oct 6, 2025

View reviewed changes

synapseclient/models/evaluation.py Outdated Show resolved Hide resolved

andrewelamb reviewed Oct 6, 2025

View reviewed changes

synapseclient/models/evaluation.py Show resolved Hide resolved

andrewelamb reviewed Oct 6, 2025

View reviewed changes

synapseclient/models/evaluation.py Show resolved Hide resolved

andrewelamb reviewed Oct 6, 2025

View reviewed changes

synapseclient/api/evaluation_services.py Outdated Show resolved Hide resolved

jaymedina added 3 commits October 6, 2025 15:34

if/elif -> if/else

b2ee087

iteratively grab the attributes instead

e42b27a

request_type is now a CREATE/UPDATE enum

e442f61

thomasyu888 reviewed Oct 13, 2025

View reviewed changes

synapseclient/api/evaluation_services.py Outdated Show resolved Hide resolved

thomasyu888 reviewed Oct 13, 2025

View reviewed changes

synapseclient/core/utils.py Outdated Show resolved Hide resolved

thomasyu888 approved these changes Oct 13, 2025

View reviewed changes

jaymedina added 2 commits October 13, 2025 15:28

feed query params directly into client httpx response call so it can …

4d84d05

…parse params with urlencode

style

86688cd

BryanFauble approved these changes Oct 17, 2025

View reviewed changes

grab the logger outside of the for-loop

198c24c

jaymedina merged commit eb6e692 into synpy-1589-evaluation-model-main Oct 17, 2025
3 of 4 checks passed

jaymedina deleted the synpy-1589-evaluation-model-functionality branch October 17, 2025 22:41

[SYNPY-1589] Implement core functionality of Evaluation OOP model #1245

[SYNPY-1589] Implement core functionality of Evaluation OOP model #1245

Uh oh!

Conversation

jaymedina commented Sep 11, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

linglp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

thomasyu888 left a comment

Choose a reason for hiding this comment

Uh oh!

BryanFauble left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants