-
Notifications
You must be signed in to change notification settings - Fork 73
[SYNPY-1351, SYNPY-1613] Implement 'Wiki2' model into OOP #1206
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
danlu1
wants to merge
74
commits into
develop
Choose a base branch
from
synpy-1351-oop-wiki2
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
74 commits
Select commit
Hold shift + click to select a range
fef882b
add Low-level Functionality to interact with wikipage2 endpoints
danlu1 977413b
remove unused module
danlu1 b2a5840
resort
danlu1 1ce73fd
add synchronous protocol parent class
danlu1 0685de4
add synchronous protocol for wiki2
danlu1 0fc3f75
remove redundant docstring
danlu1 d4a98e8
all pre-signed url to be downloaded directly
danlu1 86fa041
add rest_get_paginated_async to get paginated results from api call
danlu1 6601fdc
add unit test for rest_get_paginated_async
danlu1 b7289f2
add unit test for rest_get_paginated_async
danlu1 21a4d8d
tweak test cases
danlu1 e06a35a
use rest_get_paginated_async to get paginated results
danlu1 2fdfc71
reformat download_functions.py
danlu1 176cdf8
use specified destination for pre-signed url downloads
danlu1 c52996f
remove unused modules
danlu1 99cbb81
update function names
danlu1 47f38e0
define optional params
danlu1 a620db5
update function names
danlu1 4e89258
add tutorials
danlu1 da4dfa4
remove unwanted module name
danlu1 be5aae4
update typing hint
danlu1 1ab9888
make sure gzip file always be removed after getting the file handle id
danlu1 082df85
update filename for markdown gzip file
danlu1 43bb9a0
update tutorials
danlu1 0a5f2e6
update markdown name if it's created from text to make it more inform…
danlu1 c6077e0
reorder functions for order hint
danlu1 cef775b
update wikipage name and instructions
danlu1 e32da1b
reorder and fill in details for tutorial md
danlu1 50417a3
remove unwanted comments
danlu1 d59a9d6
make low-level functionalities importable from synapseclient.api
danlu1 a317bc9
rename post_wiki to post_wiki_page
danlu1 9ae4c81
remove debug code
danlu1 d52720c
remove unwanted decorator
danlu1 cbcbf35
refactor wikipage store function to put validation at the beginning a…
danlu1 5c66880
remove debug statement
danlu1 d8055d6
simplify get_async for wikipage
danlu1 9d923ff
remove redirect and reorganize args and kwargs
danlu1 b362230
set default values for get_wiki_history
danlu1 02b509a
add async and sync unit test for wiki model
danlu1 62f03d4
remove redirect params
danlu1 e56540e
update presigned url provider params for multithread downloader
danlu1 a46417c
remove duplicate expired presigned url checking
danlu1 5058b03
Merge branch 'develop' into synpy-1351-oop-wiki2
danlu1 7b8269b
refactor fill_from_dict, add unzip file function, use logger.info ins…
danlu1 60d06db
remove unwanted changes
danlu1 ef18316
added and updated test cases
danlu1 4079eb1
add examples
danlu1 b43175e
merge upstream changes
danlu1 0e7ea8c
add static method to unzip file and reformat attachment name, allow g…
danlu1 efca157
update tutorials as changes in code
danlu1 5e1bb91
disable redirection when pulling markdown and attachment url
danlu1 487f0b3
add integration tests
danlu1 b253eb9
publish wiki docs
danlu1 508de6c
update tutorial docs
danlu1 2dd1b68
updated unit tests
danlu1 b3b2716
fix typo and update message
danlu1 ed3522a
update example
danlu1 8b5406b
update rest_apis.md as wiki_services has been added
danlu1 fa7e332
rename the files and extend docstrings
danlu1 06bf5d9
update method reference
danlu1 51ddcb5
add reference pages
danlu1 be5586e
update module name
danlu1 216217d
add wiki reference to navigation in mkdocs.yml
BryanFauble bc2a4a4
add helper functions
danlu1 76ba272
Merge branch 'synpy-1351-oop-wiki2' of https://github.com/Sage-Bionet…
danlu1 f043033
use rest_get_paginated_async in api_client instead
danlu1 8d55729
add synapse_client
danlu1 2a98516
add wiki methods in Synapse
danlu1 79b5fc0
update wikipage protocol as the introduction of generator in outputs
danlu1 9457541
add code section formatter and remove redundant in-line comments
danlu1 2e0d4b8
refactor _cleanup by simplifying the logic and add wiki entities
danlu1 8301176
refactor test suites
danlu1 d673f33
remove unwanted changes
danlu1 a764d78
Merge branch 'develop' into synpy-1351-oop-wiki2
danlu1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| # Wiki | ||
|
|
||
| Contained within this file are experimental interfaces for working with the Synapse Python | ||
| Client. Unless otherwise noted these interfaces are subject to change at any time. Use | ||
| at your own risk. | ||
|
|
||
| ## API reference | ||
|
|
||
| ::: synapseclient.models.WikiOrderHint | ||
| options: | ||
| inherited_members: true | ||
| members: | ||
| - store_async | ||
| - get_async | ||
| ::: synapseclient.models.WikiHistorySnapshot | ||
| options: | ||
| inherited_members: true | ||
| members: | ||
| - get_async | ||
| ::: synapseclient.models.WikiHeader | ||
| options: | ||
| inherited_members: true | ||
| members: | ||
| - get_async | ||
| ::: synapseclient.models.WikiPage | ||
| options: | ||
| inherited_members: true | ||
| members: | ||
| - store_async | ||
| - restore_async | ||
| - get_async | ||
| - delete_async | ||
| - get_attachment_handles_async | ||
| - get_attachment_async | ||
| - get_attachment_preview_async | ||
| - get_markdown_file_async | ||
| - reformat_attachment_file_name | ||
| - unzip_gzipped_file | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| [](){ #wiki-reference-sync } | ||
| # Wiki | ||
| Contained within this file are experimental interfaces for working with the Synapse Python | ||
| Client. Unless otherwise noted these interfaces are subject to change at any time. Use | ||
| at your own risk. | ||
|
|
||
| ## API reference | ||
|
|
||
| ::: synapseclient.models.WikiOrderHint | ||
| options: | ||
| inherited_members: true | ||
| members: | ||
| - store | ||
| - get | ||
| ::: synapseclient.models.WikiHistorySnapshot | ||
| options: | ||
| inherited_members: true | ||
| members: | ||
| - get | ||
| ::: synapseclient.models.WikiHeader | ||
| options: | ||
| inherited_members: true | ||
| members: | ||
| - get | ||
| ::: synapseclient.models.WikiPage | ||
| options: | ||
| inherited_members: true | ||
| members: | ||
| - store | ||
| - restore | ||
| - get | ||
| - delete | ||
| - get_attachment_handles | ||
| - get_attachment | ||
| - get_attachment_preview | ||
| - get_markdown_file | ||
| - reformat_attachment_file_name | ||
| - unzip_gzipped_file |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,2 @@ | ||
| ::: synapseclient.wiki | ||
| ::: synapseclient.models.wiki | ||
danlu1 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,266 @@ | ||
| #!/usr/bin/env python3 | ||
| """ | ||
| Tutorial script demonstrating the Synapse Wiki models functionality. | ||
|
|
||
| This script shows how to: | ||
| 1. Create, read, update, and restore wiki pages | ||
| 2. Create wiki pages from markdown text and files, and download markdown content | ||
| 3. Create wiki pages with attachments, retrieve attachment handles and URLs, and download attachments and previews | ||
| 4. Retrieve wiki page hierarchy using WikiHeader | ||
| 5. Access wiki version history using WikiHistorySnapshot | ||
| 6. Get, set, and update wiki page ordering using WikiOrderHint | ||
| 7. Delete wiki pages | ||
|
|
||
| """ | ||
| import os | ||
|
|
||
| from synapseclient import Synapse | ||
| from synapseclient.models import ( | ||
| Project, | ||
| WikiHeader, | ||
| WikiHistorySnapshot, | ||
| WikiOrderHint, | ||
| WikiPage, | ||
| ) | ||
|
|
||
| syn = Synapse() | ||
| syn.login() | ||
|
|
||
| # Get the project | ||
| project = Project(name="My uniquely named project about Alzheimer's Disease").get() | ||
|
|
||
| # Section1: Create, read, and update wiki pages | ||
| # Create a new wiki page for the project with plain text markdown | ||
| root_wiki_page = WikiPage( | ||
| owner_id=project.id, | ||
| title="My Root Wiki Page", | ||
| markdown="# Welcome to My Root Wiki\n\nThis is a sample root wiki page created with the Synapse client.", | ||
| ).store() | ||
|
|
||
| # OR you can create a wiki page with an existing markdown file. More instructions can be found in section 2. | ||
| markdown_file_path = "path/to/your_markdown_file.md" | ||
| root_wiki_page = WikiPage( | ||
| owner_id=project.id, | ||
| title="My First Root Wiki Page Version with existing markdown file", | ||
| markdown=markdown_file_path, | ||
| ).store() | ||
|
|
||
| # Update the wiki page | ||
| root_wiki_page_new = WikiPage( | ||
| owner_id=project.id, | ||
| title="My First Root Wiki Page NEW", | ||
| markdown="# Welcome to My Root Wiki NEW\n\nThis is a sample root wiki page created with the Synapse client.", | ||
| id=root_wiki_page.id, | ||
| ).store() | ||
|
|
||
| # Restore the wiki page to the original version | ||
| wiki_page_restored = WikiPage( | ||
| owner_id=project.id, id=root_wiki_page.id, wiki_version="0" | ||
| ).restore() | ||
|
|
||
| # check if the content is restored | ||
| assert ( | ||
| root_wiki_page.markdown_file_handle_id == wiki_page_restored.markdown_file_handle_id | ||
| ), "Markdown file handle ID does not match after restore" | ||
| assert ( | ||
| root_wiki_page.id == wiki_page_restored.id | ||
| ), "Wiki page ID does not match after restore" | ||
| assert ( | ||
| root_wiki_page.title == wiki_page_restored.title | ||
| ), "Wiki page title does not match after restore" | ||
|
|
||
| # Get the wiki page | ||
| # Once you know the Wiki page id, you can retrieve the Wiki page with the id | ||
| retrieved_wiki = WikiPage(owner_id=project.id, id=root_wiki_page.id).get() | ||
|
|
||
| # Or you can retrieve the Wiki page with the title | ||
| retrieved_wiki = WikiPage(owner_id=project.id, title=root_wiki_page.title).get() | ||
|
|
||
| # Check if the retrieved Wiki page is the same as the original Wiki page | ||
| assert ( | ||
| root_wiki_page.markdown_file_handle_id == retrieved_wiki.markdown_file_handle_id | ||
| ), "Markdown file handle ID does not match retrieved wiki page" | ||
| assert ( | ||
| root_wiki_page.id == retrieved_wiki.id | ||
| ), "Wiki page ID does not match retrieved wiki page" | ||
| assert ( | ||
| root_wiki_page.title == retrieved_wiki.title | ||
| ), "Wiki page title does not match retrieved wiki page" | ||
|
|
||
| # Create a sub-wiki page | ||
| sub_wiki_1 = WikiPage( | ||
| owner_id=project.id, | ||
| title="Sub Wiki Page 1", | ||
| parent_id=root_wiki_page.id, | ||
| markdown="# Sub Page 1\n\nThis is a sub-page of another wiki.", | ||
danlu1 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ).store() | ||
|
|
||
| # Section 2: WikiPage Markdown Operations | ||
| # Create wiki page from markdown text | ||
| markdown_content = """# Sample Markdown Content | ||
|
|
||
| ## Section 1 | ||
| This is a sample markdown file with multiple sections. | ||
|
|
||
| ## Section 2 | ||
| - List item 1 | ||
| - List item 2 | ||
| - List item 3 | ||
|
|
||
| ## Code Example | ||
| ```python | ||
| def hello_world(): | ||
| print("Hello, World!") | ||
| ``` | ||
| """ | ||
|
|
||
| # Create wiki page from markdown text | ||
| sub_wiki_2 = WikiPage( | ||
| owner_id=project.id, | ||
| parent_id=root_wiki_page.id, | ||
| title="Sub Page 2 created from markdown text", | ||
| markdown=markdown_content, | ||
| ).store() | ||
|
|
||
| # Create a wiki page from a markdown file | ||
| markdown_file_path = "~/temp/temp_markdown_file.md.gz" | ||
|
|
||
| # Create wiki page from markdown file | ||
| sub_wiki_3 = WikiPage( | ||
| owner_id=project.id, | ||
| parent_id=root_wiki_page.id, | ||
| title="Sub Page 3 created from markdown file", | ||
| markdown=markdown_file_path, | ||
| ).store() | ||
|
|
||
| # Download the markdown file | ||
| # Note: If the markdown is generated from plain text using the client, the downloaded file will be named wiki_markdown_<wiki_page_title>.md.gz. If it is generated from an existing markdown file, the downloaded file will retain the original filename. | ||
| # Download the markdown file for sub_wiki_2 that is created from markdown text | ||
| wiki_page_markdown_2_url = WikiPage( | ||
| owner_id=project.id, | ||
| id=sub_wiki_2.id, | ||
| ).get_markdown_file( | ||
| download_file=False, | ||
| ) | ||
|
|
||
| # Download the markdown file for sub_wiki_2 that is created from markdown text | ||
| wiki_page_markdown_2 = WikiPage( | ||
| owner_id=project.id, id=sub_wiki_2.id | ||
| ).get_markdown_file(download_file=True, download_location=".") | ||
|
|
||
| # Download the markdown file for sub_wiki_3 that is created from a markdown file | ||
| wiki_page_markdown_3 = WikiPage( | ||
| owner_id=project.id, id=sub_wiki_3.id | ||
| ).get_markdown_file(download_file=True, download_location=".") | ||
|
|
||
| # Section 3: WikiPage with Attachments | ||
| # Create a temporary file for the attachment | ||
| attachment_file_name = "temp_attachment.txt" | ||
|
|
||
| # reformat '.' and '_' in the attachment file name to be a valid attachment path | ||
danlu1 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| attachment_file_name_reformatted = WikiPage.reformat_attachment_file_name( | ||
| os.path.basename(attachment_file_name) | ||
| ) | ||
| sub_wiki_4 = WikiPage( | ||
| owner_id=project.id, | ||
| parent_id=root_wiki_page.id, | ||
| title="Sub Page 4 with Attachments", | ||
| markdown=f"# Sub Page 4 with Attachments\n\nThis is a attachment: ${{previewattachment?fileName={attachment_file_name_reformatted}}}", | ||
| attachments=[attachment_file_name], | ||
| ).store() | ||
|
|
||
| # Inlucde images in the markdown file | ||
| image_file_path = "path/to/test_image.png" | ||
| # use the original file name instead of the gzipped file name for images | ||
| image_file_name = os.path.basename(image_file_path) | ||
| markdown_content = f"# Sub Page 5 with images\n\nThis is an attached image: ${{image?fileName=test_image.png&align=None&scale=100&responsive=true&altText=}}" | ||
| sub_wiki_5 = WikiPage( | ||
| owner_id=project.id, | ||
| parent_id=root_wiki_page.id, | ||
| title="Sub Page 5 with Images", | ||
| markdown=markdown_content, | ||
| attachments=[image_file_path], | ||
| ).store() | ||
|
|
||
| # Get attachment handles | ||
| attachment_handles = WikiPage( | ||
| owner_id=project.id, id=sub_wiki_4.id | ||
| ).get_attachment_handles() | ||
|
|
||
| # Get attachment URL without downloading | ||
| wiki_page_attachment_url = WikiPage( | ||
| owner_id=project.id, id=sub_wiki_4.id | ||
| ).get_attachment( | ||
| file_name=os.path.basename(attachment_file_name), | ||
| download_file=False, | ||
| ) | ||
|
|
||
| # Download an attachment | ||
| wiki_page_attachment = WikiPage(owner_id=project.id, id=sub_wiki_4.id).get_attachment( | ||
| file_name=os.path.basename(attachment_file_name), | ||
| download_file=True, | ||
| download_location=".", | ||
| ) | ||
| # Unzip the attachment file | ||
| unzipped_attachment_file_path = WikiPage.unzip_gzipped_file(wiki_page_attachment) | ||
|
|
||
| # Download an attachment preview. Instead of using the file_name from the attachmenthandle response when isPreview=True, you should use the original file name in the get_attachment_preview request. The downloaded file will still be named according to the file_name provided in the response when isPreview=True. | ||
| # Get attachment preview URL without downloading | ||
| attachment_preview_url = WikiPage( | ||
| owner_id=project.id, id=sub_wiki_4.id | ||
| ).get_attachment_preview( | ||
| file_name=os.path.basename(attachment_file_name), | ||
| download_file=False, | ||
| ) | ||
|
|
||
| # Download an attachment preview | ||
| attachment_preview = WikiPage( | ||
| owner_id=project.id, id=sub_wiki_4.id | ||
| ).get_attachment_preview( | ||
| file_name=os.path.basename(attachment_file_name), | ||
| download_file=True, | ||
| download_location=".", | ||
| ) | ||
|
|
||
| # Section 4: WikiHeader - Working with Wiki Hierarchy | ||
| # Get wiki header tree (hierarchy) | ||
| headers = WikiHeader.get(owner_id=project.id) | ||
|
|
||
| # Section 5. WikiHistorySnapshot - Version History | ||
| # Get wiki history for root_wiki_page | ||
| history = WikiHistorySnapshot.get(owner_id=project.id, id=root_wiki_page.id) | ||
|
|
||
| # Section 6. WikiOrderHint - Ordering Wiki Pages | ||
| # Set the wiki order hint | ||
| order_hint = WikiOrderHint(owner_id=project.id).get() | ||
| # As you can see from the printed message, the order hint is not set by default, so you need to set it explicitly at the beginning. | ||
| order_hint.id_list = [ | ||
| root_wiki_page.id, | ||
| sub_wiki_3.id, | ||
| sub_wiki_4.id, | ||
| sub_wiki_1.id, | ||
| sub_wiki_2.id, | ||
| sub_wiki_5.id, | ||
| ] | ||
| order_hint.store() | ||
|
|
||
| # Update wiki order hint | ||
| order_hint = WikiOrderHint(owner_id=project.id).get() | ||
| order_hint.id_list = [ | ||
| root_wiki_page.id, | ||
| sub_wiki_1.id, | ||
| sub_wiki_2.id, | ||
| sub_wiki_3.id, | ||
| sub_wiki_4.id, | ||
| sub_wiki_5.id, | ||
| ] | ||
| order_hint.store() | ||
|
|
||
| # Delete a wiki page | ||
| sub_wiki_6 = WikiPage( | ||
| owner_id=project.id, | ||
| parent_id=root_wiki_page.id, | ||
| title="Sub Page 6 to be deleted", | ||
| markdown=f"# Sub Page 6 to be deleted\n\nThis is a sub page to be deleted.", | ||
| ).store() | ||
| wiki_page_to_delete = WikiPage(owner_id=project.id, id=sub_wiki_6.id).delete() | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.