v2.7.2

What's Changed

• [SYNPY-1267] Lock down urllib3 by @thomasyu888 in #958
• [SYNPY-1268] Add deprecation warning for non-supported login arguments by @thomasyu888 in #960

Full Changelog: v2.7.1...v2.7.2

v2.7.1

What's Changed

• Mint patch release: Lock down pandas version since pandas >1.5 doesn't work by @thomasyu888 in #955
• [SYNPY-1257] Hotfix - upgrade github actions by @thomasyu888 in #956

Full Changelog: v2.7.0...v2.7.1

v2.7.0

Bug Fixes

• [SYNPY-226] - isConsistent fails as parameter for table query
• [SYNPY-562] - Make sure SQL functions, including "year", are quoted correctly
• [SYNPY-1031] - File version increments with 400 client error
• [SYNPY-1219] - Update Entity class to be compatible with the new Dataset entity
• [SYNPY-1224] - Correct SynapseUnmetAccessRestrictions message
• [SYNPY-1237] - as_table_columns function is mishandling mixed data types

Stories

• [SYNPY-63] - py: use metaclass to replace the _entity_type_to_class hack
• [SYNPY-992] - synapseutils changeFileMetadata missing syn parameter docstring
• [SYNPY-1175] - Programmatic Support for Download V2 via Py Client
• [SYNPY-1193] - Support Datasets functionality
• [SYNPY-1221] - Set up gh-action: black, the python auto formatter on the python client

Sub-Tasks

• [SYNPY-1207] - Support syn.get() on a dataset

Tasks

• [SYNPY-566] - Clarify expected list format for sync manifest
• [SYNPY-1053] - Increase documentation of forceVersion in syncToSynapse
• [SYNPY-1145] - Link to manifest format in CLI sync command usage help
• [SYNPY-1226] - Leverage ViewBase for Datasets instead of SchemaBase
• [SYNPY-1235] - Create codeql scanning workflow

Contributors

• @danlu1
• @thomasyu888
• @allaway
• @vpchung

New Contributors

• @vpchung made their first contribution in #914

Full Changelog: v2.6.0...v2.7.0

v2.6.0

Bug Fixes

• [SYNPY-1204] - Python 3.10 compatibility

Stories

• [SYNPY-728] - Improve error message when pandas is not available
• [SYNPY-974] - Documentation for generateManifest
• [SYNPY-1209] - Support for MaterializedViews in Py Client

Tasks

• [SYNPY-1174] - Add function to create Synapse config file
• [SYNPY-1176] - syncToSynapse aborted + silent failure of file upload
• [SYNPY-1184] - Add includeTypes to synapseutils.walk()
• [SYNPY-1189] - Document "maximumListLength" parameter for Column
• [SYNPY-1196] - Expose forceVersion on changeFileMetadata
• [SYNPY-1205] - Python 3.6 EOL - Remove support for 3.6
• [SYNPY-1212] - Include dataset as an entity type to return in getChildren()

Contributors: @danlu1 , @BrunoGrandePhD , @thomasyu888

Full Changelog: v2.5.1...v2.6.0-rc

v2.6.0-rc

Bug Fixes

• [SYNPY-1204] - Python 3.10 compatibility

Stories

• [SYNPY-728] - Improve error message when pandas is not available
• [SYNPY-974] - Documentation for generateManifest
• [SYNPY-1209] - Support for MaterializedViews in Py Client

Tasks

• [SYNPY-1174] - Add function to create Synapse config file
• [SYNPY-1176] - syncToSynapse aborted + silent failure of file upload
• [SYNPY-1184] - Add includeTypes to synapseutils.walk()
• [SYNPY-1189] - Document "maximumListLength" parameter for Column
• [SYNPY-1196] - Expose forceVersion on changeFileMetadata
• [SYNPY-1205] - Python 3.6 EOL - Remove support for 3.6
• [SYNPY-1212] - Include dataset as an entity type to return in getChildren()

Contributors: @danlu1 , @BrunoGrandePhD , @thomasyu888

Full Changelog: v2.5.1...v2.6.0-rc

v2.5.1

Bug Fixes

• [SYNPY-1197] - Schema is a string and strings don't have columns_to_store attributes

Stories

• [SYNPY-772] - update statement that appears on PyPi about Synapse to be consistent
• [SYNPY-997] - Typos in Views documentation

Contributors: @thomasyu888, @danlu1

New Contributors

• @danlu1 made their first contribution in #892

GitHub Changelog: v2.5.0...v2.5.1

v2.5.0

Highlights

• Added ability to generate a manifest file from your local directory structure.

  # from the command line
  # write the manifest to manifest.tsv
  synapse manifest --parent-id syn123 --manifest-file ./manifest.tsv /path/to/local/directory
  # stdout
  synapse manifest --parent-id syn123 /path/to/local/directory
  

• Added ability to pipe manifest stdout into sync function.

  # from the command line
  synapse manifest --parent-id syn123 ./docs/ | synapse sync -
  

• Added ability to return summary statistics of csv and tsv files stored in Synapse.

  # from python
  import synapseclient
  import synapseutils
  syn = synapseclient.login()
  statistics = synapseutils.describe(syn=syn, entity="syn12345")
  print(statistics)
  {
      "column1": {
          "dtype": "object",
          "mode": "FOOBAR"
      },
      "column2": {
          "dtype": "int64",
          "mode": 1,
          "min": 1,
          "max": 2,
          "mean": 1.4
      },
      "column3": {
          "dtype": "bool",
          "mode": false,
          "min": false,
          "max": true,
          "mean": 0.5
      }
  }
  

• Next major release (3.0.0) there will be major cosmetic changes to the cli such as removing all camel case or non-standard single dash long command line interface (cli) parameters.
  Example: command line arguments like -parent will become --parent. Commands that support camel case like --parentId will be changed to --parent-id.

Bug Fixes

• [SYNPY-669] - Signing of Synapse authentication header does not correctly URL encode the URL path
• [SYNPY-770] - Files failing to upload using syncToSynapse
• [SYNPY-1123] - All tables erroring when indexing
• [SYNPY-1146] - Error writing Booleans from Python dataframes into Boolean columns in a Synapse table
• [SYNPY-1156] - datetimes in a Pandas dataframe are not properly stored to Synapse

Stories

• [SYNPY-726] - mirror local folder structure for bulk upload
• [SYNPY-1163] - Expose synId with syn get -r
• [SYNPY-1165] - Generate manifest template from local folder structure
• [SYNPY-1167] - Support for Quick Summary Statistics on CSV and TSV files

Tasks

• [SYNPY-1169] - Integration tests failures in develop branch against stack-371
• [SYNPY-1172] - Passing a pandas dataframe with a column called "read" breaks the type parsing in as_table_columns()
• [SYNPY-1173] - Support DATE_LIST, ENTITYID_LIST, USERID_LIST table columns
• [SYNPY-1188] - Support piping of synapse manifest stdout in synapse sync function

@jkiang13, @linchiahuisage, @esurface, @thomasyu888, @BrunoGrandePhD, @mattfazza

New Contributors

• @esurface made their first contribution in #876
• @mattfazza made their first contribution in #877

Full Changelog: v2.4.0...v2.5.0

v2.4.0

Highlights

• Added ability to authenticate from a SYNAPSE_AUTH_TOKEN environment variable set with a valid personal access token.

# e.g. set environment variable prior to invoking a Synapse command or running a program that uses synapseclient
SYNAPSE_AUTH_TOKEN='<my_personal_access_token>' synapse <subcommand options>

The environment variable will take priority over credentials in the user’s .synapseConfig file or any credentials saved in a prior login using the remember me option.

See here for more details on usage.

• Added ability to silence all console output.

# from the command line, use the --silent option with any synapse subcommand, here it will suppress the download progress indicator
synapse --silent get <synid>

# from code using synapseclient, pass the silent option to the Synapse constructor
import synapseclient

syn = synapseclient.Synapse(silent=True)
syn.login()
syn.get(<synid>)

• Improved robustness during downloads with unstable connections. Specifically the client will automatically recover when encoutering some types of network errors that previously would have caused a download to start over as indicated by a reset progress bar.

Bug Fixes

• [SYNPY-198] - get: Unmet access requirement should not raise error if entity not downloadable
• [SYNPY-959] - FileEntity 'path' property has wrong separator in Windows
• [SYNPY-1113] - Confusing error when putting the positional FILE at the end of the synapse store command with an optional n-arg
• [SYNPY-1128] - failures downloading 14G vcf file
• [SYNPY-1130] - Migration tool trying to move URL-linked data
• [SYNPY-1134] - 500 error during part copy to AWS presigned url
• [SYNPY-1135] - Exceeding part limit during AD Migration
• [SYNPY-1136] - Connection aborted to AWS part copy to presigned url during AD Migration
• [SYNPY-1141] - synapse get command line nargs usage/error
• [SYNPY-1150] - Boolean-like string columns being reformatted (TRUE/FALSE to True/False)
• [SYNPY-1158] - race condition in test_caching.py#test_threaded_access
• [SYNPY-1159] - logging in with an email address and an authToken gives spurious error
• [SYNPY-1161] - ChunkEncodingError encountered from external collaborator during a synapseclient download

Improvements

• [SYNPY-638] - add after date to cache purge
• [SYNPY-929] - silent parameter for all functions which default to writing to stdout
• [SYNPY-1068] - Should show some progress indicator during upload md5 calculation
• [SYNPY-1125] - Allow login with environment variables

Stories

• [SYNPY-1138] - When using boto3 client to upload a file, also include ACL to give bucket owner full access

Tasks

• [SYNPY-948] - command line client set-annotations does not return proper error code when there's a problem
• [SYNPY-1024] - remove reference to deprecated 'status' field from Evaluation
• [SYNPY-1115] - Setting provenance in store ignored for tables
• [SYNPY-1143] - indicate in CLI doc's that select statement requires double quotes

v2.3.1

Highlights

• Entities can be annotated with boolean datatypes, e.g.:

  file = synapseclient.File('/path/to/file', parentId='syn123', synapse_is_great=True)
  syn.store(file)

• synapseclient is additionally packaged as a Python wheel.

Bug Fixes

• [SYNPY-829] - syn.store always updates annotations
• [SYNPY-1033] - If versionComment is left blank, previous version comment populates

Improvements

• [SYNPY-1120] - Build wheel distributions
• [SYNPY-1129] - support boolean annotations in Python client

v2.3.0

Highlights

• The index_files_for_migration and migrate_indexed_files functions are added to synapseutils to help migrate files in Synapse projects and folders between AWS S3 buckets in the same region. More details on using these utilities can be found here.

• This version supports login programatically and from the command line using personal access tokens that can be obtained from your synapse.org Settings. Additional documentation on login and be found here.

  # programmatic
  syn = synapseclient.login(authToken=<token>)

  # command line
  synapse login -p <token>

• The location where downloaded entities are cached can be customized to a location other than the user’s home directory. This is useful in environments where writing to a home directory is not appropriate (e.g. an AWS lambda).

  syn = synapseclient.Synapse(cache_root_dir=<directory path>)

• A helper method on the Synapse object has been added to enable obtaining the Synapse certification quiz status of a user.

  passed = syn.is_certified(<username or user_id>)

• This version has been tested with Python 3.9.

Bug Fixes

• [SYNPY-1039] - tableQuery asDataFrame() results with TYPE_LIST columns should be lists and not literal strings
• [SYNPY-1109] - unparseable synapse cacheMap raises JSONDecodeError
• [SYNPY-1110] - Cleanup on Windows console login
• [SYNPY-1112] - Concurrent migration of entities sharing the same file handle can result in an error
• [SYNPY-1114] - Mitigate new Rust compiler dependency on Linux via transitive cryptography dependency
• [SYNPY-1118] - Migration tool erroring when it shouldn’t

New Features

• [SYNPY-1058] - Accept oauth access token for authentication to use Synapse REST services
• [SYNPY-1103] - Multipart copy integration
• [SYNPY-1111] - Add function to get user certification status

Improvements

• [SYNPY-885] - Public interface to customize CACHE_ROOT_DIR
• [SYNPY-1102] - syncToSynapse adds empty annotation values
• [SYNPY-1104] - Python 3.9 support
• [SYNPY-1119] - Add source storage location option to storage migrate functions