v2.5.0

Highlights

• Added ability to generate a manifest file from your local directory structure.

  # from the command line
  # write the manifest to manifest.tsv
  synapse manifest --parent-id syn123 --manifest-file ./manifest.tsv /path/to/local/directory
  # stdout
  synapse manifest --parent-id syn123 /path/to/local/directory
  

• Added ability to pipe manifest stdout into sync function.

  # from the command line
  synapse manifest --parent-id syn123 ./docs/ | synapse sync -
  

• Added ability to return summary statistics of csv and tsv files stored in Synapse.

  # from python
  import synapseclient
  import synapseutils
  syn = synapseclient.login()
  statistics = synapseutils.describe(syn=syn, entity="syn12345")
  print(statistics)
  {
      "column1": {
          "dtype": "object",
          "mode": "FOOBAR"
      },
      "column2": {
          "dtype": "int64",
          "mode": 1,
          "min": 1,
          "max": 2,
          "mean": 1.4
      },
      "column3": {
          "dtype": "bool",
          "mode": false,
          "min": false,
          "max": true,
          "mean": 0.5
      }
  }
  

• Next major release (3.0.0) there will be major cosmetic changes to the cli such as removing all camel case or non-standard single dash long command line interface (cli) parameters.
  Example: command line arguments like -parent will become --parent. Commands that support camel case like --parentId will be changed to --parent-id.

Bug Fixes

• [SYNPY-669] - Signing of Synapse authentication header does not correctly URL encode the URL path
• [SYNPY-770] - Files failing to upload using syncToSynapse
• [SYNPY-1123] - All tables erroring when indexing
• [SYNPY-1146] - Error writing Booleans from Python dataframes into Boolean columns in a Synapse table
• [SYNPY-1156] - datetimes in a Pandas dataframe are not properly stored to Synapse

Stories

• [SYNPY-726] - mirror local folder structure for bulk upload
• [SYNPY-1163] - Expose synId with syn get -r
• [SYNPY-1165] - Generate manifest template from local folder structure
• [SYNPY-1167] - Support for Quick Summary Statistics on CSV and TSV files

Tasks

• [SYNPY-1169] - Integration tests failures in develop branch against stack-371
• [SYNPY-1172] - Passing a pandas dataframe with a column called "read" breaks the type parsing in as_table_columns()
• [SYNPY-1173] - Support DATE_LIST, ENTITYID_LIST, USERID_LIST table columns
• [SYNPY-1188] - Support piping of synapse manifest stdout in synapse sync function

@jkiang13, @linchiahuisage, @esurface, @thomasyu888, @BrunoGrandePhD, @mattfazza

New Contributors

• @esurface made their first contribution in #876
• @mattfazza made their first contribution in #877

Full Changelog: v2.4.0...v2.5.0