v3.0.0-rc

What's Changed

• update name of utils function: is_synapse_id by @vpchung in #949
• SYNPY-1248: add is_synapse_id function to the client by @vpchung in #950
• [SYNPY-1241] Don't remove files from downloadCart if empty list of files by @thomasyu888 in #944
• [SYNPY-1242] Refactor _loggedIn function by @thomasyu888 in #945
• [SYNPY-1246] Add Python 3.11 support by @thomasyu888 in #951
• [SYNPY-737] Remove non-standard single dash long cli parameters by @thomasyu888 in #883
• [SYNPY-1255] Update pandas to support 2.0 and deprecate support for python 3.7 by @thomasyu888 in #954
• [SYNPY-1266] Add support for GCP bucket by @thomasyu888 in #959
• [SYNPY-1229, SYNPY-1230, SYNPY-1231] Lint synapseclient and synapseutils by @thomasyu888 in #961
• Add pre-commit by @thomasyu888 in #963
• [SYNPY-1227] Lint all code with black by @thomasyu888 in #965
• [SYNPY-570] Make sure warning doesn't appear ifcollision is overwrite.local by @thomasyu888 in #967
• [SYNPY-622] Improve evaluation.py syntax by @thomasyu888 in #968
• [SYNPY-1269] Edit synapse config file to use auth token and edit tests by @thomasyu888 in #970
• [Bug fix]: Fixed test related to pandas issue on window 2019 by @linglp in #971
• [SYNPY-1270] Deprecated parameter date_parser to support pandas2.0.0 and above by @linglp in #973
• [SYNPY-1199] Implement low-level functions for JSON Schema API by @BrunoGrandePhD in #894
• pkg_resources is a deprecated API - replace with importlib.resources by @thomasyu888 in #975
• [SYNPY-645] improve file entity name docs by @thomasyu888 in #969
• [SYNPY-716] Remove asInteger() feature by @thomasyu888 in #962
• Clean up setup.py by using setup.cfg by @thomasyu888 in #974
• [SYNPY-1213, SYNPY-1245] Improve doc site by @thomasyu888 in #946

New Contributors

• @linglp made their first contribution in #971

Full Changelog: v2.7.2...v3.0.0-rc

v2.7.2

What's Changed

• [SYNPY-1267] Lock down urllib3 by @thomasyu888 in #958
• [SYNPY-1268] Add deprecation warning for non-supported login arguments by @thomasyu888 in #960

Full Changelog: v2.7.1...v2.7.2

v2.7.1

What's Changed

• Mint patch release: Lock down pandas version since pandas >1.5 doesn't work by @thomasyu888 in #955
• [SYNPY-1257] Hotfix - upgrade github actions by @thomasyu888 in #956

Full Changelog: v2.7.0...v2.7.1

v2.7.0

Bug Fixes

• [SYNPY-226] - isConsistent fails as parameter for table query
• [SYNPY-562] - Make sure SQL functions, including "year", are quoted correctly
• [SYNPY-1031] - File version increments with 400 client error
• [SYNPY-1219] - Update Entity class to be compatible with the new Dataset entity
• [SYNPY-1224] - Correct SynapseUnmetAccessRestrictions message
• [SYNPY-1237] - as_table_columns function is mishandling mixed data types

Stories

• [SYNPY-63] - py: use metaclass to replace the _entity_type_to_class hack
• [SYNPY-992] - synapseutils changeFileMetadata missing syn parameter docstring
• [SYNPY-1175] - Programmatic Support for Download V2 via Py Client
• [SYNPY-1193] - Support Datasets functionality
• [SYNPY-1221] - Set up gh-action: black, the python auto formatter on the python client

Sub-Tasks

• [SYNPY-1207] - Support syn.get() on a dataset

Tasks

• [SYNPY-566] - Clarify expected list format for sync manifest
• [SYNPY-1053] - Increase documentation of forceVersion in syncToSynapse
• [SYNPY-1145] - Link to manifest format in CLI sync command usage help
• [SYNPY-1226] - Leverage ViewBase for Datasets instead of SchemaBase
• [SYNPY-1235] - Create codeql scanning workflow

Contributors

• @danlu1
• @thomasyu888
• @allaway
• @vpchung

New Contributors

• @vpchung made their first contribution in #914

Full Changelog: v2.6.0...v2.7.0

v2.6.0

Bug Fixes

• [SYNPY-1204] - Python 3.10 compatibility

Stories

• [SYNPY-728] - Improve error message when pandas is not available
• [SYNPY-974] - Documentation for generateManifest
• [SYNPY-1209] - Support for MaterializedViews in Py Client

Tasks

• [SYNPY-1174] - Add function to create Synapse config file
• [SYNPY-1176] - syncToSynapse aborted + silent failure of file upload
• [SYNPY-1184] - Add includeTypes to synapseutils.walk()
• [SYNPY-1189] - Document "maximumListLength" parameter for Column
• [SYNPY-1196] - Expose forceVersion on changeFileMetadata
• [SYNPY-1205] - Python 3.6 EOL - Remove support for 3.6
• [SYNPY-1212] - Include dataset as an entity type to return in getChildren()

Contributors: @danlu1 , @BrunoGrandePhD , @thomasyu888

Full Changelog: v2.5.1...v2.6.0-rc

v2.6.0-rc

Bug Fixes

• [SYNPY-1204] - Python 3.10 compatibility

Stories

• [SYNPY-728] - Improve error message when pandas is not available
• [SYNPY-974] - Documentation for generateManifest
• [SYNPY-1209] - Support for MaterializedViews in Py Client

Tasks

• [SYNPY-1174] - Add function to create Synapse config file
• [SYNPY-1176] - syncToSynapse aborted + silent failure of file upload
• [SYNPY-1184] - Add includeTypes to synapseutils.walk()
• [SYNPY-1189] - Document "maximumListLength" parameter for Column
• [SYNPY-1196] - Expose forceVersion on changeFileMetadata
• [SYNPY-1205] - Python 3.6 EOL - Remove support for 3.6
• [SYNPY-1212] - Include dataset as an entity type to return in getChildren()

Contributors: @danlu1 , @BrunoGrandePhD , @thomasyu888

Full Changelog: v2.5.1...v2.6.0-rc

v2.5.1

Bug Fixes

• [SYNPY-1197] - Schema is a string and strings don't have columns_to_store attributes

Stories

• [SYNPY-772] - update statement that appears on PyPi about Synapse to be consistent
• [SYNPY-997] - Typos in Views documentation

Contributors: @thomasyu888, @danlu1

New Contributors

• @danlu1 made their first contribution in #892

GitHub Changelog: v2.5.0...v2.5.1

v2.5.0

Highlights

• Added ability to generate a manifest file from your local directory structure.

  # from the command line
  # write the manifest to manifest.tsv
  synapse manifest --parent-id syn123 --manifest-file ./manifest.tsv /path/to/local/directory
  # stdout
  synapse manifest --parent-id syn123 /path/to/local/directory
  

• Added ability to pipe manifest stdout into sync function.

  # from the command line
  synapse manifest --parent-id syn123 ./docs/ | synapse sync -
  

• Added ability to return summary statistics of csv and tsv files stored in Synapse.

  # from python
  import synapseclient
  import synapseutils
  syn = synapseclient.login()
  statistics = synapseutils.describe(syn=syn, entity="syn12345")
  print(statistics)
  {
      "column1": {
          "dtype": "object",
          "mode": "FOOBAR"
      },
      "column2": {
          "dtype": "int64",
          "mode": 1,
          "min": 1,
          "max": 2,
          "mean": 1.4
      },
      "column3": {
          "dtype": "bool",
          "mode": false,
          "min": false,
          "max": true,
          "mean": 0.5
      }
  }
  

• Next major release (3.0.0) there will be major cosmetic changes to the cli such as removing all camel case or non-standard single dash long command line interface (cli) parameters.
  Example: command line arguments like -parent will become --parent. Commands that support camel case like --parentId will be changed to --parent-id.

Bug Fixes

• [SYNPY-669] - Signing of Synapse authentication header does not correctly URL encode the URL path
• [SYNPY-770] - Files failing to upload using syncToSynapse
• [SYNPY-1123] - All tables erroring when indexing
• [SYNPY-1146] - Error writing Booleans from Python dataframes into Boolean columns in a Synapse table
• [SYNPY-1156] - datetimes in a Pandas dataframe are not properly stored to Synapse

Stories

• [SYNPY-726] - mirror local folder structure for bulk upload
• [SYNPY-1163] - Expose synId with syn get -r
• [SYNPY-1165] - Generate manifest template from local folder structure
• [SYNPY-1167] - Support for Quick Summary Statistics on CSV and TSV files

Tasks

• [SYNPY-1169] - Integration tests failures in develop branch against stack-371
• [SYNPY-1172] - Passing a pandas dataframe with a column called "read" breaks the type parsing in as_table_columns()
• [SYNPY-1173] - Support DATE_LIST, ENTITYID_LIST, USERID_LIST table columns
• [SYNPY-1188] - Support piping of synapse manifest stdout in synapse sync function

@jkiang13, @linchiahuisage, @esurface, @thomasyu888, @BrunoGrandePhD, @mattfazza

New Contributors

• @esurface made their first contribution in #876
• @mattfazza made their first contribution in #877

Full Changelog: v2.4.0...v2.5.0

v2.4.0

Highlights

• Added ability to authenticate from a SYNAPSE_AUTH_TOKEN environment variable set with a valid personal access token.

# e.g. set environment variable prior to invoking a Synapse command or running a program that uses synapseclient
SYNAPSE_AUTH_TOKEN='<my_personal_access_token>' synapse <subcommand options>

The environment variable will take priority over credentials in the user’s .synapseConfig file or any credentials saved in a prior login using the remember me option.

See here for more details on usage.

• Added ability to silence all console output.

# from the command line, use the --silent option with any synapse subcommand, here it will suppress the download progress indicator
synapse --silent get <synid>

# from code using synapseclient, pass the silent option to the Synapse constructor
import synapseclient

syn = synapseclient.Synapse(silent=True)
syn.login()
syn.get(<synid>)

• Improved robustness during downloads with unstable connections. Specifically the client will automatically recover when encoutering some types of network errors that previously would have caused a download to start over as indicated by a reset progress bar.

Bug Fixes

• [SYNPY-198] - get: Unmet access requirement should not raise error if entity not downloadable
• [SYNPY-959] - FileEntity 'path' property has wrong separator in Windows
• [SYNPY-1113] - Confusing error when putting the positional FILE at the end of the synapse store command with an optional n-arg
• [SYNPY-1128] - failures downloading 14G vcf file
• [SYNPY-1130] - Migration tool trying to move URL-linked data
• [SYNPY-1134] - 500 error during part copy to AWS presigned url
• [SYNPY-1135] - Exceeding part limit during AD Migration
• [SYNPY-1136] - Connection aborted to AWS part copy to presigned url during AD Migration
• [SYNPY-1141] - synapse get command line nargs usage/error
• [SYNPY-1150] - Boolean-like string columns being reformatted (TRUE/FALSE to True/False)
• [SYNPY-1158] - race condition in test_caching.py#test_threaded_access
• [SYNPY-1159] - logging in with an email address and an authToken gives spurious error
• [SYNPY-1161] - ChunkEncodingError encountered from external collaborator during a synapseclient download

Improvements

• [SYNPY-638] - add after date to cache purge
• [SYNPY-929] - silent parameter for all functions which default to writing to stdout
• [SYNPY-1068] - Should show some progress indicator during upload md5 calculation
• [SYNPY-1125] - Allow login with environment variables

Stories

• [SYNPY-1138] - When using boto3 client to upload a file, also include ACL to give bucket owner full access

Tasks

• [SYNPY-948] - command line client set-annotations does not return proper error code when there's a problem
• [SYNPY-1024] - remove reference to deprecated 'status' field from Evaluation
• [SYNPY-1115] - Setting provenance in store ignored for tables
• [SYNPY-1143] - indicate in CLI doc's that select statement requires double quotes

v2.3.1

Highlights

• Entities can be annotated with boolean datatypes, e.g.:

  file = synapseclient.File('/path/to/file', parentId='syn123', synapse_is_great=True)
  syn.store(file)

• synapseclient is additionally packaged as a Python wheel.

Bug Fixes

• [SYNPY-829] - syn.store always updates annotations
• [SYNPY-1033] - If versionComment is left blank, previous version comment populates

Improvements

• [SYNPY-1120] - Build wheel distributions
• [SYNPY-1129] - support boolean annotations in Python client