v0.2.0a2

This is the release notes of v0.2.0a2. See here for the complete list of solved issues and merged PRs.

Highlights

• Eager mode support, if eager mode is on, tensor will be executed immediately once it's created.
• Initial worker-level failover support.

New Features

• Replace executed tensor with fetch tensor (#180)
• Support eager mode (#200)
• Support sparse for some operators in mt.linalg (#215)
• Add TileDB support as external storage (#227)
• Add worker-level fail-over support for single graph (#207)

Enhancements

• Pre-push operands into worker when the worker can be estimated (#119)
• Separate graph analyzer from graph and add test cases (#193)
• Fix codacy problems (#204)
• Refactor executor to do operand-oriented execution and support actor operation (#212)
• Add __mars_tokenize__ protocol (#218)
• Manually add methods for Tensor class (#223)
• Use random object id in plasma store to avoid potential collision (#226)
• Assign initials given existing data in eager mode (#242)

Bug fixes

• Fix serializing graph with a tiled tensor (#191)
• Fix starting with multiple schedulers with --schedulers (#202)
• Fix process clean up when local cluster terminates (#206)
• Fix bug that TensorZeros generate same key even if their shapes are different in TensorDiag.tile (#210)
• Fix bug that chunks of ones and zeros with same shape generated different key (#216)
• Fix execute one of the multiple output tensors (#221)
• Add TensorInv operand to fix bug in mt.linalg.inv (#237)
• Fix executor's behavior of result deletions (#249)

Documentation

• Add contributing guide & remove tracing on tags (#181)

Installation

• Use fixed versions of dependencies to build wheels (#198)

Tests

• Fix CI failure under python 2.7 in MacOS (#188)
• Use conda for all configurations (#229)

v0.1.1

This is the release notes of v0.1.1. See here for the complete list of solved issues and merged PRs.

New Features

• Support sparse for part of linalg operands (#238)
• Add TileDB support as external storage (#245)

Enhancements

• Add stub for execute() and __dir__ support in Tensor to facilitate code completion (#235)

Bug fixes

• Fix starting with multiple schedulers with --schedulers (#203)
• Fix process clean up when local cluster terminates (#208)
• Fix bug that TensorZeros generate same key even if their shapes are different in TensorDiag.tile (#213)
• Fix bug that chunks of ones and zeros with same shape generated different key (#225)
• Add TensorInv operand to fix mt.linalg.inv (#241)

Installation

• Use fixed versions of dependencies to build wheels (#199)

Tests

• Fix CI failure under python 2.7 in MacOS (#189)

v0.2.0a1

This is the release notes of v0.2.0a1. See here for the complete list of solved issues and merged PRs.

Highlights

• Initial DataFrame support, currently only support create DataFrame from pandas one.

New Features

• Support QR decomposition on short-and-fat tensor(sfqr method) (#113)
• Update tensor shape after execution (#120)
• Implement __array__ protocol for Tensor (#126)
• Add IndexField, SeriesField, DataFrameField for protobuf and json serializer, remove DataTuple for dataserializer (#164)
• DataFrame initial support (#173)

Enhancements

• Refactor transfer.py to increase maintainability (#88)
• Support execute tensor without fetching data (#92)
• Use property instead of weak-key dict to store cix (#107)
• Worker refactor and schedule on worker (#89)
• Add rough_nbytes for tensor data type (#128)
• Use dataserializer to serde ndarray in protobuf and json serializers instead of numpy.save and numpy.load (#163)
• Decouple services from applications & unify worker start logic (#170)
• Add batch support for broadcasts and local meta setting (#166)

Bug fixes

• Fix setitem and getitem generate same operand key (#101)
• Check parameters for rand and randn (#100)
• Fix summation on slicing on an array with same range (#103)
• Fix compose failure after #103 (#111)
• Fix worker start fail on machines with insufficient memory (also some doc fixes) (#141)
• Fix --phy-mem arg error and BrokenPipeErrors under Python 2.7 (#159)
• Fix random execute that a random tensor can have different results when executed in different sessions (#174)

Documentation

• Typo in index.rst: import numpy should be import numpy as np (#136, thanks @mmm311!)
• Fix missing import in install.rst (#139, thanks @mmm311!)

Installation

• Change scripts into entry-points in setup.py for platform compatibility (#151)
• Include requirements-*.txt, pyx and pxd, exclude c file in MANIFEST.in (#153)

Tests

• Add test cases for etcd and tasks web page (#85)
• Add cython coverage (#121)
• Add cases and docs for ExecutionActor (#90)
• Add test cases for multiple workers and increase case coverage (#132)
• Use miniconda to test on OSX (#168)

Others

• Remove redundant web static files (#133)

v0.1.0

This is the release notes of v0.1.0. See here for the complete list of solved issues and merged PRs.

This release note only covers the difference from v0.1.0b1; for all highlights and changes, please refer to the release notes of the pre-releases:

• alpha1
• alpha2
• beta1

New Features

• Support QR decomposition on short-and-fat tensor(sfqr method) (#116)
• Update tensor shape after execution (#122)
• Implement __array__ protocol for Tensor (#127)

Enhancements

• Support execute tensor without fetching data (#95)
• Use property instead of weak-key dict to store cix (#109)
• Add rough_nbytes for tensor data type (#149)

Bug fixes

• Fix AttributeError on get_workers (#96)
• Fix summation on slicing on an array with same range (#104)
• Fix setitem and getitem generate same operand key (#108)
• Check parameters for rand and randn (#110)
• Fix compose failure after #103 (#112)
• Fix worker meta error (#130)
• Fix worker start fail on machines with insufficient memory, docs fixed also (#143)
• Fix --phy-mem arg error and BrokenPipeErrors under Python 2.7 (#162)
• Fix parse error of cache_mem (#171)
• Fix random execute that a random tensor can have different results when executed in different sessions (#178)

Installation

• Bug fixes on installation (scripts and extra requirement files in source dists) (#155)

Tests

• Add test cases for etcd and tasks web page (#86)

v0.1.0b1

This is the release notes of v0.1.0b1. See here for the complete list of solved issues and merged PRs.

Changes that break compatibility

• Rename chunks to chunk_size for all tensor expressions (#74)

New Features

• Support fetch executed tensor data from server (#80)

Enhancements

• Add multiple scheduler support & process restart support (#36)
• Use ChunkMetaActor to replace KVStoreActor (#37)
• Use number of cores as n_parallel for local session by default (#82)
• Add schedulers information page for web (#83)

Bug fixes

• Fix execute method of ExecutableTuple (#65)
• Fix repeat deletion of kv item (#73)
• Remove broadcast entries of deleted items (#76)
• Fix Index and IndexSetValue operand when new chunks without indexes parameter (#70)
• Fix bug when some of the svd results are gc collected (#79)

Tests

• Remove sudo in Travis CI (#69, thanks @cclauss!)
• Add flake8 code quality check (#78)

v0.1.0a2

This is the release notes of v0.1.0a2. See here for the complete list of solved issues and merged PRs.

Enhancements

• Support set endpoint for web session to reuse the session but with a different web (#54)

Bug fixes

• Fix chunk index of tensor of singular values calculated by svd decomposition (#58)

Tests

• Remove redundant code to increase coverage (#57)
• Add virtualenv test & fix some warnings (#60)

Others

• Make fuse logic out of graph (#61)

v0.1.0a1

This is the release notes of v0.1.0a1. See here for the complete list of solved issues and merged PRs.

New Features

• Add numpy compatible tensor API support
• Threaded, local cluster(#35) and distributed scheduling

Enhancements

• Limit shared memory to 20% when starting local cluster (#47)
• Add mars common API (#26)
• Raise TypeError if argument is a shape-tuple in mt.random.rand & mt.random.randn (#14)

Bug fixes

• Fix serialize issue when submitting calculation on parts of svd outputs (#43)
• Switch the graph into FAILED when preparation failed (#24)
• Fix stop graph failed due to failure of serialization (#44)
• Fix submitting parts of multiple outputs operand like svd (#42)
• Fix web session when submit multiply tensors (#40)
• Fix web, not patch thread to make web debuggable (#31)
• Fix web UI dashboard (#33)

Documentation

• Add installation, instructions and getting involved in README, add local cluster docs (#51)

Installation

• Add requirements-dev.txt including cython (#7)
• Try catch ImportError if mock not installed (#3)

Tests

• Add travis config (#11)
• Add appveyor and fix upload config (#46)
• Fix arg reduction test (#32)
• Fix a few test cases due to the precision of result (#29)
• Remove support code for py26 (#18)