v0.8.0b2

This is the release notes of v0.8.0b2. See here for the complete list of solved issues and merged PRs.

New Features

• Learn
  • Implements glm.LogisticRegression (#2466, thanks @Fernadoo!)
  • Implements bagging sampling (#2496)
• Services
  • Basic reschedule subtask (#2467)
• Web
  • Split tileable information and subtask graph into two tabs (#2480, thanks @RandomY-2!)
  • Include tileable property in detail api (#2493, thanks @RandomY-2!)

Enhancements

• Support specified vineyard socket and skip the launching vineyardd process (#2481)
• Refine MarsDMatrix & support more parameters for XGB classifier and regressor (#2498)

Bug fixes

• Compatible with scikit-learn 1.0 (#2486)
• Fix bug that failed to execute query when there are multiple arguments (#2490, thanks @perfumescent!)

Documentation

• Fix wrong translation in cluster deployment. (#2489, thanks @perfumescent!)

Tests

• Fix version of statsmodels to pass CI (#2497)

v0.7.4

This is the release notes of v0.7.4. See here for the complete list of solved issues and merged PRs.

New Features

• Web
  • Split tileable information and subtask graph into two tabs (#2482, thanks @RandomY-2!)
  • Include tileable property in detail api (#2499, thanks @RandomY-2!)

Enhancements

• Support specified vineyard socket and skip the launching vineyardd process (#2500)
• Refine MarsDMatrix & support more parameters for XGB classifier and regressor (#2501)

Bug fixes

• Compatible with scikit-learn 1.0 (#2487)
• Fix bug that failed to execute query when there are multiple arguments (#2491, thanks @perfumescent!)

v0.8.0b1

This is the release notes of v0.8.0b1. See here for the complete list of solved issues and merged PRs.

New Features

• DataFrame
  • Integrate Mars DataFrame with Ray Dataset (#2393, thanks @vcfgv!)
• Learn
  • Add _binary_roc_auc_score method (#2403, thanks @Divyanshu-Singh-Chauhan!)
• Web
  • Support visualizing subtask graphs on Mars Web (#2426, thanks @RandomY-2!)
• Others
  • Revisit {from,to}_vineyard for tensors and dataframes. (#2419)
  • [Ray] Reconstruct worker (#2413)
  • Make cmdline support third party modules (#2454)
  • Add nightly builds for docker images (#2456)

Enhancements

• Refine and unify subtask detail APIs (#2465, thanks @RandomY-2!)

Bug fixes

• Fix df/series.{apply, map_chunk} when some chunk output empty data (#2434)
• Fix missing DAGs when initializing with empty API results (#2439, thanks @RandomY-2!)
• Fix skew and kurt errors under MacOS (#2443)
• Add tests for public kubernetes image (#2446)
• Fix timeout error when waiting for a submitted task (#2457)
• Print the error message when error happens in TaskProcessor (#2458)
• Fix misuse of name parameter in DataFrame align (#2469, thanks @qxzhou1010!)
• Fix hang when start sub pool fails (#2468)

Installation

• Build and upload docker images in continuous deployment (#244)

Tests

• Fix coverage for Azure pipeline (#2474)

v0.7.3

This is the release notes of v0.7.3. See here for the complete list of solved issues and merged PRs.

New Features

• Learn
  • Add _binary_roc_auc_score method (#2477, thanks @Divyanshu-Singh-Chauhan!)
• Web
  • Support visualizing subtask graphs on Mars Web (#2471, thanks @RandomY-2!)
• Others
  • Revisit {from,to}_vineyard for tensors and dataframes (#2436)
  • Add nightly builds for docker images (#2462)
  • Make cmdline support third party modules (#2472)

Bug fixes

• Fix df/series.{apply, map_chunk} when some chunk output empty data (#2437)
• Fix missing DAGs when initializing with empty API results (#2442, thanks @RandomY-2!)
• Fix skew and kurt errors under MacOS (#2445)
• Fix usage of kubernetes image (#2448)
• Fix timeout error when waiting for a submitted task (#2461)
• Fix misuse of name parameter in DataFrame align (#2473, thanks @qxzhou1010!)
• Fix hang when start sub pool fails (#2476)

Tests

• Fix coverage for Azure pipeline (#2475)

v0.7.2post1

This release is a hotfix of v0.7.2 in order to fix the public docker image.

Bug fixes

• Fix usage of kubernetes image for v0.7.2 (#2447)

v0.8.0a3

This is the release notes of v0.8.0a3. See here for the complete list of solved issues and merged PRs.

New Features

• Tensor
  • Implemented hypergeometric functions (#2397, thanks @Alfa-Shashank!)
  • Implements mars.tensor.append (#2417)
• DataFrame
  • Implements Series.between (#2368, thanks @gowrijsuria!)
  • Integrate Mars DataFrame with Ray MLDataset (#2294, thanks @vcfgv!)
  • Support DataFrame.transpose() (#2327, thanks @hoarjour!)
• Learn
  • Add mars.learn.ensemble.{BlockwiseVotingClassifier, BlockwiseVotingRegressor} (#2390)
  • Implements linear_model.LinearRegression (#2260, thanks @Fernadoo!)
  • Add TensorFlow dataset (#2383, thanks @yuanchongtt!)
  • Implements mars.learn.preprocessing.{LabelBinarizer,label_binarize} (#2415)
  • Implements mars.learn.metrics.log_loss (#2418)
  • Implements mars.learn.wrappers.ParallelPostFit (#2425)
• Services
  • Initially support auto scaling (#2210)
• Web
  • API for subtask DAG structure (#2389, thanks @RandomY-2!)

Bug fixes

• Fix raising wrong error for an operand when post_execute implemented and error occurs in execute (#2395)
• [Ray] Fix occasionally failed unittest test_ownership_when_scale_in (#2401)
• [Oscar] Fix possible ActorCaller.call hang (#2404, thanks @Catch-Bull!)

Documentation

• Highlight dask-on-mars in doc (#2399)

Tests

• Improve case stability (#2381)
• Upgrade vineyard to v0.2.7 (#2193)
• Add checks for file mode changes and absolute imports (#2398)
• [Ray] Fix ray version (#2406)
• Change all tests to use relative import (#2407)

v0.7.2

This is the release notes of v0.7.2. See here for the complete list of solved issues and merged PRs.

New Features

• Tensor
  • Implements hypergeometric functions (#2408, thanks @Alfa-Shashank!)
  • Implements mars.tensor.append (#2422)
• DataFrame
  • Implements Series.between (#2382, thanks @gowrijsuria!)
  • Implements DataFrame.transpose() (#2423, thanks @hoarjour!)
• Learn
  • Add mars.learn.ensemble.{BlockwiseVotingClassifier, BlockwiseVotingRegressor} (#2391)
  • Add TensorFlow dataset (#2409, thanks @yuanchongtt!)
  • Implements linear_model.LinearRegression (#2411, thanks @Fernadoo!)
  • Implements mars.learn.preprocessing.{LabelBinarizer, label_binarize} (#2421)
  • Implements mars.learn.metrics.log_loss (#2424)
  • Implements mars.learn.wrappers.ParallelPostFit (#2427)
• Web
  • API for subtask DAG structure (#2410, thanks @RandomY-2!)

Bug fixes

• Fix raising wrong error for an operand when post_execute implemented and error occurs in execute (#2396)

Tests

• Improve case stability (#2387)
• Change all tests to use relative import (#2412)

v0.8.0a2

This is the release notes of v0.8.0a2. See here for the complete list of solved issues and merged PRs.

New Features

• DataFrame
  • Support initializing Mars objects from CUDA (#2308)
  • Support md.to_numeric (#2290, thanks @hoarjour!)
  • Gives an error when input DataFrame has unknown dtypes (#2355)
  • Added assign to DataFrame (#2362, thanks @hxri!)
  • Support reading csv file from oss (#2292, thanks @zebivy!)
• Tensor
  • Implements mars.tensor.stats.ks_2samp (#2324)
  • Implements mars.tensor.stats.ks_1samp (#2335)
• Learn
  • Support PyTorch Dataset for oscar (#2246, thanks @yuanchongtt!)
  • Add KFold support (#2363)
• Services
  • Add API to retrieve progress and status of tileables (#2357)
• Web
  • Add visualization page for tileable graphs (#2282, thanks @RandomY-2!)
  • Add storage infos in web (#2317)
  • Display tileable progress, status and dependency link type on task detail page (#2360, thanks @RandomY-2!)
• Others
  • [Ray] Rerun subtask for ray backend (#2288, thanks @keyile!)
  • Add experimental Dask-on-Mars support (#2289, thanks @loopyme!)

Enhancements

• Support setting multiple columns in DataFrame (#2303)
• Refactor tileable visualization classes (#2318)
• Create service classes to manage service and session operations (#2326)
• Improve wait_actor_pool_recovered (#2328, thanks @keyile!)
• Remove bokeh from package requirements (#2339)
• Optimize mars supervisor scheduling (#2325)

Bug fixes

• Fix hangs when worker main pool has failures. (#2286)
• Fix the error when multiple subtasks fetch the same data (#2322)
• [Ray] Fix ray ci (#2343, thanks @keyile!)
• Fix error in Dask-on-Mars when compute multiple objects (#2348, thanks @loopyme!)
• Fix KeyError when remote function returns None (#2371)
• Fix DataFrame comparison when data type is period (#2373)

Documentation

• Fix untranslated strings in doc (#2346)
• Fix docs of DataFrame.assign (#2367)

v0.7.1

This is the release notes of v0.7.1. See here for the complete list of solved issues and merged PRs.

New Features

• DataFrame
  • Support md.to_numeric (#2334, thanks @hoarjour!)
  • Gives an error when input DataFrame has unknown dtypes (#2359)
  • Implements DataFrame.assign (#2369, thanks @hxri!)
  • Support reading csv file from oss (#2374, thanks @zebivy!)
• Tensor
  • Implements mars.tensor.stats.ks_2samp (#2332)
  • Implements mt.stats.ks_1samp (#2341)
• Learn
  • Support PyTorch Dataset for oscar (#2364, thanks @yuanchongtt!)
  • Add KFold support (#2365)
• Services
  • Add API to retrieve progress and status of tileables (#2358)
• Web
  • Add visualization page for tileable graphs (#2319, thanks @RandomY-2!)
  • Add storage infos in web (#2333)
  • Display tileable progress, status and dependency link type on task detail page (#2377, thanks @RandomY-2!)

Enhancements

• Support setting multiple columns in DataFrame (#2313)
• Create service classes to manage service and session operations (#2331)
• Remove bokeh from package requirements (#2344)
• Optimize scheduling service on supervisors (#2347)
• Improve wait_actor_pool_recovered (#2350, thanks @keyile!)

Bug fixes

• Fix the error when multiple subtasks fetch the same data (#2340)
• Fix KeyError when remote function returns None (#2375)
• Fix DataFrame comparison when data type is period (#2376)

Documentation

• Fix untranslated strings in doc (#2349)

v0.8.0a1

This is the release notes of v0.8.0a1. See here for the complete list of solved issues and merged PRs.

New Features

• Tensor
  • Add partial support of bessel functions (#2258, thanks @JuntaoMa!)
  • Implements mars.tensor.in1d (#2297)
• Learn
  • Implements mars.learn.utils.multiclass.unique_label (#2295)
• Services
  • Add get_storage_level_info api (#2228)
  • Basic rerun subtask (#2198)
  • Add API to fetch tileable graph as JSON (#2253, thanks @RandomY-2!)
  • Enable running on GPU for oscar (#2284)
• Others
  • Add support for seek method in memory cases (#2250)

Enhancements

• Support choosing aggregation algorithm at runtime (#2213)
• Add support for stateless actors (#2218)
• Add status filters for Cluster service (#2214)
• Reassign subtasks and filter nodes with status (#2159, thanks @vcfgv!)
• Add methods to sessions to get web endpoint (#2236)
• Ensure range index incremental for data source op like md.read_csv (#2232)
• Use Kubernetes Service to discover Mars Supervisors (#2227)
• Record mapper meta for shuffle task (#2248)
• Support data dependency for run_script (#2251)
• Refine oscar debugging (#2252)
• Support fetch_log for web session (#2257)
• Use batch method to reduce transferring cost for shuffle tasks (#2233)
• Allow turning off actor killing (#2273)
• Assign bands given devices of subtasks (#2276)
• Add bind method to facilitate extracting batch args (#2280)
• Reduce memory estimation for specific operands (#2283)

Bug fixes

• Fix NoDataToSpill when multiple storage quota requests happen simultaneously (#2203)
• Pass logging config file name into sub pools (#2222)
• Stop using thread local to store default session. (#2217)
• Fix possible CI failure when destroying remote object for incremental index (#2239)
• Fix service errors in Windows (#2237)

Documentation

• Doc refinement for Oscar (#2234)
• Add docs for batch methods (#2293)

Installation

• Merge default & distributed requirements (#2263)

Tests

• Add separate check pipeline (#2299)
• Fix delocate version to 0.8.2 to avoid deploy error (#2305)