Benchmark Kits

This benchmark kits is a test suite designed to provides representative performance measures of FATE 1.5.0 (LTS) across three different usage scenarios.

To learn more and see the results of our tests please refer to our official report FATE 1.5.0 (LTS) performance test for all the details. For descriptions of the test types that we run, see the highlights section below.

The current LTS release of FATE is v1.5.0. All the releases are available here.

Quick Start Guide

To get started perfroming all the tests you'll need to first deploy FATE 1.5.0 (LTS).

1. Clone repo

    $ git clone [email protected]:FederatedAI/benchmark_kits.git
   

2. Change directories

    $ cd benchmark_kits/{one_of_the_three_test}/dsl
   

3. Submit test jobs by fateflow client

    $ python ${your_install_path}/fate_flow/fate_flow_client.py -f submit_job -d ${dsl} -c ${conf}
   

Highlights

1. System Requirement

Component	Operating System	Host Configuration	Storage	Networking
Exchange	CentOS 7.2	4C8G	100G	100M
Guest	CentOS 7.2	16C32G	500G	100M
Host	CentOS 7.2	16C32G	500G	100M

2. Test Type

   1） Starndard-size Dataset

   • 1 guest, 1 host

   • 5k*200 dataset

   • dsl&conf

   2） Large-scale Dataset

   • 1 guest, 1 host

   • 1m*2k dataset

   • dsl&conf

   3）Multi-host Federated

   • 1 guest, 30 hosts

   • 5k*200 dataset(modified)

   • dsl&conf