Skip to content

Latest commit

 

History

History
78 lines (57 loc) · 2.47 KB

standalone_deploy.md

File metadata and controls

78 lines (57 loc) · 2.47 KB
Raw

FATE-LLM Single-Node Deployment Guide

1. Introduction

Server Configuration:

  • Quantity: 1
  • Configuration: 8 cores / 16GB memory / 500GB hard disk / GPU Machine
  • Operating System: CentOS Linux release 7
  • User: User: app owner:apps

The single-node version provides 3 deployment methods, which can be selected based on your needs:

  • Install FATE-LLM from PyPI With FATE
  • Install FATE-LLM from PyPI with FATE, FATE-Flow, FATE-Client

2. Install FATE-LLM from PyPI With FATE

In this way, user can run tasks with Launcher, a convenient way for fast experimental using.

2.1 Installing Python Environment

  • Prepare and install conda environment.
  • Create a virtual environment:
# FATE-LLM requires Python >= 3.10
conda create -n fate_env python=3.10
conda activate fate_env

2.2 Installing FATE-LLM

This section introduces how to install FATE-LLM from pypi with FATE, execute the following command to install FATE-LLM.

pip install fate_llm[fate]==2.2.0

2.3 Usage

After installing successfully, please refer to tutorials to run tasks, tasks describe in the tutorials running will Launcher are all supported.

3. Install FATE-LLM from PyPI with FATE, FATE-Flow, FATE-Client

In this way, user can run tasks with Pipeline or Launcher.

3.1 Installing Python Environment

Please refer to section-2.1

3.2 Installing FATE-LLM with FATE, FATE-Flow, FATE-Client

pip install fate_client[fate,fate_flow,fate_client]==2.2.0

3.3 Service Initialization

mkdir fate_workspace
fate_flow init --ip 127.0.0.1 --port 9380 --home $(pwd)/fate_workspace
pipeline init --ip 127.0.0.1 --port 9380
  • ip: The IP address where the service runs.
  • port: The HTTP port the service runs on.
  • home: The data storage directory, including data, models, logs, job configurations, and SQLite databases.

3.4 Start Fate-Flow Service

fate_flow start
fate_flow status # make sure fate_flow service is started

FATE-Flow also provides other instructions like stop and restart, use only if users want to stop/restart fate_flow services.

# Warning: normal installing process does not need to execute stop/restart instructions.
fate_flow stop
fate_flow restart

3.5 Usage

Please refer to tutorials for more usage guides, tasks describe in the tutorials running will Pipeline or Launcher are all supported.