Skip to content

Latest commit

 

History

History
164 lines (141 loc) · 4.6 KB

CLIENT_CONFIGURE.md

File metadata and controls

164 lines (141 loc) · 4.6 KB
Raw

Client side configuration

Paddle Serving C++ client SDK的配置文件格式用protobuf定义，全部在configure/proto/sdk_configure.proto中。如果要增加配置字段，需要先在该protobuf文件中增加相应字段，才能被Serving SDK读取和解析。

Paddle Serving主配置文件为conf/predictors.prototxt。其中一个示例如下：

1. Sample conf

default_variant_conf {
  tag: "default"
  connection_conf {
    connect_timeout_ms: 2000
    rpc_timeout_ms: 20000
    connect_retry_count: 2
    max_connection_per_host: 100
    hedge_request_timeout_ms: -1
    hedge_fetch_retry_count: 2
    connection_type: "pooled"
  }
  naming_conf {
    cluster_filter_strategy: "Default"
    load_balance_strategy: "la"
  }
  rpc_parameter {
    compress_type: 0
    package_size: 20
    protocol: "baidu_std"
    max_channel_per_request: 3
  }
}
predictors {
  name: "ximage"
  service_name: "baidu.paddle_serving.predictor.image_classification.ImageClassifyService"
  endpoint_router: "WeightedRandomRender"
  weighted_random_render_conf {
    variant_weight_list: "50|50"
  }
  variants {
    tag: "var1"
    naming_conf {
      cluster: "list://127.0.0.1:8010"
    }
  }
  variants {
    tag: "var2"
    naming_conf {
      cluster: "list://127.0.0.1:8011"
    }
  }
}

predictors {
  name: "echo_service"
  service_name: "baidu.paddle_serving.predictor.echo_service.BuiltinTestEchoService"
  endpoint_router: "WeightedRandomRender"
  weighted_random_render_conf {
    variant_weight_list: "50"
  }
  variants {
    tag: "var1"
    naming_conf {
      cluster: "list://127.0.0.1:8010,127.0.0.1:8011"
    }
  }
}

2. 名词解释

  • 预测服务 (Predictor)：对一个Paddle预测服务的封装
  • 端点（Endpoit）：对一个预测需求的逻辑抽象，通常包含一到多个服务变体，以方便多版本模型管理；
  • 变体（Variant）：一套同质化的Paddle Serving集群服务，每个实例起一个Paddle Serving进程；

3. 配置项解释

3.1 default_variant_conf

default_variant_conf {
  tag: "default"
  connection_conf {
    connect_timeout_ms: 2000
    rpc_timeout_ms: 20000
    connect_retry_count: 2
    max_connection_per_host: 100
    hedge_request_timeout_ms: -1
    hedge_fetch_retry_count: 2
    connection_type: "pooled"
  }
  naming_conf {
    cluster_filter_strategy: "Default"  # Not used for now
    load_balance_strategy: "la"
  }
  rpc_parameter {
    compress_type: 0 
    package_size: 20
    protocol: "baidu_std"
    max_channel_per_request: 3
  }
}

其中：

connection_type: Maybe single/short/pooled, see BRPC DOC: connection_type

cluster_filter_strategy: 暂时未用

load_balance_strategy: Maybe rr/wrr/random/la/c_murmurhash/c_md5, see BRPC DOC: load_balance

compress_type: 0-None, 1-Snappy, 2-gzip, 3-zlib, 4-lz4, see BRPC DOC: compress_type

protocol: Maybe baidu_std/http/h2/h2:grpc/thrift/memcache/redis... see BRPC DOC: protocol

3.2 Predictors

可以为客户端配置多个predictor，每个predictor代表一个要访问的预测服务

predictors {
  name: "ximage"
  service_name: "baidu.paddle_serving.predictor.image_classification.ImageClassifyService"
  endpoint_router: "WeightedRandomRender"
  weighted_random_render_conf {
    variant_weight_list: "50|50"
  }
  variants {
    tag: "var1"
    naming_conf {
    cluster: "list://127.0.0.1:8010, 127.0.0.1:8011"
    }
  }
  variants {
    tag: "var2"
    naming_conf {
      cluster: "list://127.0.0.1:8011"
    }
  }
}

predictors {
  name: "echo_service"
  service_name: "baidu.paddle_serving.predictor.echo_service.BuiltinTestEchoService"
  endpoint_router: "WeightedRandomRender"
  weighted_random_render_conf {
    variant_weight_list: "50"
  }
  variants {
    tag: "var1"
    naming_conf {
      cluster: "list://127.0.0.1:8010"
    }
  }
}

其中：

service_name: 写sdk-cpp/proto/xx.proto的package name

endpoint_router: 目前只支持WeightedRandomRender

variant_weight_list: 与接下来的variants列表共用，用于表示variants之间相对权重；通过修改此数值可以调整variants调度的比重

cluster: Cluster支持的格式见 BRPC DOC: naming service