Ernie inference docs (#978)

projects/ernie/auto_export_ernie_345M_mp1.sh

@@ -0,0 +1,26 @@
+
+#! /bin/bash
+
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+log_dir=log_auto
+rm -rf $log_dir
+
+# 345M mp1 export
+python -m paddle.distributed.launch --log_dir $log_dir --devices "0" \
+    ./tools/auto_export.py \
+    -c ./ppfleetx/configs/nlp/ernie/auto/finetune_ernie_345M_single_card.yaml \
+    -o Distributed.mp_degree=1 \

projects/ernie/docs/README.md

@@ -24,14 +24,19 @@ Learnt by ERNIE：[mask] [mask] [mask] 是黑龙江的省会，国际 [mask] [ma
 ```text
 .
 ├── docs
+│   └── inference.md
 │   └── README.md
+├── auto_export_ernie_345M_mp1.sh           # 345M ernie-base模型，自动切分单卡导出
+├── auto_export_ernie_345M_mp2.sh           # 345M ernie-base模型，自动切分多卡导出
 ├── export_ernie_345M_single_card.sh        # 345M ernie-base模型，单卡导出
 ├── finetune_ernie_345M_single_card.sh      # 345M ernie-base模型，单卡finetune训练
+├── inference.py                            # ernie推理代码
 ├── pretrain_ernie_base_175B_mp8_pp16.sh    # 175B ernie-base模型，3D混合并行
 ├── pretrain_ernie_base_3D.sh               # ci测试
 ├── pretrain_ernie_base_6.7B_sharding16.sh  # 6.7B ernie-base模型，sharding16
 ├── pretrain_ernie_base.sh                  # 345M ernie-base模型，单卡
-└── pretrain_ernie_large.sh                 # ernie-large模型，单卡     
+└── pretrain_ernie_large.sh                 # ernie-large模型，单卡  
+└── run_inference.sh                        # ernie推理运行脚本    
 ```
 
 
@@ -150,3 +155,8 @@ python -m paddle.distributed.launch --log_dir $log_dir --devices "0,1,2,3,4,5,6,
     -c ./ppfleetx/configs/nlp/ernie/finetune_ernie_345M_single_card.yaml \
     -o Model.use_recompute=True
 ```
+</details>
+
+## 3. 推理部署
+
+[推理部署](inference.md)

projects/ernie/docs/inference.md

@@ -0,0 +1,31 @@
+# 推理部署
+
+模型训练完成后，可使用飞桨高性能推理引擎Paddle Inference通过如下方式进行推理部署。
+
+## 1. 模型导出
+
+以`ERNIE(345M)`模型为例
+
+
+导出单卡`ERNIE(345M)`模型：
+```bash
+sh projects/ernie/auto_export_ernie_345M_mp1.sh
+```
+
+导出多卡`ERNIE(345M)`模型：
+```bash
+sh projects/ernie/auto_export_ernie_345M_mp2.sh
+```
+
+## 2. 推理部署
+
+模型导出后，可通过`tasks/ernie/inference.py`脚本进行推理部署。
+
+`ERNIE(345M)` 推理
+```bash
+bash projects/ernie/run_inference.sh
+```
+
+## 3. Benchmark
+
+测试中