Seldon Core Deployment

This is an example project showing the deployment of an iris model in Seldon Core.

Installing Seldon

This project has the same requirements as the quickstart described in the Seldon docs.

Install Docker

First install Docker for Linux, Mac, Windows.

Install Kind

Install Kind to run kubernetes locally.

Install kubectl

Install kubectl for Linux, Mac, Windows to interact with kubernetes clusters.

Install Helm

Install Helm.

Install Istio

For Linux and macOS, the easiest way to download Istio is using the following command:

curl -L https://istio.io/downloadIstio | sh -

Move to the Istio package directory. For example, if the package is istio-1.11.4:

cd istio-1.11.4

Add the istioctl client to your path (Linux or macOS):

export PATH=$PWD/bin:$PATH

Istio provides a command line tool istioctl to make the installation process easy. The demo configuration profile has a good set of defaults that will work on your local cluster. Install it with:

istioctl install --set profile=demo -y

The namespace label istio-injection=enabled instructs Istio to automatically inject proxies alongside anything we deploy in that namespace. We’ll set it up for our default namespace:

kubectl label namespace default istio-injection=enabled

In order for Seldon Core to use Istio’s features to manage cluster traffic, we need to create an Istio Gateway by running the following command:

kubectl apply -f istio_gateway.yaml

For custom configuration and more details on installing seldon core with Istio please see the Istio Ingress page.

Install Seldon Core

First, create a new namespace for the operator to run in:

kubectl create namespace seldon-system

Create a new Kubernetes cluster and configure kubectl to use it with:

kind create cluster --name seldon
kubectl cluster-info --context kind-seldon

Then install seldon-core in our cluster

helm install seldon-core seldon-core-operator \
    --repo https://storage.googleapis.com/seldon-charts \
    --set usageMetrics.enabled=true \
    --set istio.enabled=true \
    --namespace seldon-system

To check that the controller is running do:

kubectl get pods -n seldon-system

You should see a seldon-controller-manager pod with STATUS=Running.

Local Port Forwarding

Because your kubernetes cluster is running locally, we need to forward a port on your local machine to one in the cluster for us to be able to access it externally. You can do this by running

kubectl port-forward -n istio-system svc/istio-ingressgateway 8080:80

This will forward any traffic from port 8080 on your local machine to port 80 inside your cluster.

Deploying the iris model

Seldon provides an sklearn iris model at gs://seldon-models/v1.14.0-dev/sklearn/iris. This project deploys this with the pre-packaged sklearn server provided by Seldon.

First we create a namespace to run our model in

kubectl create namespace seldon

Then we deploy our model using config from iris_server.yaml:

kubectl apply -f iris_server.yaml

Then we port forward our server.

kubectl port-forward -n istio-system svc/istio-ingressgateway 8080:80

We can now query our server by running the httprequest.py script

pip install -r requirements.txt
python httprequest.py

This guide roughly follows the Seldon instructions for installing locally and deploying a model so see those for more information.

Clearing down the model

Run the following to clear down the kubernetes setup:

kubectl delete -f iris_server.yaml
kubectl delete -f istio_gateway.yaml