Static pod ip addresses for StatefulSet

Hybridnet will keep a Pod's ip address retained if the Pod belongs to a StatefulSet. That means, once a Pod of a StatefulSet are created, its ip addresses will not change only if the owner StatefulSet is deleted.

Retaining StatefulSet Pods' addresses is enabled by default. But it also brings problem if not all the StatefulSets need their pods' ip address to be retained. For an underlay Pod, having a static ip address will limit the range of Nodes that the Pod can be scheduled.

We can change the global default behavior of retaining StatefulSet Pod ip addresses by changing the --default-ip-retain parameter of hybridnet-manager and hybridnet-webhook Pods. Obviously, it's true by default.

For example:

spec:
  template:
    spec:
      containers:
      - name: hybridnet-manager
        command:
        - /hybridnet/hybridnet-manager
        args:
        - --default-ip-retain=false

If you started with Getting Started, or you are using the hybridnet online helm chart, operations should be much easier:

# Change default network type to Underlay
helm upgrade hybridnet hybridnet/hybridnet -n kube-system --set defualtIPRetain=false

Using a special annotation of "networking.alibaba.com/ip-retain" in the StatefulSet Pod template, we can change the behavior of a specific StatefulSet too, which has a higher priority than the global default behavior.

For example:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: curl-ss
spec:
  selector:
    matchLabels:
      app: curl-ss
  replicas: 3
  serviceName: "curl"
  template:
    metadata:
      annotations:
        networking.alibaba.com/ip-retain: "false"        # Stop retaining ip addressed for the StatefulSet. 
      labels:
        app: curl-ss
    spec:
      containers:
      - args:
        - -c
        - sleep 999999999
        command:
        - /bin/sh
        name: curl
        image: tutum/curl:alpine

If a StatefulSet Pod has a retained ip address and has been change to not retain ip address any more, the exist retained ip address will be freed after the Pod is rebuilt.

Static ip addresses can also be specified manually. We can use the "networking.alibaba.com/ip-pool" annotation to specify a list of ip addresses for StatefulSet Pods in template. At the same time, we need to specify the Network by "networking.alibaba.com/specified-network" annotation and all the ip addresses in list should belong to the Network.

For example:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: curl-ss
spec:
  selector:
    matchLabels:
      app: curl-ss
  replicas: 3
  serviceName: "curl"
  template:
    metadata:
      annotations:
        networking.alibaba.com/ip-pool: "192.168.56.101,192.168.56.102,192.168.56.254".     # Specify the ip address list.
        networking.alibaba.com/specified-network: network1                                  # Specify the Network.
      labels:
        app: curl-ss
    spec:
      containers:
      - args:
        - -c
        - sleep 999999999
        command:
        - /bin/sh
        name: curl
        image: tutum/curl:alpine

The addresses in the annotation will be consumed in order from left to right, e.g., curl-ss-0 will have an address of 192.168.56.101.

It should be noticed that every ip address must belongs to a Subnet and a Subnet must belongs to a Network. That means for the Pods in underlay Networks, normally, once the ip address of which is specified, it can only be scheduled between Nodes belong to the intuitively specified Network.