scaleup_basedon_xcat_in_openshift_cluster

Scale up based on xCAT in OpenShift cluster

This document based on all-in-one OpenShift cluster is ready.

Background

Provide solution about how to run xCAT in OpenShift container to deploy other nodes and then add these nodes to OpenShift cluster.

Run xCAT in OpenShift container

Please make sure time has been synchronized.

All-in-one OpenShift is ready.

Expose xCAT ports on host

xCAT need to use 69, 3001, 3002 and http port when deploying nodes, so need to expose them by configure iptables rules. Add below lines to /etc/sysconfig/iptables:

-A OS_FIREWALL_ALLOW -p udp -m state --state NEW -m udp --dport 69 -j ACCEPT
-A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 3001:3002 -j ACCEPT

For HTTP port, the original is 80 and the 80 port is used by OpenShift. So need to plan a new one and expose it, for example 8898:

-A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 8898 -j ACCEPT

Restart iptables to valid iptables configuration. (This will trigger all pods restarted)

# systemctl restart iptables

Install xCAT in OpenShift container

• Change working project to planned by oc project <project_name>

• Create service account to privilege for example: xcatadmin

# oc create serviceaccount <service_account> -n <project_name>
# oc adm policy add-scc-to-user privileged system:serviceaccount:<project_name>:<service_account>

• Pull xcat/xcat2 docker image by oc new-app xcat/xcat2

• Modify xcat2 deploy configuration by oc edit dc/xcat2

  • Add privileged, hostNetwork info

  ......
    spec:
      containers:
        securityContext:
          privileged: true
      hostNetwork: true
  

  • Add service account created just now

  ......
    spec:
      serviceAccount: <service_account>
      serviceAccountName: <service_account>
  

• Check xcat2 deploy pod status, if ready copy image iso to pod by oc rsync <iso_path> <xcat_deploy_pod>:<target_iso_path>

• Enter it by oc rsh <xcat_deploy_pod> to configure xCAT

  • Modify http/apache configuration to listen 8898, restart httpd/apache service. For example:

  # cat /etc/httpd/conf/httpd.conf
  ....
  Listen 8898
  

  • modify httport in site table

  # chdef -t site httpport=8898
  

  • mknb <arch> to enable new http port in xCAT files

Deploy new nodes as xCAT documents

Please deploy new nodes according xCAT document.

Configure OpenShift master and new nodes

• Enable ssh without password on all nodes

• Configure DNS on all nodes, and restart dnsmasq service by systemctl restart dnsmasq.service on master

• Install base packages on new nodes

Configure OpenShift inventory file (used when install OpenShift on master)

[OSEv3:children]
.....
new_nodes

[new_nodes]
node.example.com openshift_node_group_name='node-config-[infra|compute]'

[OSEv3:vars]
openshift_schedulable=true

If master node has label node-role.kubernetes.io/infra=true, openshift_schedulable=true must be configured. Could view master label by oc describe node <master_node>

Run ansible-playbook to scale up

# ansible-playbook -i inventory/<inventory_file> playbooks/openshift-node/scaleup.yml