The limits on numbers of nodes in a k3s cluster?

[william-wang]

Hi there,

I'd like to deploy k8s in a production cluster, what's the official largest cluster size k3s can handle?

Thanks.

[dereknola]

The same as upstream K8s https://kubernetes.io/docs/setup/best-practices/cluster-large/. We recently tested up to 2000 nodes, but there is nothing stopping you from scaling to 5K with properly resourced CP nodes. See https://docs.k3s.io/installation/requirements#server-sizing-guide for some guidelines on those requirements.

[william-wang]

@dereknola Thanks for your quick response:)
One of the biggest bottleneck of k8s is etcd. As k3s can play with mysql. I am wondering whether k3s can handle even bigger size cluster e.g. 10k nodes if I replace etcd with mysql and use k3s.

[dereknola]

No, you can't go beyond upstream limits, since in this case it's not the DB backend that's limiting you, but rather the kube-apiserver. I'm curious about the use case you imagine for a single cluster of 10k nodes.

[william-wang]

In my prod, my api-server are enhanced to be deployed with autoscaling ability enabled. It seems no bottleneck in api-server. GKE support 65k nodes by integrating with a SQL like database spanner. So I am wondering what's upper limit of k3s with SQL database?

[dereknola]

We don't have the resources to test the scale your describing. GKE is not upstream K8s, they are building in additional functionality to support more than 5K nodes.