Challenges and Strategies for Kubernetes Controlplane Management on Flatcar

[ugur99]

Hi everyone,

I'm seeking insights on managing the controlplane setup for Flatcar nodes in an immutable infrastructure setup. Let's suppose we have successfully migrated all controlplanes to Flatcar nodes, and our goal is to maintain all infrastructure components in an immutable manner. This means we avoid using tools like kubespray, preferring instead to reprovision the controlplane nodes for upgrades or reconfigurations. This approach seems feasible with minimal effort for maintaining existing setups.

However, I foresee potential challenges when it comes to scaling up our Flatcar controlplane nodes. The process involves numerous kubeadm commands, managing etcd certificates, and other complexities that are not trivial to handle in an immutable setup without a dedicated application to manage these aspects during scale up or scale down operations.

Does anyone have experience or advice on scaling immutable Flatcar controlplane setups? Any shared experiences or thoughts would be greatly appreciated!

Thanks in advance for your feedback!

[tormath1]

Hi @ugur99,

Did you have a look to the Cluster API project? I get your point of avoiding tools but I think Cluster API would be a good trade-off and it seems to perfectly meet your needs regarding control plane scaling.

Please note that Flatcar is well integrated to the Cluster API ecosystem.

If you are more into Terraform deployments, you can have a look to Typhoon which is fully integrated with Flatcar on multiple cloud providers too.

I think both of these approaches are fine because they don't bring more complexity on top of Kubernetes, they rely on standard and they just take care of "kubeadm commands, managing etcd certificates, and other complexities" nothing more.

[ugur99]

Hi @tormath1! Thanks for the explanation and the hint; yes, I heard; I actually used some capi providers in the past. But I didn't know there were bare metal providers for this as well. I'll have a look. Thanks for your help.

[ader1990]

FYI, Tinkerbell baremetal provider also exist if you prefer a more baremetal enterprise grade approach: https://github.com/tinkerbell/cluster-api-provider-tinkerbell, which works well with Flatcar.