How many consumers can KIC handle

[zzdas]

I have been using KIC in AKS for little over 3 months and soon will be deploying in baremetal Kubernetes cluster. I have the following requirement and wanted to know whether this is a scalable/feasible approach.

Current Versions:
CRD: controller-gen.kubebuilder.io/version: v0.7.0
kong/kubernetes-ingress-controller:2.2.0
kong:2.7

Requirement

Backend service with paths /foo/user123A. /foo/user123B There can be 5 K such path, one per client.
Authentication using API KEY
One client cannot use their api key to access another clients resource , e.g user123A should not access /foo/user123B , so acl plugin needed

Required Kong Configuration

Create 1 kong api auth plugin
Create 5k secret for api key
Create 5k secret for acl
Create 5k kong consumer and add acl/apikey secret pair ref to each

Create 5k kong acl plugin
Create 5k ingress with separate path/route and acl plugin in the annotation.

With so many objects being created , will kong be able to handle this many number of consumers and secrets. ( I saw this open issue too #2382 w.r.t the huge number of secrets causing kong startup issues )

Alternate approach is to use a central authority to get the role/actions the client is supposed to do, but this requires a lot of changes on the backend so would prefer to have this configured via Kong

[shaneutt]

We do not currently have performance tests on various platforms such that we could provide data on this (e.g. "on GKE with instance types X we deployed Y secrets with Z performance characteristics"). We have explored adding such performance tests and data in the past (see #1197) but currently we recommend that end-users test their requirements in their own environments prior to any production deployments.

As you referenced already we are aware of some performance problems reported with large numbers of Secrets with somewhat large contents (RE: #2382) and as those issues appear to be upstream of us we're investigating potential improvements there. Even with those resolved without performance data we would still recommend individual testing for your circumstances.