how to deploy Kong Gateway with KIC

[balait4]

Hi All,

I'm using kong(3.5.0)/KIC(3.0.2) version in okd (openshift) cluster, We only use the KIC in our usecases and never deployed kong gateway. Now I want to add kong gateway as well without disturbing the existing setup. In documentation it says "Install Gateway API CRDs before installing Kong Ingress Controller", so how to do that for existing setup? Thanks
kong.txt

Even I tried installing on top of this and I see below result. But how to verify now the gateway is working?

gatewayclass.gateway.networking.k8s.io/kong created
gateway.gateway.networking.k8s.io/kong created

[chobits]

In documentation it says "Install Gateway API CRDs before installing Kong Ingress Controller", so how to do that for existing setup?

hi @pmalek @tao12345666333 could you guys take a look at this issue? or is there some one who can take a look at it. thanks

[pmalek]

@balait4 Can you share the installation steps that you're following?

We only use the KIC in our usecases and never deployed kong gateway

That's surprising to hear. KIC is used to configure Kong (as in the API Gateway instances). What's your use case if you're not using Kong Gateway together with KIC?

Did you mean Gateway API by any chance?

Are you trying to use the operator that's described in https://docs.konghq.com/gateway-operator/?

[balait4]

Thanks for the reply! yes Gateway API here https://docs.konghq.com/kubernetes-ingress-controller/latest/concepts/gateway-api/

Gateway API resources will only be reconciled when the Gateway API CRDs are installed in your cluster before Kong Ingress Controller is started.

[pmalek]

The page that you linked has a link to Gateway API installation doc, for KIC 3.0.x it is https://docs.konghq.com/kubernetes-ingress-controller/3.0.x/get-started/#install-the-gateway-apis.

[balait4]

ok this doc also mentioned that "Install the Gateway API CRDs before installing Kong Ingress Controller." So we have existing kong with KIC. Now I installed this CRDs and created " gatewayclass.gateway.networking.k8s.io/kong" and
"gateway.gateway.networking.k8s.io/kong". How can I check whether this works or any tuning required?

[pmalek]

You can either browse through KIC logs and look for the respective controllers running:

2024-08-06T09:47:47Z    info    controllers.Gateway    Starting EventSource    {"v": 0, "source": "kind source: *v1.Gateway"}
2024-08-06T09:47:47Z    info    controllers.Gateway    Starting EventSource    {"v": 0, "source": "kind source: *v1.GatewayClass"}
2024-08-06T09:47:47Z    info    controllers.Gateway    Starting EventSource    {"v": 0, "source": "kind source: *v1.Service"}
2024-08-06T09:47:47Z    info    controllers.Gateway    Starting EventSource    {"v": 0, "source": "kind source: *v1.HTTPRoute"}
2024-08-06T09:47:47Z    info    controllers.Gateway    Starting EventSource    {"v": 0, "source": "kind source: *v1beta1.ReferenceGrant"}
2024-08-06T09:47:47Z    info    controllers.Gateway    Starting Controller    {"v": 0}
2024-08-06T09:47:47Z    info    controllers.Gateway.V1GatewayClass    Starting EventSource    {"v": 0, "source": "kind source: *v1.GatewayClass"}
2024-08-06T09:47:47Z    info    controllers.Gateway.V1GatewayClass    Starting Controller    {"v": 0}
2024-08-06T09:47:47Z    info    controllers.Gateway.V1GatewayClass    Starting workers    {"v": 0, "worker count": 1}
2024-08-06T09:47:47Z    info    controllers.Gateway    Starting workers    {"v": 0, "worker count": 1}

Or inspect your Gateway or GatewayClass status field

kubectl get gateway  -o yaml  -w -o jsonpath-as-json='{.status}'

and verify if they have been accepted by the controller.

---

If those steps do not prove that Gateway API controllers are running then you can always restart KIC pods and that will cause it to pick up new CRDs ( which should be picked up in runtime anyway but this can help in some situations ).

[balait4]

we have 2 kong running in same namespace for review and staging. Getting below error in pod logs.
staging: KIC env
CONTROLLER_PUBLISH_SERVICE: kong/kong-proxy-staging
CONTROLLER_INGRESS_CLASS: kong

review: KIC env
CONTROLLER_PUBLISH_SERVICE: kong/kong-review-public
CONTROLLER_INGRESS_CLASS: kong-review-public

2024-08-06T10:12:11Z	error	controllers.Gateway	Could not determine service for gateway	
    {"GatewayV1Gateway": {"name":"kong","namespace":"kong"}, "namespace": "kong", "name": "kong", "error": "service
     ref kong/kong-review-public did 
    not match controller manager ref kong/kong-proxy-staging or /"}
    2024-08-06T10:12:11Z	info	controllers.Gateway	Warning: Reconciler returned both a non-zero result and a non-nil error. 
    The result will always be ignored if the error is non-nil and the non-nil error causes reqeueuing with exponential backoff. 
    For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler	{"reconcileID": 
    "936ed8d2-1cf9-422a-9a6c-078fc5e823df", "v": 0}

[balait4]

Do we need to create 2 gateway one with review KIC name and one with staging KIC name?

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kong
spec:
  gatewayClassName: kong-review-public
  listeners:
  - name: proxy
    port: 80
    protocol: TCP
 addresses:
    - type: IPAddress
      value: <Review_SERVICE_IP>
  serviceRef:
    name: kong-proxy-review
    namespace: kong

[pmalek]

KIC 3.1.0 introduced a flag --gateway-to-reconcile which allows you to set a single Gateway which it should reconcile.

You can use that to designate each of those KIC instances/deployments to reconcile their own isolated Gateway.

You can also use an env variable as described in https://docs.konghq.com/kubernetes-ingress-controller/latest/reference/cli-arguments/#environment-variables

[balait4]

thanks a lot! setting CONTROLLER_GATEWAY_TO_RECONCILE this in KIC seems working.

[balait4]

Only the below config is working. But what is I want to use this from other namespace apart from kong namespace.

gatewayClassName: kong
listeners:
  - allowedRoutes:
      namespaces:
        from: Same
    name: proxy
    port: 80
    protocol: HTTP

I tried setting from: All which is breaking the kong/KIC deployment. Also tried label the namespace as gateway-access=allowed and used selector&matchLabels as this also breaking.

 127.0.0.1 - - [08/Aug/2024:12:51:12 +0000] "POST /config?check_hash=1&flatten_errors=1 HTTP/1.1" 400 660 "-" "Go-http-client/1.1"

2024-08-08T12:50:53Z	error	Failed parsing resource errors	{"url": "http://127.0.0.1:8002", "update_strategy": "InMemory", "error": "could not unmarshal config error: json: cannot unmarshal object into Go struct field ConfigError.flattened_errors of type []sendconfig.FlatEntityError"}
2024-08-08T12:50:53Z	error	dataplane-synchronizer	Could not update kong admin	{"error": "performing update for http://127.0.0.1:8002 failed: failed posting new config to /config: got status code 400"}

I have this gateway only for staging using env CONTROLLER_GATEWAY_TO_RECONCILE. when the above change is breaking both staging and reveiw kong deployment. Eventhough there is no CONTROLLER_GATEWAY_TO_RECONCILE in reveiw to use this gateway. Also the Gateway status is Accepted not coming to Ready state(this unmanaged gateway has been picked up by the controller and will be processed).

[pmalek]

Which version are you using? 3.1? 3.2?

---

Parsing Gateway response errors should be greatly improved in 3.2

Just looking at this now I suggest enabling config dumps via dump-config CLI flag (or the corresponding env as described in https://docs.konghq.com/kubernetes-ingress-controller/latest/reference/cli-arguments/#environment-variables)

kubernetes-ingress-controller/internal/manager/config.go

Line 292 in 313234c

flagSet.BoolVar(&c.EnableConfigDumps, "dump-config", false, fmt.Sprintf("Enable config dumps via web interface host:%v/debug/config.", DiagnosticsPort))

With this enabled you'll be able to access

• "/debug/config/successful"
• "/debug/config/failed"
• "/debug/config/fallback"
• "/debug/config/raw-error"

where you're mostly interested in the last one. This is accessible via the DiagnosticsPort: 10256

kubernetes-ingress-controller/internal/manager/consts.go

Lines 20 to 21 in bf5fb88

	// DiagnosticsPort is the default port of the manager's diagnostics service listens on.
	const DiagnosticsPort = 10256

[balait4]

I tried in version 3.2 as well, the Gateway is not coming to ready state. In KIC log I don't see the httproute accepted for my application route. I enabled the dump-config, but I'm using openshift cluster where 10256 port is used by openshift-sdn, is there any env variable to use different port? I tried env CONTROLLER_DIAGNOSTICS_PORT but no effect.

2024-08-09T09:02:15Z	info	Diagnostics server is starting to listen	{"v": 0, "addr": 10256}
2024-08-09T09:02:15Z	error	Could not start diagnostics server	{"error": "listen tcp :10256: bind: address already in use"}
2024-08-09T09:02:15Z	error	Unable to start diagnostics server	{"error": "listen tcp :10256: bind: address already in use"}

I feel this issue comes when we have 2 deployments (review/staging) in same kong namespace. Somehow its not routing even we set CONTROLLER_GATEWAY_TO_RECONCILE

[pmalek]

You can change the port using diagnostic-server-port flag

kubernetes-ingress-controller/internal/manager/config.go

Line 294 in 313234c

flagSet.IntVar(&c.DiagnosticServerPort, "diagnostic-server-port", DiagnosticsPort, "The port to listen on for the profiling and config dump server.")

Can you provide the full PodSpec of both of those KIC Deployments?

[balait4]

Here is my deployment file for both instance of my test cluster.. My developement cluster is still KIC 3.0.2 where facing the same issue.
deployment.txt

[pmalek]

Can you also attach KIC logs? Where you able to get anything from the debug endpoints? Especially the /debug/config/raw-error which would return the raw error which KIC receives from Kong Gateway.

[balait4]

Thanks a lot for the support. After I upgrade my kong and KIC to latest version, I could see the HTTPRoutes and able to route now. Thanks