Unable to Register New Nodes in HostedCluster When Adding API Server Certificate

[alfredtm]

Hello! When adding API server certificate to the HostedCluster

apiVersion: hypershift.openshift.io/v1beta1
kind: HostedCluster
metadata:
  name: test
  namespace: test
spec:
  configuration:
    apiServer:
      servingCerts:
        namedCertificates:
        - servingCertificate:
            name: tls-secret

The bootstrap-kubeconfig is no longer trusted

Unable to connect to the server: tls: failed to verify certificate: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "root-ca")

This seems to be a issue when adding new nodes (using kubevirt) to the cluster. They dont register/show up on the HostedCluster.

When not adding certificate to the apiServer the nodes register fine.

[alfredtm]

I resolved this by setting servicePublishingStrategy.loadBalancer.hostname to the apiServer's internal service URL.

The certificate-authority-data in bootstrap-kubeconfig is then trusted.

[alfredtm]

Reopening this issue
We are experiencing issues using the apiserver internal service url at a later stage in our deployment.

Ideally we would like to use the external apiserver fqdn in servicePublishingStrategy.loadBalancer.hostname. But adding apiserver certificate the bootstrap-kubeconfig is no longer trusted.

Is there any way to modify the bootstrap-kubeconfig?

I also found this issue that seems to be the same issue I am experiencing
https://issues.redhat.com/browse/OCPBUGS-19067

[KiqoCode]

We have the same issue..

[omlet05]

Same issue here, does anyone solved it?

[jparrill]

Hey @alfredtm @KiqoCode @omlet05 Is this a day-2 action?

[omlet05]

@jparrill Same with a brand new cluster or an update to an existing one.

[alfredtm]

@jparrill This happens with newly created clusters. And existing clusters when adding APIServerCertificate at a later stage.

[jparrill]

Thanks for the info folks, that really helped. I will try to take a look to the issue.

[jparrill]

/assign @jparrill

[KiqoCode]

Hello,

There is an open Jira with Redhat now I know which is planned to be fixed in 4.18 I think.
For now it seems custom API Certificate is not supported on HostedCluster.
Documentation have also been updated I know:
https://docs.openshift.com/container-platform/4.16/security/certificates/api-server.html

"In hosted control plane clusters, you cannot replace self-signed certificates from the API."