Implement annotations proxy-ssl-client-secret and proxy-ssl-ca-configmap

[juliocamarero]

What this PR does / why we need it:

Right now, if you specify proxy-ssl-secret annotation in an ingress resource, it serves 2 different purposes:

• If is contains a ca.crt, then that value is used as the proxy_ssl_trusted_certificate (CA bundle)
• If the secret contains tls.crt and tls.key, then those values are used as proxy_ssl_certificate and proxy_ssl_certificate_key, i.e. they are used for mutual TLS.

This is very convenient for simple scenarios where we just use a secret obtained using cert-manager (which includes the tls.crt/tls.key to sign requests, and also the ca.crt used to generate that cert, which is usually the one we need to trust). However, if we need to trust a CA bundle (to support more complex scenarios such a CA rotation), this becomes more complex since now we need to create a custom secret with the tls.crt/tls.key from a secret and the ca.crt from another location. It is becoming more and more common to use trust-manager to distribute CA bundles, which are by default distributed in a configmap since they just contain certificate public keys (see docs).

This PR tries to implement the solution described in #9415 by adding two annotations that could eventually replace the existing one:

• proxy-ssl-client-secret - a secret which contains tls.crt/tls.key
• proxy-ssl-ca-configmap - a configmap which contains ca.crt (since configmap is the default output for trust-manager)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• CVE Report (Scanner found CVE and adding report)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation only

Which issue/s this PR fixes

#9415

How Has This Been Tested?

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I've read the CONTRIBUTION guide
• I have added unit and/or e2e tests to cover my changes.
• All new and existing tests passed.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: juliocamarero
Once this PR has been reviewed and has the lgtm label, please assign tao12345666333 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: juliocamarero / name: Julio Camarero (76624fc, 7e70470, b112acd, b62721d, 59851d6, 351d89d, 316494b, 1d05507, 7e51421, 1b86d8f, bf220e6, 561ee18)

[k8s-ci-robot]

Welcome @juliocamarero!

It looks like this is your first PR to kubernetes/ingress-nginx 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/ingress-nginx has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @juliocamarero. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[netlify]

✅ Deploy Preview for kubernetes-ingress-nginx canceled.

Name	Link
🔨 Latest commit	561ee18
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-ingress-nginx/deploys/67586182cde45d0008d28bf4

[longwuyuan]

@juliocamarero thanks for your contribution.

I request that you document and explain the scenario and use-case for non-experts who are users of ssl. For example ;

• In the issue, or even better, here in this PR description, write a description of the difference between terminating TLS on the controller and sending plaintext to backend, VS termination TLS on controller and then initiating yet another TLS connection to the backend. As in the latter use-case, all this issue description and this PR description becomes relevant. Significant aspect being the backend-protocol annotation key will need to be set to a value of "HTTPS", to use certs (and hence the annotation that this PR is introducing)
• Follow it up kindly, with simple text diagramatic illustration of NON-TLS and TLS connections
• The design of the implied mTLS is deliberate as per my guess. If the backend is listening on a TLS port and hence serving a cert, then the CA in that backend webserver has to be a well-known CA (normally) and hence it would be known to the controller, out of the box. The need for providing a CA cert etc only arises when the CA of the backend's cert is not well-known. And then only the existence of mTLS requirement, would come into consideration (normally).