Template: Fix faulty CORS headers handling.

[elizabeth-dev]

What this PR does / why we need it:

CORS headers handling implementation has been producing inconsistent results for a bit due to the unusual way Nginx handles if directives (essentially, they are to include return and rewrite directives only inside them). This change recreates the exact same logic we were trying to apply whe handling CORS, but using NJS scripting.

This also includes the NJS module into our custom build of Nginx, since it was going to be necessary for #12383 anyway.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• CVE Report (Scanner found CVE and adding report)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation only

Which issue/s this PR fixes

fixes #10267

How Has This Been Tested?

Test cases including multiple origin, headers, and methods have been run using curl, in order to ensure all the different CORS headers behave accordingly to the different annotation values we allow. This also included the specific configuration a user reported failing in the linked issue.

Additionally, most of the CORS e2e test suite has been revised and improved, relying more on actually checking the result of HTTP requests rather than just checking the contents of the nginx.conf file.

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I've read the CONTRIBUTION guide
• I have added unit and/or e2e tests to cover my changes.
• All new and existing tests passed.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: elizabeth-dev
Once this PR has been reviewed and has the lgtm label, please assign strongjz for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[k8s-ci-robot]

Hi @elizabeth-dev. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[netlify]

✅ Deploy Preview for kubernetes-ingress-nginx ready!

Name	Link
🔨 Latest commit	49ab05e
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-ingress-nginx/deploys/6754cff8c331250008d3c4f0
😎 Deploy Preview	https://deploy-preview-12424--kubernetes-ingress-nginx.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[Gacko]

I'd not call it a fix if we need to introduce NJS for it. We also already have a different PR for introducing NJS but as this needs a working NGINX base image build, we first need to fix this, introduce NJS and then can rebase this PR on top.

[elizabeth-dev]

@Gacko in that case we can prioritize and merge the NJS PR first, and then add this on top of it as just a fix

[Gacko]

Yes, makes sense! Sadly we can first release it in v1.13. Also I wouldn't word it as a fix then. Does this make sense?

[elizabeth-dev]

@Gacko hmm, not sure if I understand. you wouldn't tag the other PR (adding NJS + some migrations) as a fix, I agree with that, but once that is merged whenever, we can put this change on top, then it'd just be the scripting part, and that would pass as "just a fix" for me, since no new modules would be added and the logic would stay the same (but working as expected)

or do you prefer to pass both changes as new features?

[Gacko]

Hm, yeah, you're right. We can consider this a fix. It's just a bit complex as we are also changing the whole framework the implementation is based on.

[elizabeth-dev]

well, that's settled then. we'll see if we can fix the build issues soon

/hold