detect unused annotation rules

[danwinship]

Tests get removed and renamed but we never update our annotation rules...

This makes it so that update-test-annotations.sh will actually fail if there are unused rules, though it could be made a warning instead.

This also removes all currently-unused rules. In some cases it's clear that a test just got renamed, and in some cases it seems pretty likely that weird tests got removed or recategorized upstream. But I didn't do any investigation of any of them.

Presumably you'd want this in origin too...

/assign @atiratree

(Previously: #830)

[openshift-ci-robot]

@danwinship: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• 0b028ad|Remove no-longer-matched annotation rules: does not specify an upstream backport in the commit message
• 4918215|Fail updating annotations if any rules match 0 tests: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: danwinship
Once this PR has been reviewed and has the lgtm label, please assign deads2k for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• DOWNSTREAM_OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

@danwinship: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• 53fb58b|UPSTREAM: : Remove no-longer-matched annotation rules: does not specify an upstream backport in the commit message
• 815d932|UPSTREAM: : Fail updating annotations if any rules match 0 tests: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[openshift-ci-robot]

@danwinship: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• 5bb8da6|UPSTREAM: : Remove no-longer-matched annotation rules: does not specify an upstream backport in the commit message
• 7d1f521|UPSTREAM: : Fail updating annotations if any rules match 0 tests: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[openshift-ci]

@danwinship: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-ovn-serial	7d1f521	link	true	/test e2e-aws-ovn-serial
ci/prow/e2e-aws-ovn-fips	7d1f521	link	true	/test e2e-aws-ovn-fips

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.