Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reconciliation stuck because of Sealed Secrets #4969

Open
1 task done
jmsariron opened this issue Sep 5, 2024 · 1 comment
Open
1 task done

Reconciliation stuck because of Sealed Secrets #4969

jmsariron opened this issue Sep 5, 2024 · 1 comment

Comments

@jmsariron
Copy link

Describe the bug

We are having this issue, where, not sure how exactly, but sometimes the Flux reconciliation is failing because of Sealed Secrets, which are (As far as I know) correct.

Kustomization gets stuck in Reconciliation in Progress, getting the details I get a:

Warning  HealthCheckFailed  19m   kustomize-controller  health check failed after 59m30.097886897s: timeout waiting for: [SealedSecret/myapp/myapp-secrets status: 'InProgress'] 

Checking the kustomize-controllerpod logs, it shows a server-side apply completed with all elements being unchanged and then the Reconciation Failed log message, I'm kinda confused.

Going to the SealedSecret on the Namespace it shos as correctly applied and Synced, with the corresponding regular Secret generated.

This is happening only sometimes and we can't figure out why. Since I can't reproduce it 100% of time I'm looking for some help to debug this behaviour.

Steps to reproduce

We are using SealedSecrets on other projects and AFAIK the same way, sometimes it fails sometimes not, so I don't know how to reproduce it exactly

Expected behavior

I should reconcile just fine

Screenshots and recordings

No response

OS / Distro

N/A

Flux version

N/A

Flux check

N/A

Git provider

No response

Container Registry provider

No response

Additional context

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@jmsariron
Copy link
Author

Adding a bit more info. for this case particular I:

  1. Deleted the Sealed Secret inside Kubernetes
  2. Deleted the kustomize-controller pod

When the new kustomize-controller started syncing everything, it fixed the kustomization and now appears as Synced with the Sealed Secret recreated from the git repo, so it seems to be something related to he controller?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant