Improve the clean playbook to respect ongoing deployments

[sampaccoud]

Bug Report

Expected behavior/code
The clean playbook should only delete objects that are not connected and thus useless.

Actual Behavior
If the clean playbook is run at the same time as a deployment, it happens to delete objects that are not yet connected but useful...

Steps to Reproduce

1. start a deployment
2. once the deployment has started creating services, pods, etc., start the clean playbook

Environment

• Arnold version: 4.3.1

Possible Solution
Check that no deployment is running before actually deleting objects.

[jmaupetit]

🤔 looks like we need some sort of lock file for ongoing deployments.

[jmaupetit]

We can maybe use a redis instance per project for that? Used as a core app like acme or redirect?

[madmatah]

Maybe we can use kubernetes to store the deployment state of our applications, instead of deploying a new app only for this purpose ?

K8s uses etcd internally as a k/v store, but I think we can't use etcd directly for security reasons.
However, if we store this information in a kubernetes object, it will be persisted and we will be able to query these data easily from our playbooks.

The question is... in which kind of k8s object can we store this information ?
I see 2 options :

1. A ConfigMap per application.
2. In the metadata of the route (or service) of an each application.

(If you think of other options, I'm interested !)

I don't like the second solution because we can have multiple routes (or services) per application and the information will be duplicated.

What do you think about this idea ?

[jmaupetit]

Using a dedicated configMap is a good idea, but we need to refactor ConfgMap management first! See #253