diff --git a/demos/dynatrace-demo/instructions.md b/demos/dynatrace-demo/instructions.md index 9fcdb5c1..36192c45 100644 --- a/demos/dynatrace-demo/instructions.md +++ b/demos/dynatrace-demo/instructions.md @@ -1,7 +1,7 @@ # Ansible Rulebook + Dynatrace DEMO ## Description -In this demo we will register a host to Dynatrace and set up a webhook to send +In this demo, we will register a host to Dynatrace and set up a webhook to send problem notifications to ansible-rulebook CLI. Dynatrace monitors the availability of a process. Upon receiving a problem notification the ansible- rulebook CLI runs a remedy playbook to restart the process. @@ -30,7 +30,7 @@ crashing. 3. Update `inventory.yml` with correct ip and user to access the client node 4. Start the rulebook CLI: ``` - ansible-rulebook -i demos/dynatrace-demo/inventory.yml --rules demos/dynatrace-demo/rulebook.yml + ansible-rulebook -i demos/dynatrace-demo/inventory.yml -r demos/dynatrace-demo/rulebook.yml ``` This rulebook starts an alertmanager source that listens on port 5050 diff --git a/demos/kubernetes/example_playbook.yml b/demos/kubernetes/example_playbook.yml new file mode 100644 index 00000000..7229df2b --- /dev/null +++ b/demos/kubernetes/example_playbook.yml @@ -0,0 +1,12 @@ +--- +- name: Example triggered playbook + hosts: localhost + tasks: + - name: Print a message + ansible.builtin.debug: + msg: | + We received a resource with the attributes: + Type: {{ type }} + Kind: {{ kind }} + ApiVersion: {{ apiversion }} + Name: {{ name }} diff --git a/demos/kubernetes/instructions.md b/demos/kubernetes/instructions.md new file mode 100644 index 00000000..24a4333e --- /dev/null +++ b/demos/kubernetes/instructions.md @@ -0,0 +1,137 @@ +# Ansible Rulebook + Kubernetes + +## Description + +In this demo, we will configure a single-node K8s development environment, +and we will set up ansible-rulebook CLI to consume events from the +cluster using the Kubernetes event source. Upon triggering this watcher +method the ansible-rulebook CLI will run a playbook to execute any additional +action in response to this event. + +## Instructions + +### Installing a development environment + +The method to install a small development environment is straightforward. The reasoning +for choosing K3D is based on the fact that GitHub actions support Ubuntu with Docker so +the next steps can be easily integrated into a functional end-to-end GitHub actions workflow. + +``` +# Installing kubectl +curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" +chmod +x ./kubectl +sudo mv ./kubectl /usr/local/bin/kubectl +# Installing K3D +curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash +# Creating a cluster +k3d cluster create testcluster --api-port 6443 --servers 1 --agents 1 --port "30500-31000:30500-31000@server:0" +# Test the cluster +kubectl get nodes +``` + +### Rulebook return parameters + +When an event is triggered and returned, there are two keys in the +result dictionary, `type`, and `resource`. + +These output parameters come from the +[stream method](https://github.com/kubernetes-client/python/blob/master/kubernetes/base/watch/watch.py#L116) +in the watch class monitoring for changes in the state of the cluster. + +The possible values of type are the types of event such as "ADDED", "DELETED", etc. +The resource value is a dictionary representing the watched object. + +For example, a condition that can be monitored is: + +``` +condition: event.type == "ADDED" and event.resource.kind == 'Pod' +``` + +The previous condition will be met after there was `ADDED` a new object of the kind `Pod`. + +For the further implementation of additional rules, the user must know the +representation of the watched object. +This logic will be handled in the rulebook rules and not in the event source plugin +enabling users to use the Python Kubernetes client without restrictions. + +### Monitoring resources + +For a better reference review the main list of APIs and methods that are +[supported](https://github.com/kubernetes-client/python/blob/master/kubernetes/README.md). + +#### Monitoring for new ADDED Pods + +1. Have ansible-rulebook CLI and its dependencies installed +2. Have ansible.eda collection installed with the Kubernetes event source +3. Start the rulebook CLI: +``` + # Go to the Kubernetes demos folder and run: + ansible-rulebook -i inventory.yml -r rulebook_monitor_pods.yml +``` +4. Once the ansible-rulebook is running, add a pod to the cluster by running: +``` +kubectl apply -f k8s_deployment_no_namespace.yml +``` + +An event for the new Pod creation should be triggered and +the `demos/kubernetes/example_playbook.yml` playbook executed. + +#### Monitoring for a new ADDED deployment + +It is possible also to monitor deployments, for instance, +run the rulebook to check the status of the deployments: + +``` +ansible-rulebook -i inventory.yml -r rulebook_monitor_deployment.yml +``` + +Or by namespaces: + +``` +ansible-rulebook -i inventory.yml -r rulebook_monitor_deploymentns.yml +``` + +And then create the deployments: + +``` +kubectl apply -f k8s_deployment_no_namespace.yml +``` + +Or create a deployment in a namespace: + +``` +kubectl apply -f k8s_deployment_namespace.yml +kubectl apply -f k8s_deployment_with_namespace.yml + +``` + +#### Monitoring for a new ADDED custom resource + +Custom resources can be monitored also, any resource as long it has +a valid API and method to watch for changes should work without any +major change. + +Let's watch for changes in the CR. + +``` +ansible-rulebook -i inventory.yml -r rulebook_monitor_cr.yml +``` + +Create a custom resource definition with its custom object: +``` +kubectl apply -f k8s_crontab_crd.yml +kubectl apply -f k8s_crontab_cr.yml +``` + +#### Monitoring for a new ADDED config maps + +It is possible also for looking for changes in con + +``` +ansible-rulebook -i inventory.yml -r rulebook_monitor_configmaps.yml +``` + +Create a custom resource definition with the configmap: +``` +kubectl apply -f k8s_configmap.yml +``` \ No newline at end of file diff --git a/demos/kubernetes/inventory.yml b/demos/kubernetes/inventory.yml new file mode 100644 index 00000000..97537d9e --- /dev/null +++ b/demos/kubernetes/inventory.yml @@ -0,0 +1,5 @@ +--- +hosts: + vars: + ansible_connection: local + ansible_python_interpreter: "{{ansible_playbook_python}}" diff --git a/demos/kubernetes/k8s_configmap.yml b/demos/kubernetes/k8s_configmap.yml new file mode 100644 index 00000000..3c6d05a7 --- /dev/null +++ b/demos/kubernetes/k8s_configmap.yml @@ -0,0 +1,8 @@ +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: configmap-demo +data: + parameter_1: "value_1" + parameter_2: "value_2" diff --git a/demos/kubernetes/k8s_crontab_cr.yml b/demos/kubernetes/k8s_crontab_cr.yml new file mode 100644 index 00000000..6e66452e --- /dev/null +++ b/demos/kubernetes/k8s_crontab_cr.yml @@ -0,0 +1,7 @@ +apiVersion: "stable.example.com/v1" +kind: CronTab +metadata: + name: my-new-cron-object +spec: + cronSpec: "* * * * */5" + image: my-awesome-cron-image diff --git a/demos/kubernetes/k8s_crontab_crd.yml b/demos/kubernetes/k8s_crontab_crd.yml new file mode 100644 index 00000000..316d2517 --- /dev/null +++ b/demos/kubernetes/k8s_crontab_crd.yml @@ -0,0 +1,41 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + # name must match the spec fields below, and be in the form: . + name: crontabs.stable.example.com +spec: + # group name to use for REST API: /apis// + group: stable.example.com + # list of versions supported by this CustomResourceDefinition + versions: + - name: v1 + # Each version can be enabled/disabled by Served flag. + served: true + # One and only one version must be marked as the storage version. + storage: true + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object + properties: + cronSpec: + type: string + image: + type: string + replicas: + type: integer + # either Namespaced or Cluster + scope: Namespaced + names: + # plural name to be used in the URL: /apis/// + plural: crontabs + # singular name to be used as an alias on the CLI and for display + singular: crontab + # kind is normally the CamelCased singular type. Your resource manifests use this. + kind: CronTab + # shortNames allow shorter string to match your resource on the CLI + shortNames: + - ct diff --git a/demos/kubernetes/k8s_deployment_namespace.yml b/demos/kubernetes/k8s_deployment_namespace.yml new file mode 100644 index 00000000..a097d140 --- /dev/null +++ b/demos/kubernetes/k8s_deployment_namespace.yml @@ -0,0 +1,7 @@ +--- +kind: Namespace +apiVersion: v1 +metadata: + name: nginx-deployment-namespaced + labels: + name: nginx-deployment-namespaced diff --git a/demos/kubernetes/k8s_deployment_no_namespace.yml b/demos/kubernetes/k8s_deployment_no_namespace.yml new file mode 100644 index 00000000..97b79a39 --- /dev/null +++ b/demos/kubernetes/k8s_deployment_no_namespace.yml @@ -0,0 +1,20 @@ +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: nginx-deployment +spec: + selector: + matchLabels: + app: nginx + replicas: 2 + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 diff --git a/demos/kubernetes/k8s_deployment_with_namespace.yml b/demos/kubernetes/k8s_deployment_with_namespace.yml new file mode 100644 index 00000000..a2f33d9f --- /dev/null +++ b/demos/kubernetes/k8s_deployment_with_namespace.yml @@ -0,0 +1,21 @@ +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: nginx-deployment-namespaced + namespace: nginx-deployment-namespaced +spec: + selector: + matchLabels: + app: nginx + replicas: 2 + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 diff --git a/demos/kubernetes/rulebook_monitor_configmaps.yml b/demos/kubernetes/rulebook_monitor_configmaps.yml new file mode 100644 index 00000000..43470244 --- /dev/null +++ b/demos/kubernetes/rulebook_monitor_configmaps.yml @@ -0,0 +1,23 @@ +--- +- name: Listen for changes in a custom resource + hosts: localhost + sources: + - name: Check for configmaps in the cluster + ansible.eda.kubernetes: + api: CoreV1Api + method: list_config_map_for_all_namespaces + params: {} + rules: + - name: A configmap was found + condition: > + event.type == "ADDED" and + event.resource.kind == 'ConfigMap' and + event.resource.metadata.name == 'configmap-demo' + action: + run_playbook: + name: example_playbook.yml + extra_vars: + type: "{{ event.type }}" + kind: "{{ event.resource.kind }}" + apiversion: "{{ event.resource.apiVersion }}" + name: "{{ event.resource.metadata.name }}" diff --git a/demos/kubernetes/rulebook_monitor_cr.yml b/demos/kubernetes/rulebook_monitor_cr.yml new file mode 100644 index 00000000..470dd1f8 --- /dev/null +++ b/demos/kubernetes/rulebook_monitor_cr.yml @@ -0,0 +1,23 @@ +--- +- name: Listen for changes in a custom resource + hosts: localhost + sources: + - name: Check for pods in the cluster + ansible.eda.kubernetes: + api: CustomObjectsApi + method: list_cluster_custom_object + params: + group: stable.example.com + version: v1 + plural: crontabs + rules: + - name: A CustomResource was found + condition: event.type == "ADDED" + action: + run_playbook: + name: example_playbook.yml + extra_vars: + type: "{{ event.type }}" + kind: "{{ event.resource.kind }}" + apiversion: "{{ event.resource.apiVersion }}" + name: "{{ event.resource.metadata.name }}" diff --git a/demos/kubernetes/rulebook_monitor_deployment.yml b/demos/kubernetes/rulebook_monitor_deployment.yml new file mode 100644 index 00000000..d652b000 --- /dev/null +++ b/demos/kubernetes/rulebook_monitor_deployment.yml @@ -0,0 +1,20 @@ +--- +- name: Listen for changes in pods on a Kubernetes cluster + hosts: localhost + sources: + - name: Check for deployments in the cluster + ansible.eda.kubernetes: + api: AppsV1Api + method: list_deployment_for_all_namespaces + params: {} + rules: + - name: A deployment creation was found with a name nginx-deployment + condition: event.type == "ADDED" and event.resource.kind == 'Deployment' and event.resource.metadata.name == 'nginx-deployment' + action: + run_playbook: + name: example_playbook.yml + extra_vars: + type: "{{ event.type }}" + kind: "{{ event.resource.kind }}" + apiversion: "{{ event.resource.apiVersion }}" + name: "{{ event.resource.metadata.name }}" diff --git a/demos/kubernetes/rulebook_monitor_deploymentns.yml b/demos/kubernetes/rulebook_monitor_deploymentns.yml new file mode 100644 index 00000000..01e77a01 --- /dev/null +++ b/demos/kubernetes/rulebook_monitor_deploymentns.yml @@ -0,0 +1,21 @@ +--- +- name: Listen for changes in pods on a Kubernetes cluster + hosts: localhost + sources: + - name: Check for deployments in the cluster + ansible.eda.kubernetes: + api: AppsV1Api + method: list_namespaced_deployment + params: + namespace: nginx-deployment-namespaced + rules: + - name: A deployment condition matched + condition: event.type == "ADDED" and event.resource.metadata.namespace == 'nginx-deployment-namespaced' + action: + run_playbook: + name: example_playbook.yml + extra_vars: + type: "{{ event.type }}" + kind: "{{ event.resource.kind }}" + apiversion: "{{ event.resource.apiVersion }}" + name: "{{ event.resource.metadata.name }}" diff --git a/demos/kubernetes/rulebook_monitor_pods.yml b/demos/kubernetes/rulebook_monitor_pods.yml new file mode 100644 index 00000000..a095c7fe --- /dev/null +++ b/demos/kubernetes/rulebook_monitor_pods.yml @@ -0,0 +1,20 @@ +--- +- name: Listen for changes in pods on a Kubernetes cluster + hosts: localhost + sources: + - name: Check for pods in the cluster + ansible.eda.kubernetes: + api: CoreV1Api + method: list_pod_for_all_namespaces + params: {} + rules: + - name: A pod condition was found + condition: event.type == "ADDED" and event.resource.kind == 'Pod' + action: + run_playbook: + name: example_playbook.yml + extra_vars: + type: "{{ event.type }}" + kind: "{{ event.resource.kind }}" + apiversion: "{{ event.resource.apiVersion }}" + name: "{{ event.resource.metadata.name }}" diff --git a/extensions/eda/plugins/event_source/kubernetes.py b/extensions/eda/plugins/event_source/kubernetes.py new file mode 100644 index 00000000..2af87364 --- /dev/null +++ b/extensions/eda/plugins/event_source/kubernetes.py @@ -0,0 +1,172 @@ +"""kubernetes.py. + +An ansible-rulebook event source plugin +that can fetch dinamically any Kubernetes resource +supported in the official Kubernetes Python client. + +Arguments: +--------- + api - the API instance to be invoked + (i.e. CoreV1Api or CustomObjectsApi) + method - the state of the resource + (i.e. list_pod_for_all_namespaces or list_namespaced_custom_object) + params - the method's paramters + (i.e. {} in the case of fetching a namespaced pods list) + +Examples: +-------- + - name: Check the state of a custom resourceeee + ansible.eda.kubernetes: + api: CustomObjectsApi + method: list_namespaced_custom_object + params: + group: metrics.k8s.io + version: v1beta1 + namespace: default + plural: pods + + - name: Check a Kubernetes resource content (get pods) + ansible.eda.kubernetes: + api: CoreV1Api + method: list_pod_for_all_namespaces + params: {} + +""" + +import asyncio +import inspect +import logging +import os +from typing import Any + +from kubernetes import client, config, watch +from kubernetes.client.rest import ApiException + +logger = logging.getLogger(__name__) + + +async def main( + queue: asyncio.Queue, + args: dict[str, Any], +) -> None: # pylint: disable=R0914 + """Watch for Kubernetes events.""" + k8s_event_api = args.get("api", "") + k8s_event_method = args.get("method", "") + k8s_event_params = args.get("params", {}) + + # We make sure we can connect to the Kubernetes cluster + load_kubernetes_config() + + api_instance, w = load_kubernetes_api(k8s_event_api) + + # Each method has different parameters we will need to + # define in the watch stream call, we make sure we get a list of + # those method parameters, like at least the instance itself + # (self),or i.e. the namespace + resource_method = getattr(api_instance, k8s_event_method) + resource_method_parameters = inspect.getfullargspec(resource_method).args + resource_method_parameters.remove("self") + method_params = k8s_event_params + # We make sure the method parameters are consistent with what it wass passed + check_method_parameters(resource_method_parameters, method_params.keys()) + + last_resource_version = 0 + + extra_parameters = { + "watch": True, + "timeout_seconds": 10, + "resource_version": last_resource_version, + } + watcher_params = dict(method_params, **extra_parameters) + + try: + while True: + # We watch for the method passed unpacking the parameters + for event in w.stream(resource_method, **watcher_params): + logger.info("Object found :: %s", event) + # In the case we find an object we return it + await queue.put( + { + "type": event["type"], + "resource": event["raw_object"], + }, + ) + await asyncio.sleep(1) + watcher_params["resource_version"] = event["raw_object"]["metadata"][ + "resourceVersion" + ] + except ApiException as e: + err_t = 404 + if e.status == err_t: + # Unless we have objects we shouldnt be doing anything + pass + else: + logging.info("Error while watching for event stream :: %s", e) + raise + + +def load_kubernetes_api(k8s_event_api: str) -> dict: + """Get the main client class instance with no parameters. + + All the clients have the following + syntax i.e. client.AppsV1Api() or client.AppsV1Api + """ + api_instance = getattr(client, k8s_event_api)() + w = watch.Watch() + return api_instance, w + + +def check_method_parameters( + resource_method_parameters: dict, + method_params: dict, +) -> None: + """Check the parameters keys. + + This method makees sure the method parameters are consistent with respect the input + """ + if set(resource_method_parameters) != set(method_params): + logger.error("The parameters %s do not match", resource_method_parameters) + return + + +def load_kubernetes_config() -> None: + """Load the initial kubeconfig details. + + We load the config depending where we execute the events source from + """ + try: + if "KUBERNETES_PORT" in os.environ: + config.load_incluster_config() + elif "KUBECONFIG" in os.environ: + config.load_kube_config(os.getenv("KUBECONFIG")) + else: + config.load_kube_config() + except Exception: + logging.exception( + "---\n" + "The Python Kubernetes client could not be configured" + "at this time. You need a working Kubernetes environment" + "to make this event source to work, Check the following:\n" + "Use the env var KUBECONFIG like:\n" + " export KUBECONFIG=~/.kube/config\n" + "Or run ADA from within the cluster.", + ) + raise + + +if __name__ == "__main__": + """MockQueue if running directly.""" + + class MockQueue: + """A fake queue.""" + + async def put(self: str, event: str) -> str: + """Print the event.""" + print(event) # noqa: T201 + + asyncio.run( + main( + MockQueue(), + {"api": "CoreV1Api", "method": "list_pod_for_all_namespaces", "params": {}}, + ), + ) diff --git a/requirements.txt b/requirements.txt index 50a9219a..7ea0d0c1 100644 --- a/requirements.txt +++ b/requirements.txt @@ -7,3 +7,4 @@ systemd-python dpath pyyaml dpath +kubernetes