Merge pull request #3 from vdcloudcraft/metric-aggregation

Added location grouping

README.md

@@ -6,7 +6,8 @@ NOTE: This software assumes your fail2ban jail definitions are all in a single f
 
 ## Disclaimer
 
-This exporter goes deliberately against best practices and ist not suitable for deployments at scale. It's intended to be used in a homelab alike setting and won't even provide any sane metric to alert on. This may change in the future, but it more than likely will not. 
+This exporter goes deliberately against best practices and ist not suitable for deployments at scale. It's intended to be used in a homelab alike setting and won't even provide any sane metric to alert on. This may change in the future, but it more than likely will not.
+By enabling grouping in the `conf.yml`, the growth of label cardinality can be reduced, but this is still far from ideal.
 
 ## Metrics
 
@@ -15,6 +16,12 @@ Default labels are: `jail` and `ip` with their respective values.
 
 More labels can be provided by enabling geoIP annotation. At this point, the maxmind provider adds `city`, `latitude`, and `longitude` in addition the default labels.
 
+If you enable grouping in the `conf.yml`, you will instead receive two sets of metrics and no more data about single IPs. Instead you get a gauge `fail2ban_location` which counts the number of banned IPs per location. The labels are the same as above, just without `jail` and `ip`.
+
+The second metric is `fail2ban_jailed_ips` which is a gauge, that displays all currently banned IPs per jail. `jail` is the only label in this metric.
+
+It's highly recommended to enable grouping, in order to reduce the cardinality of your labels.
+
 A small guide to creating your own geoIP provider can be found in the [Extensibility](#Extensibility) section of this README.
 
 ## Installation
@@ -57,10 +64,11 @@ Now open `conf.yml` and you should see something like this
 ```yaml
 server:
     listen_address:
-    port: 
+    port:
 geo:
     enabled: True
     provider: 'MaxmindDB'
+    enable_grouping: False
     maxmind:
         db_path: '/f2b-exporter/db/GeoLite2-City.mmdb'
 f2b:
@@ -71,6 +79,7 @@ f2b:
 Just plug in the port and IPv4 address you want your exporter to be listening on. If you want to enable geotagging, there is only one method at this time and for that you will need to sign up for a free account at https://www.maxmind.com, download their city database and plug the path to the db in `geo.maxmind.db_path`. Their paid tier claims to have increased accuracy and is interchangable with their free database, so that should work as a data source for this exporter as well. At the time of writing I can neither deny, nor confirm these claims.
 
 When that is all done, run following commands and your exporter is running and will survive reboots:
+
 ```bash
 sudo systemctl daemon-reload
 sudo systemctl enable fail2ban-geo-exporter.service
@@ -105,15 +114,18 @@ Currently there is only one way to geotag IPs and that is with the Maxmind db. B
 If you wish to implement your own method for annotating your IPs, you need to create a Python class and save the module in `./geoip_provider/`
 
 You need to ensure following requirements are fullfilled:
+
 - Your module name is a lower case version of your class name. E.g.: Class `FancyProvider`, module `./geoip_providers/fancyprovider.py`
 - Your class has a constructor that accepts a single parameter. This will be the parsed `conf.yml` that will be passed to the class.
 - Your class implements a method `annotate(ip)`, that takes in a single IP as a string and returns a dictionary with all additional labels for Prometheus to use. Do not include the IP itself as a label.
 - Your class implements a method `get_labels()` that returns a list of strings with the label names it's going to provide.
 
 When all that is given, you can just put your class name with[!] capitalization into the configuration under `geo.provider` and enjoy the fruits of your labour.
 
+Be aware that the `enable_grouping` setting will use only the labels provided by your class to aggregate locations.
+
 If you do create another provider class and think other people might find it useful too, I'll gladly review pull requests.
 
 ## Grafana dashboard
 
-The file `grafana-dash.json` includes a complete definition to display your banned IPs on a worldmap and count all banned IPs per jail.
+The files `dashboard-*.json` include a complete definition for either grouping configuration to display your banned IPs on a worldmap and count all banned IPs per jail.

conf.yml

@@ -4,6 +4,7 @@ server:
 geo:
     enabled: True
     provider: 'MaxmindDB'
+    enable_grouping: False
     maxmind:
         db_path: '/f2b-exporter/db/GeoLite2-City.mmdb'
 f2b:

dashboard-with-grouping.json

@@ -0,0 +1,251 @@
+{
+  "__inputs": [
+    {
+      "name": "DS_PROMETHEUS",
+      "label": "Prometheus",
+      "description": "",
+      "type": "datasource",
+      "pluginId": "prometheus",
+      "pluginName": "Prometheus"
+    }
+  ],
+  "__requires": [
+    {
+      "type": "grafana",
+      "id": "grafana",
+      "name": "Grafana",
+      "version": "7.0.0"
+    },
+    {
+      "type": "panel",
+      "id": "grafana-worldmap-panel",
+      "name": "Worldmap Panel",
+      "version": "0.3.2"
+    },
+    {
+      "type": "datasource",
+      "id": "prometheus",
+      "name": "Prometheus",
+      "version": "1.0.0"
+    },
+    {
+      "type": "panel",
+      "id": "stat",
+      "name": "Stat",
+      "version": ""
+    }
+  ],
+  "annotations": {
+    "list": [
+      {
+        "builtIn": 1,
+        "datasource": "-- Grafana --",
+        "enable": true,
+        "hide": true,
+        "iconColor": "rgba(0, 211, 255, 1)",
+        "name": "Annotations & Alerts",
+        "type": "dashboard"
+      }
+    ]
+  },
+  "editable": true,
+  "gnetId": null,
+  "graphTooltip": 0,
+  "id": null,
+  "iteration": 1590951136241,
+  "links": [],
+  "panels": [
+    {
+      "datasource": "${DS_PROMETHEUS}",
+      "description": "",
+      "fieldConfig": {
+        "defaults": {
+          "custom": {
+            "align": null,
+            "displayMode": "color-text"
+          },
+          "decimals": 0,
+          "mappings": [],
+          "noValue": "0",
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": null
+              }
+            ]
+          },
+          "unit": "none"
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 4,
+        "w": 6,
+        "x": 0,
+        "y": 0
+      },
+      "id": 4,
+      "options": {
+        "colorMode": "value",
+        "graphMode": "none",
+        "justifyMode": "auto",
+        "orientation": "auto",
+        "reduceOptions": {
+          "calcs": [
+            "last"
+          ],
+          "values": false
+        }
+      },
+      "pluginVersion": "7.0.0",
+      "repeat": "jail",
+      "repeatDirection": "h",
+      "targets": [
+        {
+          "expr": "sum (fail2ban_jailed_ips{jail=\"$jail\"}) by (jail)",
+          "format": "time_series",
+          "instant": true,
+          "interval": "",
+          "legendFormat": "",
+          "refId": "A"
+        }
+      ],
+      "timeFrom": null,
+      "timeShift": null,
+      "title": "$jail",
+      "transformations": [
+        {
+          "id": "organize",
+          "options": {
+            "excludeByName": {
+              "Time": true
+            },
+            "indexByName": {},
+            "renameByName": {
+              "Value": "Banned IPs"
+            }
+          }
+        }
+      ],
+      "type": "stat"
+    },
+    {
+      "circleMaxSize": "30",
+      "circleMinSize": "2",
+      "colors": [
+        "#C0D8FF",
+        "#73BF69",
+        "#FADE2A",
+        "#C4162A"
+      ],
+      "datasource": "${DS_PROMETHEUS}",
+      "decimals": 0,
+      "esMetric": "Count",
+      "fieldConfig": {
+        "defaults": {
+          "custom": {}
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 24,
+        "w": 24,
+        "x": 0,
+        "y": 8
+      },
+      "hideEmpty": false,
+      "hideZero": false,
+      "id": 2,
+      "initialZoom": "3",
+      "locationData": "table",
+      "mapCenter": "custom",
+      "mapCenterLatitude": "27",
+      "mapCenterLongitude": 14,
+      "maxDataPoints": 1,
+      "mouseWheelZoom": true,
+      "showLegend": false,
+      "stickyLabels": false,
+      "tableQueryOptions": {
+        "geohashField": "geohash",
+        "labelField": "city",
+        "latitudeField": "latitude",
+        "longitudeField": "longitude",
+        "metricField": "Value",
+        "queryType": "coordinates"
+      },
+      "targets": [
+        {
+          "expr": "sum(fail2ban_location) by (latitude, longitude, city)",
+          "format": "table",
+          "instant": true,
+          "interval": "",
+          "legendFormat": "",
+          "refId": "A"
+        }
+      ],
+      "thresholds": "0,10,100",
+      "timeFrom": null,
+      "timeShift": null,
+      "title": "Banned IP locations",
+      "transformations": [],
+      "type": "grafana-worldmap-panel",
+      "unitPlural": "IPs",
+      "unitSingle": "",
+      "unitSingular": "IP",
+      "valueName": "current"
+    }
+  ],
+  "refresh": "10s",
+  "schemaVersion": 25,
+  "style": "dark",
+  "tags": [],
+  "templating": {
+    "list": [
+      {
+        "allValue": "",
+        "current": {},
+        "datasource": "${DS_PROMETHEUS}",
+        "definition": "fail2ban_jailed_ips",
+        "hide": 2,
+        "includeAll": true,
+        "label": null,
+        "multi": false,
+        "name": "jail",
+        "options": [],
+        "query": "fail2ban_jailed_ips",
+        "refresh": 1,
+        "regex": "/.*jail=\"([^\"]*).*/",
+        "skipUrlSync": false,
+        "sort": 5,
+        "tagValuesQuery": "",
+        "tags": [],
+        "tagsQuery": "",
+        "type": "query",
+        "useTags": false
+      }
+    ]
+  },
+  "time": {
+    "from": "now-5m",
+    "to": "now"
+  },
+  "timepicker": {
+    "refresh_intervals": [
+      "10s",
+      "30s",
+      "1m",
+      "5m",
+      "15m",
+      "30m",
+      "1h",
+      "2h",
+      "1d"
+    ]
+  },
+  "timezone": "",
+  "title": "fail2ban banned locations",
+  "uid": "pGlJmeRGk",
+  "version": 24
+}

fail2ban-exporter.py

@@ -5,6 +5,7 @@
 from prometheus_client import make_wsgi_app
 from prometheus_client.core import GaugeMetricFamily, REGISTRY
 from wsgiref.simple_server import make_server
+from collections import defaultdict
 
 class Jail:
     def __init__(self, name):
@@ -64,15 +65,44 @@ def collect(self):
         self.get_jailed_ips()
         self.assign_location()
 
+        if conf['geo']['enable_grouping']:
+            yield self.expose_grouped()
+            yield self.expose_jail_summary()
+        else:
+            yield self.expose_single()
+
+    def expose_single(self):
         metric_labels = ['jail','ip'] + self.extra_labels
-        ip_gauge = GaugeMetricFamily('fail2ban_banned_ip', 'IP banned by fail2ban', labels=metric_labels)
+        gauge = GaugeMetricFamily('fail2ban_banned_ip', 'IP banned by fail2ban', labels=metric_labels)
+
+        for jail in self.jails:
+            for entry in jail.ip_list:
+                values = [jail.name, entry['ip']] + [ entry[x] for x in self.extra_labels ]
+                gauge.add_metric(values, 1)
+
+        return gauge
+
+    def expose_grouped(self):
+        gauge = GaugeMetricFamily('fail2ban_location', 'Number of currently banned IPs from this location', labels=self.extra_labels)
+        grouped = defaultdict(int)
 
         for jail in self.jails:
             for entry in jail.ip_list:
-                values = [jail.name, entry['ip']] + [entry[x] for x in self.extra_labels]
-                ip_gauge.add_metric(values, 1)
+                location_key = tuple([ entry[x] for x in self.extra_labels ])
+                grouped[location_key] += 1
+
+        for labels, count in grouped.items():
+            gauge.add_metric(list(labels), count)
+
+        return gauge
+
+    def expose_jail_summary(self):
+        gauge = GaugeMetricFamily('fail2ban_jailed_ips', 'Number of currently banned IPs per jail', labels=['jail'])
+
+        for jail in self.jails:
+            gauge.add_metric([jail.name], len(jail.ip_list))
 
-        yield ip_gauge
+        return gauge
 
 if __name__ == '__main__':
     with open('conf.yml') as f: