Grafana is an open source metric analytics & visualization suite. It is most commonly used for visualizing time series data for infrastructure and application analytics but many use it in other domains including industrial sensors, home automation, weather, and process control. This integration extends the ability to alert the person on call to a group and pause or unpause an alert coming out of Grafana using our closed-loop integration. Check out the sweet video!
An updated version of this integration is available. You can install the new one-way version right from the Workflow Templates directory within your xMatters instance. Learn more.
- GrafanaAlert.zip - Workflow containing the Grafana Steps.
An alert fires a webhook to the xMatters Workflow HTTP Trigger. The HTTP Trigger parses the incoming JSON and builds the event, then fires the event to notify the default recipients. Alternatively subscriptions can be set up to notify the desired parties. From the alert, you can pause the alert in Grafana to investigate the issue. When the issue is fixed, you can resume the alerting rule. When the alerting status of the Grafana alert goes to "ok", then the flow terminates the existing xMatters events.
- Login as the integration user in xMatters and navigate to the Developer Tab. (NOTE: Make sure the integration user has REST Web Services User Role. Reference General Notes section on how to create API user in xMatters.
- Import the GrafanaAlert.zip Steps to a workflow.
- Click on the workflow, and then Edit>Layout on the Grafana Alert to modify the alert recipients.
Click for image
- Save changes and return to the previous page. Click on Flows, and then Grafana Alert.
Click for image
- Select the "Grafana Alert - Inbound from Grafana" step.
Click for image
- Copy the Initiation URL, this will be used for setting up an Alert Notification Channel in Grafana.
Click for image
- Insert the API Key you generated in Grafana into
Grafana Tokenconstant, Which is accessible in the Flow Designer under Components>Constants. Reference General Notes section on how to generate the API key
Click for image
- Add the Grafana Endpoint by clicking on Components>Endpoints in the Flow Designer. Add an endpoint and name it "Grafana". Add the URL to your Grafana instance. Make sure you save changes.
Click for image
- You first need to set up a notification channel in Grafana which can be found under the Alerting section
Click for image
- Click on "New Channel" and create a new webhook found under "type". Name it, set Http Method to "POST", set URL as the URL you saved. Reference step 6 in the xMatters set up for the URL. Your channel should look something like this. NOTE: If you want to test out and see if the webhook works, you can click "Send Test" at the bottom.
Click for image
- Go to the chart you want to be alerted on in Grafana, click "edit", click "Alert", then click "Notifications". Click the plus icon and add the Channel you just added in the previous step.
Click for image
- How to set up the Grafana API Key to plug into xMatters. PLEASE NOTE: Grafana will only show the API you created ONCE. Suggestion is to copy and paste it in a notepad somewhere so you can get to it if you ever need it again
Click for image
- How to create API User in xMatters Click "Users", click "Invite Users" or "Add Users". Information should be something like this.
Click for image
Do an action in your Grafana application or infrastructure that will trigger the alert conditions set up in the Alert. This will fire the webhook into the workflow and an event will be created, targeting the default recipients. After it targets the right person, you can use our 2-way integration by responding with "Pause Alert" to pause the alerting process in Grafana. Once you fix the issue in Grafana, you can come back to the original xMatters alert and use the response option "Resume Alert" to resume the alerting rule.
Click for image
If it doesn't work, then access the log to find out what's going wrong.
Click the Activity button in the upper right of the screen.
Click for image
Select a flow element such as "Grafana Alert - Inbound from Grafana".
Click for image
Select "Log" in order to view the log.
Click for image
