SignalFx Integration

SignalFx is a cloud-based monitoring solution offers flexibility to any size operation. It aggregates and analyzes the metrics needed to detect outliers, manage capacity, and spot service-level trends before they affect performance. OpsGenie is an alert and notification management solution that is highly complementary to SignalFx.

What does OpsGenie offers to SignalFx users?

SignalFx detectors monitor signals and send alerts when they cross defined thresholds. With SignalFx Integration, OpsGenie acts as a dispatcher for these alerts, determining the right people to notify based on on-call schedules, notifying them using email, text messages (SMS), phone calls and iPhone & Android push notifications, and escalating alerts until the alert is acknowledged or closed.

OpsGenie has a specific API for SignalFx Integration, SignalFx sends alerts to OpsGenie and OpsGenie handles automatic creation of alerts.


Functionality of the integration

  • SignalFx detectors monitor signals and send alerts when they cross defined thresholds. When an alert is created in SignalFx, an alert is also created in OpsGenie automatically through the integration.
  • When values is back to normal, SignalFx detectors resolves the alert. When an alert is resolved in SignalFx, an alert is also closed in OpsGenie.

Add SignalFx Integration in OpsGenie

  1. Please create an OpsGenie account if you haven't done already
  2. Go to OpsGenie SignalFx Integration page,
  3. Specify who should be notified for SignalFx alerts using the "Teams" and "Recipients" fields. Auto-complete suggestions will be provided as you type. 
  4. Copy the Api URL by clicking on the copy button or selecting.
  5. Click on "Save Integration".

Configuration in SignalFx

  1. In SignalFx, open hosts view
  2. If you want to create alert for total host metrics open System Metrics tab or to create alerts for a specific host, click a host and open system metrics tab
  3. Choose a metric you want to create alert and click the bell icon upper-right corner
  4. Click New Detector From Chart
  5. In Alert screen; populate name, severity, conditions and trigger duration of the alert
  6. From Notifications dropdown, choose Webhook
  7. Paste into URL field and click UPDATE
  8. Click SAVE

Sample payloads sent from SignalFx to OpsGenie

Sources field of the below content is a string representation of a JSON object. Content of the sources object can differ. That's why whole sources object is added to the available draggable fields to make it possible to extract custom fields from sources object.

Let's say you wanted to put the dsname field of the sources object object to the alert: {{sources.dsname}} solves the problem.

Create Alert payload:

{
    "severity": "Critical",
    "sources": "{\"AWSUniqueId\":\"i-274c99e1_us-west-2_974705804932\",\"dsname\":\"value\",\"host\":\"ip-172-31-41-66.us-west-2.compute.internal\",\"plugin\":\"signalfx-metadata\",\"plugin_instance\":\"utilization\",\"sf_metric\":\"memory.utilization\"}",
    "rule": "Create OpsGenie Alert",
    "alertTimestamp": 1463567930000,
    "eventType": "_SF_PLOT_KEY_Ciu5eDlAgAc_9_3",
    "incidentId": "CiYI3ipAgnA",
    "detector": "Memory Used % Detector",
    "detectorUrl": "https://app.signalfx.com/#/detector/Ciu5eDlAgAc/edit",
    "currentValue": "4.07361626150254",
    "status": "too high"
}

Close Alert payload:

{
    "severity": "Critical",
    "sources": "{\"AWSUniqueId\":\"i-274c99e1_us-west-2_974705804932\",\"dsname\":\"value\",\"host\":\"ip-172-31-41-66.us-west-2.compute.internal\",\"plugin\":\"signalfx-metadata\",\"plugin_instance\":\"utilization\",\"sf_metric\":\"memory.utilization\"}",
    "rule": "Create OpsGenie Alert",
    "alertTimestamp": 1463568222190,
    "eventType": "_SF_PLOT_KEY_Ciu5eDlAgAc_9_3",
    "incidentId": "CiYI3ipAgnA",
    "detector": "Memory Used % Detector",
    "detectorUrl": "https://app.signalfx.com/#/detector/Ciu5eDlAgAc/edit",
    "currentValue": "4.07361626150254",
    "status": "ok"
}

Sample alert