Configure Magic Tunnel health alerts

Magic Transit customers can configure Magic Tunnel health alerts to receive email, webhook, and PagerDuty notifications when the percentage of successful health checks for a Magic Tunnel drops below the selected service-level objective (SLO).

Magic Tunnel health alerts will monitor the health check success rate of each Magic Tunnel included in the alert that has actively transferred customer traffic (excluding health check traffic) over the past six hours. Customers can define an SLO threshold for the percentage of health checks that must be successful for each Magic Tunnel. If the number of successful health checks for the Magic Tunnel(s) included in the alert drops below the SLO threshold, then an alert will fire.

Alert data

If a Magic Tunnel health alert is fired, customers can expect the following data in the email, webhook, and PagerDuty notification:

• Cloudflare account name
• Cloudflare account ID
• Alert type
• Tunnel name
• Tunnel ID
• Tunnel status
• Alert SLO
• Timestamp

Set up Magic Tunnel health alerts

Dashboard

1. Log in to your Cloudflare dashboard ↗, and select your account.
2. Select Notifications > Add.
3. Select Magic Transit > Magic Tunnel Health Check Alert > Select to add a notification.
4. Enter a name and description for the notification.
5. Add webhooks or an email address for the person who should receive the notification, and select Next.
6. Choose the tunnels you want to receive alerts for.
7. Select the Alert Sensitivity Level threshold. It is predefined for Medium, but you can choose between High, Medium, and Low. If you need more than three sensitivity levels, use the API.
8. Select Create when you are done.

API

Note

Refer to the documentation for Notifications to learn about specific permissions you need to access the service via the API.

Send a POST request to create a Magic Transit tunnel health alert. You can set tunnel health alerts with any SLO value between 0 and 99.99.

curl https://api.cloudflare.com/client/v4/accounts/account_id/alerting/v3/policies \
--header 'Authorization: Bearer <YOUR_TOKEN>' \
--header 'Content-Type: application/json' \
--data '{
  "alert_type": "magic_tunnel_health_check_event",
  "description": "<DESCRIBE_POLICY>",
  "enabled": true,
  "filters": {
    "slo": [
      "99.9"
    ]
  },
  "mechanisms": {
    "email": [
      {
        "id": "EMAIL_ADDRESS"
      }
    ],
  "name": "<DESCRIBE_ALERT>"
}'

{
  "result": [
    {
      "id": "f174e90a-fafe-4643-bbbc-4a0ed4fc8415",
      "name": "<POLICY_NAME>",
      "description": "<POLICY_DESCRIPTION>",
      "enabled": true,
      "alert_type": "magic_tunnel_health_check_event",
      "mechanisms": {
        "email": [
          {
            "id": "<YOUR_EMAIL>"
          }
        ]
      },
      "created": "2024-09-11T14:13:29.585658Z",
      "modified": "2024-09-11T14:13:29.585658Z",
      "conditions": {
        "and": [
          {
            "or": [
              {
                "<=": [
                  {
                    "var": "slo"
                  },
                  "99.9"
                ]
              }
            ]
          }
        ]
      },
      "filters": {
        "slo": ["99.9"]
      }
    }
  ],
  "success": true,
  "errors": [],
  "messages": []
}

Recommended SLO thresholds

Currently, there are three SLO threshold values that are supported by the API. The SLO threshold for Magic Tunnel health alerts can be defined as the percentage of health checks that must be successful for each of the Magic Tunnel(s) included in the alert:

Alert Sensitivity Level	Recommended SLO threshold
High	99.0
Medium	98.0
Low	97.0

With these settings, at 100% failure Cloudflare will send alerts at the following time frames, after a problem is detected:

• High sensitivity: First alert within 10 minutes.
• Medium sensitivity: First alert within 20 minutes.
• Low sensitivity: First alert within 30 minutes.

Refer to the Magic tunnels background information page for more information on this topic.

Test SLOs

To test whether a specific alert sensitivity level works for your use case:

1. Create an alert with a specific sensitivity level for a tunnel with active traffic within the past six hours. If you are not sure of what tunnels to choose, refer to Network Analytics to learn how you can view real-time and historical data about your network.
2. Disable the tunnel you are testing, so there is 100% health check failure.
3. The time it takes for Cloudflare to send you an alert will depend on the sensitivity you chose for your alerts (High, Medium or Low).