Instant Logs

Instant Logs allows Cloudflare customers to access a live stream of the traffic for their domain from the Cloudflare dashboard or from a command-line interface (CLI). Seeing data in real time allows you to investigate an attack, troubleshoot, debug or test out changes made to your network. Instant Logs is lightweight, simple to use and does not require any additional setup.

Availability

	Free	Pro	Business	Enterprise
Availability	No	No	Yes	Yes

Instant Logs via Cloudflare Dashboard

1. Log in to the Cloudflare dashboard ↗ and select the zone you want to use with Instant Logs.

2. Go to Analytics > Instant Logs.

3. Select Start streaming.

4. (optional) Select Add filter to narrow down the events to be shown.

Fields supported in our HTTP requests dataset can be used when you add filters. Some fields with additional subscriptions required are not supported in the dashboard, you will need to use CLI instead.

Once a filter is selected and the stream has started, only log lines that match the filter criteria will appear. Filters are not applied retroactively to logs already showing in the dashboard.

Instant Logs via CLI

1. Create an Instant Logs Job

Create a session by sending a POST request to the Instant Logs job endpoint with the following parameters:

• Fields - List any field available in the HTTP requests dataset.

• Sample - The sample parameter is the sample rate of the records set by the client: "sample": 1 is 100% of records "sample": 10 is 10% and so on.

Note

Instant Logs has a maximum data rate supported. For high volume domains, we sample server side as indicated in the "sampleInterval" parameter returned in the logs.

• Filters - Use filters to drill down into specific events. Filters consist of three parts: key, operator and value.

All supported operators can be found in the Filters page.

Below we have three examples of filters:

# Filter when client IP country is not Canada:
"filter": "{\"where\":{\"and\":[{\"key\":\"ClientCountry\",\"operator\":\"neq\",\"value\":\"ca\"}]}}"

# Filter when the status code returned from Cloudflare is either 200 or 201:
"filter": "{\"where\":{\"and\":[{\"key\":\"EdgeResponseStatus\",\"operator\":\"in\",\"value\":\"200,201\"}]}}"

# Filter when the request path contains "/static" and the request hostname is "example.com":
"filter": "{\"where\":{\"and\":[{\"key\":\"ClientRequestPath\",\"operator\":\"contains\",\"value\":\"/static\"}, {\"where\":{\"and\":[{\"key\":\"ClientRequestHost\",\"operator\":\"eq\",\"value\":\"example.com\"}]}}"

Example request using cURL:

curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/edge/jobs \
--header "X-Auth-Email: <EMAIL>" \
--header "X-Auth-Key: <API_KEY>" \
--header "Content-Type: application/json" \
--data '{
  "fields": "ClientIP,ClientRequestHost,ClientRequestMethod,ClientRequestURI,EdgeEndTimestamp,EdgeResponseBytes,EdgeResponseStatus,EdgeStartTimestamp,RayID",
  "sample": 100,
  "filter": "",
  "kind": "instant-logs"
}'

Response:

The response will include a new field called destination_conf. The value of this field is your unique WebSocket address that will receive messages from Cloudflare’s global network.

{
  "errors": [],
  "messages": [],
  "result": {
    "id": <JOB_ID>,
    "fields": "ClientIP,ClientRequestHost,ClientRequestMethod,ClientRequestURI,EdgeEndTimestamp,EdgeResponseBytes,EdgeResponseStatus,EdgeStartTimestamp,RayID",
    "sample": 100,
    "filter": "",
    "destination_conf": "wss://logs.cloudflare.com/instant-logs/ws/sessions/<SESSION_ID>",
    "kind": "instant-logs"
  },
  "success": true
}

2. Connect to WebSocket

Using a CLI utility like Websocat ↗, you can connect to the WebSocket and start immediately receiving logs.

websocat wss://logs.cloudflare.com/instant-logs/ws/sessions/<SESSION_ID>

Response:

Once connected to the websocket, you will receive messages of line-delimited JSON.

Angle Grinder

Now that you have a connection to Cloudflare’s websocket and are receiving logs from Cloudflare’s global network, you can start slicing and dicing the logs. A handy tool for this is Angle Grinder ↗. Angle Grinder lets you apply filtering, transformations and aggregations on stdin with first class JSON support. For example, to get the number of visitors from each country you can sum the number of events by the ClientCountry field.

awebsocat wss://logs.cloudflare.com/instant-logs/ws/sessions/<SESSION_ID> | agrind '* | json | sum(sampleInterval) by ClientCountry'

Response:

ClientCountry	_sum
pt	4
fr	3
us	3
om	2
ar	1
au	1

Datasets available

For the moment, HTTP requests is the only dataset supported. In the future, we will expand to other datasets.

Export

You can download the table of logs that appears in the dashboard, in JSON format via the Export button.

Limits

Instant Logs has three limits set in place:

• Only one active Instant Logs session per zone.
• Maximum session time is 60 minutes.
• If you stop listening to a socket for more than five minutes.

If either of these limits are reached, the logs stream will automatically stop.

Connect with us

If you have any feature requests or notice any bugs, share your feedback directly with us by joining the Cloudflare Developers community on Discord ↗.