Get started

Note

Pub/Sub is currently in private beta. You can sign up for the waitlist ↗ to register your interest.

Pub/Sub is a flexible, scalable messaging service built on top of the MQTT messaging standard, allowing you to publish messages from tens of thousands of devices (or more), deploy code to filter, aggregate and transform messages using Cloudflare Workers, and/or subscribe to topics for fan-out messaging use cases.

This guide will:

• Instruct you through creating your first Pub/Sub Broker using the Cloudflare API.
• Create a <broker>.<namespace>.cloudflarepubsub.com endpoint ready to publish and subscribe to using any MQTT v5.0 compatible client.
• Help you send your first message to the Pub/Sub Broker.

Before you begin, you should be familiar with using the command line and running basic terminal commands.

Prerequisite: Create a Cloudflare account

In order to use Pub/Sub, you need a Cloudflare account. If you already have an account, you can skip this step.

1. Enable Pub/Sub

During the Private Beta, your account will need to be explicitly granted access. If you have not, sign up for the waitlist, and we will contact you when you are granted access.

2. Install Wrangler (Cloudflare CLI)

Note

Pub/Sub support in Wrangler requires wrangler 2.0.16 or above. If you’re using an older version of Wrangler, ensure you update the installed version.

Installing wrangler, the Workers command-line interface (CLI), allows you to init, dev, and publish your Workers projects.

To install wrangler ↗, ensure you have npm installed ↗, preferably using a Node version manager like Volta ↗ or nvm ↗. Using a version manager helps avoid permission issues and allows you to easily change Node.js versions. Then run:

npm

npm install wrangler --save-dev

yarn

yarn add --dev wrangler

Validate that you have a version of wrangler that supports Pub/Sub:

wrangler --version

2.0.16 # should show 2.0.16 or greater - e.g. 2.0.17 or 2.1.0

With wrangler installed, we can now create a Pub/Sub API token for wrangler to use.

3. Fetch your credentials

To use Wrangler with Pub/Sub, you’ll need an API Token that has permissions to both read and write for Pub/Sub. The wrangler login flow does not issue you an API Token with valid Pub/Sub permissions.

Note

This API token requirement will be lifted prior to Pub/Sub becoming Generally Available.

1. From the Cloudflare dashboard ↗, click on the profile icon and select My Profile.
2. Under My Profile, click API Tokens.
3. On the API Tokens ↗ page, click Create Token
4. Choose Get Started next to Create Custom Token
5. Name the token - e.g. “Pub/Sub Write Access”
6. Under the Permissions heading, choose Account, select Pub/Sub from the first drop-down, and Edit as the permission.
7. Select Add More below the newly created permission. Choose User > Memberships from the first dropdown and Read as the permission.
8. Select Continue to Summary at the bottom of the page, where you should see All accounts - Pub/Sub:Edit as the permission.
9. Select Create Token and copy the token value.

In your terminal, configure a CLOUDFLARE_API_TOKEN environmental variable with your Pub/Sub token. When this variable is set, wrangler will use it to authenticate against the Cloudflare API.

export CLOUDFLARE_API_TOKEN="pasteyourtokenhere"

Warning

This token should be kept secret and not committed to source code or placed in any client-side code.

With this environmental variable configured, you can now create your first Pub/Sub Broker!

4. Create your first namespace

A namespace represents a collection of Pub/Sub Brokers, and they can be used to separate different environments (production vs. staging), infrastructure teams, and in the future, permissions.

Before you begin, consider the following:

• Choose your namespace carefully. Although it can be changed later, it will be used as part of the hostname for your Brokers. You should not use secrets or other data that cannot be exposed on the Internet.
• Namespace names are global; they are globally unique.
• Namespaces must be valid DNS names per RFC 1035. In most cases, this means only a-z, 0-9, and hyphens are allowed. Names are case-insensitive.

For example, a namespace of my-namespace and a broker of staging would create a hostname of staging.my-namespace.cloudflarepubsub.com for clients to connect to.

With this in mind, create a new namespace. This example will use my-namespace as a placeholder:

wrangler pubsub namespace create my-namespace

{
  "id": "817170399d784d4ea8b6b90ae558c611",
  "name": "my-namespace",
  "description": "",
  "created_on": "2022-05-11T23:13:08.383232Z",
  "modified_on": "2022-05-11T23:13:08.383232Z"
}

If you receive an HTTP 403 (Forbidden) response, check that your credentials are correct and that you have not pasted erroneous spaces or characters.

5. Create a broker

A broker, in MQTT terms, is a collection of connected clients that publish messages to topics, and clients that subscribe to those topics and receive messages. The broker acts as a relay, and with Cloudflare Pub/Sub, a Cloudflare Worker can be configured to act on every message published to it.

This broker will be configured to accept TOKEN authentication. In MQTT terms, this is typically defined as username:password authentication. Pub/Sub uses JSON Web Tokens (JWT) that are unique to each client, and that can be revoked, to make authentication more secure.

Broker names must be:

• Chosen carefully. Although it can be changed later, the name will be used as part of the hostname for your brokers. Do not use secrets or other data that cannot be exposed on the Internet.
• Valid DNS names (per RFC 1035). In most cases, this means only a-z, 0-9 and hyphens are allowed. Names are case-insensitive.
• Unique per namespace.

To create a new MQTT Broker called example-broker in the my-namespace namespace from the example above:

wrangler pubsub broker create example-broker --namespace=my-namespace

{
  "id": "4c63fa30ee13414ba95be5b56d896fea",
  "name": "example-broker",
  "authType": "TOKEN",
  "created_on": "2022-05-11T23:19:24.356324Z",
  "modified_on": "2022-05-11T23:19:24.356324Z",
  "expiration": null,
  "endpoint": "mqtts://example-broker.namespace.cloudflarepubsub.com:8883"
}

In the example above, a broker is created with an endpoint of mqtts://example-broker.my-namespace.cloudflarepubsub.com. This means:

• Our Pub/Sub (MQTT) Broker is reachable over MQTTS (MQTT over TLS) - port 8883
• The hostname is example-broker.my-namespace.cloudflarepubsub.com
• Token authentication is required to clients to connect.

6. Create credentials for your broker

In order to connect to a Pub/Sub Broker, you need to securely authenticate. Credentials are scoped to each broker and credentials issued for broker-a cannot be used to connect to broker-b.

Note that:

• You can generate multiple credentials at once (up to 100 per API call), which can be useful when configuring multiple clients (such as IoT devices).
• Credentials are associated with a specific Client ID and encoded as a signed JSON Web Token (JWT).
• Each token has a unique identifier (a jti - or JWT ID) that you can use to revoke a specific token.
• Tokens are prefixed with the broker name they are associate with (for example, my-broker) to make identifying tokens across multiple Pub/Sub brokers easier.

Note

Ensure you do not commit your credentials to source control, such as GitHub. A valid token allows anyone to connect to your broker and publish or subscribe to messages. Treat credentials as secrets.

To generate two tokens for a broker called example-broker with a 48 hour expiry:

wrangler pubsub broker issue example-broker --namespace=NAMESPACE_NAME --number=2 --expiration=48h

You should receive a success response that resembles the example below, which is a map of Client IDs and their associated tokens.

{
  "01G3A5GBJE5P3GPXJZ72X4X8SA": "eyJhbGciOiJFZERTQSIsImtpZCI6IkpEUHVZSnFIT3Zxemxha2tORlE5a2ZON1dzWXM1dUhuZHBfemlSZG1PQ1UifQ.
  not-a-real-token.ZZL7PNittVwJOeMpFMn2CnVTgIz4AcaWXP9NqMQK0D_iavcRv_p2DVshg6FPe5xCdlhIzbatT6gMyjMrOA2wBg",
  "01G3A5GBJECX5DX47P9RV1C5TV": "eyJhbGciOiJFZERTQSIsImtpZCI6IkpEUHVZSnFIT3Zxemxha2tORlE5a2ZON1dzWXM1dUhuZHBfemlSZG1PQ1UifQ.also-not-a-real-token.WrhK-VTs_IzOEALB-T958OojHK5AjYBC5ZT9xiI_6ekdQrKz2kSPGnvZdUXUsTVFDf9Kce1Smh-mw1sF2rSQAQ",
}

Each token allows you to publish or subscribe to the associated broker.

7. Subscribe and publish messages to a topic

Your broker is now created and ready to accept messages from authenticated clients. Because Pub/Sub is based on the MQTT protocol, there are client libraries for most popular programming languages. Refer to the list of recommended client libraries.

Note

You can view a live demo available at demo.mqtt.dev ↗ that allows you to use your own Pub/Sub Broker and a valid token to subscribe to a topic and publish messages to it. The JWT field in the demo accepts a valid token from your Broker.

The example below uses MQTT.js ↗ with Node.js to subscribe to a topic on a broker and publish a very basic “hello world” style message. You will need to have a supported Node.js ↗ version installed.

# Check that Node.js is installed
which node
# Install MQTT.js
npm i mqtt --save

Set your environment variables.

export CLOUDFLARE_API_TOKEN="YourAPIToken"
export CLOUDFLARE_ACCOUNT_ID="YourAccountID"
export DEFAULT_NAMESPACE="TheNamespaceYouCreated"
export BROKER_NAME="TheBrokerYouCreated"

We can now generate an access token for Pub/Sub. We will need both the client ID and the token (a JSON Web Token) itself to authenticate from our MQTT client:

curl -s -H "Authorization: Bearer ${CLOUDFLARE_API_TOKEN}" -H "Content-Type: application/json" "https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT_ID}/pubsub/namespaces/namespace/brokers/is-it-broken/credentials?type=TOKEN&topicAcl=#" | jq '.result | to_entries | .[0]'

This will output a key representing the clientId, and a value representing our (secret) access token, resembling the following:

{
  "key": "01HDQFD5Y8HWBFGFBBZPSWQ22M",
  "value": "eyJhbGciOiJFZERTQSIsImtpZCI6IjU1X29UODVqQndJbjlFYnY0V3dzanRucG9ycTBtalFlb1VvbFZRZDIxeEUifQ....NVpToBedVYGGhzHJZmpEG1aG_xPBWrE-PgG1AFYcTPEBpZ_wtN6ApeAUM0JIuJdVMkoIC9mUg4vPtXM8jLGgBw"
}

Copy the value field and set it as the BROKER_TOKEN environmental variable:

export BROKER_TOKEN="<VALUE>"

Create a file called index.js , making sure that:

• brokerEndpoint is set to the address of your Pub/Sub broker.
• clientId is the key from your newly created access token
• The BROKER_TOKEN environmental variable populated with your access token.

Note

Your BROKER_TOKEN is sensitive, and should be kept secret to avoid unintended access to your Pub/Sub broker. Avoid committing it to source code.

const mqtt = require("mqtt");

const brokerEndpoint = "mqtts://my-broker.my-namespace.cloudflarepubsub.com";
const clientId = "01HDQFD5Y8HWBFGFBBZPSWQ22M"; // Replace this with your client ID

const options = {
  port: 8883,
  username: clientId, // MQTT.js requires this, but Pub/Sub does not
  clientId: clientId, // Required by Pub/Sub
  password: process.env.BROKER_TOKEN,
  protocolVersion: 5, // MQTT 5
};

const client = mqtt.connect(brokerEndpoint, options);

client.subscribe("example-topic");
client.publish(
  "example-topic",
  `message from ${client.options.clientId}: hello at ${Date.now()}`,
);
client.on("message", function (topic, message) {
  console.log(`received message on ${topic}: ${message}`);
});

Run the example. You should see the output written to your terminal (stdout).

node index.js

> received message on example-topic: hello from 01HDQFD5Y8HWBFGFBBZPSWQ22M at 1652102228

Your client ID and timestamp will be different from above, but you should see a very similar message. You can also try subscribing to multiple topics and publishing to them by passing the same topic name to client.publish. Provided they have permission to, clients can publish to multiple topics at once or as needed.

If you do not see the message you published, or you are receiving error messages, ensure that:

• The BROKER_TOKEN environmental variable is not empty. Try echo $BROKER_TOKEN in your terminal.
• You updated the brokerEndpoint to match the broker you created. The Endpoint field of your broker will show this address and port.
• You correctly installed MQTT.js ↗.

Next Steps

What’s next?

• Connect a worker to your broker to programmatically read, parse, and filter messages as they are published to a broker
• Learn how PubSub and the MQTT protocol work
• See example client code for publishing or subscribing to a PubSub broker