Mosquitto Alternative: Moving to a Clustered Broker

What Mosquitto is built for

Mosquitto is a single lightweight binary that runs on a router, a single-board computer or a spare VM, and it implements MQTT 5.0. That combination is what makes it the usual choice for a first deployment — and it is a design that commits, deliberately, to one process on one host.

That commitment is the whole story of this page. What Mosquitto does well follows from staying small, and what it cannot do follows from the same decision. So the question is not whether it is a good broker — it is whether your deployment has outgrown a single one.

One scoping note: Mosquitto here means the open-source Eclipse project. Cedalo's commercial Pro Edition is a separate product with its own feature set and license, and is not what the sections below compare.

The three walls

Deployments outgrow Mosquitto in three specific ways. They are worth naming precisely, because only the first one is really about scale.

The cluster wall

Mosquitto runs as a single broker. It can bridge to another broker, and a bridge does forward messages — but it shares no session state and no subscription state. Two bridged brokers are two brokers, not one logical broker: a client's session lives on the node it connected to, and if that node is gone, the session is gone with it.

Within one broker, vertical scaling is the only headroom you have, and less of it than it looks: all message handling runs on a single thread, so a bigger machine buys memory for more connections, not cores for more throughput. Splitting clients across independent brokers adds capacity when traffic partitions cleanly — but that is several brokers, not one.

The durability wall

When Mosquitto returns a PUBACK, the message is in that process's memory — and at the defaults that is the only place it ever is: persistence is off, so even a clean restart comes back empty.

Enable persistence and queued and retained messages do reach disk, but only after the acknowledgement. With the built-in persistence true, that happens at the next autosave, thirty minutes by default. With 2.1's SQLite persistence plugin, which is used instead of persistence true, it happens at the next commit, within five seconds — though at the plugin's default sync normal, a power cut can still roll back the latest commits. Both also write on a clean shutdown, so a crash or a power cut takes whatever changed since the last durable write: queued messages, retained messages and session state. The acknowledgement your publisher already holds is therefore a promise about one process on one machine.

The mixed-traffic wall

This one is the least obvious of the three, and it is usually what actually hurts. IoT traffic is not one kind of traffic. Thousands of devices publish telemetry continuously. A handful of backend applications subscribe to high-volume streams and must not miss anything. Commands target one specific client and have to arrive even when that client is briefly offline.

Mosquitto gives all three the same treatment: one queue policy for every session, held in memory. A command for a device that is briefly offline waits in that same memory and, at the defaults, does not survive a restart. A backend consumer that falls behind does not slow the devices down — once 1,000 messages are queued for it, the broker drops the rest, while each publisher still gets its PUBACK. The client that must not miss anything is the one that loses messages.

How TBMQ answers each one

TBMQ is built on Kafka for durability and distribution, Netty for non-blocking transport, and an actor per client for concurrency. Each of the three sections below answers one wall.

One cluster, no coordinator

Every TBMQ node is identical. There is no master and no leader election, so no broker node becomes a bottleneck or a single point of failure — Kafka, Valkey/Redis and PostgreSQL need their own replication for that to hold end to end. Persistent-session and subscription state is shared through Kafka, so when a persistent client reconnects after a node failure, any node in the cluster can resume its session. Adding a node needs no resharding and no downtime: consumer groups rebalance and load spreads across the larger cluster. At the defaults the shared message topic has 16 partitions and each node runs four consumers on it, so past four nodes, raise the partition count as well.

Identical nodes are also what keeps the deployment ordinary. TBMQ ships an official Helm chart, with setup guides for EKS, AKS, GKE and minikube, and every node exposes a health endpoint — so draining and replacing one is a Kubernetes rollout rather than a broker-specific procedure. No node has to go last.

Acknowledged means stored

TBMQ does not send a PUBACK or PUBREC until Kafka has confirmed the write; if Kafka cannot take it, the publisher never gets a success — an MQTT 5.0 client receives an error reason code, and an MQTT 3.1.1 client is disconnected. Once the publisher holds that acknowledgement, the message no longer depends on the node that received it: if that node dies, another picks up from Kafka and continues delivery. Being precise about the limit — the shipped defaults keep a single copy (acks=1, replication factor 1), so surviving the loss of a Kafka broker means running a replicated Kafka with matching producer acknowledgements. The guarantee is real; how far it extends is something you configure.

Device and application traffic are delivered separately

TBMQ splits persistent clients into two types, which you assign to match how each one behaves. DEVICE clients publish often and subscribe to little; while one is offline its messages wait in a per-client queue in Valkey/Redis — up to 10,000 messages for seven days by default, oldest dropped first — which absorbs high write rates and hands the queue back quickly on reconnect.

APPLICATION clients subscribe to high-volume topics, and each one gets a dedicated Kafka topic and its own consumer, so a backlog accumulates safely in Kafka and is replayed in order from that client's committed offset, kept for up to seven days or about 1 GB, whichever comes first, by default. A slow application subscriber's backlog stays in its own topic, so it cannot hold up delivery to devices. Every publish still enters through one shared Kafka topic, though, so a large enough device burst delays everything behind it, on TBMQ as on any broker.

What else you get

Two things that are not answers to a wall, and one licensing note.

Shared subscriptions that add throughput

Mosquitto implements MQTT 5.0 shared subscriptions and so does TBMQ, so this is not a support difference — it is a difference in what adding a member does. Register an application shared subscription and the group gets its own Kafka topic; each subscribing APPLICATION client joins that topic's consumer group, so an extra instance picks up a share of the partitions and the group rebalances when one dies. Messages keep accumulating in Kafka while every member is offline, and delivery resumes from the committed offset. Shared subscriptions covers the DEVICE-client behavior and the load-balancing strategies.

You can change it while it runs, not just watch it

Mosquitto 2.1 added a dashboard over the $SYS counters, so seeing broker state is no longer the gap it was. What it reports is totals and rates — how many clients are connected, how much traffic moved, how many messages are retained. What it cannot tell you is which: there is no client list, no subscription list, no topic browser. And it changes nothing, because it is read-only.

Configuration is a file. A reload signal does re-read much of it, bridges included, but it cannot add or remove a listener, and it cannot load a plugin or change a plugin's options — although a plugin that is already loaded does re-read its own password or ACL file. In 2.1 that plugin limit reaches further than it sounds, because plugins are now the recommended route for authentication, authorization and persistence.

Adding the password-file plugin to a running broker, for example, is a restart, not a reload. Ask for it with a reload and allow_anonymous false takes effect while the plugin holding the passwords does not, which locks out everyone until the restart. (The deprecated password_file option still reloads.) Runtime change is limited to authentication and authorization, driven over $CONTROL topics with mosquitto_ctrl and the dynamic-security plugin; 2.1's broker control API is off by default and only lists plugins and listeners.

In TBMQ those are entities, not file entries. Credentials and authorization rules, integrations, shared subscriptions, blocked clients and users are created and edited while the cluster runs, from the web UI or the REST API, with no restart and no file to ship to every node. Authentication providers are switched on or off, configured and reordered the same way — in Mosquitto, where authentication is a plugin, that change takes a restart. The rest of the broker-level tuning, such as listeners and size and rate limits, is a config file in both: Mosquitto reloads part of it on a signal, and TBMQ applies it on a restart.

The same views answer the operational questions — which sessions are connected and what each is subscribed to, which connection attempts were rejected and why — plus a built-in WebSocket client for publishing a test message without leaving the browser, and metrics export to Prometheus.

Open source, and what Professional Edition adds

Everything above is Community Edition, Apache License 2.0, commercial use included. Professional Edition adds OAuth 2.0 single sign-on, role-based access control, audit logs, white labeling and a dropped messages view that records every message the broker permanently lost, with the reason, client and topic — on the same broker, not a different one.

What the architecture is built to do

Each figure comes from a published TBMQ test, with its hardware and load profile documented.

On methodology: these are TBMQ tests on AWS, not a same-hardware comparison. Mosquitto publishes no benchmark results, so there is no official head-to-head number to quote.

Full benchmark methodology and results →

At a glance

Where Mosquitto still fits

Three cases where the move is not worth making yet — provided you accept what each one implies. If your deployment is a few thousand clients on one host, and you can live with every broker restart being an outage for all of them and a crash losing whatever had not reached disk, TBMQ's Kafka, Valkey/Redis and PostgreSQL dependencies cost you operational surface and buy you nothing. If the hardware is a gateway or an edge box with a few hundred megabytes of RAM, TBMQ is not built for that envelope — keep a small broker there and bridge it into a central TBMQ cluster, which is the usual shape once there is more than one site. And if you want the protocol and nothing else — no cluster, no admin API, no integrations — and are content to run the broker from config files, reload signals and restarts, the extra moving parts are a cost rather than a feature.

Moving over without a flag day

You do not have to cut over at once. One Mosquitto bridge in MQTT 5.0 mode carries the bridged topics in both directions, so clients on either broker see each other's messages while you move them across in batches.

Point one Mosquitto bridge at TBMQ

Add a bridge connection to Mosquitto with bridge_protocol_version mqttv50, notifications false and a topic … both line for each topic tree you are moving, and give it its own APPLICATION credentials in TBMQ. Every existing client keeps its current endpoint and configuration.

Let the same bridge carry traffic back

Messages published on TBMQ return over the same connection, so consumers that still read from Mosquitto keep working with no coordinated change. In MQTT 5.0 mode the bridge sets the No Local flag on its own subscription to TBMQ, so it never receives its own messages back — an MQTT 3.1.1 bridge in both directions delivers each message twice, and adding TBMQ's MQTT Integration as a second leg in the opposite direction loops it endlessly.

Repoint clients in batches, then remove the bridge

Devices first, backend consumers last, and no maintenance window at any point. Sessions do not cross the bridge: a repointed client starts a fresh session on TBMQ, and anything still queued for it on Mosquitto stays there, so move persistent clients while they are connected and caught up.

The parts that are easy to forget

Seven things that are not in the diagram and are the usual source of surprises.

Quickstart →  ·  Cluster setup →  ·  Try the live demo →  ·  Talk to us →