Dropped messages

The Dropped messages page records the messages TBMQ has permanently lost, together with the reason for each loss and the client and topic involved. It answers what the drop counters on the Monitoring page cannot: which client, which topic, and why.

The page is available to a system administrator.

An entry is created only for a permanent loss — a message the broker will not retry:

• Retained messages are never counted.
• A QoS 1 or 2 message addressed to a persistent session is not counted when its delivery fails, because it remains recoverable from the store: Kafka for an APPLICATION client, Redis for a DEVICE client. If the broker eventually gives up on those retries, that give-up is recorded once instead.
• A QoS 0 message is counted even for a persistent session, since it is never stored.

Drops are aggregated rather than logged one by one. Every drop that shares the same client ID, reason, topic, and direction collapses into a single entry: the Drops counter accumulates, the first and last timestamps bound the period, and the remaining fields — QoS, details, username, certificate CN, and client type — hold the values from the most recent occurrence.

Each entry also has a direction:

• Reception — the message was lost on its way into the broker. The entry is keyed by the publishing client.
• Delivery — the message was lost on its way out to a subscriber. The entry is keyed by the intended subscriber.

Each row of the table shows:

• Last dropped time — when the most recent drop in this group occurred.
• Reason — why the message was dropped. See Reasons.
• Direction — Reception or Delivery.
• Topic — the topic of the dropped message.
• Client ID — the publisher for a reception drop, the subscriber for a delivery drop.
• Drops — how many messages this row represents.

Rows are sorted by Last dropped time, newest first, and the time window defaults to for all time. Use the search field to match a topic or a client ID, and the Filter panel to narrow the table by topic, client ID, QoS, reason, direction, and client type.

Two actions are available on every row:

• Dropped message details — opens the full record.
• Client session — opens the session of the client named in the row. If that client has no active session, TBMQ reports that instead.

The Filter panel is the same on the table and on Analytics, and the filter travels between the two: opening Analytics from the table keeps what you set, and drilling from Analytics back into the table keeps it as well.

Topic and Client ID each carry a match mode:

• Contains — the default. Matches every entry whose value contains the text, ignoring case.
• Exact — matches only an entry whose value is exactly the text. It is case-sensitive, because MQTT topics and client IDs are.

The difference matters in a topic tree: under Contains, traffic/city/center also matches traffic/city/center/density; under Exact it matches traffic/city/center and nothing else. Explore with Contains, isolate a single topic or client with Exact. Note that a value typed in the wrong case returns no rows under Exact, with nothing on screen explaining why.

Reason has two shortcuts above the chips — All selects every reason, Invert replaces the current selection with everything it left out.

Client type filters on the type stored with the entry, Device or Application. Selecting both is the same as selecting neither: no client-type filter is applied. That is deliberate, because the type is not always recorded — TBMQ takes it from the client’s cached session at the moment the drop is written, so a drop recorded after that session has left the cache carries no type at all. Selecting a single type hides those entries; selecting both, or none, keeps them.

Click a row to open its details, grouped into three sections:

• Event — the direction and the reason, with the broker’s own free-text detail underneath it (the name of the rate limit that fired, an exception message, and so on), plus the first and last time this drop occurred summarized as N drops over the period.
• Client — client ID, client type, username, and certificate CN.
• Message — topic and QoS.

Username and Certificate CN identify the client only where the drop happened on that client’s own connection. Drops that occur on an internal queue or on a cross-node delivery path carry no session context, so both fields stay empty there. Certificate CN is filled in only for a client that authenticated with X.509 certificate chain credentials, so it stays empty for every other authentication method.

Clicking the direction, reason, client ID, or topic inside the dialog opens Analytics filtered by that value.

The Analytics view summarizes the same data instead of listing it. Open it with the Analytics button in the table header; the filter set on the table comes with you.

It shows three totals — Total dropped, Reception, and Delivery — a Drops by reason chart, and a Detailed metrics table with one row per reason and direction:

• Reason and Direction — the group being summarized.
• Drops — how many messages were lost in that group.
• Ratio — that group’s share of the total currently in view.
• Cause analysis — a short explanation of what produces this reason and what to check.

The view has its own time window and filter, and both apply to everything on the page — the totals, the chart, the metrics table, and the two rollups below them — so the numbers always describe what you are looking at rather than the whole table.

Two cards below the metrics table roll the same entries up by a single key.

Top topics has one row per topic:

• Topic — the topic being summarized.
• Clients — how many distinct client IDs dropped messages on it.
• Drops — how many messages were lost across the whole group.
• Last dropped time — the most recent drop in the group.

Top clients has one row per client ID, where Topics counts the distinct topics that client dropped messages on. Every column sorts in both cards, and the default order is Drops, highest first.

Together they separate two failure shapes that the flat table cannot. A table row is a single client and topic pairing, so one dead topic published to by a thousand clients fills the list with a thousand near-identical rows. The same situation is one Top topics row with a Clients count of a thousand — a subscriber-side problem, a subscription never created or created on the wrong topic filter. The mirror image, one Top clients row with a high Topics count, is a publisher-side problem: one client publishing where nothing subscribes.

Each card is queried on its own, so its controls affect only it:

• Search matches the topic on Top topics, and the client ID on Top clients.
• Refresh reloads that card; the refresh button in the toolbar reloads the whole page.
• The header line reports how the rows are grouped, the filter in effect, how many of the matching rows are on screen, and the drops those rows account for. That drop count covers the current page only, not every matching group.

Click a row to open the table filtered to that topic or client under an Exact match, with the Analytics time window carried across, so the row you clicked and the list you land on cover the same period.

Both cards rank only the entries the broker still holds, not all traffic ever dropped — see Retention.

Reason	Direction	Cause	What to check
No subscribers	Reception	No subscriber topic filter matched the published topic.	Often expected. If the messages are meant to be consumed, compare the subscribers’ topic filters against this topic.
Rate limits	Reception, Delivery	A message rate limit was exceeded and the surplus discarded.	The details field names the limit that fired. Raise that limit or slow the sender — see backpressure.
Packet too large	Reception	The packet exceeded the listener’s max_payload_size (65536 bytes by default), so the broker rejected the frame before decoding it.	Raise the limit for that listener, or reduce the payload size.
Quota exceeded	Reception	The client published before its CONNECT completed and filled the pre-connect queue (MQTT_PRE_CONNECT_QUEUE_MAX_SIZE, 1000 by default). Everything queued was dropped and the client was disconnected.	Make the client wait for CONNACK before it starts publishing.
Inflight window exceeded	Reception, Delivery	Messages awaiting a QoS 1 or 2 acknowledgement outgrew the in-flight window — the subscriber’s Receive Maximum on delivery, the broker’s MQTT_MAX_IN_FLIGHT_MSGS (1000 by default) on reception. On reception the client is also disconnected.	Acknowledgements are lagging. Check how promptly the client acknowledges, and its throughput.
Channel not writable	Delivery	The subscriber’s outbound buffer passed the write-buffer high water mark, so Netty reported the channel unwritable.	Usually a slow consumer or a saturated network link — see backpressure.
Delivery failed	Delivery	Writing the PUBLISH to the subscriber’s connection failed, usually because the connection dropped mid-send.	Check the subscriber’s connection stability. Only permanent losses appear here; a QoS 1 or 2 copy for a persistent session stays recoverable.
Processing give up	Reception, Delivery	The broker exhausted its acknowledgement-strategy retries for a pack of messages and skipped what was still unacknowledged.	Check downstream availability and the broker logs, or raise the retries setting.
Internal error	Reception, Delivery	The broker could not hand the message to its internal queue, so it was never routed. Covers client publishes, last will messages, and cross-node delivery.	Check Kafka health and the broker logs.

Entries are not kept indefinitely. A cleanup job runs every hour (MQTT_DROPPED_MSG_CLEANUP_CRON, in the MQTT_DROPPED_MSG_CLEANUP_ZONE time zone, UTC by default) and applies two limits:

• Age — an entry is removed once MQTT_DROPPED_MSG_RETENTION_DAYS (7 by default) have passed since its last occurrence.
• Count — MQTT_DROPPED_MSG_MAX_ROWS (100000 by default) caps the table as a safety net. Entries beyond the cap are removed, oldest first.

Because an entry’s age is measured from its last occurrence, a recurring drop stays visible for as long as it keeps happening.