Global – October 29, 2026 -- An MQTT connection can recover while the historian still has a permanent data gap, because reconnection says nothing about where each measurement was durably held during the outage. Store and forward lowers that risk by retaining data near the failure point and replaying it after recovery, but it cannot restore a measurement that was never collected or never persisted at any boundary.
Four pipeline stages each own a separate loss boundary
An industrial MQTT path commonly runs through publisher, broker, collector and historian. A source-side buffer covers a site-to-broker interruption. A broker-side queue helps an offline subscriber receive messages the broker already accepted. A collector-side buffer covers messages after collection but before the historian confirms storage. None of these mechanisms automatically covers the others.
QoS 1 and persistent sessions do not replace historian storage
QoS describes delivery behavior on a single MQTT hop. A persistent session preserves eligible subscription state and queued deliveries only within the broker's configured limits. A gateway that publishes at QoS 1 and reconnects after a WAN outage leaves the broker with nothing to deliver if it discarded samples while offline. A collector can also fail after consuming a message and before the historian write succeeds.
TDengine's taosX-Agent buffers MQTT data to local disk
For TDengine TSDB Enterprise, the documented taosX-Agent Store and Forward capability applies to MQTT real-time collection tasks. When enabled, the Agent writes collected data to a local persistent queue during an Agent-to-taosX interruption and resumes from a confirmed breakpoint after recovery. The documentation states the trade-off: disk-backed buffering adds end-to-end latency and consumes local disk capacity.
The Cache Realtime Data setting enables the buffering and is disabled by default. It protects data only along the Agent-to-taosX path, not measurements lost before they reach the collector. Edition, connector version, cache directory and failure behavior need confirmation before deployment.
Replay must preserve event time and handle duplicates
Payloads should carry a source timestamp where the source can supply one, with unit, time zone convention and clock quality validated. Ingestion time can be kept separately to diagnose delay. Sorting by arrival order during recovery can distort trends and late-data calculations.
Resends can follow an ambiguous failure, so stable event identifiers or sequence numbers should be stored with each record. Deduplicating by value alone is unsafe because two identical readings can be two legitimate observations. Ordering rules by event time, source sequence or arrival time should differ between alarm-like events and high-frequency telemetry.
Buffer sizing starts from the worst credible outage
Required storage is derived from the worst credible outage, sustained message rate, average serialized message size and an operating margin, including retries, serialization overhead and free-space alarms. The governing question is how many hours of each source can be retained locally before the buffer rejects or overwrites data. A capacity policy must be explicit: slow collection, discard by priority, raise an OT alarm or require operator intervention.
Recovery tests must prove history completeness
Testing uses a controlled stream with a known source timestamp and a monotonically increasing sequence number, with one failure injected at a time:
- Disconnect the publisher or gateway from the broker.
- Keep the broker reachable but stop the collector.
- Break the collector-to-historian route while collection continues.
- Restore each route and query the historian for sequence gaps, duplicates, late arrivals and timestamp correctness.
Teams should record buffer depth, disk consumption, reconnect time and expected versus stored record counts. Tests should be repeated with an outage longer than a routine reconnect and with a collector restart. The pass condition is an explainable, queryable historical record set, not a green connection state.