MQTT

MQTT

MQTT stands for "Message Queueing Telemetry Transport". It uses the publisher-subscriber pattern and it's especially suited for small messages when networks bandwidth is limited or when the network is not reliable. It has become a de facto standard for sensor messaging over TCP.

Check MQTT.org for more info.

ESPurna is built by default with support for MQTT v3.1. To build an image without MQTT support set the MQTT_SUPPORT setting to 0. The current version also supports MQTT over SSL but it is disabled by default since this feature has a heavy memory footprint and is not compatible with other features in the firmware. See "SSL support" below.

Configuration

You can configure MQTT via the web interface or the terminal. Check the commands available in the Terminal page.

Topic format

Root topic

Every MQTT message that ESPurna publishes starts with the root topic you define in the "MQTT Root Topic" setting. That root topic is then complemented by the magnitude (like "temperature", "rfin" or "relay"), an index when there is more that one of such elements (more than one relay) and a trailing particle to tell commands from states.

Commands and states

A state topic is what ESPurna broadcasts telling every listener out there about something that happened ("the temperature is 18.3C"). A command topic is one ESPurna subscribes to to listen to requests from other services. The default state topic particle is "" (empty string, meaning no trailing particle). The default command topic particle is "/set".

As an example, a board with one relay will publish the relay status when it changes to:

{root topic}/relay/0

And will listen to commands to modify the relay status at:

{root topic}/relay/0/set

Placeholders

The root topic may include one of these placeholders:

Placeholder	Value
{hostname}	The hostname of the board as defined in the General tab
{mac}	The MAC of the ESP8266
{magnitude}	The magnitude particle (+1.12.4)

The {magnitude} placeholder is a special one. It indicates where the magnitude particle will be inserted. If you don't specify a location for the magnitude it will be inserted after the root topic. For instance, if you have a temperature sensor called "garden", and you set the root topic to sensor/{magnitude}/{hostname} the messages will be sent to sensor/temperature/garden. In the documentation all topic examples assume the magnitude placeholder is either not used or placed at the end of the root topic.

JSON payload

When the "Use JSON Payload" option is enabled, messages will be grouped in a JSON payload. Internally, messages will be enqueued and sent after a certain time (100 milliseconds). Any message that is also enqueued during that time lapse will reset the count down. When the count down is done all enqueued messages are grouped in a JSON payload and sent to the data specific message topic along with some extra info.

For instance, a sensor that reports temperature and humidity will publish two topics every X seconds like this:

{root topic}/temperature => 18.3
{root topic}/humidity => 65

With the "Use JSON payload" option enabled only one message will be sent:

{root topic}/data => {'temperature': 18.3, 'humidity': 65, 'datetime': '2018-01-31 23:46:17', 'mac': '00:11:22:33:44:55', 'hostname': 'MINI', 'ip': '192.168.1.105', 'id': 37}

Messages

Heartbeat

Heartbeat messages are only state messages and are sent every X seconds (5 minutes by default). These messages report the status of the device and some useful info.

State topic	Example payload	Notes
{root topic}/status	1	see note 1 below
{root topic}/app	ESPURNA
{root topic}/version	1.12.3
{root topic}/hostname	MINI
{root topic}/desc	Kitchen floor light	empty by default
{root topic}/ip	192.168.1.105
{root topic}/mac	00:11:22:33:44:55
{root topic}/uptime	3215	seconds
{root topic}/datetime	2018-02-01 00:03:25	only if NTP synced
{root topic}/freeheap	22056	bytes

(1) This is also the will topic. Upon disconnection and after the keepalive timeout the broker should publish a payload 0 to this topic.

Relay and light status are also sent along with the heartbeat. Check topics for those below.

Actions

An ESPurna device will subscribe to some generic topics listening for different actions to be performed. At the moment these topics are:

Command topic	Payload	Notes
{root topic}/action/set	reboot	Reboots the device
{root topic}/action/set	heartbeat	Force the device to send heartbeat messages
{root topic}/action/set	factory_reset	Erase device's runtime settings storage

Relays

The relay module publishes the relay state and subscribes to command topics to manage the relays via MQTT. The specific message topic will always end with a 0-based index (first relay is index 0).

State topic	Example payload	Notes
{root topic}/relay/0	1	0 for 'off', 1 for 'on'

Command topic	Example payload	Notes
{root topic}/relay/0/set	2 or toggle	see note 1

Command topic	Example payload	Notes
{root topic}/pulse/0/set	0.5	see note 2

(1) Relay command payloads accept both numbers (0 for off, 1 for on and 2 for toggle) or words (on, off, toggle or query; case insensitive). Accepted and sent payload strings are configured by the relayPayloadOn, relayPayloadOff and relayPayloadToggle settings and RELAY_MQTT_ON, RELAY_MQTT_OFF, RELAY_MQTT_TOGGLE string definitions.

(2) Pulse command will toggle the relay status and will schedule another toggle after a specified time in seconds.

Lights

By default, topics are configured as:

• {root topic}/{state topic}
• {root topic}/{command topic}/set (ref. MQTT_SETTER build flag)

We support the following state & command topics:

• brightness
• channel/{number} (refer to the channels assigment table on the Lights page)
• rgb
• hsv
• mired
• kelvin (note: only command topic)

Common

State topic	Example payload
{root topic}/brightness	35 1
{root topic}/channel/{number}	128

Command topic	Example payload
{root topic}/brightness/set	200 1
{root topic}/channel/{number}/set	0

⁽¹⁾ Value ranges from 0 to 255 (ref. LIGHT_{MIN,MAX}_BRIGHTNESS build flag).

Color

State topic	Example payload
{root topic}/rgb	#FF0000 or 255,0,0 2
{root topic}/hsv	300,100,100 3
{root topic}/mired	320 5

Command topic	Example payload
{root topic}/rgb/set	#FFFFFF or 255,255,255 2 3 4
{root topic}/hsv/set	200,50,25 3
{root topic}/mired/set	400 5
{root topic}/kelvin/set	4000

⁽²⁾ HEX color prefixed with # if "Use CSS style" / useCSS setting is enabled, comma-separated values otherwise.

⁽³⁾ #RRGGBBAA values are interpreted like RGB + brightness.

⁽⁴⁾ If more than 3 channels are configured: ch0,ch1,ch2,ch3,.... ch0->R, ch1->G, ch2->B, ch3->(white or WW, or ...)

⁽⁵⁾ Value ranges from 153 to 500 (ref. LIGHT_{COLD,WARM}WHITE_MIRED build flag and / or light{Cold,Warm}Mired setting).

openHAB

Dimmer should be configured as:

Type dimmer : dimmer "Dimmer" [stateTopic = "{root topic}/brightness", commandTopic = "{root topic}/brightness/set", min=0, max=255 ]

The min/max will cause the binding to scale the value of 0-100 to match that of the ESPurna's min-max.

(ref. #1460, thanks to @CrispinP for clarification)

Sensors

The sensors module (see Sensors Configuration) provides a common interface with the rest of the firmware for all sensor defined in the sensors folder. See Sensors for more info about the available sensors.

Sensors publish magnitudes (temperature, power, current, co2,...). If there is more than one magnitude of the same type and index will be added to the magnitude topic (0-based). Available magnitude topics will depend on the available sensors. At the moment they are:

State topic	Example payload	Notes
{root topic}/temperature	18.3	in Celcius or Farenheit
{root topic}/humidity	65	in %
{root topic}/pressure	1018.52	in hPa
{root topic}/current	0.35	in A
{root topic}/voltage	227	in V
{root topic}/power	430	active power, in W
{root topic}/apparent	320	apparent power, in W
{root topic}/reactive	100	reactive power, in W
{root topic}/factor	95	power factor, in %
{root topic}/energy	253654	aggregated, in Ws (J) or kWh
{root topic}/energy_delta	60	since last report, in Ws (J) or kWh
{root topic}/analog	780	from 0 to 1023
{root topic}/digital	1	0 (low) or 1 (high)
{root topic}/event	18	since last report, count
{root topic}/pm1dot0	180	in ppm
{root topic}/pm2dot5	13	in ppm
{root topic}/pm10	5	in ppm
{root topic}/co2	65	in ppm
{root topic}/lux	430	in lux
{root topic}/distance	0.128	in meters
{root topic}/hcho	18	in ppm
{root topic}/ldr_cpm	24	events
{root topic}/ldr_uSvh	0.108	in microsievert
{root topic}/count	12	events

Reported values can configured in the Sensors tab of the web UI or by manually setting the corresponding settings keys in the Debug tab.

Buttons

The button module publishes button events (ref. Buttons & Switches page). The specific message topic will always end with a 0-based index (first button is index 0).

State topic	Example payload	Notes
{root topic}/button/0	3	see table below

The payload value is the one defined by the BUTTON_EVENT_* settings and will be sent as text. See buttons configuration page for more info.

• pressed
• released
• click
• double-click
• long-click
• looong-click
• triple-click

Button will send different event types, depending on whether button is configured as switch or pushbutton.

LEDs

When the LED mode is set to "MQTT Managed", it will accept a command to change current state (ref. LEDs). Valid values are the same as for a relay (0, 1, 2, 'on', 'off' or 'toggle'). The specific message topic will always end with a 0-based index (meaning, the first LED is index 0).

Command topic	Example payload	Notes
{root topic}/led/{index}/set	2	toggle LED status

Since version 1.15.0, we also support setting a repeating pattern via the ledPattern# key. When key is present and is a valid pattern, instead of just turning the LED ON it will be used instead.

Sonoff RF Bridge

The Sonoff RF Bridge has a dedicated module (see RFBRIDGE) that provides an MQTT to RF bridge functionality via the rfin and rfout topics.

State topic	Example payload	Notes
{root topic}/rfin	26C0013603CA511451	received code

Command topic	Example payload	Notes
{root topic}/rflearn/0/set	1	see note 1 below
{root topic}/rfout/set	26C0013603CA511451	send code
{root topic}/rfout/set	26C0013603CA511451,3	send code N times
{root topic}/rfraw/set		see note 2 below

(1) Triggers a learn action. The index after the "learn" magnitude indicates the relay the code will be linked to. The payload of the message indicates the action (0 for off, 1 for on).
(2) Raw codes require a special firmware in the EFM8BB1. See issue #386 for more info.

IR

The IR module enables different MQTT messages in and out via the irin and irout topics.

State topic	Example payload	Notes
{root topic}/irin	2:121944:32	<type>:<code>:<bits>

Command topic	Example payload	Notes
{root topic}/irout/set	2:121944:32:1	<type>:<code>:<bits>[:<repeat(1)>]

The IR module also supports RAW messages when IR_USE_RAW is enabled but format is much more complex. Read the module header for more info.

UART-MQTT module

The UART-MQTT module enables a transparent bridge to and from MQTT to a hardware UART. This is especially interesting for UART sensors like barcode scanners or RFID readers.

State topic	Example payload	Notes
{root topic}/uartin	245324234

Command topic	Example payload	Notes
{root topic}/uartout/set	245324234

Features based on MQTT

Relay & color synchronization across devices

The mqttGroup and mqttGroupInv topics are set per each relay in the board and let you synchronize relays between devices. You can use the web UI or the terminal to set them. Any relay of any device with the same mqttGroup topic (and connected to the same broker, of course) will be synch'd. If you want the relay to be in opposite state set the topic in the mqttGroupInv key instead.

Home Assistant auto-discovery

The Home Assistant module provides two features: output the configuration code to copy-paste it in the config file and send autodiscovery messages via MQTT using the MQTT Discovery feature.

This second feature sends an MQTT message to a certain topic (homeassistant/light/livingroom/config for instance) with the required parameters for HA to configure the light (in this case). It also works for switches and sensors.

Please mind HA does not store these settings in any way (database or YAML config file) but instead it saved them in memory. To avoid sending the message every now and then it is recommended to set the RETAIN flag to ON (in the MQTT tab) so HA will receive the same configuration every time it restarts. In this case, if this feature is not enabled it will send a message to the same topic but empty payload to effectively "erase" the configuration from the broker retained messages.

Domoticz

Domoticz integration is done using the MQTT protocol specification for Domoticz. Currently only for relays and sensors.

Implementation

AsyncMqttClient vs PubSubClient

ESPurna uses the AsyncMqttClient library by Marvin Roger, based on the ESPAsyncTCP library by Hristo Gochkov. This library provides a framework to send and receive MQTT message in an asynchronous way, no blocking sends and event-based messages.

Alternatively, you can compile ESPurna using the PubSubClient library by Nick O'Leary. Please mind that this option is not tested regularly.

MQTT module API

Going a bit deep in the code, the MQTT module provides an API other modules can use. It's not my intention to provide an extensive documentation here but I'd like to show you the main methods so you have a place to start studying the code.

• mqttSend and mqttSendRaw to send messages. They have different signatures but the main difference between them is that mqttSend uses a predefined topic structure (with a topic template, a root topic and getter/setter particles) and the mqttSendRaw will send the message to the topic you specify.

• mqttRegister expects a function with a specific signature that will be called whenever an MQTT event happens. These events can be connection, disconnection and message arriving.

Payload size

The PubSubClient library requires a little modification in order to work with long MQTT message payloads (like when using Domoticz integration). You will need to edit the 'PubSubClient.h' file (for me that file is under the 'C:\Users\xose\Documents\Arduino\libraries\arduino_281549\src\PubSubClient.h' folder), line 26 and change the MQTT_MAX_PACKET_SIZE to at least 512. If you intend to use JSON payload, increase value to 1024. For example:

// MQTT_MAX_PACKET_SIZE : Maximum packet size
#ifndef MQTT_MAX_PACKET_SIZE
#define MQTT_MAX_PACKET_SIZE 512
#endif

The Arduino-MQTT library also uses #define MQTT_MAX_PACKET_SIZE ..., but you can specify it in "code/espurna/config/arduino.h" instead of modifying library source code.

Secure connection

MQTT over SSL / TLS is avaiable through custom build flags.

#define SECURE_CLIENT   SECURE_CLIENT_BEARSSL
#define MQTT_LIBRARY    MQTT_LIBRARY_ARDUINOMQTT
#define MQTT_SECURE_CLIENT_CHECK    SECURE_CLIENT_CHECK_CA
#define MQTT_SECURE_CLIENT_MFLN     1024
#define MQTT_SECURE_CLIENT_INCLUDE_CA   1

Note that TLS encryption requires a lot of both ROM and RAM space, so it's not available in the default binaries. This feature is experimental.

In case INCLUDE_CA flag is enabled, you will need to provide a custom CA certificate in .pem format in the `code/espurna/static/mqtt_client_trusted_root_ca.h' defined as a PROGMEM C string:

const char PROGMEM _mqtt_client_trusted_root_ca[] = R"EOF(
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
)EOF";

See letsencrypt_isrgroot_pem.h in the same directory. By default we include Let's Encrypt ISRG X3 certificate.