Node to Cloud Communication
This page outlines how nodes securely interact with the cloud using MQTT over TLS. Each node possesses a unique identifier and credentials established during the claiming process, ensuring secure, authenticated communication. The node configuration, detailing attributes like devices and parameters, is sent to the cloud upon startup, enabling clients such as mobile apps to interpret and utilize this data. Understanding this communication framework is essential for developers to effectively integrate and manage IoT devices within the ESP RainMaker ecosystem.
All communication between the node and cloud occurs over secure MQTT.
How Does a Node Ensure Secure Communication With the Cloud?
A node ensures secure communication with the cloud by providing:
- Transport Layer Security (TLS)
- MQTT Policies
TLS
The communication between a node and the cloud is secured using TLS encryption with X.509 certificate-based mutual authentication, ensuring only trusted devices can connect.
A generic workflow to establish secure communication:
- The node first generates its own private key and then creates a Certificate Signing Request (CSR), which is like applying for an official ID.
- This request is sent to a Certification Authority (CA), which verifies it, signs it, and returns a client certificate—essentially an authenticated ID badge.
- During the Claiming process, the node receives this certificate, allowing it to establish a secure, encrypted connection with the cloud.
- Before any data exchange, both the node and the cloud verify each other's certificates to confirm they are communicating with the correct entity.
- This ensures privacy, security, and protection against unauthorized access, preventing fake or malicious devices from infiltrating the system.
- Ensures that only trusted nodes can connect to the cloud.
- Prevents hackers or fake devices from impersonating real nodes.
- Keeps all communication private and secure.
MQTT Policies
The MQTT policy ensures that a node can only connect to the cloud if its client ID matches its node ID, preventing unauthorized access. Additionally, the node is restricted to publishing and subscribing only to topics that start with node/<node_id>/, meaning it cannot access topics from other nodes.
For example, if a node has an ID of 1234, it can only interact with topics like node/1234/status or node/1234/command, but it cannot subscribe to node/5678/status.
This policy acts as a security mechanism to keep each node's communication isolated and protected, ensuring that data remains private and that nodes cannot interfere with each other.
Summary
| Security Aspect | Description |
|---|---|
| TLS | Secures the communication channel so that no one can eavesdrop or tamper with messages. |
| MQTT Policies | Defines rules on what a node is allowed to communicate within the secured channel. |
MQTT Topics What Is an MQTT Topic?
In ESP RainMaker, an MQTT topic acts as a communication path where a node publishes (sends) updates and subscribes (receives) commands or configurations. Here are the different types of MQTT topics in ESP RainMaker.
- Node Configuration
- Node Parameters
- User-Node Mapping
- Time-Series
- Alert and Notify
- Command Response
- OTA
Node Configuration
In ESP RainMaker, node configuration is a collection of all the information about a node (e.g., ESP32-S3). When the node starts up, it sends this configuration to the ESP RainMaker Cloud and also reports it to clients, such as mobile apps and command-line interfaces (CLI). This configuration contains everything needed to define the node's identity, features, and capabilities.
Below is the node configuration layout that ESP RainMaker firmware and phone apps currently use in the order from top to bottom.
Node Configuration Layout (Toggle View)
- Boxes in RED are mandatory entries in a node configuration.
- Boxes in GREEN are mandatory parameters of an entry.
- The strings below each entry indicate the corresponding key (JSON representation, Data type).
1. Node ID
2. Config Version
2. Information
3. Node Attributes
4. Devices & Params
MQTT
- Topic:
node/<node_id>/config - Operation:
PUBLISH - Data:
Node configuration JSON object described below
JSON Payload (Toggle View)
{
"node_id": "5d898491-a498-48ec-9476-a4e23519fe31",
"config_version": "2019-02-27",
"info": {
"name": "My_Bridge",
"fw_version": "1.0",
"type": "Bridge"
},
"attributes": [{
"name": "serial_num",
"value": "123abc"
}, {
"name": "model",
"value": "myBridge-2019"
}],
"devices": [{
"name": "Switch",
"primary": "power",
"params": [{
"name": "name",
"type": "esp.param.name",
"data_type": "string",
"properties": ["read", "write"]
}, {
"name": "power",
"data_type": "bool",
"properties": ["read", "write", "time_series"],
"ui_type": "esp.ui.toggle"
}]
}, {
"name": "Light",
"type": "esp.device.lightbulb",
"attributes": [{
"name": "serial_number",
"value": "012345"
}, {
"name": "mac",
"value": "xx:yy:zz:aa:bb:cc"
}],
"primary": "power",
"params": [{
"name": "name",
"type": "esp.param.name",
"data_type": "string",
"properties": ["read", "write"]
}, {
"name": "power",
"data_type": "bool",
"properties": ["read", "write", "time_series"],
"ui_type": "esp.ui.toggle"
}, {
"name": "brightness",
"data_type": "int",
"properties": ["read", "write"],
"bounds": {
"min": 0,
"max": 100
},
"ui_type": "esp.ui.slider"
}]
}]
}
By default, the device reports once after powering on and connecting to the cloud. It will also report when esp_rmaker_report_node_details is called.
Firmware API: esp_rmaker_report_node_details()
Node Parameters
All parameter changes are managed through specific node parameter topics.
Initial Reporting
Initial reporting is done every time the node boots up. Values of all the parameters of all the devices are reported in this.
MQTT
- Topic:
node/<node_id>/params/local/init - Operation:
PUBLISH - Data:
{"<device-name">:{"<param-name>":<value>,...},...}
JSON Payload (Toggle View)
{
"Lightbulb": {
"name": "Bedroom Light",
"power": true,
"brightness": 55
}
}
When the node powers on for the first time and connects to the cloud, it will report the current values of all parameters.
The node can also call the esp_rmaker_report_node_state API to report all parameter values.
Firmware API: esp_rmaker_report_node_state()
Subsequent Reporting
Any subsequent changes on the node, either due to remote control or a local change, are reported on this topic. Only the parameter that has changed is required in the payload. It is used to report parameter values to the cloud.
MQTT
- Topic:
node/<node_id>/params/local - Operation:
PUBLISH - Data:
{"<device-name">:{"<param-name>":<value>,...},...}
JSON Payload (Toggle View)
{
"Lightbulb": {
"brightness": 75
}
}
When device parameters change and need to be reported to the cloud, call the esp_rmaker_param_update_and_report API.
Firmware API: esp_rmaker_param_update_and_report()
Remote Control
A node must always stay subscribed to this topic, listening for updates triggered by the clients like phone apps or CLI. Any change in parameter values as per the updates must be reported back using the Subsequent Reporting.
MQTT
- Topic:
node/<node_id>/params/remote - Operation:
SUBSCRIBE - Data:
{"<device-name">:{"<param-name>":<value>,...},...}
JSON Payload (Toggle View)
{
"Lightbulb": {
"brightness": 100
}
}
After the node powers on and connects to the cloud, it will automatically subscribe to this topic.
User-Node Mapping
Before the devices under a node can be monitored and controlled remotely, the node should be first mapped to a user so that only that specific user has permission to access it. This happens in the background during the Wi-Fi provisioning stage. The node sends its node_id to the client and the client sends the user_id and secret_key to the node for mapping. Once connected to the RainMaker Cloud, the node sends the mapping request. Please check User-Node Mapping for a detailed workflow.
MQTT
- Topic:
node/<node_id>/user/mapping - Operation:
PUBLISH - Data:
{"node_id":"<node_id>","user_id":"<user_id>","secret_key": "<secret_key>"}
JSON Payload (Toggle View)
{
"node_id": "112233AABBCC",
"user_id": "02e95749-8d9d-4b8e-972c-43325ad27c63",
"secret_key": "9140ef1d-72be-48d5-a6a1-455a27d77dee"
}
Triggered when the device is bound or unbound to the network. The user can also manually trigger it by calling the esp_rmaker_start_user_node_mapping API. Alternatively, if you would like user-node mapping to happen automatically during Wi-Fi setup, call esp_rmaker_user_mapping_endpoint_register().
Firmware API: esp_rmaker_start_user_node_mapping() or esp_rmaker_user_mapping_endpoint_register()
Time-Series
There are two kinds of time-series services that ESP RainMaker offers. One is the fully featured time-series and the other is simple time-series. For the differences between these two services, please refer to Time-Series Data.
Time-Series Data
Time-series data is used for reporting values that require complete time-stamped history and operations to find min, max, average, etc. in a given time window. Fully featured time-series allows the reporting of multiple parameters and multiple records in a single MQTT message. It is used for historical record functionality. Compared to the below Simple Time-Series Data, Time-Series Data uses AWS's native functionality, supporting data statistics and calculations, but at a higher cost.
MQTT
- Topic:
node/<node_id>/tsdata - Operation:
PUBLISH
Data Template (Toggle View)
{
"ts_data_version": "2021-09-13",
"ts_data": [
{
"name": "<param_name>",
"dt": "<int/float/bool/string>",
"records": [
{
"t": <epoch_timestamp>,
"v": <value>
},
{
more records...
}
]
},
{
more parameters...
}
]
}
JSON Payload (Toggle View)
{
"ts_data_version": "2021-09-13",
"ts_data": [
{
"name": "Temperature Sensor.Temperature",
"dt": "float",
"records": [
{
"t": 1699468430,
"v": 26.5
}
]
}
]
}
When a device attribute is set to PROP_FLAG_TIME_SERIES, and the esp_rmaker_param_update_and_report() API is called, time-series data for that parameter will be reported.
Simple Time-Series Data
This allows reporting only a single parameter's single data point in an MQTT message. It is used for historical record functionality. Compared to Time-Series Data, Simple Time-Series Data only stores raw data and does not support cloud-side statistics and calculations, making it less expensive.
MQTT
- Topic:
node/<node_id>/simple_tsdata - Operation:
PUBLISH
Data template (Toggle View)
{
"name": "<param_name>",
"dt": "<int/float/bool/string>",
"t": <epoch_timestamp>,
"v": <value>
}
JSON Payload (Toggle View)
{
"name": "Temperature Sensor.Temperature",
"dt": "float",
"t": 1704189730,
"v": 25.5
}
When a device attribute is set to PROP_FLAG_SIMPLE_TIME_SERIES, and the esp_rmaker_param_update_and_report() is called, simple time-series data for that parameter will be reported.
Note that the parameter names in the examples are a combination of device name and parameter name. This is just for the phone apps to link to the appropriate parameter in the node configuration. However, at a specification level, time-series data is independent of other RainMaker concepts like device, parameters, etc., allowing it to be used independently, if required.
Alert and Notify
This updates a parameter's value and notifies the cloud and connected clients or triggers an alert for a specific event or condition. After publishing, the mobile app will show a notification popup.
MQTT
- Topic:
node/<node_id>/alert - Operation:
PUBLISH
When the node is required to send an alert or notify, it calls either esp_rmaker_param_update_and_notify or esp_rmaker_raise_alert.
Firmware API: esp_rmaker_param_update_and_notify() or esp_rmaker_raise_alert()
Command-Response
Command-Response can be used when a node is offline. The cloud can configure the device via the app, and when the node comes online, it will receive the pushed configuration. The cloud can have a command-response configuration which can be added and modified via the Swagger API at Command-Response Requests, and it will push the configuration to the node after the node powers on.
To Node
MQTT
- Topic:
node/<node_id>/to-node - Operation:
SUBSCRIBE
After enabling the command-response feature, the device will automatically subscribe to this topic after powering on and establishing a cloud connection.
From Node
Used for responding to commands from the cloud.
MQTT
- Topic:
node/<node_id>/from-node - Operation:
PUBLISH
After receiving the command from the platform, the node will process it and internally respond via the topic stated above.
OTA
OTA uses MQTT topics to manage the OTA process, including reporting OTA status, fetching OTA tasks, and listening for OTA updates.
Report OTA Task Status
Reports the status during the OTA process.
MQTT
- Topic:
node/<node_id>/otastatus - Operation:
PUBLISH
During the OTA process, the device will report the current OTA status. The user can also call the esp_rmaker_ota_report_status API to report the status.
Firmware API: esp_rmaker_ota_report_status()
Get OTA Task
The device queries OTA tasks. After the platform forces an upgrade or the user confirms the upgrade, if the device is offline, it can fetch the OTA tasks and perform the upgrade when it comes online.
MQTT
- Topic:
node/<node_id>/otafetch - Operation:
PUBLISH
After the device powers on and connects to the cloud, it will actively fetch the OTA tasks. By default, it fetches the OTA tasks every 24 hours. The user can also manually trigger it by calling the esp_rmaker_ota_fetch or esp_rmaker_ota_fetch_with_delay API.
Firmware API: esp_rmaker_ota_fetch() or esp_rmaker_ota_fetch_with_delay()
Listen OTA Task
When a forced OTA or user-approved OTA is created on the dashboard, and the user triggers the upgrade via the app, the cloud will push the OTA URL, firmware version, firmware size, and other information to this topic.
MQTT
- Topic:
node/<node_id>/otaurl - Operation:
SUBSCRIBE
After the device connects to the cloud, it will automatically subscribe to the otaurl topic to listen for OTA tasks issued by the cloud.