Skip to main content
Documentation Hub

Backend Explanation

Backend Components​

What are the backend components of a ESP RainMaker deployment?

In an ESP RainMaker deployment, the backend components provide cloud connectivity, device management, storage, and user account services. Here’s a breakdown of the main backend components:

1. MQTT Broker (AWS IoT Core)​

  • What it does: Provides real-time, bi-directional communication between IoT devices and the cloud using the MQTT protocol.

  • How it works: It uses a publish/subscribe model — devices and cloud services can publish messages to topics or subscribe to topics to receive messages.

    AWS IoT Core also handles authentication, authorization, and message routing securely.

  • Example: A device publishes sensor data to the topic device/abc123/status → AWS IoT Core receives the message → a subscribed cloud service (like a Lambda function) processes it → optionally stores it in a database or triggers an alert.

2. Lambda (AWS Lambda)​

  • What it does: Runs backend logic/functions without needing to manage a server.

  • How it works: It is event-driven — gets triggered when something happens, such as a new MQTT message, an API call, or a scheduled event.

  • Example: A user taps a button in the app to turn on a device → triggers a Lambda function → the function sends a command to the MQTT Broker → the device turns on.

3. S3 (Amazon Simple Storage Service)​

  • What it does: Stores static files such as firmware binaries, logs, configuration files, or assets.

  • How it works: Devices or services can upload, download, or read files from S3 via secure APIs or signed URLs. Often used to host OTA firmware updates in IoT systems.

  • Example: The backend generates a signed download URL for firmware-v2.1.0.bin stored in S3 → the device uses that URL to securely download and perform an OTA update.

4. API Gateway​

  • What it does: Acts as the entry point for HTTP requests from apps or web clients.

  • How it works: It receives API calls and routes them to the appropriate backend service, usually Lambda.

  • Example: The mobile app sends a request to /device/123/on → API Gateway receives it → triggers Lambda → sends MQTT command to the device.

5. DynamoDB​

  • What it does: A fast, scalable NoSQL database used to store structured data such as device metadata, user preferences, and state history.

  • How it works: Lambda functions or backend services read/write data to DynamoDB tables using the AWS SDK. It supports key-value and document-style storage, with low-latency performance.

  • Example: A device reports its current temperature → a Lambda function receives the data via MQTT → the function writes { device_id: "abc123", temp: 24.5 } into the DeviceStatus table in DynamoDB → the mobile app reads it later to show the latest status.

6. Cognito​

  • What it does: Manages user authentication and authorization, allowing secure sign-in and access control for mobile/web apps and IoT devices.

  • How it works: Users log in via Cognito using email/password, phone, or third-party providers (like Google/Facebook). Cognito issues secure tokens (JWT) that can be used to access backend APIs or MQTT topics.

  • Example: A user logs in using their email and password → Cognito verifies the credentials → issues an identity token → the mobile app uses this token to call API Gateway or subscribe to secure MQTT topics.

Backend Configuration​

Since ESP RainMaker is a flexible and complete solution with a relatively high degree of freedom, customers can enable and disable functions and adapt to different usage through rich configurations. In the deployment guide, we have given the recommended default configuration required by the deployer. If you want to consult or directly change certain configuration items, please contact esp-rainmaker-support@espressif.com

AWS Service Defaults and Configurable Features in RainMaker​

Espressif’s RainMaker cloud platform is designed to be easy to deploy and scale. The underlying AWS services are configured with default values that are suitable for most customer use cases. Based on extensive testing and deployment experience, the following guidelines and recommendations are provided for reference:

AWS Service Configurations by Default​

  • AWS Lambda Concurrency
    As a standard pre-deployment step, we recommend increasing the AWS Lambda concurrency limit to 1,000. This ensures sufficient performance for operations such as device onboarding, message processing, and scheduled tasks. For deployments expected to scale beyond millions of devices, this limit should be reviewed and adjusted accordingly.

  • Lambda Memory Configuration
    Lambda memory settings have been optimized based on Espressif’s internal benchmarks. The current configuration supports large-scale deployments, including those with millions of devices, without requiring further changes.

  • Amazon DynamoDB Read and Write Capacity
    The default provisioned throughput of 40,000 units is typically sufficient. For higher throughput demands, we recommend enabling auto-scaling or manually adjusting capacity based on usage patterns.

  • Amazon Cognito
    Default Cognito settings are adequate for most device identity and user authentication needs. No additional configuration is generally required.

  • AWS IoT Core
    The default IoT Core service limits meet the needs of standard deployments. For systems approaching or exceeding millions of connected devices, we recommend assessing relevant limits such as the number of device connections and messaging throughput.

  • Other AWS Services
    RainMaker makes use of additional AWS services as part of its full-stack implementation. These services can generally be deployed with their default values. However, for large-scale deployments, especially those targeting millions of devices, it is recommended to review all related service limits to ensure optimal scalability.

Configurable Settings and Fetures in RainMaker​

RainMaker provides extensive configurability to meet a wide range of deployment and product requirements. Below is a non-exhaustive list of features that can be customized by customers:

  • Message templates
  • Device Defender configuration
  • API rate limiting
  • Push notification settings
  • Logging levels
  • Voice assistant integrations
  • API passthrough service
  • Internal user management
  • Webhooks

Additional customization options are available depending on specific use cases and system requirements.

info

For more detailed documentation or assistance with large-scale deployments, please contact esp-rainmaker-support@espressif.com.

Differences between China and Global Deployment Backend​

When deploying the RainMaker platform in China, there are some differences and limitations compared to the global cloud backend, primarily due to regional availability of certain AWS services. Below is a summary of key considerations:

Service Availability and Design Considerations​

  • AWS Cognito
    Not available in the China region.
    → Alternatives must be considered for user identity management.

  • Authentication Protocol
    OAuth 2.0 is the standard authentication mechanism used as mandatory in the China region.

  • AWS SES (Simple Email Service)
    Not available in the China region.
    → Alternative email delivery services should be explored if email-based notification and communication are required.

  • AWS SNS for OTP (One-Time Password) via SMS
    Not supported in China region.
    → Local SMS gateway providers or other alternatives can be evaluated for delivering OTPs to users.

  • Cloud-to-Cloud Integration
    Available in China, but cannot rely on AWS Cognito due to its unavailability.
    → Other identity federation or integration methods can be explored if cloud-to-cloud features are needed.

Notice

For customers targeting the China market, we recommend reviewing these service differences early in the design phase and planning alternative implementations where needed. The Espressif team is available to assist with suitable options and architectural guidance.

On this page