MQTT Tutorial The Ultimate MQTT Tutorial: From Basics to Advanced

What is MQTT?

MQTT (Message Queuing Telemetry Transport) is a lightweight, publish-subscribe network protocol designed for constrained devices and low-bandwidth, high-latency, or unreliable networks. Originally developed by IBM in the late 1990s for monitoring oil pipelines via satellite, MQTT has since become the de facto standard for IoT communications.

Key characteristics of MQTT include:

• Lightweight: Minimal packet overhead makes it ideal for IoT and M2M communications
• Publish/Subscribe Pattern: Decouples message senders (publishers) from receivers (subscribers)
• Quality of Service: Three levels to guarantee message delivery as needed
• Persistent Sessions: Allows for disconnected operation in unreliable networks
• Small Code Footprint: Perfect for embedded devices with limited resources

Core MQTT Concepts

Broker

The broker is the central hub of any MQTT system. It receives all messages from clients, filters them, and distributes them to the appropriate subscribers. The broker is responsible for:

• Managing client connections
• Receiving published messages
• Filtering messages based on topics
• Distributing messages to subscribed clients
• Handling QoS levels for message delivery
• Managing retained messages and persistent sessions

Clients

MQTT clients are any devices that run an MQTT library and connect to the broker over a network. A client can be:

• Publisher: Sends messages to specific topics
• Subscriber: Receives messages from topics it has subscribed to
• Both: Most clients both publish and subscribe to topics

Topics & Topic Structure

Topics are the addressing mechanism in MQTT. They are UTF-8 strings organized in a hierarchical structure with levels separated by forward slashes (/). For example:

home/livingroom/temperature
building/floor1/room3/humidity
device/12345/status

This hierarchical structure allows for flexible message filtering using wildcards:

• Single-level wildcard (+): Matches exactly one topic level

  home/+/temperature    # Matches home/livingroom/temperature but not home/livingroom/kitchen/temperature

• Multi-level wildcard (#): Matches any number of topic levels (must be at the end)

  home/#    # Matches home/livingroom/temperature, home/kitchen/light, etc.

Best practices for topic design:

• Use specific topic structures that reflect your application's hierarchy
• Include device identifiers or types in topics for better organization
• Avoid unnecessarily deep topic levels (more than 5-7 levels)
• Consider using topic namespaces to separate different applications

Quality of Service (QoS) Levels

MQTT provides three Quality of Service (QoS) levels that guarantee different levels of message delivery:

Higher QoS levels provide more reliable delivery but require more bandwidth and introduce more latency. Choose the appropriate QoS level based on your application's requirements for reliability versus efficiency.

MQTT Messages

Message Format

An MQTT message consists of:

• Topic: The destination for the message
• Payload: The actual content of the message (can be in any format)
• QoS: The Quality of Service level
• Retain Flag: Whether the message should be retained by the broker

Retained Messages

When a client publishes a message with the retain flag set to true, the broker stores the message. When new clients subscribe to that topic, they immediately receive the last retained message, even if it was published before they subscribed.

This is useful for:

• Device state information (so new subscribers know the current state)
• Configuration values that should be immediately available to clients
• Default settings or initialization values

Last Will and Testament (LWT)

MQTT allows clients to register a "last will" message with the broker during connection. If the client disconnects ungracefully (without sending a proper DISCONNECT message), the broker will publish this last will message to a specified topic.

This is particularly useful for:

• Notifying other clients when a device goes offline unexpectedly
• Implementing presence detection for IoT devices
• Triggering failover mechanisms

Implementation Guide

Setting Up a Connection

Python Example (using Paho MQTT)

import paho.mqtt.client as mqtt

# Callbacks
def on_connect(client, userdata, flags, rc):
    print(f"Connected with result code {rc}")
    # Subscribe to topics on successful connect
    client.subscribe("home/temperature")

def on_message(client, userdata, msg):
    print(f"Received message on {msg.topic}: {msg.payload.decode()}")

# Create client instance
client = mqtt.Client()

# Assign callbacks
client.on_connect = on_connect
client.on_message = on_message

# Optional: Set username and password
client.username_pw_set("username", "password")

# Optional: Enable TLS/SSL
# client.tls_set(ca_certs="ca.crt")

# Connect to broker
client.connect("mqtt.example.com", 1883, 60)

# Start networking loop
client.loop_forever()

JavaScript Example (using MQTT.js)

const mqtt = require('mqtt');

// Connect to broker
const client = mqtt.connect('mqtt://mqtt.example.com:1883', {
  username: 'username',
  password: 'password',
  clientId: 'mqttjs_client_' + Math.random().toString(16).substr(2, 8)
});

// Handle connection events
client.on('connect', function () {
  console.log('Connected to MQTT broker');
  
  // Subscribe to topics
  client.subscribe('home/temperature', function (err) {
    if (!err) {
      // Publish a message
      client.publish('home/status', 'Hello MQTT!');
    }
  });
});

// Handle incoming messages
client.on('message', function (topic, message) {
  console.log(`Received message on ${topic}: ${message.toString()}`);
});

Publishing Messages

Python Example

# Publish a non-retained message with QoS 0
client.publish("home/temperature", "22.5", qos=0, retain=False)

# Publish a retained message with QoS 1
client.publish("home/status", "online", qos=1, retain=True)

# Publish a message with QoS 2
client.publish("home/alert", "Motion detected!", qos=2, retain=False)

JavaScript Example

// Publish a non-retained message with QoS 0
client.publish('home/temperature', '22.5', { qos: 0, retain: false });

// Publish a retained message with QoS 1
client.publish('home/status', 'online', { qos: 1, retain: true });

// Publish a message with QoS 2
client.publish('home/alert', 'Motion detected!', { qos: 2, retain: false });

Subscribing to Topics

Python Example

# Subscribe to a single topic with QoS 1
client.subscribe("home/temperature", qos=1)

# Subscribe to multiple topics with different QoS levels
client.subscribe([
    ("home/temperature", 0),
    ("home/humidity", 1),
    ("home/+/status", 2)  # Using wildcard
])

JavaScript Example

// Subscribe to a single topic with QoS 1
client.subscribe('home/temperature', { qos: 1 });

// Subscribe to multiple topics with different QoS levels
client.subscribe({
  'home/temperature': { qos: 0 },
  'home/humidity': { qos: 1 },
  'home/+/status': { qos: 2 }  // Using wildcard
});

MQTT Security

Securing MQTT communications is crucial, especially for IoT applications that may transmit sensitive data or control critical systems. Here are key security considerations:

Authentication

MQTT supports username/password authentication, which should be enabled to prevent unauthorized access:

• Use strong, unique passwords for each client
• Consider implementing token-based authentication for better security
• For higher security, use client certificate authentication (with TLS)

Encryption with TLS/SSL

Transport Layer Security (TLS) should be used to encrypt MQTT traffic:

• Use port 8883 for MQTT over TLS (instead of the standard 1883)
• Implement proper certificate validation
• Keep certificates updated and manage their lifecycle

Authorization & Access Control

Implement topic-level access control to restrict which clients can publish/subscribe to specific topics:

• Define clear access control lists (ACLs)
• Structure topics to facilitate access control
• Apply principle of least privilege - grant only necessary permissions

Network Security

Additional network security measures should be considered:

• Use firewalls to restrict access to the MQTT broker
• Implement IP filtering where possible
• Consider using VPNs for additional security layers

Troubleshooting Common Issues

Advanced MQTT Concepts

Clean vs. Persistent Sessions

When a client connects to a broker, it can choose whether to start a clean session or resume a persistent session:

• Clean Session (true): All subscription information and queued messages are discarded when the client disconnects
• Persistent Session (false): The broker stores subscription information and undelivered QoS 1 and 2 messages for the client

Persistent sessions are useful for clients with intermittent connectivity, as they won't miss messages published while they were offline.

Bridging and Federation

For large-scale MQTT deployments, you might need to connect multiple brokers:

• Bridging: Direct connection between two brokers, with configurable topic mapping
• Federation: Looser coupling between brokers, where messages are selectively shared

These approaches help build distributed, scalable MQTT systems that can span multiple networks or geographic regions.

MQTT 5.0 Features

The latest version of MQTT (5.0) introduces several new features:

• Shared Subscriptions: Allow load balancing among multiple subscribers
• Message Expiry: Specify how long messages are valid
• Topic Aliases: Reduce bandwidth by using numeric identifiers for topics
• User Properties: Add custom key-value pairs to messages
• Request/Response Pattern: Simplifies implementation of request/response interactions

Real-World MQTT Applications

Using MQTT with MQTT.pro

MQTT.pro provides a fully-managed, serverless MQTT broker service that simplifies implementation and scaling of MQTT-based applications:

Getting Started with MQTT.pro

1. Create an Instance: Set up your MQTT broker instance in the MQTT.pro dashboard
2. Configure Security: Set up authentication and access control for your broker
3. Connect Clients: Use the connection details provided to connect your MQTT clients
4. Monitor Usage: Track connections, message throughput, and other metrics in the dashboard

MQTT.pro Benefits

• Serverless Architecture: No infrastructure to maintain
• Automatic Scaling: Handles changing connection and message loads
• High Availability: Distributed architecture ensures reliability
• Security Features: Built-in authentication, TLS/SSL, and access control
• Simple Management: User-friendly dashboard for monitoring and control

Conclusion

MQTT has become the protocol of choice for IoT and messaging applications due to its efficiency, flexibility, and reliability. This tutorial has covered the essentials of MQTT, from basic concepts to advanced features and real-world implementations.

By understanding topics, QoS levels, security considerations, and implementation best practices, you're now equipped to build robust MQTT-based systems for various applications.

With MQTT.pro's serverless broker service, you can focus on building your application rather than managing infrastructure, while ensuring security, scalability, and reliability for your MQTT communications.