MQTT Communication Between NodeMCU and Raspberry Pi 3 B+

Using MQTT, NodeMCU, DHT22, RaspberryPi and IoT MQTT Panel to monitor temperature and humidity.

Things used in this project

Hardware components

Raspberry Pi 3 Model B+ ×1

NodeMCU ESP8266 Breakout Board ×9

DFRobot DHT22 Temperature and Humidity Sensor ×9

Female/Female Jumper Wires ×27

Software apps and online services

Raspberry Pi Raspbian

Thonny Python IDE

IoT MQTT Panel

Story

Using a Raspberry Pi 3 B+ as a broker for several NodeMCU with DHT-22 sensors measuring temperature and humidity and monitor on IoT MQTT Panel App. I made the algorithm for NodeMCU and Raspberry escalable. Just change the topics published and subscribed and add on IoT MQTT Panel App to have all the data at your hand anytime.

I have searched the internet for a lot of information as I have no knowledge of raspberry, python and MQTT. So, I have summarized everything I've learned and have given credit to those websites.

If your internet connection is lost your sensors will keep sending data to your broker. Meaning you can save your data!!! (of course you need to do some programming)

Please follow me for any updates. Soon I will post a video of everything running! :)

1. Connecting things:

How your network it will look like:

2. Programming things:

First you have to be sure you have all libraries installed on your Arduino IDE and on your Raspberry Pi 3B+.

2.1 Arduino

Install the libraries on your Arduino IDE.

MQTT - https://github.com/knolleary/pubsubclient/releases/tag/v2.7

DHT Sensor Library: https://github.com/adafruit/DHT-sensor-library

Adafruit Unified Sensor Lib: https://github.com/adafruit/Adafruit_Sensor

Esp8266 - https://github.com/esp8266/Arduino/tree/master/libraries/ESP8266WiFi

2.2 Raspberry Pi 3 b+

Install the libraries on your Python IDE.

time - Native library on Python

Paho MQTT - type "pip install paho-mqtt" on your LX terminal to install the MQTT library. If any doubts this website is AWESOME!! http://www.steves-internet-guide.com/mqtt/

2.3 Uploading codes

Upload the codes respectively to your NodeMCU and Raspberry.

2.4 Explaining the code - ARDUINO IDE

Those are the libraries mentioned before to be installed on your Arduino IDE.

#include <ESP8266WiFi.h>                              // Esp8266/NodeMCU Library
#include <PubSubClient.h>                             // MQTT Library
#include "DHT.h"                                      // DHT Sensor

Variables declared to be used through out the code:

const char* mqtt_server = "Broker_IP_Address";       // MQTT Server IP Address

mqtt_server: to get the IP address on Raspberry pi open a terminal screen and type in:

On this example the IP address would be 192.168.1.200

const char* clientID = "room1";     // The client id identifies the NodeMCU device.

clientID: Any name or number to identify the NodeMCU you are using. In this case it will be located at room1. So it is named room1.

const char* topicT = "/room1/temperature";  // Topic temperature
const char* topicH = "/room1/humidity";     // Topic humidity

topicT: Topic to publish temperature. In this example for room1 temperature the topic will be "/room1/temperature".

topicH: Topic to publish humidity. In this example for room1 humiditythe topic will be "/room1/humidity".

const char* willTopic = "/room1/status";    // Topic Status
const char* willMessage = "0";              // 0 - Disconnecetd

willTopic: Topic to publish the will testament. This will be used to check if the NodeMCU is connected/on. If it disconnects it will publish the willMessage to the willTopic. In this case "/room1/status"

willMessage: Message to be published on willTopic if the NodeMCU is disconnected/turned off.

int willQoS = 0;
boolean willRetain = true;

willQoS: Used to set the quality of service. In this case 0.

willRetain: Used to retain will message in case of disconnection. Set to True.

int counter = 0;                            // Used to reconnect to MQTT server
const char* swversion = "1.0";              // Software version

counter: counter used on reconnect routine.

swversion: used to control my software revision.

WiFiClient wifiClient;
PubSubClient client(mqtt_server, 1883, wifiClient); // 1883 is the listener port for the Broker

wifiClient: Creates a client that can connect to to a specified internet IP address and port as defined in client.connect().

client(): Client is the base class for all WiFi client based calls. It is not called directly, but invoked whenever you use a function that relies on it.

DHT dhtA(2, DHT22);  // DHT instance named dhtA, Pin on NodeMCU D4 and sensor type

DHT: Creates an instance named dhtA and assign pin 2 of the NodeMCU V3 (D4) of sensor DHT-22. As per schematics below. If you want to use another pin change the value to the correct pin. Before changing the pin used check the pinout below to assign the correct pin.

Using pin D2 -> GPIO4

DHT dhtA(4, DHT22)

If you are using DHT-11 it would be:

Using pin D2 -> GPIO4

DHT dhtA(4, DHT11)

NOTE: If you are using the same library as I am. If you use a different library please check the library documentation to check how you should declare pin and sensors used.


void setup() : Here we initialize things.

void setup() {

Serial.begin(9600);     // Debug purposes check if DHT and connection with MQTT Broker are working
Serial.print(swversion);// Debug. Software version
dhtA.begin();           // Starting the DHT-22

Connecting to MQTT broker.

delay(2000);               // Delay to allow first connection with MQTT Broker

delay(2000): increase time if first connection fails. In this case 2000 allows the NodeMCU to connect to the Broker.

if (client.connect(clientID,"","", willTopic, willQoS, willRetain, willMessage, true)) {  // Connecting to MQTT Broker

client.connect(): explained below. Taken from:

https://pubsubclient.knolleary.net/api.html#connect5

____________________________________________________________

boolean connect (clientID, username, password, willTopic, willQoS,willRetain, willMessage, cleanSession)

Connects the client with a Will message, username, password and clean-session flag specified.

Note : even if the cleanSession is set to false/0 the client will not retry failed qos 1 publishes. This flag is only of use to maintain subscriptions on the broker.

Parameters

clientID : the client ID to use when connecting to the server.

username : the username to use. If NULL, no username or password is used (const char[])

password : the password to use. If NULL, no password is used (const char[])

willTopic : the topic to be used by the will message (const char[])

willQoS : the quality of service to be used by the will message (int : 0,1 or 2)

willRetain : whether the will should be published with the retain flag (int : 0 or 1)

willMessage : the payload of the will message (const char[])

cleanSession : whether to connect clean-session or not (boolean)

Returns

false - connection failed.

true - connection succeeded.

____________________________________________________________

Serial.print(clientID);
    Serial.println(" connected to MQTT Broker!");
}
else {
    Serial.print(clientID);
    Serial.println(" connection to MQTT Broker failed...");
}

client.publish(): explained below. Taken from: https://pubsubclient.knolleary.net/api.html#connect5
____________________________________________________________
int publish (topic, payload, retained)
Publishes a string message to the specified topic.
Parameters
topic - the topic to publish to (const char[])
payload - the message to publish (const char[])
retained - whether the message should be retained (boolean)
false - not retained
true - retained
Returns
false - publish failed, either connection lost, or message too large
true - publish succeeded
____________________________________________________________

client.publish(willTopic, "1", true);

Serial.print(): printing the clientID so we know which NodeMCU is alive. Also printing a message if connection is successful or not.

Serial.println("IP address: ");                     // Print IP address
Serial.println(WiFi.localIP());
}

Printing the IP Address of the NodeMCU. Take note, just in case. :)

void loop() {
    climateRoutine();          // Climate routines
    if (!client.connected()) { // Reconnecting to MQTT server if connection is lost
        reconnect();
    }
}

void loop(): Main loop. Here we start the climateRoutine() and check we are still connected. If not we will try to reconnect calling reconnect() function.

void climateRoutine() {
    
    char temp[16];
    dtostrf(dhtA.readTemperature(), 3, 2, temp);   // To convert float into char
    client.publish(topicT, temp);                  // Publishing temperature
    
    char humi[16];
    dtostrf(dhtA.readHumidity(), 3, 2, humi);      // To convert float into char
    client.publish(topicH, humi);                  // Publishing humidity
    
    if (isnan(temp[16]) || isnan(humi[16])) {      // Check if there DHT is working
        Serial.println("Failed to read from DHT sensor!");
        return;
    }

delay(1000);                            // Delay to avoid flooding system with data
}

void climateRoutine(): Here we get the temperature and humidity from the DHT-22 sensor.

char temp[16];
    dtostrf(dhtA.readTemperature(), 3, 2, temp);   // To convert float into char
    client.publish(topicT, temp);                  // Publishing temperature

To publish the temperature we have to convert the value from float to char. So we use dtostrf() to convert the value float dhtA.readTemperature() to char temp[]. After that we publish the value temp on topicT.

char humi[16];
    dtostrf(dhtA.readHumidity(), 3, 2, humi);      // To convert float into char
    client.publish(topicH, humi);                  // Publishing humidity

To publish the humidity we have to convert the value from float to char. So we use dtostrf() to convert the value float dhtA.readHumidity() to char humi[]. After that we publish the value humi on topicH.

void reconnect(): if we get disconnected from broker it will try to connect again.

void reconnect() {            
    while (!client.connected()) {
        Serial.print("Attempting MQTT connection...");
        if (client.connect(clientID,"","", willTopic, willQoS, willRetain, willMessage, true)) { 
            client.publish(willTopic, "1", true);  
            Serial.println("Connected to MQTT Broker!");
            counter = 0;
        } 
        else {
            Serial.println("Connection to MQTT Broker failed. Trying again in 2 seconds");
            ++counter;
            if (counter > 180) ESP.reset();
            delay(2000);
        }
    }
}

int counter: we use this counter to skip the reconnect() function and reset the NodeMCU in case we can't get a connection with the broker.

delay(): as previously we use the delay() to give some time before we try to reconnect to broker. Increase time if you can't get connection.

counter = 0;
        } 
        else {
            Serial.println("Connection to MQTT Broker failed. Trying again in 2 seconds");
            ++counter;
            if (counter > 180) ESP.reset();
            delay(2000);
        }
    }
}

2.5 Explaining the code - RaspberryPi - Python

We start importing the libraries we will use.

import time
import paho.mqtt.client as mqtt

time: so we can delay the process and allow

paho.mqtt.client: to administrate the MQTT.

# Don't forget to change the variables for the MQTT broker!
mqtt_broker_ip = "Broker_IP_Address"
client=mqtt.Client()

qqt_broke_IP: assigning the broker IP address. This value you can get by typing ifconfing on your raspberrypi terminal screen.

client: we use it to print the client name when it publishes something.

def on_connect(client, userdata, flags, rc):

def on_connect(): Setup callback functions that are called when MQTT events happen link connecting to the server or receiving data from a subscribed feed.

Info below from: https://www.eclipse.org/paho/clients/python/docs/

____________________________________________________________

Callbacks

on_connect()

on_connect(client, userdata, flags, rc)

Called when the broker responds to our connection request.

client the client instance for this callback userdata the private user data as set in Client() or user_data_set() flags response flags sent by the broker rc the connection result flags is a dict that contains response flags from the broker: flags['session present'] - this flag is useful for clients that are using clean session set to 0 only. If a client with clean session=0, that reconnects to a broker that it has previously connected to, this flag indicates whether the broker still has the session information for the client. If 1, the session still exists.

The value of rc indicates success or not:

0: Connection successful 1: Connection refused - incorrect protocol version 2: Connection refused - invalid client identifier 3: Connection refused - server unavailable 4: Connection refused - bad username or password 5: Connection refused - not authorised 6-255: Currently unused.
____________________________________________________________

print("Connected with result code" + str(rc))
    client.subscribe("/#")

print(): we print the result code when connecting

client.subscribe(): Here we subscribe to all topics by using "/#".

def on_message(): The callback for when a PUBLISH message is received from the server.

Info below from: https://www.eclipse.org/paho/clients/python/docs/

____________________________________________________________

on_message()

on_message(client, userdata, message)

Called when a message has been received on a topic that the client subscribes to and the message does not match an existing topic filter callback. Use message_callback_add() to define a callback that will be called for specific topic filters. on_message will serve as fallback when none matched.

client the client instance for this callback userdata the private user data as set in Client() or user_data_set() message an instance of MQTTMessage. This is a class with members topic, payload, qos, retain.
____________________________________________________________
msg.payload: to decode the message we assign this variable to msg.payload.decode("utf-8"). Thus removing the code b and ' ' that come attached to the message.

# Show status of sensors  
    #room1
    if msg.topic == "/room1/status":
        if msg.payload == "1":
            print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))
    if msg.topic == "/room1/status":
        if msg.payload == "0":
            print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))
    #room2
    if msg.topic == "/room2/status":
        if msg.payload == "1":
            print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))
    if msg.topic == "/room2/status":
        if msg.payload == "0":
            print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))
    #room3
    if msg.topic == "/room3/status":
        if msg.payload == "1":
            print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))
    if msg.topic == "/room3/status":
        if msg.payload == "0":
            print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))
    #room4
    if msg.topic == "/room4/status":
        if msg.payload == "1":
            print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))
    if msg.topic == "/room4/status":
        if msg.payload == "0":
            print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))

Here we print willMessage from sensors. In this case the status:

1 - when ON/Connected

2 - when OFF/Disconnected

When the client disconnects for any reason the willMessage already on the broker will be published under the topic /status. Thus we know if the NodeMCU is connected or not.

#Print humidity and temperature values
    if msg.topic == "/room1/temperature":
        print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))
    if msg.topic == "/room1/humidity":
        print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))
    if msg.topic == "/room2/temperature":
        print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))
    if msg.topic == "/room2/humidity":
        print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))
    if msg.topic == "/room3/temperature":
        print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))
    if msg.topic == "/room3/humidity":
        print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))
    if msg.topic == "/room4/temperature":
        print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))
    if msg.topic == "/room4/humidity":
        print ("Topic: ", msg.topic + "\nMessage: " + str(msg.payload))

Here we are printing all messages received from 4 different sensors. We have temperature and humidity.

client.on_connect = on_connect
client.on_message = on_message

client.on_connect and client.on_message: Here, we are telling the client which functions are to be run on connecting, and on receiving a message.

client.connect(mqtt_broker_ip, 1883)

client.connect(): Once everything has been set up, we can (finally) connect to the broker 1883 is the listener port that the MQTT broker is using.

client.loop_forever()
client.disconnect()
time.sleep(30)

client.loop_forever(): Once we have told the client to connect, let the client object run itself.

Info below from: https://www.eclipse.org/paho/clients/python/docs/

____________________________________________________________
loop_forever()

loop_forever(timeout=1.0, max_packets=1, retry_first_connection=False)

This is a blocking form of the network loop and will not return until the client calls disconnect(). It automatically handles reconnecting.

Except for the first connection attempt when using connect_async, use retry_first_connection=True to make it retry the first connection. Warning: This might lead to situations where the client keeps connecting to an non existing host without failing.

The timeout and max_packets arguments are obsolete and should be left unset.
____________________________________________________________

client.disconnect(): For the client to disconnect clean without triggering a willMessage.
Info below from: https://www.eclipse.org/paho/clients/python/docs/

__________________________________________________________

disconnect()

disconnect()

Disconnect from the broker cleanly. Using disconnect() will not result in a will message being sent by the broker.
Disconnect will not wait for all queued message to be sent, to ensure all messages are delivered, wait_for_publish() from MQTTMessageInfo should be used. See publish() for details.
____________________________________________________________

3. IoT MQTT Panel

Install the IoT MQTT Panel on your phone (Android).

4. Hardware

There is only need to connect the DHT-22 Sensors to your NodeMCUs.

NodeMCU DHT-22

3.3V -> +
GND -> -
D4 -> OUT

5. Tips

Weld the connections!! I have had a hard time with faulty connections. :(

If your NodeMCU can't connect check a few things.

Open the serial monitor and upload the script. Check if the IP address printed is not like that 0.0.0.0. If it is, copy and paste the following code to your code and run it.

const char* ssid="SSID"; // The name of the Wi-Fi network you want to connect to
const char* password = "PASSWORD"; // The password of the Wi-Fi network

void setup() {
  Serial.begin(115200);
  delay(10);
  Serial.println('\n');
  
  WiFi.begin(ssid, password);             
  Serial.print("Connecting to ");
  Serial.print(ssid); Serial.println(" ...");

  int i = 0;
  while (WiFi.status() != WL_CONNECTED) { 
    delay(1000);
    Serial.print(++i); Serial.print(' ');
  }

  Serial.println('\n');
  Serial.println("Connection established!");  
}

Both const char* should be copied and pasted before the void setup().

Everything that is inside void setup() copy and paste inside your code void setup().

After running once you can delete this part of the code from your original code.

6. Celebrate

If you have a beer can and some meat this might be useful! How to start a charcoal grill with your can of beer! :)

Schematics

NodeMCU + DHT-22 (Click here to download)

This Project copied from hackster.io, Project Maker: Cmtelesann