ESP32 Tutorial (Arduino Websocket server): Receiving and parsing JSON content

DFRobot Nov 10 2017 2637

The objective of this esp32 tutorial is to explain how to receive and parse a JSON message on a Websocket Server running on the ESP32. We will use the Arduino core as programming framework. The tests of this ESP32 tutorial were performed using a DFRobot’s ESP-WROOM-32 device integrated in a ESP32 development board.

Introduction

The objective of this post is to explain how to receive and parse a JSON message on a Websocket Server running on the ESP32. We will use the Arduino core as programming framework.

In terms of Websockets, the code shown here will be based on this previous tutorial. The tutorial explains how to install the Arduino library needed to work with Websockets and the Python module needed for the testing client we are going to create here.

For the JSON parsing functionalities, you can check this tutorial. It also explains how to install the JSON parsing library needed for the Arduino code.

The tests of this ESP32 tutorial were performed using a DFRobot’s ESP-WROOM-32 device integrated in a ESP32 FireBeetle board.

The Python client code

We will start the code by importing the websocket module, so we can have access to all the functions needed to connect to the ESP32 Websocket server.

1 import websocket

We will also import the json module, which will allow us to convert a Python dictionary (one of the language’s basic structures) into a JSON string.

1 import json

Next, we will create an object of class WebSocket, which has the methods needed to connect to the server and exchange data with it.

1 ws = websocket.WebSocket()

To connect to the ESP32 websocket server, we call the connect method on this object, passing as input a string with the destination server in the format “ws://{ESP32 IP}/”, changing the {ESP32 IP} by the local IP that will be assigned to your ESP32 in the WiFi network.

Note that we will be printing the IP of the ESP32 on the network on the Arduino code, so for now you can leave a dummy value and change it later.

1 ws.connect("ws://192.168.1.78/")

Now we will create a Python dictionary with some key-value pairs representing a sensor measurement. Note that this is a testing data structure but in a real application use case our ESP32 could be acting as a gateway, receiving data from simpler sensors, processing it and sending it to the cloud.

myDict = {"sensor": "temperature", "identifier":"SENS123456789", "value":10, "timestamp": "20/10/2017 10:10:25"}

We will then send the data to the Websocket server by calling the send method of our WebSocket object. This method receives as input the data we want to send.

In our case, the data will be the JSON string representation of the previously declared Python dictionary. To convert a dictionary to a JSON string, we call the dumps function of the json module, passing as input the dictionary.

1 ws.send(json.dumps(myDict))

After that, we will get a response from the server by calling the recv method on our WebSocker object and print the result.

result = ws.recv()
print(result)

To finalize, we will close the connection by calling the close method on the same object.

ws.close()

The final complete Python code can be seen below.

import websocket
import json
 
ws = websocket.WebSocket()
ws.connect("ws://192.168.1.78/")
 
myDict = {"sensor": "temperature", "identifier":"SENS123456789", "value":10, "timestamp": "20/10/2017 10:10:25"}
 
ws.send(json.dumps(myDict))
result = ws.recv()
print(result)
 
ws.close()

The Arduino code

Library includes and global variables
As usual, we will start the Arduino code with some library includes. For this tutorial, we will need the following ones:
WiFi.h: Library needed to connect the ESP32 to a WiFi network and thus allow the websocket client to reach the server.
WebSocketServer.h: Library needed to set the Websocket server and handle the exchange of data with the client.
ArduinoJson.h: Library needed for parsing and accessing the data from the JSON content sent by the client.

#include <WiFi.h>
#include <WebSocketServer.h>
#include <ArduinoJson.h>

In order to get our Websocket server running, we need to declare an object of class WiFiServer, which we will use to set a TCP server. The Websocket server will then operate on top of our TCP server.

The constructor of the WiFiServer object receives as input the number of the port where the TCP server will be listening for incoming client connections. We will use port 80, which is the default HTTP port.

We will also need an object of class WebSocketServer for all the Websocket protocol related functionalities.

WiFiServer server(80);
WebSocketServer webSocketServer;

Finally we will need some variables to store the credentials of the WiFi network to which we are going to connect. Note that we could pass them directly to the function that is used to connect to the WiFi network, but it’s cleaner and easier to maintain if they are declared here.

const char* ssid = "yourNetworkName";
const char* password = "yourNetworkPassword";

The client message handling function

Since we are going to need to parse the JSON message that comes from the client, we are going to create a dedicated function for that. This way, we keep the code clean and encapsulated, rather that having a lot of logic spread across the Arduino main loop function.

So, our function to handle the data from the client will receive as parameter a String with the data to be parsed. Remember from the previous post about setting the Websockets server on the ESP32 that when we receive data from the client its data type is String.

void handleReceivedMessage(String message){
  // message handling code
}

Now, to parse the message, we will start by declaring an object of class StaticJsonBuffer. This object will be used by the JSON library as a pre-allocated memory pool to store the JSON object tree.

We will need to specify a size for this memory pool, which is done as a template parameter. You can consult this tool for a more precise way of calculating the size needed for this buffer accordingly to the structure of the JSON. Nonetheless, for this simple example, we will use a value of 500, which will be more than enough for the structure we are going to parse.

StaticJsonBuffer<500> JSONBuffer; //Memory pool

Next, to parse the JSON message we received as input of our function, we will call the parseObject method of the StaticJsonBuffer object we have just created.

This method receives as input the message we want to parse and returns a reference to an object of class JsonObject. We will use this object below to access the parsed values but before that we will first check if the parsing procedure has occurred successfully.

To do so, we simply call the success method on this JsonObject reference we have just obtained. This method receives no arguments and returns a Boolean value indicating if the parsing was successful (true) or not (false).

If the parsing was not successful we will just finish the execution of our function since we would not be able to access any values of the message.

JsonObject& parsed = JSONBuffer.parseObject(message); //Parse message
if (!parsed.success()) {   //Check for errors in parsing
 
    Serial.println("Parsing failed");
    return;
 
}

If the parsing is successful, we can then access the values from the message. In our case, we will simply get each variable from the JSON structure and print them to the serial console.

To access each value from the parsed JSON message, we simply use the subscript operator (square brackets) and use a string with the name of the key in the JSON object.

Note that we already know the structure of the JSON from the Python code, and thus we know each key and can easily map each variable to a suitable data type.

const char * sensorType = parsed["sensor"]; //Get sensor type value
const char * sensorIdentifier = parsed["identifier"]; //Get sensor type value
const char * timestamp = parsed["timestamp"]; //Get timestamp
int value = parsed["value"]; //Get value of sensor measurement

After obtaining each value, we will simply print it to the serial port in a nicely formatted way. You can check below the full code for our message handling function, already including these prints.

void handleReceivedMessage(String message){
 
  StaticJsonBuffer<500> JSONBuffer;                     //Memory pool
  JsonObject& parsed = JSONBuffer.parseObject(message); //Parse message
 
  if (!parsed.success()) {   //Check for errors in parsing
 
    Serial.println("Parsing failed");
    return;
 
  }
 
  const char * sensorType = parsed["sensor"];           //Get sensor type value
  const char * sensorIdentifier = parsed["identifier"]; //Get sensor type value
  const char * timestamp = parsed["timestamp"];         //Get timestamp
  int value = parsed["value"];                          //Get value of sensor measurement
 
  Serial.println();
  Serial.println("----- NEW DATA FROM CLIENT ----");
 
  Serial.print("Sensor type: ");
  Serial.println(sensorType);
 
  Serial.print("Sensor identifier: ");
  Serial.println(sensorIdentifier);
 
  Serial.print("Timestamp: ");
  Serial.println(timestamp);
 
  Serial.print("Sensor value: ");
  Serial.println(value);
 
  Serial.println("------------------------------");
}

The setup

Our Arduino setup function will be used for initializations. First of all, we will open a serial connection, so we are able to print the outputs of our program.
Next, we will connect the ESP32 to a WiFi network. We are going to use the same code we have been using in a lot of previous tutorials and you can check here a detailed explanation of each function.

Note that after establishing the connection to the network, we will print the local IP assigned to the ESP32, which is the one we should use on the Python code Websocket connect method.

Once the connection to the WiFi network is finished, we will initialize the TCP server by calling the begin method on the WiFiServer global object we declared on the beginning of the program. This function receives no arguments and returns void.

You can check below the full setup function code, which contains all the initializations mentioned before.

void setup() {
 
  Serial.begin(115200);
delay(2000);
WiFi.begin(ssid, password); 
 
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }
 
  Serial.println("Connected to the WiFi network");
  Serial.println(WiFi.localIP());
 
  server.begin();
  delay(100);
}

The main loop

To finalize our example, we still need to create the code for handling the client data, which will done on the Arduino main loop.

So, the first thing we need to do is checking if we have a client available. To do it, we will call the available method of the WiFiServer object we declared globally and initialized in the setup function.

As mentioned in the previous tutorials, we are still working at the TCP level and only latter we will work in the Websockets protocol layer.
The available method call receives no arguments and returns an object of class WiFiClient. After storing this object in a variable, we then need to call the connected method, which will return a Boolean value indicating if the client is connected or not.

If the client is connected, we will need to perform the Websocket handshake procedure, which is the initial part of the protocol. To do it, we simple call the handshake method of the WebSocketServer object we declared in the beginning of the program.

The handshake method receives as input the WiFiClient object we obtained before and will use it in its implementation to communicate with the client. This method call returns true if the handshake procedure is done correctly and false otherwise.

Thus, we should only try to exchange data with the client if the handshake method returns true.

WiFiClient client = server.available();
 
if (client.connected() && webSocketServer.handshake(client)) {
   // Handling data exchange with the Websocket client
}

Inside the previous conditional block, we will now declare a String buffer to hold the data received from the client and then do a loop while the client is connected. We can check if the client is still connected by calling again the connected method on the WiFiClient and use the returning value as our while loop condition.

Then, inside our loop, we get the data that the client sends by calling the getData method of the WebSocketServer object. This method receives no arguments and returns a String with the data sent by the client.

Then, if the size of our data is greater than zero (the client may have not sent any data) we call our previously defined message handling function, which will take care of parsing it.

Just for testing purposes, we will also send the message back to the client with a call to the sendData method, so we have something to print on the Python program. Naturally, an expected behavior for a real application scenario would be returning a message indicating if everything was correctly parsed.

You can check below the full Arduino main loop function, which already includes these function calls. Note that each iteration of this internal loop for exchanging data with the client needs to have a small delay. Otherwise, we will experience problems receiving the data from the client.

void loop() {
 
  WiFiClient client = server.available();
 
  if (client.connected() && webSocketServer.handshake(client)) {
 
    String data;      
 
    while (client.connected()) {
 
      data = webSocketServer.getData();
 
      if (data.length() > 0) {
         handleReceivedMessage(data);
         webSocketServer.sendData(data);
      }
 
      delay(10); // Delay needed for receiving the data correctly
   }
 
   Serial.println("The client disconnected");
   delay(100);
  }
 
  delay(100);
}

The final code

The final ESP32 Arduino source code can be seen below.

#include <WiFi.h>
#include <WebSocketServer.h>
#include <ArduinoJson.h>
 
WiFiServer server(80);
WebSocketServer webSocketServer;
 
const char* ssid = "yourNetworkName";
const char* password =  "yourNetworkPassword";
 
void handleReceivedMessage(String message){
 
  StaticJsonBuffer<500> JSONBuffer;                     //Memory pool
  JsonObject& parsed = JSONBuffer.parseObject(message); //Parse message
 
  if (!parsed.success()) {   //Check for errors in parsing
 
    Serial.println("Parsing failed");
    return;
 
  }
 
  const char * sensorType = parsed["sensor"];           //Get sensor type value
  const char * sensorIdentifier = parsed["identifier"]; //Get sensor type value
  const char * timestamp = parsed["timestamp"];         //Get timestamp
  int value = parsed["value"];                          //Get value of sensor measurement
 
  Serial.println();
  Serial.println("----- NEW DATA FROM CLIENT ----");
 
  Serial.print("Sensor type: ");
  Serial.println(sensorType);
 
  Serial.print("Sensor identifier: ");
  Serial.println(sensorIdentifier);
 
  Serial.print("Timestamp: ");
  Serial.println(timestamp);
 
  Serial.print("Sensor value: ");
  Serial.println(value);
 
  Serial.println("------------------------------");
}
 
void setup() {
 
  Serial.begin(115200);
 
  delay(2000);
 
  WiFi.begin(ssid, password); 
 
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }
 
  Serial.println("Connected to the WiFi network");
  Serial.println(WiFi.localIP());
 
  server.begin();
  delay(100);
}
 
void loop() {
 
  WiFiClient client = server.available();
 
  if (client.connected() && webSocketServer.handshake(client)) {
 
    String data;      
 
    while (client.connected()) {
 
      data = webSocketServer.getData();
 
      if (data.length() > 0) {
         handleReceivedMessage(data);
         webSocketServer.sendData(data);
      }
 
      delay(10); // Delay needed for receiving the data correctly
   }
 
   Serial.println("The client disconnected");
   delay(100);
  }
 
  delay(100);
}

Testing the code

To test the whole system, we start by compiling and uploading the Arduino code to our ESP32. After the procedure is finished, simply open the serial monitor and wait for the connection to the WiFi network.

When the connection is finished, the local IP address of the ESP32 should have been printed to the console. Copy that address and use it on the Python code on the connect method.

Then, run the Python code. You should get an output similar to figure 1, where the JSON message we sent to the server was echoed back.

Figure 1 – JSON message echoed back to the Python WebSocket client.

After that, go back to the Arduino serial monitor. You should get a result similar to figure 2, which shows the parsed values from the JSON message getting printed to the console.

Figure 2 – Parsed values from the JSON message.

NOTE: This esp32 tutorial is written by Nuno Santos who is an kindly Electronics and Computers Engineer. live in Lisbon, Portugal. you could check the original article here.
He had written many useful tutorials and projects about ESP32, ESP8266, If you are interested, you could check his blog to know more.

DFRobot supply lots of esp32 arduino tutorials and esp32 projects for makers to learn.

ESP32 / ESP8266 Projects ESP32 / ESP8266 Products