LattePanda Facial Recognition Robot with OpenCV

This tutorial explains how to use the LattePanda with python language to use OpenCV machine vision library to create a facial recognition robot and this example will be using the Windows 10 OS on the LattePanda.

Time 1 hour

Difficulty: * *

Materials needed are: LattePanda, screen, keyboard, mouse, webcam.

1. Software preparation

Install Python with related packages

First of all, please go to Python's website to download the latest version of the installation file. Just to inform you, one is 2.7x version and another version is 3x. Python 2.7x version is the older version, and officially is not updated by the website, so this example uses the new version of Python, the 3x one. Please note that do not choose the wrong version. The latest version is 3.6.1, as shown below. Then follow the steps below

Please install the Python file, towards the end of the installation, make sure to "Add Python 3.6 to PATH"  meaning to add Python to the Windows operating system environment variables. (Shown Below)

After the installation is complete, in order to test whether our computer can run Python normally , please call run prompt on your computer, windows users please press the shortcut key " win + R " will be in the lower left corner of the implementation window, then enter " cmd ". This way we can call out the windows command prompt.

Then please enter " Python " above, calling the Python interactive interface. If you can successfully see the " >>> " symbol, it means that has been successfully called the Python interface and the user can directly enter Python. If  its not properly displaying the interface, then you may have had a problem in the installation process, please return to the previous installation steps, you can try to do the installation again.

Then enter " exit () " to exit out of this Python interactive interface, back to the windows command prompt characters.

Then enter the " pip install numpy “, using Python packages administrator pip to install numpy this package.

After installing numpy, use pip to install the OpenCV suite. Please enter " pip install opencv-python ".

Finally, enter " pip install pyserial ", installed in python used to do Serial communication pyserial suite.

Prepare the Python IDE

 Now we write Python programs In addition to playing code , but also need to implement the environment, so I recommend here is a very easy to use PythonIDE , please go to the following website ( http://thonny.org/ ) download And install Thonny.

After installing the necessary Python suite, we are going to set Thonny's program path as the default Python implementation program in our computer system. Please click Tools and then click Options to enter Thonny's preferences. 

Then select the interpreter, that is, Python's interpreter for the system default path.

2. Hardware Setup 

This example uses the LogitechC170's webcam face recognition camera lens, the other in order to be able to track the location of a human face, we also need to set up a rotating platform, as shown by the bottom of a small servo motor (MG90come) which control the camera’s position. That the small motor control signal line is connected to the bottom of the LattePanda left of the 3 Pinconnector, remember to pick on the dark ground, this is very important!!

3. Software implementation Arduino program code

To control the servo motor on a rotating platform, we need to use the Arduino Leonardo chip on LattePanda. Please open the Arduino IDE and upload the following code and upload it to the Arduino side on LattePanda (remember the COM port).
Arduino side of the control action is actually very simple, it will listen to Serial characters from the command, if it is lowercase letters a or b will be rotated about once. Then to achieve the effect of rotating the lens, and servo motor control signal pin in the In the example, use pin 9, connect to the above diagram, if you want to take other pins please remember to change the Arduino program code in the pin definition (s.attach (9)).

//Arduino sketch //waiting command from LattePanda (Python) to control servo #include <Servo.h> Servo s; int degree = 90; void setup() {    Serial.begin(115200);    s.attach(9); } void loop() {    if(Serial.available())    {        int cmd = Serial.read();        if(cmd == 'a')        {            degree++;            if(degree >= 180)                degree = 180;            s.write(degree);        }        else if(cmd == 'b')        {            degree--;            if(degree <= 0)   //lower                degree = 0;            s.write(degree);        }        else            Serial.println("Press a/b to turn the camrea!");    } }

Python program code

In front of the already prepared Python will need to use the library with the IDE. Then we will come to the final implementation of the key face recognition and tracking, because we need to identify the face model. SO go to OpenCV github website ( https://github.com/opencv/opencv/tree/master/data/haarcascades ) Download the following haarcascade_frontalface_default.xml and haarcascade_eye.xml files. These two .xml files, respectively, are used to identify people's face and eyes.

The actual code can be split up into several important parts so let’s try to understand it step by step!

First, we will use cv2.VideoCapture (0) to read the Webcam image information, and set the initial resolution is 640 × 480, followed by the port COM5 to establish a Serial channel, used to instructions to the Arduino side to control the direction of the lens.

The while loop which contains the main identification process, the identification method is actually very simple; the first frame of each photo of the first gray-scale photo, then goes through the front of the face and eye recognition module set into the face (face_cascde And eye_cascade) to do the most likely face detection. (the method used here is to take the largest / near face area)

Finally, in order to track the face, we need to calculate the center position of the face; (position = x + w / 2.0), where x is the corner of the face box [marks the x coordinate], w is the width [so take half of the width] will get the x coordinate position in the person. Then we just compare the image center line, and then left and right to rotate the lens to do tracking, and the center line position can be set from the previous resolution of 640 × 480. Calculate that half of the width of the program can be obtained in the 320, if the reader wants to try other resolution. Please remember to modify the lens tracking center line.

#include Servo s; int degree = 90; void setup() {    Serial.begin(115200);    s.attach(9); } void loop() {    if(Serial.available())    {        int cmd = Serial.read();        if(cmd == 'a')        {            degree++;            if(degree >= 180)                degree = 180;            s.write(degree);        }        else if(cmd == 'b')        {            degree--;            if(degree <= 0)                degree = 0;            s.write(degree);        }        else            Serial.println("Press a/b to turn the camrea!");    } }

Related Reading:

[LattePanda] Use C # to make Bluetooth 4.0 iBeacon's door lock system [ http://www.dfrobot.com.cn/community/forum.php?mod=viewthread&tid=27240&extra=  ]

Credits

CAVEDU Education

DFRobot