Detect Faces in Images using Nodejs

In this article, we will study how to Detect Faces in Images using Nodejs and OpenCV. If you are familiar with Computer Vision and OpenCV, Then go ahead download the code and start playing with it.
But if you aspire to understand basics of Face Detection, how it works and how it detects the faces in Images and Videos, then read the below content, it is truly a good read.

 Download

1. Computer Vision

=> Face Detection has been one of the heated topics of computer vision for the past few years. But now it is possible due to Computer Vision.

=>Computer Vision! wait, what? Yes, it is a field/technology that made this possible. In Computer Vision we humans teach machines to recognize and detect things as we humans do.

Mind you, Face recognition and Face Detection are completely different things. So, try not to confuse yourself with it.

=> Computer Vision is a very lengthy topic, so we will adhere our selves to Image Detection only. You may google about it and get more knowledge about it.

But how does it works? The reason is OpenCV.

2. OpenCV

=> OpenCV is a very famous library in Computer Vision for face detection and recognition. it is originally written in C/C++, it now provides bindings for Nodejs as well thanks to Vincent Mühler.

=> Detecting and Recognizing faces is not just peace of cake, it takes a lot to do. Basically, OpenCV applies a lot of Machine Learning Algorithm under the hood. These algorithms are composed of several small algorithms and thus gives the desired output.

=> Since images are nothing but digital blocks, These algorithms perform different tasks on each block and ultimately detects the face in an image. These individual tasks are called as Classifier.

3. Classifier and Cascades

=> In solid words, the classifier is a small set of program, that determines if the image or a block of the image is a Face. Now the other question begs is how does a Classifier manage to do all this? Well, they use the Cascades.

=> In Machine Learning term, you can call it a Model. A model which is trained to identify and detect the faces.
Now there few sets of combinations to identify and detect the faces, these cascades are .xml files. and they provided by OpenCV library by default.

=>There are two types of Cascades first is the Haar Classifier and the second one is the LBP Classifier.

I have created a demo application so that you can understand it and apply it in your own projects.Just make sure, you know what code you are writing and why you are writing that code.

Though I will try to explain each line of code, So let’s start.

4. Installing OpenCV and opencv4nodejs NPM module

=> To install OpenCV, follow the this how to install guide.

My suggestion is that don’t just start installing. First read it and read it again until you understand the steps (My personal experience).

=>Once understood then, do what you need to install. After installing the OpenCV, the next step would be to install NPM module called as opnecv4nodejs, which provides the binding of the NodeJs. Run the below command to install it.

npm install --save opencv4nodejs

5. Creating a new Nodejs project and Project Goals

=>Let’s start off by creating a new Nodejs project by usingnpm initcommand. This command will create a new package.json file.

=>After that copy the dependencies from below package.json and paste it in your file and runnpm install.

Below is my package.json file for this application.

package.json:

{
  "name": "Face-detection-NodeJs-Opencv",
  "version": "1.0.0",
  "description": "Detect Faces in Images using Nodejs and opencv4nodejs",
  "main": "server.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "ejs": "^2.6.1",
    "express": "^4.16.3",
    "opencv4nodejs": "^4.5.1"
  }
}

The goal of this Project is very simple, here we will do three operations which are listed below,

1. We will detect the Face in an Image.
2. We will render the Images with Detected Face, on the Web Page.
3. Also, at the last, we will save the image with the detected face.

So let’s get our hands dirty.

6. Project structure and Files

=>If you have followed my previousNodejs articles, you might familiar with folder structure I create. I try to keep the file structure as realistic as they can be so that you can easily integrate them into your projects. Also, we follow the object-oriented style, so that makes our code more elegant and clean.
=>The below images represents the file structure of our project. There are four files that we are going to use, let’s understand the purpose of each file,

1.config.js: In this file, we will do the setup of our application.
2.opencv-helpers.js: Here we will write the method that will help to detect the faces in an Image.
3.routes.js: In this file, we will register the routes of the application.
4.server.js: In this file, you will setup your NodeJs server and application configurations.

4. Setting Up the Nodejs server and writing Routes for the application.

=>So here we will first set up the nodejs and then we will initialize the routes of the application. As I said in the last section we will do our setup in theserver.jsfile.

=>So open theserver.jsfile and add the below code, the below code is very straightforward and easy to understand so I will leave to up to you. Let me know in the comment box if you don’t understand something.

server.js:

/**
 * @author Shashank Tiwari
 * Detect Faces Images using Nodejs
 */
'use strict';

const express = require("express");
const http = require('http');
const bodyParser = require('body-parser');


const routes = require('./utils/routes'); 
const config = require('./utils/config'); 


class Server{

    constructor(){
        this.port =  process.env.PORT || 3000;
        this.host = `localhost`;
        
        this.app = express();
        this.http = http.Server(this.app);
    }

    appConfig(){        
        this.app.use(
            bodyParser.json()
        );
        new config(this.app);
    }

    /* Including app Routes starts*/    includeRoutes(){
        new routes(this.app).routesConfig();
    }
    /* Including app Routes ends*/  

    appExecute(){

        this.appConfig();
        this.includeRoutes();

        this.http.listen(this.port, this.host, () => {
            console.log(`Listening on http://${this.host}:${this.port}`);
        });
    }

}

const app = new Server();
app.appExecute();

=> Since, we are rendering the Image with the detect face on Web Page and for that, we will make AJAX call and get the Image from the server. That’s all.

=> In the below code, we are have written two routes,

1. /image-face-detect: Will render the HTML file.
2. /detect-faces: This route will send the Image with the detected face.

routes.js:

/**
 * @author Shashank Tiwari
 * Detect Faces Images using Nodejs
 */
 'use strict';
const opencvHelpers = require('./opencv-helpers');

class Routes{

constructor(app,socket){
this.app = app;
this.io = socket;
}

appRoutes() {

this.app.get('/image-face-detect', (request, response) => {
response.render('image-face-detect');
});

this.app.get('/detect-faces', (request, response) => {
opencvHelpers.detectFace()
.then((result) => {
response.statusCode = 200;
response.send(result);
response.end();
}).catch((err) => {
response.statusCode = 500;
response.send(null);
response.end();
});
});
}

routesConfig(){
this.appRoutes();
}
}
module.exports = Routes;

5. Detect Faces in Images using Nodejs

=> The below class OpencvHelpers does all the magic. In this class, we have written all the methods to Detect faces and Save it to the disk. This class we will be consumed by the routes of the application.

=>The /detect-faces route we will basicallydetectFace() method and then the rest of the process will start.

opencv-helpers.js:

/**
 * @author Shashank Tiwari
 * Detect Faces Images using Nodejs
 */
const cv = require('opencv4nodejs');
const path = require('path');

class OpencvHelpers {

constructor() {}


getDataFilePath(fileName) {
return path.resolve(path.resolve(__dirname, '../data'), fileName);
}

drawRect(image, rect, color) {
image.drawRectangle(
rect,
color,
2,
cv.LINE_4
);
}

drawFaceBorder(image, rect) {
this.drawRect(image, rect, new cv.Vec(0, 255, 0));
}

detectFace() {
return new Promise( (resolve, reject) => {
try {
const image = cv.imread(this.getDataFilePath('g.r.l.jpeg'));
const classifier = new cv.CascadeClassifier(cv.HAAR_FRONTALFACE_DEFAULT);

// detect faces
const {
objects
} = classifier.detectMultiScale(image.bgrToGray());

if (!objects.length) {
throw new Error('No faces detected!');
}

objects.forEach((rect, i) => {
this.drawFaceBorder(image, rect);
});

this.saveFaceDetectedImage(image);

resolve(cv.imencode('.jpg', image));

} catch (error) {
console.log(error);
reject(null);
}
});
}

saveFaceDetectedImage(data) {
cv.imwrite(this.getDataFilePath('g.r.l2.jpeg'), data)
}
}

module.exports = new OpencvHelpers();

Explanation:

1. So let’s start withdetectFace(), The first thing to notice is we are returning Promise and the result of the Promise, we will send a response to the client.

2. Then in the next line(shown below), we are doing two things as listed below,

const image = cv.imread(this.getDataFilePath('g.r.l.jpeg'));

• Using,this.getDataFilePath('g.r.l.jpeg'), we will get the absolute path of g.r.l.jpegfile.
• And then cv.imreadwill give the image matrix on which we will apply rest of the operations.

3. After that, we have to select our cascade. Our model to detect the faces in an image, Remeber? Here I have used HAAR_FRONTALFACE_DEFAULT. Though you can any other HAAR cascade, it’s completely up to you.

4. In the next line, we remove the colors of the image and the remains of the images passed to the cascade, so it can detect the Faces in an image.

5. The last step is to stand out the detected faces in an Image. So, for that, we can draw a border around the face.

6. In the above code, on line number 39; we get objects constant which contains the data of detected face. Now we will apply the Loop on that objects constant. Why? because there can be the possibility of detecting multiple faces in an image.

7. The drawFaceBorder() method will basically call the drawRect() method. Which expects three parameters as shown below,

1. image: Full Image Matrix.
2. rect: Detected face from the Image Matrix.
3. color: Color of the border.

8. And lastly, we call drawRectangle() method on the Image matrix, which again expects 4 fours parameters, which are gain listed below,

1. rect: Detected face from the Image Matrix.
2. color: Color of the border.
3. thickness: Here thickness means, the thickness of the border. By default, I am passing 2 as thickness.
4. lineType: Here you can use any kind of line type such as LINE_4,LINE_8 andLINE_AA.

These were the steps to detect the faces in an image. Now let’s save the image with the detected faces.

9. saveFaceDetectedImage() method will save the image in the disk. we can use imwrite() method, the first parameter will the absolute path and the second parameter will be the Image matrix.

10. In the end, we will resolve the Promise by sending Image Buffer usingimencode()method.

Rendering the Image on the webpage

=> I will keep this part sweet and short. In this section, we will take a look at two more files first image-face-detect.html file where we will render the image.

=> Then we will take a look at the Javascript script which will make Ajax call and render the response. Below is the Markup, where we will render the response of the image with the detected face.

image-face-detect.html:

<!DOCTYPE html>
<html>
    <head>
        <meta name="viewport" content="initial-scale=1" />
        <title>Face detection Node Opencv</title>
        <link rel="stylesheet" href="/css/style.css">
    </head>
    <body>
        <div class="face-detect">
            <div class="example">
                <img src='g.r.l.jpeg' width="640" height="480">
            </div>
            <div class="example">
                <canvas id="canvas-video" width="640" height="480"></canvas>
            </div>
        </div>
        <script src="/js/image-face-detect.js"></script>
    </body>
</html>

The above markup needs no explanation. Here we have included the image-face-detect.js file in our HTML file. which will do all the operations behind the scene operations.

Now open theimage-face-detect.js and add the below code,

image-face-detect.js:

/**
 * @author Shashank Tiwari
 * Detect Faces Images using Nodejs
 */
'use strict';

class FaceDetect {
detectFace() {
fetch('/detect-faces')
.then((resp) => resp.arrayBuffer())
.then((data) => {
this.drawFace(data);
})
.catch((error) => {
console.log(error);
});
}

drawFace(data) {
const canvas = document.getElementById('canvas-video');
const context = canvas.getContext('2d');
const img = new Image();

context.fillStyle = '#333';
context.fillText('Loading...', canvas.width / 2 - 30, canvas.height / 3);


const base64String = btoa(new Uint8Array(data).reduce(function (data, byte) {
return data + String.fromCharCode(byte);
}, ''));

img.onload = function () {
context.drawImage(this, 0, 0, canvas.width, canvas.height);
};
img.src = 'data:image/jpg;base64,' + base64String;
}
}

const faceDetect = new FaceDetect();
faceDetect.detectFace();

Explanation:

1. The FaceDetect class has 2 methods,detectFace() anddrawFace().
2. detectFace() will basically call the Nodejs server and fetch the Image. Once the response is received, it will convert the text response to Array Buffer.
3. drawFace(), This method will basically by converting the array buffer to base64image string.

Conclusion and What next?

Just a recap! In this article, we studied below-listed topics,

1. The little background of the computer vision.
2. How to Detect faces in Image using Nodejs and OpenCV.
3. How to save the image with the detected face.
4. Lastly, we rendered the desired output on the web page.

The next step after this would be extending to WebCam. Yes, in the next article we will implement a Real-time face detection. So stay tuned for that. And if you have something in your mind, that you want me to implement do comment below, I would gladly look into it.

Am sure after reading this article, you will be enough confidence to start using OpenCV and face detection stuff in NodeJs. If You have any doubts feel free to comment below and if you like the above content to share with others.

Credits

The Photo used in the banner of this article is clicked by Samad Ismayilov, and taken from Pexels.com.