Skip to main content

Building a sentiment analysis app with Node.js

Building A Sentiment Analysis App With Node.js
In this post, we’ll use Node.js to build a sentiment analysis application that analyzes text data from user reviews and uses natural language processing (NLP) to determine the user’s sentiment.
Our final application will look like this:
Sentiment Analysis Application Preview
Before we get started, let’s understand what sentiment analysis and natural language processing mean.

What is sentiment analysis?

Sentiment analysis is the process of analyzing text data and deriving its emotional tone. An example would be classifying a customer’s review of a product into either happy, unhappy, or neutral. To automate this process, we’ll be using natural language processing, a branch of artificial intelligence.

What is natural language processing?

Unlike programming languages, natural languages are often ambiguous and were not designed to be understood by computers — hence the need for a technology that handles its processing in order to derive meaningful and actionable data from it. SAS puts it succinctly:
Natural language processing is a branch of AI that gives computers the ability to interpret, derive meaning from, and manipulate human languages.

Setting up

Let’s start by building a new Node.js application using the Express framework. We’ll generate a scaffold app with the express-generator CLI tool.
First, we’ll ensure that we have Node installed by running the following command on our terminal:
node --version
If it returns an error message, click here to see Node installation instructions. With Node installed, let’s run the following command on our terminal:
npm install -g express-generator
The express-generator is what we’ll use to scaffold a new Node app. To do this, we’ll run:
express node_nlp --no-view
To start our application, let’s navigate to our new app directory and run npm start:
cd node_nlp
npm start
In our new generated app directory, let’s navigate to ./package.json. We’ll need to set up nodemon to help us automatically restart our application whenever we save new changes. On your terminal, run:
npm install --save nodemon
Next, we’ll add a new script to start our application via nodemon. Under scripts in package.json, add the following code:
"dev": "nodemon ./bin/www"
Moving on, we can start our application by running the following command on our terminal:
npm run dev
Now that we’ve successfully set up our application, let’s implement our sentiment analysis functionality using NLP.
We’ll start by installing Natural, a Node.js package that supports most of the NLP algorithms we’ll be using for our project. Let’s run the following command on our terminal:
npm install --save natural
Next, in our routes directory, we’ll create a new file and call it nlp.js. This is where we’ll house our NLP-related routes for our API. In our new file, ./routes/nlp.js, let’s import the following packages:
const express = require('express');
const natural = require('natural');
After this, we’ll create a new route and give it the path s-analyzer. When users send POST requests to our route with the product review in their request body, they should receive a response containing its sentiment analysis.
To create our new route, let’s modify our ./routes/nlp.js file:
const express = require('express');
const natural = require('natural');

const router = express.Router();

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
});
Notice that we’ve destructured the user’s review since we’ll be expecting it from our request.body object.

Data preprocessing

The raw data we get from our user is often filled with a lot of noise and is likely to contain many errors, hence the need to transform it into an understandable/usable format for our NLP algorithm. This step is known as data preprocessing.

Converting contractions into standard lexicon

To maintain uniform structure in our text data, we need to convert contractions (e.g., I’m, you’re, etc.) to their standard lexicon (i.e., I am, you are, etc.). To do this, let’s install the package apos-to-lex-form by running the following command on our terminal:
npm install --save apos-to-lex-form
Next, we’ll import it in our /routes/nlp.js file and use it for our data conversion:
const express = require('express');
const aposToLexForm = require('apos-to-lex-form');
const natural = require('natural');

const router = express.Router();

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
  const lexedReview = aposToLexForm(review);
});

Converting our text data to lowercase

During our sentiment analysis, we want all the data in a uniform format. This step ensures that our algorithm treats good and GOOD as the same words. We’ll do this by using JavaScript’s default toLowerCase() function:
...

const router = express.Router();

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
  const lexedReview = aposToLexForm(review);
  const casedReview = lexedReview.toLowerCase();
});

Removing non-alphabetical and special characters

To improve our accuracy in classifying the user’s sentiment, we’ll remove special characters and numerical tokens since they don’t contribute to sentiment. This process will ensure that our text data is left with only alphabetical characters.
Let’s use JavaScript’s default replace() function to achieve this:
...

const router = express.Router();

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
  const lexedReview = aposToLexForm(review);
  const casedReview = lexedReview.toLowerCase();
  const alphaOnlyReview = casedReview.replace(/[^a-zA-Z\s]+/g, '');
});

Tokenization

This is the process of splitting a text into its individual meaningful units. We can think of a word as a token of a sentence, and a sentence as a token of a paragraph.
For our next step, we’ll be using the WordTokenizer from our imported Natural package:
...

const router = express.Router();

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
  const lexedReview = aposToLexForm(review);
  const casedReview = lexedReview.toLowerCase();
  const alphaOnlyReview = casedReview.replace(/[^a-zA-Z\s]+/g, '');

  const { WordTokenizer } = natural;
  const tokenizer = new WordTokenizer();
  const tokenizedReview = tokenizer.tokenize(alphaOnlyReview);
});

Correcting misspelled words

Since the product reviews will be manually written by our users, there is a high chance of typographic errors. Before passing our data to our sentiment analysis algorithm, let’s use the spelling-corrector package to correct misspelled words, so that if our user inputs lov by mistake, the correct spelling, love, will be passed to our algorithm.
Let’s start by installing it with the following command:
npm install --save spelling-corrector
Next, we’ll add the following highlighted lines to our ./routes/nlp.js file:
...
const SpellCorrector = require('spelling-corrector');

const router = express.Router();

const spellCorrector = new SpellCorrector();
spellCorrector.loadDictionary();

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
  const lexedReview = aposToLexForm(review);
  const casedReview = lexedReview.toLowerCase();
  const alphaOnlyReview = casedReview.replace(/[^a-zA-Z\s]+/g, '');

  const { WordTokenizer } = natural;
  const tokenizer = new WordTokenizer();
  const tokenizedReview = tokenizer.tokenize(alphaOnlyReview);

  tokenizedReview.forEach((word, index) => {
    tokenizedReview[index] = spellCorrector.correct(word);
  })
});

Removing stop words

Stop words are generally the most common words in a language, which are filtered out before processing. Some examples of stop words include but, a, or, and what. Since these words have no effect on a user’s sentiment, removing them will help us focus on the important keywords.
To do this, we’ll use the stopword package. Let’s install it by running the following command on our terminal:
npm install --save stopword
Next, we’ll add the following highlighted line to our ./routes/nlp.js file:
...
const SW = require('stopword');

const router = express.Router();

const spellCorrector = new SpellCorrector();
spellCorrector.loadDictionary();

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
  const lexedReview = aposToLexForm(review);
  const casedReview = lexedReview.toLowerCase();
  const alphaOnlyReview = casedReview.replace(/[^a-zA-Z\s]+/g, '');

  const { WordTokenizer } = natural;
  const tokenizer = new WordTokenizer();
  const tokenizedReview = tokenizer.tokenize(alphaOnlyReview);

  tokenizedReview.forEach((word, index) => {
    tokenizedReview[index] = spellCorrector.correct(word);
  })
  const filteredReview = SW.removeStopwords(tokenizedReview);
});

Stemming

This is a process of word normalization in NLP that is used to convert derived or inflected words to their base or root form. For example, a stemmer algorithm is expected to reduce the words “giving,” “gave,” and “giver” to their root word, “give.”
For our application, we’ll not be performing this process separately because the SentimentAnalyzer from the Natural library provides us with the option of supplying a stemmer as a parameter while calling it. During its analysis, the individual words will be converted to their root form.

Sentiment analysis with the Natural library

Now that we have the text data in our desired state, we can use the SentimentAnalyzer from Natural to make an analysis of our user’s review.
The sentiment analysis algorithm from the Natural library is based on a vocabulary that assigns polarity to words. For example, the word “good” has a polarity of 3, while “bad” has a polarity of -3. The algorithm does its sentiment calculation by summing the polarity of each word in a piece of text and normalizing with the length of a sentence.
This is why preprocessing and removing all the noise from our data was a necessary step to achieve a more accurate result. The text’s sentiment is considered negative if our algorithm returns a negative value, positive if it returns a positive value, and neutral if it returns 0.
The SentimentAnalyzer constructor has three parameters:
  • The language of the text data
  • The stemmer
  • The vocabulary (currently supports AFINN, Senticon, and Pattern)
Here’s a link to the official sentiment analysis documentationfrom the Natural library.
To use the algorithm in our application, let’s add the following highlighted code to our ./routes/nlp.js file:
...

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
  const lexedReview = aposToLexForm(review);
  const casedReview = lexedReview.toLowerCase();
  const alphaOnlyReview = casedReview.replace(/[^a-zA-Z\s]+/g, '');

  const { WordTokenizer } = natural;
  const tokenizer = new WordTokenizer();
  const tokenizedReview = tokenizer.tokenize(alphaOnlyReview);

  tokenizedReview.forEach((word, index) => {
    tokenizedReview[index] = spellCorrector.correct(word);
  })
  const filteredReview = SW.removeStopwords(tokenizedReview);

  const { SentimentAnalyzer, PorterStemmer } = natural;
  const analyzer = new SentimentAnalyzer('English', PorterStemmer, 'afinn');
  const analysis = analyzer.getSentiment(filteredReview);

  res.status(200).json({ analysis });
});

module.exports = router;
In our newly added lines, we destructured the SentimentAnalyzerand PorterStemmer methods from the Natural library, then created a new variable, analyzer and assigned the result of our sentiment analysis to it.
Notice that in the SentimentAnalyzer constructor, we supplied the parameters English (since this is the language we’ll be expecting from our users), PorterStemmer (the type of stemmer we chose for our analysis), and afinn (the type of vocabulary for our analysis).

Connecting our NLP route to our server

With our sentiment analysis route set up, the next step will be connecting it to our Express server. To do this, we’ll import the nlp router to our ./app.js file and add it as a route with the /api/nlp path.
Let’s add the following highlighted lines to our ./app.js file:
var express = require('express');
var path = require('path');
var cookieParser = require('cookie-parser');
var logger = require('morgan');

var indexRouter = require('./routes/index');
var nlpRouter = require('./routes/nlp');

var app = express();

app.use(logger('dev'));
app.use(express.json());
app.use(express.urlencoded({ extended: false }));
app.use(cookieParser());
app.use(express.static(path.join(__dirname, 'public')));

app.use('/', indexRouter);
app.use('/api/nlp', nlpRouter);

module.exports = app;

Working with our frontend

Now that we have our route set up, let’s connect it with the frontend of our application. We’ll be adding a simple form that collects our user’s review and a JavaScript function for making our API call.
Let’s modify our ./public/index.html file to look like this:
<html>

<head>
  <title>Sentiment Analyzer</title>
  <link rel="stylesheet" href="/stylesheets/style.css">
</head>

<body>
  <h1 id="title">Please write a review for this product:</h1>
  <form id="reviewForm">
    <textarea id="review" rows="4" cols="50"></textarea>
  </form>

  <div id="emojiSection"></div>

  <script type="text/javascript" src="./javascripts/index.js"></script>
</body>

</html>
Next, in the /public/javascripts folder, let’s create a new file index.js and paste the following lines code in it:
const submitReview = (e) => {
  e.preventDefault();
  const review = document.getElementById('review').value;
  const options = {
    method: 'POST',
    body: JSON.stringify({ review }),
    headers: new Headers({ 'Content-Type': 'application/json' })
  }

  const emojiSection = document.getElementById('emojiSection');
  const title = document.getElementById('title');
  const outline = document.querySelector(':focus');

  fetch('/api/nlp/s-analyzer', options)
    .then(res => res.json())
    .then (({ analysis }) => {
      if (analysis < 0) {
        emojiSection.innerHTML = '<img src="https://img.icons8.com/emoji/96/000000/angry-face.png">';
        title.style.color = 'red';
        outline.style.borderColor = 'red';
      };
      if (analysis === 0) {
        emojiSection.innerHTML = '<img src="https://img.icons8.com/officel/80/000000/neutral-emoticon.png">';
        title.style.color = '#00367c';
        outline.style.borderColor = '#00367c';
      }
      if (analysis > 0) {
        emojiSection.innerHTML = '<img src="https://img.icons8.com/color/96/000000/happy.png">';
        title.style.color = 'green';
        outline.style.borderColor = 'green'
      }
    })
    .catch(err => {
      emojiSection.innerHTML = 'There was an error processing your request!'
    })
}

document.getElementById('review').addEventListener('keyup', submitReview);
document.getElementById('reviewForm').addEventListener('submit', submitReview);
Notice that we’re rendering an emoji to the emojiSection divwe created in our index.html file. We are also changing the color of our application based on the sentiment value received from our API: results less than 0 are considered negative, greater than 0positive, and equal to 0 neutral.
Now when we start our application and navigate to http://localhost:3000/, it should be able to calculate the sentiment analysis of our product review based on our form input, just like the demo below:
Sentiment Analysis Application Preview

Conclusion

In this article, we’ve covered the basics of natural language processing using Node.js and have built a sentiment analysis application that calculates a user’s sentiment based on the text data received from their review.

Comments

Post a Comment

Popular posts from this blog

How to use Ngx-Charts in Angular ?

Charts helps us to visualize large amount of data in an easy to understand and interactive way. This helps businesses to grow more by taking important decisions from the data. For example, e-commerce can have charts or reports for product sales, with various categories like product type, year, etc. In angular, we have various charting libraries to create charts.  Ngx-charts  is one of them. Check out the list of  best angular chart libraries .  In this article, we will see data visualization with ngx-charts and how to use ngx-charts in angular application ? We will see, How to install ngx-charts in angular ? Create a vertical bar chart Create a pie chart, advanced pie chart and pie chart grid Introduction ngx-charts  is an open-source and declarative charting framework for angular2+. It is maintained by  Swimlane . It is using Angular to render and animate the SVG elements with all of its binding and speed goodness and uses d3 for the excellent math functio...
Post a Comment
Read more

Understand Angular’s forRoot and forChild

  forRoot   /   forChild   is a pattern for singleton services that most of us know from routing. Routing is actually the main use case for it and as it is not commonly used outside of it, I wouldn’t be surprised if most Angular developers haven’t given it a second thought. However, as the official Angular documentation puts it: “Understanding how  forRoot()  works to make sure a service is a singleton will inform your development at a deeper level.” So let’s go. Providers & Injectors Angular comes with a dependency injection (DI) mechanism. When a component depends on a service, you don’t manually create an instance of the service. You  inject  the service and the dependency injection system takes care of providing an instance. import { Component, OnInit } from '@angular/core'; import { TestService } from 'src/app/services/test.service'; @Component({ selector: 'app-test', templateUrl: './test.component.html', styleUrls: ['./test.compon...
Post a Comment
Read more

How to solve Puppeteer TimeoutError: Navigation timeout of 30000 ms exceeded

During the automation of multiple tasks on my job and personal projects, i decided to move on  Puppeteer  instead of the old school PhantomJS. One of the most usual problems with pages that contain a lot of content, because of the ads, images etc. is the load time, an exception is thrown (specifically the TimeoutError) after a page takes more than 30000ms (30 seconds) to load totally. To solve this problem, you will have 2 options, either to increase this timeout in the configuration or remove it at all. Personally, i prefer to remove the limit as i know that the pages that i work with will end up loading someday. In this article, i'll explain you briefly 2 ways to bypass this limitation. A. Globally on the tab The option that i prefer, as i browse multiple pages in the same tab, is to remove the timeout limit on the tab that i use to browse. For example, to remove the limit you should add: await page . setDefaultNavigationTimeout ( 0 ) ;  COPY SNIPPET The setDefaultNav...
Post a Comment
Read more