Skip to main content

Building a sentiment analysis app with Node.js

Building A Sentiment Analysis App With Node.js
In this post, we’ll use Node.js to build a sentiment analysis application that analyzes text data from user reviews and uses natural language processing (NLP) to determine the user’s sentiment.
Our final application will look like this:
Sentiment Analysis Application Preview
Before we get started, let’s understand what sentiment analysis and natural language processing mean.

What is sentiment analysis?

Sentiment analysis is the process of analyzing text data and deriving its emotional tone. An example would be classifying a customer’s review of a product into either happy, unhappy, or neutral. To automate this process, we’ll be using natural language processing, a branch of artificial intelligence.

What is natural language processing?

Unlike programming languages, natural languages are often ambiguous and were not designed to be understood by computers — hence the need for a technology that handles its processing in order to derive meaningful and actionable data from it. SAS puts it succinctly:
Natural language processing is a branch of AI that gives computers the ability to interpret, derive meaning from, and manipulate human languages.

Setting up

Let’s start by building a new Node.js application using the Express framework. We’ll generate a scaffold app with the express-generator CLI tool.
First, we’ll ensure that we have Node installed by running the following command on our terminal:
node --version
If it returns an error message, click here to see Node installation instructions. With Node installed, let’s run the following command on our terminal:
npm install -g express-generator
The express-generator is what we’ll use to scaffold a new Node app. To do this, we’ll run:
express node_nlp --no-view
To start our application, let’s navigate to our new app directory and run npm start:
cd node_nlp
npm start
In our new generated app directory, let’s navigate to ./package.json. We’ll need to set up nodemon to help us automatically restart our application whenever we save new changes. On your terminal, run:
npm install --save nodemon
Next, we’ll add a new script to start our application via nodemon. Under scripts in package.json, add the following code:
"dev": "nodemon ./bin/www"
Moving on, we can start our application by running the following command on our terminal:
npm run dev
Now that we’ve successfully set up our application, let’s implement our sentiment analysis functionality using NLP.
We’ll start by installing Natural, a Node.js package that supports most of the NLP algorithms we’ll be using for our project. Let’s run the following command on our terminal:
npm install --save natural
Next, in our routes directory, we’ll create a new file and call it nlp.js. This is where we’ll house our NLP-related routes for our API. In our new file, ./routes/nlp.js, let’s import the following packages:
const express = require('express');
const natural = require('natural');
After this, we’ll create a new route and give it the path s-analyzer. When users send POST requests to our route with the product review in their request body, they should receive a response containing its sentiment analysis.
To create our new route, let’s modify our ./routes/nlp.js file:
const express = require('express');
const natural = require('natural');

const router = express.Router();

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
});
Notice that we’ve destructured the user’s review since we’ll be expecting it from our request.body object.

Data preprocessing

The raw data we get from our user is often filled with a lot of noise and is likely to contain many errors, hence the need to transform it into an understandable/usable format for our NLP algorithm. This step is known as data preprocessing.

Converting contractions into standard lexicon

To maintain uniform structure in our text data, we need to convert contractions (e.g., I’m, you’re, etc.) to their standard lexicon (i.e., I am, you are, etc.). To do this, let’s install the package apos-to-lex-form by running the following command on our terminal:
npm install --save apos-to-lex-form
Next, we’ll import it in our /routes/nlp.js file and use it for our data conversion:
const express = require('express');
const aposToLexForm = require('apos-to-lex-form');
const natural = require('natural');

const router = express.Router();

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
  const lexedReview = aposToLexForm(review);
});

Converting our text data to lowercase

During our sentiment analysis, we want all the data in a uniform format. This step ensures that our algorithm treats good and GOOD as the same words. We’ll do this by using JavaScript’s default toLowerCase() function:
...

const router = express.Router();

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
  const lexedReview = aposToLexForm(review);
  const casedReview = lexedReview.toLowerCase();
});

Removing non-alphabetical and special characters

To improve our accuracy in classifying the user’s sentiment, we’ll remove special characters and numerical tokens since they don’t contribute to sentiment. This process will ensure that our text data is left with only alphabetical characters.
Let’s use JavaScript’s default replace() function to achieve this:
...

const router = express.Router();

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
  const lexedReview = aposToLexForm(review);
  const casedReview = lexedReview.toLowerCase();
  const alphaOnlyReview = casedReview.replace(/[^a-zA-Z\s]+/g, '');
});

Tokenization

This is the process of splitting a text into its individual meaningful units. We can think of a word as a token of a sentence, and a sentence as a token of a paragraph.
For our next step, we’ll be using the WordTokenizer from our imported Natural package:
...

const router = express.Router();

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
  const lexedReview = aposToLexForm(review);
  const casedReview = lexedReview.toLowerCase();
  const alphaOnlyReview = casedReview.replace(/[^a-zA-Z\s]+/g, '');

  const { WordTokenizer } = natural;
  const tokenizer = new WordTokenizer();
  const tokenizedReview = tokenizer.tokenize(alphaOnlyReview);
});

Correcting misspelled words

Since the product reviews will be manually written by our users, there is a high chance of typographic errors. Before passing our data to our sentiment analysis algorithm, let’s use the spelling-corrector package to correct misspelled words, so that if our user inputs lov by mistake, the correct spelling, love, will be passed to our algorithm.
Let’s start by installing it with the following command:
npm install --save spelling-corrector
Next, we’ll add the following highlighted lines to our ./routes/nlp.js file:
...
const SpellCorrector = require('spelling-corrector');

const router = express.Router();

const spellCorrector = new SpellCorrector();
spellCorrector.loadDictionary();

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
  const lexedReview = aposToLexForm(review);
  const casedReview = lexedReview.toLowerCase();
  const alphaOnlyReview = casedReview.replace(/[^a-zA-Z\s]+/g, '');

  const { WordTokenizer } = natural;
  const tokenizer = new WordTokenizer();
  const tokenizedReview = tokenizer.tokenize(alphaOnlyReview);

  tokenizedReview.forEach((word, index) => {
    tokenizedReview[index] = spellCorrector.correct(word);
  })
});

Removing stop words

Stop words are generally the most common words in a language, which are filtered out before processing. Some examples of stop words include but, a, or, and what. Since these words have no effect on a user’s sentiment, removing them will help us focus on the important keywords.
To do this, we’ll use the stopword package. Let’s install it by running the following command on our terminal:
npm install --save stopword
Next, we’ll add the following highlighted line to our ./routes/nlp.js file:
...
const SW = require('stopword');

const router = express.Router();

const spellCorrector = new SpellCorrector();
spellCorrector.loadDictionary();

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
  const lexedReview = aposToLexForm(review);
  const casedReview = lexedReview.toLowerCase();
  const alphaOnlyReview = casedReview.replace(/[^a-zA-Z\s]+/g, '');

  const { WordTokenizer } = natural;
  const tokenizer = new WordTokenizer();
  const tokenizedReview = tokenizer.tokenize(alphaOnlyReview);

  tokenizedReview.forEach((word, index) => {
    tokenizedReview[index] = spellCorrector.correct(word);
  })
  const filteredReview = SW.removeStopwords(tokenizedReview);
});

Stemming

This is a process of word normalization in NLP that is used to convert derived or inflected words to their base or root form. For example, a stemmer algorithm is expected to reduce the words “giving,” “gave,” and “giver” to their root word, “give.”
For our application, we’ll not be performing this process separately because the SentimentAnalyzer from the Natural library provides us with the option of supplying a stemmer as a parameter while calling it. During its analysis, the individual words will be converted to their root form.

Sentiment analysis with the Natural library

Now that we have the text data in our desired state, we can use the SentimentAnalyzer from Natural to make an analysis of our user’s review.
The sentiment analysis algorithm from the Natural library is based on a vocabulary that assigns polarity to words. For example, the word “good” has a polarity of 3, while “bad” has a polarity of -3. The algorithm does its sentiment calculation by summing the polarity of each word in a piece of text and normalizing with the length of a sentence.
This is why preprocessing and removing all the noise from our data was a necessary step to achieve a more accurate result. The text’s sentiment is considered negative if our algorithm returns a negative value, positive if it returns a positive value, and neutral if it returns 0.
The SentimentAnalyzer constructor has three parameters:
  • The language of the text data
  • The stemmer
  • The vocabulary (currently supports AFINN, Senticon, and Pattern)
Here’s a link to the official sentiment analysis documentationfrom the Natural library.
To use the algorithm in our application, let’s add the following highlighted code to our ./routes/nlp.js file:
...

router.post('/s-analyzer', function(req, res, next) {
  const { review } = req.body;
  const lexedReview = aposToLexForm(review);
  const casedReview = lexedReview.toLowerCase();
  const alphaOnlyReview = casedReview.replace(/[^a-zA-Z\s]+/g, '');

  const { WordTokenizer } = natural;
  const tokenizer = new WordTokenizer();
  const tokenizedReview = tokenizer.tokenize(alphaOnlyReview);

  tokenizedReview.forEach((word, index) => {
    tokenizedReview[index] = spellCorrector.correct(word);
  })
  const filteredReview = SW.removeStopwords(tokenizedReview);

  const { SentimentAnalyzer, PorterStemmer } = natural;
  const analyzer = new SentimentAnalyzer('English', PorterStemmer, 'afinn');
  const analysis = analyzer.getSentiment(filteredReview);

  res.status(200).json({ analysis });
});

module.exports = router;
In our newly added lines, we destructured the SentimentAnalyzerand PorterStemmer methods from the Natural library, then created a new variable, analyzer and assigned the result of our sentiment analysis to it.
Notice that in the SentimentAnalyzer constructor, we supplied the parameters English (since this is the language we’ll be expecting from our users), PorterStemmer (the type of stemmer we chose for our analysis), and afinn (the type of vocabulary for our analysis).

Connecting our NLP route to our server

With our sentiment analysis route set up, the next step will be connecting it to our Express server. To do this, we’ll import the nlp router to our ./app.js file and add it as a route with the /api/nlp path.
Let’s add the following highlighted lines to our ./app.js file:
var express = require('express');
var path = require('path');
var cookieParser = require('cookie-parser');
var logger = require('morgan');

var indexRouter = require('./routes/index');
var nlpRouter = require('./routes/nlp');

var app = express();

app.use(logger('dev'));
app.use(express.json());
app.use(express.urlencoded({ extended: false }));
app.use(cookieParser());
app.use(express.static(path.join(__dirname, 'public')));

app.use('/', indexRouter);
app.use('/api/nlp', nlpRouter);

module.exports = app;

Working with our frontend

Now that we have our route set up, let’s connect it with the frontend of our application. We’ll be adding a simple form that collects our user’s review and a JavaScript function for making our API call.
Let’s modify our ./public/index.html file to look like this:
<html>

<head>
  <title>Sentiment Analyzer</title>
  <link rel="stylesheet" href="/stylesheets/style.css">
</head>

<body>
  <h1 id="title">Please write a review for this product:</h1>
  <form id="reviewForm">
    <textarea id="review" rows="4" cols="50"></textarea>
  </form>

  <div id="emojiSection"></div>

  <script type="text/javascript" src="./javascripts/index.js"></script>
</body>

</html>
Next, in the /public/javascripts folder, let’s create a new file index.js and paste the following lines code in it:
const submitReview = (e) => {
  e.preventDefault();
  const review = document.getElementById('review').value;
  const options = {
    method: 'POST',
    body: JSON.stringify({ review }),
    headers: new Headers({ 'Content-Type': 'application/json' })
  }

  const emojiSection = document.getElementById('emojiSection');
  const title = document.getElementById('title');
  const outline = document.querySelector(':focus');

  fetch('/api/nlp/s-analyzer', options)
    .then(res => res.json())
    .then (({ analysis }) => {
      if (analysis < 0) {
        emojiSection.innerHTML = '<img src="https://img.icons8.com/emoji/96/000000/angry-face.png">';
        title.style.color = 'red';
        outline.style.borderColor = 'red';
      };
      if (analysis === 0) {
        emojiSection.innerHTML = '<img src="https://img.icons8.com/officel/80/000000/neutral-emoticon.png">';
        title.style.color = '#00367c';
        outline.style.borderColor = '#00367c';
      }
      if (analysis > 0) {
        emojiSection.innerHTML = '<img src="https://img.icons8.com/color/96/000000/happy.png">';
        title.style.color = 'green';
        outline.style.borderColor = 'green'
      }
    })
    .catch(err => {
      emojiSection.innerHTML = 'There was an error processing your request!'
    })
}

document.getElementById('review').addEventListener('keyup', submitReview);
document.getElementById('reviewForm').addEventListener('submit', submitReview);
Notice that we’re rendering an emoji to the emojiSection divwe created in our index.html file. We are also changing the color of our application based on the sentiment value received from our API: results less than 0 are considered negative, greater than 0positive, and equal to 0 neutral.
Now when we start our application and navigate to http://localhost:3000/, it should be able to calculate the sentiment analysis of our product review based on our form input, just like the demo below:
Sentiment Analysis Application Preview

Conclusion

In this article, we’ve covered the basics of natural language processing using Node.js and have built a sentiment analysis application that calculates a user’s sentiment based on the text data received from their review.

Comments

Post a Comment

Popular posts from this blog

4 Ways to Communicate Across Browser Tabs in Realtime

1. Local Storage Events You might have already used LocalStorage, which is accessible across Tabs within the same application origin. But do you know that it also supports events? You can use this feature to communicate across Browser Tabs, where other Tabs will receive the event once the storage is updated. For example, let’s say in one Tab, we execute the following JavaScript code. window.localStorage.setItem("loggedIn", "true"); The other Tabs which listen to the event will receive it, as shown below. window.addEventListener('storage', (event) => { if (event.storageArea != localStorage) return; if (event.key === 'loggedIn') { // Do something with event.newValue } }); 2. Broadcast Channel API The Broadcast Channel API allows communication between Tabs, Windows, Frames, Iframes, and  Web Workers . One Tab can create and post to a channel as follows. const channel = new BroadcastChannel('app-data'); channel.postMessage(data); And oth...
Post a Comment
Read more

Certbot SSL configuration in ubuntu

  Introduction Let’s Encrypt is a Certificate Authority (CA) that provides an easy way to obtain and install free  TLS/SSL certificates , thereby enabling encrypted HTTPS on web servers. It simplifies the process by providing a software client, Certbot, that attempts to automate most (if not all) of the required steps. Currently, the entire process of obtaining and installing a certificate is fully automated on both Apache and Nginx. In this tutorial, you will use Certbot to obtain a free SSL certificate for Apache on Ubuntu 18.04 and set up your certificate to renew automatically. This tutorial will use a separate Apache virtual host file instead of the default configuration file.  We recommend  creating new Apache virtual host files for each domain because it helps to avoid common mistakes and maintains the default files as a fallback configuration. Prerequisites To follow this tutorial, you will need: One Ubuntu 18.04 server set up by following this  initial ...
Post a Comment
Read more

Working with Node.js streams

  Introduction Streams are one of the major features that most Node.js applications rely on, especially when handling HTTP requests, reading/writing files, and making socket communications. Streams are very predictable since we can always expect data, error, and end events when using streams. This article will teach Node developers how to use streams to efficiently handle large amounts of data. This is a typical real-world challenge faced by Node developers when they have to deal with a large data source, and it may not be feasible to process this data all at once. This article will cover the following topics: Types of streams When to adopt Node.js streams Batching Composing streams in Node.js Transforming data with transform streams Piping streams Error handling Node.js streams Types of streams The following are four main types of streams in Node.js: Readable streams: The readable stream is responsible for reading data from a source file Writable streams: The writable stream is re...
Post a Comment
Read more