Skip to main content

https://medium.com/javascript-in-plain-english/improve-observability-of-nodejs-http-request-response-a12f8629ccd0

 NodeJS being IO intensive, http requests form a major part of the application lifecycle. For monitoring and observability of a NodeJS application, understanding its behavior of requests made to other systems is an integral part of a micro-services architecture. Inspired by the tweet below, we decided to share how to improve the Observability in context of HTTP request-response lifecycle in a NodeJS application.

NodeJS HTTP request-response phases

Image for post

In NodeJS, both HTTP Request and Response are NodeJS streams, they implement the EventEmitter interface and emit events for specific lifecycle events. We can add event listeners to these events in order to measure the important metrics for request and response. The events emitted by NodeJS for the parts of the above HTTP lifecycle phases are

The  starts with the creation of net.Socket object which emits the  event once the socket is created. Now this socket can be used to listen to the DNS Lookup, TCP and TLS events. We will use  which marks High Resolution timestamp to mark the events and would find the duration from start of the request to each of those timestamps.

const request = https.request(options, callback)
const startTime = process.hrtime();
let dnsLookup, tcpConnection, tlsHandshake, responseBodyStart, responseBodyEnd;request.on("socket", socket => {
    // Socket created for dnslookup
    socket.on("lookup", () => {
        dnsLookup = process.hrtime();
    });    // TCP Connection established
    socket.on("connect", () => {
        tcpConnection = process.hrtime();
    });    // TLS Handshake complete
    socket.on("secureConnect", () => {
        tlsHandshake = process.hrtime();
    });
});

Once the request completes, the server would start sending responses. These response bytes can be listened on the http.Response object

https.request(requestOptions, res => {
    // The first time response bytes are transferred
    res.once("data", () => {
        responseBodyStart = process.hrtime();
    });
    res.on("data", chunk => (response += chunk));
    res.on("end", () => {
        responseBodyEnd = process.hrtime();
        callback(null, {
            body: response,
            duration: getEventDuration()
        });
    });
});

Now you can find the duration of each of these events marked with a timestamp to find the duration of important parts of the lifecycle.

DNS lookup = dnsLookup - startTime
TCP connection = tcpConnection - (dnsLookup || startTime)
TLS connection = tlsHandshake - tcpConnection
First byte = responseBodyStart - tlsHandshake
Content transfer = responseBodyEnd - responseBodyStart
Total = responseBodyEnd - startTime

This information can now be added to Monitoring and Distributed tracing to help improve the Observability of your systems.

The whole implementation for this is published as a npm package  and the whole code is available on github/http-duration-client

Comments

Post a Comment

Popular posts from this blog

4 Ways to Communicate Across Browser Tabs in Realtime

1. Local Storage Events You might have already used LocalStorage, which is accessible across Tabs within the same application origin. But do you know that it also supports events? You can use this feature to communicate across Browser Tabs, where other Tabs will receive the event once the storage is updated. For example, let’s say in one Tab, we execute the following JavaScript code. window.localStorage.setItem("loggedIn", "true"); The other Tabs which listen to the event will receive it, as shown below. window.addEventListener('storage', (event) => { if (event.storageArea != localStorage) return; if (event.key === 'loggedIn') { // Do something with event.newValue } }); 2. Broadcast Channel API The Broadcast Channel API allows communication between Tabs, Windows, Frames, Iframes, and  Web Workers . One Tab can create and post to a channel as follows. const channel = new BroadcastChannel('app-data'); channel.postMessage(data); And oth...
Post a Comment
Read more

Certbot SSL configuration in ubuntu

  Introduction Let’s Encrypt is a Certificate Authority (CA) that provides an easy way to obtain and install free  TLS/SSL certificates , thereby enabling encrypted HTTPS on web servers. It simplifies the process by providing a software client, Certbot, that attempts to automate most (if not all) of the required steps. Currently, the entire process of obtaining and installing a certificate is fully automated on both Apache and Nginx. In this tutorial, you will use Certbot to obtain a free SSL certificate for Apache on Ubuntu 18.04 and set up your certificate to renew automatically. This tutorial will use a separate Apache virtual host file instead of the default configuration file.  We recommend  creating new Apache virtual host files for each domain because it helps to avoid common mistakes and maintains the default files as a fallback configuration. Prerequisites To follow this tutorial, you will need: One Ubuntu 18.04 server set up by following this  initial ...
Post a Comment
Read more

Working with Node.js streams

  Introduction Streams are one of the major features that most Node.js applications rely on, especially when handling HTTP requests, reading/writing files, and making socket communications. Streams are very predictable since we can always expect data, error, and end events when using streams. This article will teach Node developers how to use streams to efficiently handle large amounts of data. This is a typical real-world challenge faced by Node developers when they have to deal with a large data source, and it may not be feasible to process this data all at once. This article will cover the following topics: Types of streams When to adopt Node.js streams Batching Composing streams in Node.js Transforming data with transform streams Piping streams Error handling Node.js streams Types of streams The following are four main types of streams in Node.js: Readable streams: The readable stream is responsible for reading data from a source file Writable streams: The writable stream is re...
Post a Comment
Read more