Skip to main content

How to set up a Headless Chrome Node.js server in Docker

How to Set Up a Headless Chrome Node.js Server in Docker
Headless browsers have become very popular with the rise of automated UI tests in the application development process. There are also countless use cases for website crawlers and HTML-based content analysis.
For 99 percent of these cases, you don’t actually need a browser GUI because it is fully automated. Running a GUI is more expensive than spinning up a Linux-based server or scaling a simple Docker container across a microservices cluster such as Kubernetes.
But I digress. Put simply, it has become increasingly critical to have a Docker container-based headless browser to maximize flexibility and scalability. In this tutorial, we’ll demonstrate how to create a Dockerfile to set up a Headless Chrome browser in Node.js.

Headless Chrome with Node.js

Node.js is the main language interface used by the Google Chrome development team, and it has an almost native integrated library for communicating with Chrome called Puppeteer.js. This library uses WebSocket or a System Pipe-based protocol over a DevTools interface, which can do all kinds of things such as take screenshots, measure page load metrics, connection speeds, and downloaded content size, and more. You can test your UI on different device simulations and take screenshots with it. Most importantly, Puppeteer doesn’t require a running GUI; it can all be done in a headless mode.
const puppeteer = require('puppeteer');
const fs = require('fs');

Screenshot('https://google.com');

async function Screenshot(url) {
   const browser = await puppeteer.launch({
       headless: true,
       args: [
       "--no-sandbox",
       "--disable-gpu",
       ]
   });

    const page = await browser.newPage();
    await page.goto(url, {
      timeout: 0,
      waitUntil: 'networkidle0',
    });
    const screenData = await page.screenshot({encoding: 'binary', type: 'jpeg', quality: 30});
    fs.writeFileSync('screenshot.jpg', screenData);

    await page.close();
    await browser.close();
}
Shown above is the simple actionable code for taking a screenshot over Headless Chrome. Note that we are not specifying Google Chrome’s executable path because Puppeteer’s NPM module comes with a Headless Chrome version embedded inside. Chrome’s dev team did a great job of keeping the library usage very simple and minimizing the required setup. This also makes our job of embedding this code inside the Docker container much easier.

Google Chrome inside a Docker container

Running a browser inside a container seems simple based on the code above, but it’s important not to overlook security. By default, everything inside a container runs under the root user, and the browser executes JavaScript files locally.
Of course, Google Chrome is secure, and it doesn’t allow users to access local files from browser-based script, but there are still potential security risks. You can minimize many of these risks by creating a new user for the specific purpose of executing the browser itself. Google also has sandbox mode enabled by default, which restricts external scripts from accessing the local environment.
Below is the Dockerfile sample responsible for the Google Chrome setup. We will choose Alpine Linux as our base container because it has a minimal footprint as a Docker image.
FROM alpine:3.6

RUN apk update && apk add --no-cache nmap && \
    echo @edge http://nl.alpinelinux.org/alpine/edge/community >> /etc/apk/repositories && \
    echo @edge http://nl.alpinelinux.org/alpine/edge/main >> /etc/apk/repositories && \
    apk update && \
    apk add --no-cache \
      chromium \
      harfbuzz \
      "freetype>2.8" \
      ttf-freefont \
      nss

ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true

....
....
The run command handles the edge repository for getting Chromium for Linux and libraries required to run chrome for Alpine. The tricky part is to make sure we don’t download Chrome embedded inside Puppeteer. That would be a useless space for our container image, which is why we are keeping the PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true environment variable.
After running the Docker build, we get our Chromium executable: /usr/bin/chromium-browser. This should be our main Puppeteer Chrome executable path.
Now let’s jump to our JavaScript code and complete a Dockerfile.

Combining Node.js Server and Chromium container

Before we continue, let’s change a little bit of our code to fit as a microservice for taking screenshots of given websites. For that, we’ll use Express.js to spin a basic HTTP server.
// server.js
const express = require('express');
const puppeteer = require('puppeteer');

const app = express();

// /?url=https://google.com
app.get('/', (req, res) => {
    const {url} = req.query;
    if (!url || url.length === 0) {
        return res.json({error: 'url query parameter is required'});
    }

    const imageData = await Screenshot(url);

    res.set('Content-Type', 'image/jpeg');
    res.set('Content-Length', imageData.length);
    res.send(imageData);
});

app.listen(process.env.PORT || 3000);

async function Screenshot(url) {
   const browser = await puppeteer.launch({
       headless: true,
       executablePath: '/usr/bin/chromium-browser',
       args: [
       "--no-sandbox",
       "--disable-gpu",
       ]
   });

    const page = await browser.newPage();
    await page.goto(url, {
      timeout: 0,
      waitUntil: 'networkidle0',
    });
    const screenData = await page.screenshot({encoding: 'binary', type: 'jpeg', quality: 30});

    await page.close();
    await browser.close();

    // Binary data of an image
    return screenData;
}
This is the final step to complete a Dockerfile. After running docker build -t headless:node, we’ll have an image with Node.js service and a Headless Chrome browser for taking screenshots.
Taking screenshots is fun, but there are countless other use cases. Fortunately, the process described above applies to almost all of them. For the most part, only minor changes to the Node.js code would be required. The rest is pretty standard environmental setup.

Common problems with Headless Chrome

Google Chrome eats a lot of memory during execution, so it’s no surprise that Headless Chrome does the same on the server side. If you keep a browser open and reuse the same browser instance many times, your service will eventually crash.
The best solution is to follow the principle of one connection, one browser instance. While this is more expensive than managing multiple pages per browser, sticking to just one page and one browser will make your system more stable. Of course, this all depends on personal preference and your particular use case. Depending on your unique needs and goals, you may be able to find a middle ground.
Take, for example, the official website for performance monitoring tool Hexometer. The environment includes a remote browser service that contains hundreds of idle browser pools. These are designed to pick up new connections over WebSocket when there is a need for execution, but it strictly follows the principle of one page, one browser. This makes it a stable and efficient way to not only keep running browsers idle, but keep them alive.
Puppeteer connection over WebSocket is pretty stable, and you can do something similar by making a custom service like browserless.io (there is an open-source version as well).
...
...

const browser = await puppeteer.launch({
    browserWSEndpoint: `ws://repo.treescale.com:6799`,
});

...
...
This will connect to the headless Chrome DevTools socket using the same browser management protocol.

Conclusion

Having a browser running inside a container provides a lot of flexibility and scalability. It’s also a lot cheaper than traditional VM-based instances. Now we can simply use a container service such as AWS Fargate or Google Cloud Run to trigger container execution only when we need it and scale to thousands of instances within a seconds.

Comments

Post a Comment

Popular posts from this blog

4 Ways to Communicate Across Browser Tabs in Realtime

1. Local Storage Events You might have already used LocalStorage, which is accessible across Tabs within the same application origin. But do you know that it also supports events? You can use this feature to communicate across Browser Tabs, where other Tabs will receive the event once the storage is updated. For example, let’s say in one Tab, we execute the following JavaScript code. window.localStorage.setItem("loggedIn", "true"); The other Tabs which listen to the event will receive it, as shown below. window.addEventListener('storage', (event) => { if (event.storageArea != localStorage) return; if (event.key === 'loggedIn') { // Do something with event.newValue } }); 2. Broadcast Channel API The Broadcast Channel API allows communication between Tabs, Windows, Frames, Iframes, and  Web Workers . One Tab can create and post to a channel as follows. const channel = new BroadcastChannel('app-data'); channel.postMessage(data); And oth...
Post a Comment
Read more

Certbot SSL configuration in ubuntu

  Introduction Let’s Encrypt is a Certificate Authority (CA) that provides an easy way to obtain and install free  TLS/SSL certificates , thereby enabling encrypted HTTPS on web servers. It simplifies the process by providing a software client, Certbot, that attempts to automate most (if not all) of the required steps. Currently, the entire process of obtaining and installing a certificate is fully automated on both Apache and Nginx. In this tutorial, you will use Certbot to obtain a free SSL certificate for Apache on Ubuntu 18.04 and set up your certificate to renew automatically. This tutorial will use a separate Apache virtual host file instead of the default configuration file.  We recommend  creating new Apache virtual host files for each domain because it helps to avoid common mistakes and maintains the default files as a fallback configuration. Prerequisites To follow this tutorial, you will need: One Ubuntu 18.04 server set up by following this  initial ...
Post a Comment
Read more

Working with Node.js streams

  Introduction Streams are one of the major features that most Node.js applications rely on, especially when handling HTTP requests, reading/writing files, and making socket communications. Streams are very predictable since we can always expect data, error, and end events when using streams. This article will teach Node developers how to use streams to efficiently handle large amounts of data. This is a typical real-world challenge faced by Node developers when they have to deal with a large data source, and it may not be feasible to process this data all at once. This article will cover the following topics: Types of streams When to adopt Node.js streams Batching Composing streams in Node.js Transforming data with transform streams Piping streams Error handling Node.js streams Types of streams The following are four main types of streams in Node.js: Readable streams: The readable stream is responsible for reading data from a source file Writable streams: The writable stream is re...
Post a Comment
Read more