How to listen for webhooks with Node.js and Express

Have you ever been building an application and thought: "I can make requests to this service's API, but is there a way for them to let my app know when X happens? You could try calling the API on a set interval. Take the response, compare it to the last, and go from there. This is polling, but it is inefficient and can be an easy way to hit rate limits. Instead, some APIs and services offer what's known as a webhook. Instead of contacting them, they contact you. In this tutorial we will go over analyzing a webhook, setting up a local testing environment, and creating a listener for the webhook in Node.js and the Express framework.

How webhooks work

Webhooks are a way to send a notification. They are essentially one-way. The sender doesn't care what you do with it, or in some cases even who you are. At most they only want a response letting them know that your app received it. Many services, like the Stripe API, SendGrid API, GitHub API, and Bearer 🐻 offer the ability to notify you programmatically when an event occurs. The webhook makes a POST request to the URL of your choice with a payloadcontaining details about the event.

If this sounds familiar, that's because it is. A webhook ends up looking like an API call, but in reverse. Rather than you calling the API and requesting data, the API is calling you to let you know something has happened. All services will require at least one thing: the URL, or endpoint, to send the payload to. Think of it like the mailing address. Others may also offer configuration for securing and authenticating webhooks.

If you've ever built a REST API that your application or other apps use, you know nearly everything you need to get started.

For our example, the flow looks like this:

1. Web service sends payload to your app's endpoint.
2. Your app receives payload.
3. Your app responds and confirms receipt.
4. Your app acts upon the payload data.

If you want to test the shape of a webhook before building an application to consume it, you can use a service like Requestbin.

Set up your project

For this example we'll be using Node.js and Express, but the concepts carry over to other languages and frameworks. At the time of this writing, we'll be using Node.js v13.1 and Express v4.17.1. If you're adding webhook consumption to an existing project, skip over the setup below.

To get started, initialize a new node project:

npm init -y

Next install express and body-parser:

npm install express body-parser

The project uses Express to run our server and handle the webhook route. Body parser is a package that makes handling request payloads easier.

Next we'll set up a minimal express server at index.js:

// Require express and body-parser
const express = require("express")
const bodyParser = require("body-parser")

// Initialize express and define a port
const app = express()
const PORT = 3000

// Tell express to use body-parser's JSON parsing
app.use(bodyParser.json())

// Start express on the defined port
app.listen(PORT, () => console.log(`🚀 Server running on port ${PORT}`))

This may look familiar if you've spent some time with express. If your configuration is more complex, that is okay. The same concepts apply to simple and complex express configurations.

Set up a route for the webhook

Consuming a webhook starts the same way as creating a new endpoint for your own API. In express this means creating a new route to handle the incoming call.

//...
app.use(bodyParser.json())

app.post("/hook", (req, res) => {
  console.log(req.body) // Call your action on the request here
  res.status(200).end() // Responding is important
})

//...

Above is our simplified route. We create a new endpoint, http://ourapp.com/hookto handle the request. It is important to respond quickly with a HTTP 2xx (200, 201, 202, etc.) status code. The Slack API requires this response within three seconds. Some APIs like SendGrid and Slack will continue to retry sending the payload if they don't receive an acceptable response code in a reasonable amount of time. Check the docs for the APIs you rely on for specifics.

Configure the webhook and start testing locally

With everything set up on your end, it's time to tell the API provider where to send their event. While normally found in your application's API settings under "webhooks", you will sometimes see it located in "events" or "event subscriptions". Note: Some APIs, like Zeit's Now, require you to programmatically create webhooks through their REST API.

For Bearer, we offer webhooks as a way to receive incident notifications. You can set them up in a rule's configuration:

For local testing, you'll need a way send these requests to your locally running server.

When we run our app (e.g., node index.js) our endpoint becomes http://localhost:3000/hook. That won't work, so instead we'll need to expose the local server. We'll use ngrok to handle this, but there are other solutions like localtunnel and localhost.run available if you prefer.

Sign up for your ngrok account and follow the instructions to download, install, authenticate, and connect. This normally means unzipping their file, placing it in your user folder, running the command they supply, and then running ./ngrok http 3000. The 3000 portion needs to match the port your app is running on. Ngrok provides you with a url that looks something like http://4ds23d1.ngrok.io

To test your webhook, enter your newly created url into the proper area in the API's settings. Don't forget to include your app's webhook endpoint. /hook from our example. It should look something like http://4ds23d1.ngrok.io/hook.

If we test the webhook from one of Bearer's rules using the "Send Test" button, we'll receive a json payload with details about the rule, the API affected, start and end times, and more.

What can you do with this information?

Each API provides different types of events. Webhooks shine when dealing with some event that needs your app to take action. This is valuable when data stored by your application relies on data that might change outside of your app's interface, or when you need to know that an action occurred. Webhooks are popular for connecting services to chat applications, like Slack or Discord, because they can send messages when an event occurs.

For our Rules & Incidents system in Bearer, webhooks allow your application to make decisions when an API isn't performing as expected. For example:

Let's say you send email through service A. Emails for accounts, password resets, etc. You like service A, but it is good to have a backup in case it has performance issues. You have a rule set up that notifies your app when the error rate of service A exceeds 50% over a short span of time. This might be the sign of an unexpected outage. Your app receives the webhook, checks the outage time, and swaps over to your fallback service B until the incident is over.