Skip to main content

Posts

Request Id Tracing in Node.js Applications

 If you ever wrote back-end applications in Node.js, you know that tracing the same HTTP request through log entries is a problem. Usually your logs look something like this: [07/Nov/2018:15:48:11 +0000] User sign-up: starting request validation [07/Nov/2018:15:48:11 +0000] User sign-up: starting request validation [07/Nov/2018:15:48:12 +0000] User sign-up: request validation success [07/Nov/2018:15:48:13 +0000] User sign-up: request validation failed. Reason: ... Here, log entries are mixed up and there is no way to determine which of them belong to the same request. While you would probably prefer to see something like this: [07/Nov/2018:15:48:11 +0000] [request-id:550e8400-e29b-41d4-a716-446655440000] User sign-up: starting request validation [07/Nov/2018:15:48:11 +0000] [request-id:340b4357-c11d-31d4-b439-329584783999] User sign-up: starting request validation [07/Nov/2018:15:48:12 +0000] [request-id:550e8400-e29b-41d4-a716-446655440000] User sign-up: request validation success...

Querying MongoDB Like an SQL DB Using Aggregation Pipeline

 What Are Aggregations? Aggregation operations process data records and return computed results. Aggregation operations group values from multiple documents together and can perform a variety of operations on the grouped data to return a single result. In the db.collection.aggregate method and db.aggregate method, pipeline stages appear in an array. Documents pass through the stages in sequence. We will go through some of the stages to achieve a relational DB like results. $match (WHERE) Filters the documents to pass only the documents that match the specified condition(s) to the next pipeline stage. It has the following prototype: { $match: { <query> } } It is the equivalent of WHERE in SQL queries. Let us take an example to make things clear. This example uses a collection named articles with the following documents: { "_id" : ObjectId("512bc95fe835e68f199c8686"), "author" : "dave", "score" : 80, "views" : 100 } { ...

How to upload/store images in MongoDB using Node.js, Express & Multer

 Node.js upload/store image in MongoDB We’re gonna show you how to build this Node.js app step by step. Project Structure Now look at our project structure: node-js-upload-store-images-mongodb-project-structure-revise – views/index.html: contains HTML form for user to upload images. – routes/web.js: defines routes for endpoints that is called from views, use controllers to handle requests. – controllers: home.js returns views/index.html upload.js handles upload & store images with middleware function. – middleware/upload.js: initializes Multer GridFs Storage engine (including MongoDB) and defines middleware function. – server.js: initializes routes, runs Express app. Setup Node.js modules Open command promt, change current directory to the root folder of our project. Install Express, Multer, Multer GridFs Storage with the following command: npm install express multer multer-gridfs-storage The package.json file will look like this: {   "name": "upload-multiple-files...

Multi Tenancy Plugin for Mongoose

  Multi Tenancy Plugin for Mongoose Prelude There are 3 ways of implementing multi-tenancy in mongoDB: on document level (cheap and easy to administer but only secured by app logic) on collection level (not recommended, due to breaking mongoDB concepts) on database level (very flexible and secure but expensive) About The mongo tenant is a highly configurable mongoose plugin solving multi-tenancy problems on document level (for now...). It creates a tenant-reference field and takes care of unique indexes. Also it provides access to tenant-bound model-classes, that prohibit the exploid of the given tenant scope. Last but not least the "MAGIC" can be disabled so that shipping of the same code in single- and multi-tenancy environment (on premis vs. cloud hosted) is a question of a single line of config. Requirements Mongo tenant is compatible with mongoose 4 and 5. Incompatibilities Mongo Tenant does not work with mongoose 4.8.1-4.8.2 see  Automattic/mongoose#4947 . Install $ npm...

NodeJS excel file parser & builder

 Node XLSX Excel file parser/builder that relies on js-xlsx. Usage Installation npm install node-xlsx --save Examples Parsing a xlsx from file/buffer, outputs an array of worksheets import xlsx from 'node-xlsx'; // Or var xlsx = require('node-xlsx').default; // Parse a buffer const workSheetsFromBuffer = xlsx.parse(fs.readFileSync(`${__dirname}/myFile.xlsx`)); // Parse a file const workSheetsFromFile = xlsx.parse(`${__dirname}/myFile.xlsx`); Building a xlsx import xlsx from 'node-xlsx'; // Or var xlsx = require('node-xlsx').default; const data = [[1, 2, 3], [true, false, null, 'sheetjs'], ['foo', 'bar', new Date('2014-02-19T14:30Z'), '0.3'], ['baz', null, 'qux']]; var buffer = xlsx.build([{name: "mySheetName", data: data}]); // Returns a buffer Custom column width import xlsx from 'node-xlsx'; // Or var xlsx = require('node-xlsx').default; const data = [[1, 2, 3], [true, fa...

PiProxy: NodeJS Web Reverse Proxy

 Features Native HTTP2 support as front-end Can proxy to HTTP2, HTTPS or HTTP destinations ACME/LetsEncrypt support for automatic creation and renewal of free, valid and signed SSL certificates No-IP support for automatic dynamic IP updates such as ddns.net and others Configurable passthrough compression using Brotli algorithm Optional rate limiting using sliding window TLS version protection (TLS v1.2 and higher are allowed) Helmet and CSP protection GeoIP reverse lookups on access (large package size is due to GeoIP lite databases included in the package) Custom error handling Agent analysis on access Text file and DB logging Performance and size measurements Built-in statistcs Run Simply install and run: Make sure you have NodeJS already installed Clone repository git clone https://github.com/vladmandic/piproxy or download and unpack from https://github.com/vladmandic/piproxy/releases/ Install using ./setup.js This will install all dependencies as needed Run using: npm start or ...

Building a Post Scheduler for Facebook Pages with Node.js and React

 In this tutorial, we are building a Post Scheduler for Facebook pages. We’ll start off by creating an Express.js web server and use Passport.js to authenticate users with Facebook. After setting up authentication, we’ll build a react app to provide users with an interface to schedule text and images posts for Facebook pages. Getting Started Let’s start by initializing a package.json file by running npm init in your working directory and add these dependencies. "dependencies": {   "body-parser": "^1.13.2",   "config": "^1.30.0",   "connect-ensure-login": "^0.1.1",   "cookie-parser": "^1.3.5",   "ejs": "^2.3.3",   "express": "^4.13.1",   "express-fileupload": "^0.4.0",   "express-session": "^1.11.3",   "mongoose": "^5.1.4",   "morgan": "^1.6.1",   "passport": "^0.2....