Building Scalable Web Apps with Node.js and Express

CHAPTER 1

Introduction to Node.js

Introduction

The popularity of Node.js is booming day by day in the IT market worldwide. Through this book, any JavaScript developer can easily learn about Node.js from basic to advanced levels. This chapter talks about the Node.js basics and architecture. We will also learn how to write a simple Node.js program.

Structure

In this chapter, we will be covering the following topics:

Defining Node.Js and Where It is Used

Pros and Cons of Node.Js

Installing Node.Js on Various Platforms

Understanding Event-Driven Programming

Node.js Architectures

Writing HTTP and HTTPS Server

Using the Cluster Module

Defining Node.Js

When Ryan Dahl demonstrated his remarkable work, Node.js at JSConf 2009, it was the beginning of a new era. He stated that the concurrency was achieved by threads in most of the top languages and that using threads has certain problems as context switching between threads is costly. Using the event loop, he showed that Node.js achieves way higher concurrency than any of the existing languages. People at the conference welcomed the idea and applauded Dahl. Thus began the much-awaited shift in the programming world.

Node.js is an open source cross-platform JavaScript runtime environment. It states that anyone can use it free of cost on any operating system such as Windows, Linux, Unix, Mac, and more. JavaScript is the foundation of Node.js. The code of any node.js application is written in JavaScript. This code runs on Google Chrome’s V8 JavaScript engine which converts source code to machine code directly without interpreting and then it gets executed without the need for a web browser. Node.js provides the necessary environment for the code to run.

Figure 1.1: Node.js Architecture

Figure 1.1 shows a high level architecture of Node.js. Even though Node.js is single threaded, it still handles lots of concurrent requests at a time through a mechanism of asynchronous non-blocking I/O operations which provides hidden threads from the libuv library that executes itself as multi-threaded.

Due to non-blocking I/O operation, Node.js is fast compared to other languages because the request does not wait for its response and parallelly executes another request. All requests are first sent to the event queue, processed in the event loop, and then sent back to the V8 engine through the queue as displayed in Node.js architecture. More about the event loop is elaborated in the section, "Event-driven Mechanism" later in this chapter.

Node.js is not only written for backend-side server programming applications but also developed as node modules and used on the client side, which is beneficial for developers as the same language is used on both sides.

Applications of Node.js

The usage of Node.js has been growing at a fast pace in IT Industries due to its features and different types of application. Here are a few examples of different types of applications related to Node.js.

Single-Page Applications

Currently, there are a lot of organizations and enterprises that provide complex and real time solutions to their clients by developing Node.js applications as server side applications through single-page applications. For example, Gmail, Twitter, Facebook, Trello, and many more applications are developed as SPA (Single-Page Applications). A single-page application communicates with the user’s actions by rewriting data on a single web page instead of reloading the whole web page.

Real-time Applications

Node.js is an ideal model for real-time applications because it gives responses to numerous requests at the same time. If there are a large number of users that need real-time response, Node.js is a better choice. You can use Node.js with WebSockets for continuous connection and to provide a faster response time. Applications such as audio, video, chat, multiplayer games, and stock trading are developed in this manner.

IoT Devices Applications

Due to its faster response time and ability to handle large numbers of requests concurrently, Node.js is a good choice for Internet of Things (IoT) apps where devices or sensors are connected to the internet and send huge amounts of data continuously. IoT use cases such as fire detection, noise pollution measure, fitness tracker, health monitoring are many such applications where Node.js is playing a big role.

Data Streaming Application

Node.js allows working with an abstract interface as a stream for data streaming. Large media files are divided into small chunks and sent as buffers. These buffers are transformed into meaningful data. Netflix-like streaming services use Node.js where data is transferred in chunks instead of whole large streams that reduce loading and delay while streaming happens.

The uses of Node.js are not limited to the preceding types of applications but also many more types of applications developed in Node.js such as making proxy or signaling servers, monitoring data-based applications, and more.

Pros of Node.js

Node.js is a very powerful runtime environment for JavaScript. It allows developers to build high performance and scalable applications. Some of the key advantages that Node.js offers are as follows:

Cross platform: Node.js comes up with cross-platform functionality so the application can be easily developed on any OS and deployed on any platform. Key platforms supported by Node.js are Windows, Mac(Intel), Mac(ARM), and Linux (Intel/ARM). Probably every major platform is supported.

High performance: Node.js offers high performance due to asynchronous non-blocking I/O operations which execute requests in parallel without waiting for the response of any other request.

Easy to scale: Node.js is itself single threaded but in heavy traffic, it handles a lot of requests at the same time to scale up with the "cluster" module which creates child processes and reduces the load on the application.

Caching: Node.js allows storing data in temporary memory that is not updated frequently, which reduces loading time and saves database transactions. This is called caching.

Huge community: Ever since Node.js appeared, its community size has been increasing day by day. The language used for programming is JavaScript which has been the backbone of the internet and almost every front-end developer was already familiar with it. This made learning easy and made the community grow rapidly. There are more than 1.3 million open-source libraries available for use.

There are many other advantages as well such as cost-effectiveness, ease of learning, and adaptability. Node.js is a technique that has really made a difference.

Cons of Node.js

There are some disadvantages of Node.js too. However many of these can be overcome using best practices.

Single threaded: Node.js is single threaded, which is an advantage as well as a pitfall because it is unable to process heavy CPU oriented computation quickly. When requests which need more CPU for processing come in the event loop, they keep piling up because until it finishes one request, it will not pick other requests from the event queue. However, this happens only when there are only CPU centric tasks. If a request needs some IO to happen, another request will be picked while a request waits for IO to complete. CPU centric tasks make the performance low and delay the response. For example, for searching algorithms and mathematical calculations where complexity is high at that time, Node.js is not recommended due to poor performance.

Callback hell: Asynchronous programming in Node.js can be challenging for some developers, especially when using callbacks. Callback hell is a situation when callback functions are nested. This can make code difficult to read and maintain. However to avoid this, developers can use promises, async-await, or libraries such as Async.js.

Library compatibility: Although there are more than a million libraries available yet those being open sourced by individual developers might not be up-to-date to the latest versions. This sometimes makes it difficult to use those libs in projects.

Installing Node.js

Now when we have a high level understanding of what Node.js is and what it offers, let us jump to the setup. There are different ways to download and install Node.js in your system but here, we give the easiest and best way. Download the LTS (Long Term Support) Version of Node.js from its official site (https://nodejs.org/en/download) based on your operating system. On the site, there will be LTS and Current Version, choose the LTS version because it is stable and recommended for complex projects.

At the time of writing, Node.js version 20 is ACTIVE.

Figure 1.2: Node.js Versions

The preceding up-to-date release schedule can be seen on Node.js GitHub page— https://github.com/nodejs/release#release-schedule.

Installing Node For Linux/Ubuntu

NPM (Node Package Manager) is the default package manager for node.js and also a library of JavaScript software packages. It is open source so that developers can install other modules in their project via npm free of cost.

Node Version Manager (NVM) is a shell script that manages multiple node versions and uses it on different projects.

We can highlight the importance of installing Node.js through NVM with real use cases.

Using NVM, you can easily manage multiple Node.js versions on the same machine. Here is how it helps:

Version Management: NVM allows you to install multiple versions of Node.js on your system. This means you can switch between different versions seamlessly based on the requirements of your projects.

Isolated Environments: Each Node.js version installed through NVM is isolated from others. This ensures that changes made to one version won’t affect the others. It is particularly useful when you are working on projects with conflicting dependencies or when you need to maintain compatibility with older versions.

Flexibility: With NVM, you have the flexibility to switch between Node.js versions effortlessly. This allows you to test your applications across different versions, ensuring compatibility and stability.

Project-specific Versioning: NVM allows you to specify the Node.js version required for a particular project. This ensures that each project uses the correct version of Node.js without interfering with others.

Easy Updates: NVM simplifies the process of updating Node.js to the latest version. You can easily upgrade or downgrade Node.js versions with a single command, ensuring that your development environment stays up-to-date.

NVM for managing Node.js versions provides a streamlined and efficient workflow, enhancing productivity and reducing potential conflicts between projects. It is an essential tool for developers working on multiple Node.js projects simultaneously.

Based on the advantages of NVM, we will install Node.js through NVM on different platforms.

Let us first install NVM and then install Node.js by NVM. Open terminal/console or cmd and follow three given steps:

Update your system with the latest versions of packages.

$ sudo apt-get update

Download and install NVM using this command:

$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash

Before running the preceding command, make sure curl is installed on the system. If it is not installed then run the following commands to install and verify:

$ sudo apt install curl $ curl –version

Verify NVM version:

$ nvm –version

Install Node.js as follows:

$ nvm install node

This command will install the latest stable version of node.

To install Node.js with LTS version, use this command:

$ nvm install -lts

If anyone wants specific version of node, then add specific version at end of NVM and install as follows:

$ nvm install 18.15.0

Or

$ nvm install 18.x

In the preceding commands, 18.15.0 is a specific version of node.js and 18.x means it will consider the highest version of 18 above and below 19.

Verify Node.js version as follows:

$ node –version

After successfully executing the above steps, you can expect the following output to be displayed in the command prompt for your reference.

Figure 1.3: Linux Node.js Installation

Once Node.js installed, by default NPM is also installed with Node.js installation package, which can be verified with $ npm –version

Other NVM commands which can be helpful for developers to play with node versions on different projects are as follows:

$ nvm ls - Checks list of node version in the system

$ nvm use 18.x – For specific use of node version on the project

$ nvm alias default 18.x – It is to set default for all projects in the system

$ nvm uninstall 18.x- It will uninstall that 18.x version from system

Installing Node.js for Windows

While we have covered Linux installation, now let us proceed with the installation process on Windows. You can follow the steps outlined below for Windows installation.

Installing Node.js through NVM on Windows via the command prompt (cmd) requires the usage of a specialized tool called "nvm-windows". Here are the steps to install Node.js on Windows using nvm-windows via the command prompt:

Download NVM for Windows:

Go to the GitHub repository of nvm-windows: nvm-windows. You can explore more nvm for windows on https://github.com/coreybutler/nvm-windows.

Download the latest installer (.zip file) from the Releases section from following link: https://github.com/coreybutler/nvm-windows/releases Here, we will download nvm-setup.zip file

Figure 1.4: Windows Node.js Download Zip File

Extract the Zip File:

Extract the downloaded .zip file to a directory on your system.

Install NVM for Windows:

Open the Command Prompt as an administrator (right-click and select "Run as administrator"). Navigate to the directory where you extracted the nvm-windows files.

Figure 1.5: NVM Install Select Location

Click Finish to complete the process:

Figure 1.6: NVM Finish Install Process

Run the nvm-setup.zip executable to start the installation process. Follow the on-screen instructions to complete the installation.

Verify NVM Installation:

Close and reopen the Command prompt as administrator. Run the command NVM version to ensure that NVM is installed correctly.

Install Node.js:

Once NVM is installed, you can use it to install Node.js. To install a specific version of Node.js, use the command nvm install (for example, nvm install 18.0.0).

After the installation is complete, you can switch between Node.js versions using the nvm use command.

Verify Node.js Installation:

Run the command node -v to verify that Node.js is installed and the correct version is active.

Installing Node.js for mac

Installation of Node.js on Mac OS is similar to that of Linux. Follow the given steps:

Install NVM: To install NVM, we just need to run the following command:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash

Please make sure that curl is available.

Install Node.js using NVM. In case, we just want to install latest Node.js, run the following command:

nvm install node

This command will automatically download and install the latest Node.js version. In case you want to install the LTS (Long term support) version, run this command:

nvm install –-lts

(Please note that there are two '-’ hyphens before lts).

Verify the installation by opening the console and run the following command:

node --version

This will output the version installed, for example:

v20.0.0

We can also check the npm version using this command:

npm -- version

9.6.4

Event-Driven Mechanism

Node.js is an asynchronous non-blocking event-driven programming. Any action that happens is called an event and it is either done by the user or the system itself. Node.js provides an inbuilt module Event, which is an instance of EventEmitter. Event is an I/O request that is first sent to the Event Queue. If there are multiple concurrent requests coming to the queue, then the queue passes it to the event loop.

Figure 1.7: Event-Driven Diagram

Event loop monitors the event queue, collects the events from it, then processes it and executes based on blocking and non-blocking functions. Blocking functions are executed sequentially, one after the other, and the second function is not called until the first one responds. Sometimes it depends on external resources and waits for its response which takes longer time, whereas non-blocking functions do not need to wait for any response. It executes asynchronously, which means multiple functions run parallel at a time so that they are not depending on one another. Blocking and non-blocking functions send thread and I/O Pool to its pools, respectively. Once the actual operation is done, the response of that request is sent back to the event queue via event loop. In nutshell, events are emitted and then registered or unregistered through a queue, which is monitored by the event loop, binding the appropriate handlers accordingly.

Node.js follows a non-blocking asynchronous model even though it has a single thread that handles multiple requests at a time, without blocking its call respective handlers.

Example of Event Programming

Create a file save with event_index.js and paste the following code:

// Import 'events' module

const events = require('events');

// Initiate an EventEmitter object

const eventEmitter = new events.EventEmitter();

// Binds event handler for send message

eventEmitter.on('send_message', function () {

console.log('Hi, This is my first message');

});

// Handler associated with the connection event

const connectHandler = function connected() {

console.log('Connection is created');

// Trigger the corresponding event

eventEmitter.emit('send_message');

};

// Binds the event with handler

eventEmitter.on('connection', connectHandler);

// Trigger the connection event

eventEmitter.emit('connection');

console.log(Finish);

Run the file with $ node event_index.js and the following output will be shown:

Connection is created

Hi, This is my first message

Finish

Example of Synchronous Code

Create one file named hello.txt and paste the following text in it:

Hello, I am Developer

Create another file named index.js within the same folder, paste the following code and save it:

const fs = require('fs');

console.log('Start');

const data = fs.readFileSync('hello.txt');

console.log(data.toString());

console.log('End');

Run the code as follows:

$ node sync_index.js

You will get an output as follows:

Start

Hello, I am Developer

End

Here, fs is a file system module importing it and fs.readFileSync() is a synchronous function which waits until file read is complete and assigns that response to the data variable. It prints line-by-line and executes synchronously.

Example of Asynchronous code

Create one file named hello.txt and paste the following text:

Hello, I am Developer

Create another file named index.js within the same folder, paste the following code and save it:

const fs = require('fs');

console.log('Start');

fs.readFile('hello.txt', function (err, data) {

if (err) {

return console.error(err);

}

console.log(data.toString());

});

console.log('End');

Run the code as follows:

$ node async_index.js

You will get an output as follows:

Start

End

Hello, I am Developer

Here, fs is a file system module. We import it, and fs.readFile() is an asynchronous function. This function does not wait for the file read to complete. Instead, it has a callback function. Once the file read operation is finished, the callback function executes and prints the data. Therefore, the line after the callback is executed asynchronously.

Types of Node.js Architectures

When we start developing the applications using Node.js, it is important to decide how your application should be structured. There are many ways to structure your Node.js application with different kinds of architecture. Let us discuss those briefly here.

Monolithic Architecture

In this architecture, all components or modules of business logic are blended together in a single unit. Almost all web servers or server-side frameworks are built using monolithic architecture (see Figure 1.8), which is the easiest way for developers:

Figure 1.8: Monolithic Architecture

For smaller applications that do not require extensive scalability, this architecture can be suitable. However, it may not be well-suited for larger and more complex applications. As the traffic load on your server-side application grows, you will need to scale it to handle the increased demand. In this architecture, you have a single main Node.js server file that routes all API requests to controllers and services, managing database transactions.

You can scale a monolithic architecture using clusters to reduce the load. However, there are instances when a single server is unable to handle the incoming traffic. In such cases, you can deploy the same code on multiple servers, run application servers, and employ a load balancer like Nginx. The load balancer, using a round-robin approach, becomes a reliable solution, especially for very large and heavily used applications. We will delve into this aspect in more detail in the deployment section. The biggest drawback of this structure is that if there is a small change needed on any component, then it needs to be done in all servers and rebuilt and redeployed again.

Microservice Architecture

A microservices architecture is a type of architecture that is developed as a collection of services. The framework provided here allows us to develop and deploy the microservices along with maintaining them independently. Microservices sort out the challenges of monolithic systems by fragmenting the application from a whole into several smaller parts. It is reliable and suitable for large and complex applications such as e-commerce platforms, social sites where multiple features are provided to millions of users at same time. Hence, during maintenance or while adding new features, it does not interrupt other existing features and deploy only updated services. Nowadays, it is trending more and more for its flexibility where multiple developers work individually and are only responsible for their own small code instead of whole system code.

In this architecture, all components or modules of business logic are individual. Many large enterprises use this kind of microservice architecture (see Figure 1.9):

Figure 1.9: Microservice Architecture

As per the preceding diagram, client as user or UI sends a request, which is collected by API Gateway and passed to the respective microservice, which has its own function (Lambda Function). This function connects to the database and gives back a response accordingly. Each microservice can be easily changed and deployed without affecting each other. In addition, these microservices also call each other through API HTTP service or gRPC (Google Remote Procedure Call) which is a generic flow of microservice architecture. However, while it is a cost-effective and time-saving architecture for development, it may not be suitable for smaller applications. This is because it relies on cloud-based solutions, which can become expensive even with minimal setup requirements. This cost issue can often be mitigated by adopting more budget-friendly solutions offered