Node.js: The Big Picture
Node.js is an open-source, event-driven JavaScript runtime environment used for executing JavaScript code outside the browser. In this multi-part series, you’ll learn what Node.js is all about, its benefits and drawbacks, core modules, and a bit of how it works under the hood.
Where does Node.js shine?
- Web servers
- Microservices
- Serverless
- Real-Time Applications
- CLI Applications
- Streams
Who uses Node.js?
- Wallmart
- Netflix
- Paypal
- Other
The Power of Node.js comes from its ingenious design.
JavaScript is a scripting language designed to run on a single thread in the browser. Node.js brings new things to the table:
- Non-Blocking I/O
- Event-Driver Arhitecture
- Multi-Processing
- Worker Threads
- Makes use of Streams
- Designed around the separation of concerns (through Modules)
Where do these innovations come from?
V8 Engine
Node.js is built on the V8 JavaScript engine, written in C++, using the same engine that powers the Google Chrome browser. V8 is what allows developers to compile JavaScript on a computer.
Libuv
Libuv is a C library designed around Node.js to handle I/O operations. Libuv provides Node.js with additional threads to handle asynchronous and I/O operations without blocking the main thread.
Make no mistake, Node.js still operates on a main (single) JavaScript thread, but it finds ways to offload tasks from it whenever possible.
console.log('Frodo') // EXECUTED FIRST
setTimeout(() => console.log('Sam'), 1000) // EXECUTED LAST
setTimeout(() => console.log('Pippin'), 0) // EXECUTED THIRD
console.log('Merry') // EXECUTED SECONDHere, you can see that the program doesn’t pause and wait in response to an asynchronous operation or a timeout. Rather than obstructing the execution, Node.js offloads items to a separate API for processing and displays the result only after the synchronous code has finished running.
Modern Web browsers have adopted these features as well.
Node.js is designed not to block while:
- Processing HTTP requests
- Performing I/O operations on the File System (using asynchronous API)
- Handling API Communication (Fetch API)
- Using streams
- Sending and receiving Events
- Performing DNS Lookup
- Using Timeouts (
setTimeout,setImmediate) - Applying Zlib compression
- Encryption using
PBKDF2,randomBytes, etc.
Under The Hood
The scheduling of non-blocking operations is handled through the Event Loop.
The Event Loop
When an asynchronous or I/O request comes up in the call stack, it is offloaded to a separate environment where it is processed by one of Libuv threads (or Browser APIs) and then put back into the call stack to hand off to the user.
This means less work for the main thread and ultimately less blocking. There is even an order of precedence with micro and macro tasks. More on this below,
Essentially Libuv boosts the performance of Node.js by executing JavaScript outside the main thread, which is why Node.js apps should make use of it whenever possible.
It’s also important to note that Node.js allows synchronous executions and it recommends using it when you need something to happen immediately. Those can be improved using Worker Threads API.
In addition to this, Node.js has a built-in orchestrator (Cluster) that can spawn a separate instance for each CPU on your machine or additional Worker threads for each instance.
Combined with a vast ecosystem of built-in and third-party tools and the same language on the front and the back, Node.js is a must-know skillset for any JavaScript developer.
Drawbacks of Node.js
Unfortunately, it’s not all sun and rainbows.
Running CPU Intensive Tasks
While Node.js can concurrently manage a series of non-blocking tasks, it pales in comparison to the CPU-intensive tasks that block the execution.
Using a separate process or a worker thread to handle intensive tasks helps, but it’s nowhere near the capabilities of other languages.
Heavy memory usage
Even though the V8 engine brings many adversities to Node.js, but it's also very memory-hungry.
Limited Standard Library
Node.js intentionally keeps its standard library small, encouraging developers to use external modules (NPM) for additional functionality.
Node Package Manager
The NPM is often referred to both as a hero and villain. The latter comes from a series of interconnected modules that heavily depend on each other. If an NPM package is removed from the registry, deprecated, or blocked for security reasons, all other packages that rely on it suffer.
And since anyone can publish just about anything, you should beware of malware and unmaintained packages.
Although Node.js runs on JavaScript, it is not 100% JavaScript you see in your browser. While web JavaScript makes use of:
- Browser APIs
- DOM structure
- Client Events (clicks, hover, and user input)
Node.js replaces these with its APIs to:
- Host servers (HTTP, HTTP2)
- Perform DNS Lookups
- Emit and Listen to Server events
- Read and Write files
- Interact with the database
- Use streams
- Execute CLI scripts
- Inspect Operating System
Ultimately this makes JavaScript run on all fronts, on the web, desktop, IoT, mobile, and server.
Node.js uses events (Event-Driven Architecture) to handle inter-process communication, like notifying processes about specific events or requesting certain behaviors.
const { EventEmitter } = require('events');
const myEmitter = new EventEmitter();
// CREATE THE EVENT
myEmitter.emit('someEvent', 'Hello World');
// LISTEN TO AND ACT UPON THE EVENT
myEmitter.on('someEvent', message => {
console.log(message); // Hello World
});This pattern of listening and responding to an event is reused when:
- Reading from or writing to files,
- Handling HTTP requests
- Working with real-time events (Sockets)
- Interacting with the network
- Transfer state between Child/Worker processes as well as other stuff you can read about:
// USING EVENTS WITH HTTPS MODULE
const https = require('https');
https.get('https://jsonplaceholder.typicode.com/todos/1', (res) => {
console.log('res.statusCode :>> ', res.statusCode); // 200
let responseBody;
// DATA ARRIVES IN CHUNKS WHILE WE LISTEN TO THE EVENT
res.on('data', (incomingData) => {
responseBody += incomingData;
});
// THIS EVENT IS TRIGGERED WHEN DATA COMPLETES
res.on('end', () => {
console.log('responseBody :>> ', responseBody); // {...}
})
});The HTTP module handles requests in a non-blocking manner with streams under the hood. Streams provide an abstraction that allows you to work with data in chunks, making it more memory-efficient and responsive, especially for handling large amounts of data. Read more:
The Process
In a Node.js application, everything starts with a Process.
The Process is a global object that lives in your Node.js instance and retrieves information about the active process in the application, such as:
The Process object makes use of POSIX signals to transfer information, as well as shut down child processes. This is particularly useful when an exception occurs.
// SIGNAL INTERRUPTED (When you hit CTRL+C)
process.on('SIGINT', () => {
console.log('App is shutting down!');
});
// BEFORE EVENT OCCURS
process.on('beforeExit', () => {
// Close database, queue connections
});
// WHEN ERROR IS THROWN BUT NOT HANDLED
process.on('uncaughtException', (error) => {
console.error(error);
process.exit(1); // Terminate the active process with signal 1
});Knowing how processes work lets you manage your application nicely, by transferring information or releasing the allocated resources to it, like database connections or file handles.
Environment Variables
A recent addition to Node.js is an inclusion of reading environment variables from the .env file. Previously you’d need to rely on dotenv npm package. In the latest version, this is supported out of the box.
- Create a .env file in your project root directory
- Add desired variables and assign them values
- use
process.env.VARIABLE_NAMEto access a value
The process.env can also be used to read any environment variable on your system.
At the heart of Node.js lie the Modules. The module is a piece of code or functionality you can separate from the codebase and then import anywhere to reuse when needed.
Characteristics of Node.js modules:
- A module is just another JavaScript file that has exported members, that you import elsewhere.
- Node.js uses the CommonJS module system by default, although EcmaScript modules are also supported.
- In OOP terms, a module is a singleton.
- In a module, all functions, variables, and objects that you export are like public class members. Everything else is treated as a private member.
- To import a module, you point to its location on your machine.
- Core and NPM modules are imported by name. Custom modules are imported with the
./prefix.
Also, Core modules can be imported usingnode:module-nameprefix.
Learn more about modules:
Node.js is designed to build distributed applications with many nodes, hence the name Node.js.
Although Node.js lets you create custom modules, the real meat of Node.js is the Core modules.
Here are some of the Native modules (Node.js standard library):
Then there is the Node Package Manager (NPM).
NPM (Node Package Manager) is one of the largest ecosystems of open-source libraries in the world. Npm is a package manager for the JavaScript language and is the default package manager for the JavaScript runtime environment Node.js.
As a Node.js developer, you’ll probably find yourself using NPM more often than core Node.js modules.
With a plethora of NPM tools to choose from, you open up a terminal (CMD), pick any module, and install in your project using:
$ npm install module-nameIf you’re starting a new project, be sure to first initialize the NPM in your project folder, using:
$ npm init -y (y stands for yes to all questions)This will generate a package.json file in your project. The purpose of this file is to track third-party dependencies in the project, as well as package-lock.json which tracks changes and versions of the installed packages.
NPM can also be used to automate certain operations using scripts, which we’ll dive into in the later chapters.
Summary
Node.js is a popular and powerful platform for building scalable and high-performance server-side applications. Despite the drawbacks, Node.js has proven to be a strong choice for many types of applications, especially those that prioritize scalability, real-time capabilities, and a lightweight, event-driven architecture.
Ready, Steady, Node!
Start using Node.js today.
Go to the official page and click the button to download the LTS version. Be sure to stick to the official documentation for support.
This was just a small slice of everything that Node.js has to offer. In the next chapters, we’ll dive deeper into the core modules of Node.js.
