Node JS: Architecture & Event Loop

👷‍♂️ Software Architecture Series — Part 20.

This article is focused towards providing a structured understanding of how Node JS works internally. How does its event loop get processed and what are its dependencies? So, let’s dive into the topic in an orderly fashion.

Prerequisite: Basic understanding of JavaScript and how Node programs are run. Article is intended towards intermediate or advanced developers.

Node JS

As mentioned on Node JS official guide, ‘Node JS is a JavaScript runtime built on Chrome’s V8 JavaScript engine’.

Node JS is empowered by asynchronous nature of JavaScript and is designed to build scalable network applications. Its concept was incepted to run JS programs outside browser on your machine as a standalone application.

Node JS uses a lot of dependencies to process the program. Google V8 engine and libuv library are two of the most prominent ones among them. V8 engine helps JS run the code outside browser environment. Libuv library is used to access operating system and underlying file system and networks. Node also provides a nice wrapper of APIs such as http, fs, crypto, path etc. which internally point to functionalities inside libuv library.

Node JS: Event loop of a Single Threaded program

Whenever we run a Node Program, an instance of the program is created by Operating System, as we know it as process, and this process is assigned a single main thread which internally communicates with CPU core to run the scheduled instructions and I/O operations. Although, there is just one main thread on the surface, there are a bunch of auxiliary threads in the system kernel that Node JS can utilize for extensive disk and network-based async operations. This group of threads constitutes (what is known as) the worker pool.

When Node JS performs I/O operations, such as reading from a network, accessing a database, or interacting with the filesystem, instead of blocking the main thread and wasting CPU cycles waiting, it continues the execution of other code instructions rather than waiting for I/O operations to complete. Event loop comes into the picture as it decides what the single most thread assigned to run a Node program will be doing at a given time. It takes care of the basic processing, but for I/O operations, it can offload the processing to the worker pool in the system kernel. The worker pool is implemented in libuv and can spawn and manage multiple threads as per the requirement. These threads can individually run their assigned tasks in a synchronous manner and return their response to the event loop whenever ready. Meanwhile, the event loop can continue operating as usual, concurrently catering to other requests.

JavaScript Runtime Model

JavaScript internally maintains a message queue, and with each message in the queue, a function is associated, which is called to handle the message. Each time a function related to a message in message queue is called, a new stack frame is created, and processing continues till the stack frame is empty. After completion, the next message from the queue will be processed by event loop. JavaScript allocates objects on heap, which are unstructured region of memory.

While running function from stack frame, some I/O operations or events (such as reading from a file, making an HTTP request, or querying a database) might get triggered and an event listener is attached. Task is delegated to the system’s kernel or a separate thread from the thread pool and event loop continues to execute other code that is not dependent on the completion of the I/O operation. Once the I/O operation is completed, a callback function associated with that operation is placed in the event loop’s callback queue.. For e.g. ‘setTimeOut’. It takes 2 arguments, a message to add to the queue and a time value in milliseconds which represents minimum delay after which message will be pushed to the message queue. The event loop periodically checks the callback queue for completed tasks and executes the associated callback functions. Once stack frame is empty, event loop executes the added message from the queue. This mechanism allows Node.js to efficiently handle multiple I/O operations simultaneously without blocking the main thread.

Instead of using multiple threads, Node.js achieves concurrency through its event-driven model and asynchronous I/O. While one I/O operation is in progress, the main thread can handle other tasks or respond to events, making it appear as if multiple tasks are being executed concurrently.

Event Loop

Event loop synchronously processes message from message queue and keeps waiting for messages in case queue is empty. Let us dive into how event loop processes a message internally in a sequential way.

Each tick of event loop encloses certain phases or checkpoints.

1. Event loop will check for any pending timers such as ‘setTimeOut’ or ‘setInterval’,

2. Now, Event loop will check for any pending OS tasks (such as server listening to port) or long running operations (such as reading a file).

If any of the above checkpoints are true, Node will enter the event loop. Following operations will be performed sequentially.

1. If pending timers (‘setTimeOut’ or ‘setInterval’) are available, Node will check for related callbacks

2. If any pending OS tasks or long running operations are available, Node will call their respective callbacks.

3. Node enters the polling phase and pauses execution temporarily. It will continue if a new OS task or long running operation is completed or if a timer is about to complete and relevant callback function needs to be called.

4. Node checks for a specific timer called ‘setImmediate’, and its relevant callbacks.

5. Node checks for any ‘close’ operations (which might be some clean up events) and executes them.

Once the actions listed above are done, one tick of event loop is closed and if there are any messages in queue, Node will start its next tick of event loop in the same pattern.

Node.js is known for its scalability, especially in scenarios with a large number of concurrent connections (e.g., web servers handling multiple client requests simultaneously). The asynchronous, non-blocking nature allows it to efficiently manage many concurrent connections without the need for a large number of threads.

NOTE: However, we claim Node is single threaded, one thread executes one event loop, in reality, some of the operations (some Node frameworks or standard library) run outside event loop and hence starts the debate whether Node is single threaded or not? My view is of that event loop is single threaded, not all operations of Node. While Node.js is single-threaded in terms of its event loop, it can still take advantage of multiple cores in a system using the cluster module or by spawning child processes for CPU-intensive tasks. The cluster module allows Node.js to create multiple child processes, each running on a separate core, and share the network ports between them to enable load balancing over the cores. This enables Node.js to handle more concurrent requests and utilize the processing power of multi-core systems effectively. However, the primary concurrency model is centered around the event loop and asynchronous I/O operations.