I’ve just completed the Introduction to Node.js course on Frontend Masters and am posting my notes from the course and additional study here for future reference.

The absolute basics

Node is a JavaScript runtime based on Chrome’s V8. It is most easily installed and managed via the the Node Version Manager (NVM) because this allows you to easily install and switch Node versions while simultaneously avoiding the EACCES permissions problems that are so familiar to those of us who’ve worked on Node projects.

Node can be executed via:

  • the Node REPL (Read, Evaluate, Print, Loop) by typing node in a terminal window. You exit the REPL with Ctrl + C or typing .exit
  • executing a file, for example with node index.js

Basic components

Globals

Some globals to mention are:

  • global is the global namespace object (think of it like window in the browser)
  • __dirname and __filename - give you the current file and directory (both of which are extremely helpful). Note that these are only available when you are using CommonJS modules. There are equivalents for ECMAScript modules, such as import.meta.url to get the current path.
  • process is like a Swiss Army knife. It stores everything about the environment, including the global environment variables on your computer.
  • globals that relate to modules exports, module and require (you see these a lot in Node.js)

Modules

Modules provide a mechanism for including scripts within other scripts. This was originally done by loading multiple <script> tags in a page, then we moved onto bundling. Now we have modules, which are transportable, self-contained pieces of code.

“In Node, each file is an independent module with a private namespace. Constants, variables, functions, and classes defined in one file are private to that file unless the file exports them. And values exported by one module are only visible in another module if that module explicitly imports them.” JavaScript: The Definitive Guide, 7th Edition

There are two different kinds of modules supported in Node.js: Common JS (older but still common) and ECMAScript modules.

Common js module syntax

The Common JS syntax is the default module syntax for Node.js and ships with Node.js.

// A Common JS export
const action = () => {
    console.log('Action');
}

module.exports = action;
// A Common JS require
const action = require('./action');

action();

The newer ECMAScript (ES) module syntax

ES module syntax seeks to make everything standard across Node and browsers. It has been available in Node for quite a few versions now, but you need to tell Node if you want to use this syntax. There are different ways this can be achieved:

  • using the .mjs file extension for your modules makes it explicit that you’re using ES module syntax. This is preferred.
  • setting it in package.json (which means you can keep the standard .js syntax.

This is what ES module export syntax looks like

// This creates a named export
export const action = () => {
    console.log('Action');
}

This is what ES module import syntax looks like

import { action } from './action'

action();

While you can get these module types to interplay with each other (i.e. have a file that exports using one system import using another), the graph nature of modules and their dependencies means it’s not necessarily easy to switch between the two.

Internal modules

These include:

  • fs
  • path
  • child_process
  • http

File system

If you don’t use the sync version of methods, it will be executed asynchronously. If you prefer to block, use the sync version. In most cases you’ll be using async, but not always (the synchronous versions can be useful upon server startup and initialisation, for example).

Error handling

If you don’t handle errors well in Node.js, your whole app will crash for all your users.

Generally in programming, the terms error and exception are often used interchangeably. Within the Node environment, these two concepts are not identical. Errors and exceptions are different. Additionally, the definition of error and exception within Node does not necessarily align with similar definitions in other languages and development environments.

Conventionally, an error condition in a Node program is a non-fatal condition that should be caught and handled, seen most explicitly in the Error as first argument convention displayed by the typical Node callback pattern. An exception is a serious error (a system error) that a sane environment should not ignore or try to handle. - Mastering Node.js - Second Edition by Sandro Pasquali, Kevin Faaborg

For practical purposes, errors are objects that describe what wen’t wrong. When errors are thrown they become exceptions, which stops execution.

When an exception is thrown in Node.js, the current process will exit with a code of 1, which means something went wrong. You can mimic this with process.exit(1) . process.exit(0) means it just finished, and nothing went wrong. You will never use these in practice because it doesn’t allow you to see what the error was.

Sometimes you do want your app to crash in response to an error, but often you will not. Your responsibility in Node.js is to know what exceptions you want to throw and what exceptions you do not want to throw. If you throw an error in Node.js and do not catch it, the app will crash.

Async handling

This was the way errors tended to be handled in the past, but you might still encounter this.

Via a callback
import { readFile } from "fs";

readFile(new URL("doesNotExist.js", import.meta.url), "utf-8", (err, data) => {
    if (err) {
        // Throwing the error here will cause the app to crash
        // But logging it instead will not cause a crash
        throw err;
    } else {
        // Do the thing
    }
    console.log("This will not run");
});
Via a promise
import { readFile } from "fs/promises";

readFile(new URL("doesNotExist.js", import.meta.url), "utf-8").catch((e) => {
    console.log(e);
    console.log("Because we have a .catch(), this line will be logged");
});
Via await
import { readFile } from "fs/promises";

try {
    const result = await readFile(
        new URL("doesNotExist.js", import.meta.url),
        "utf-8"
    );
} catch (e) {
    console.error(e);
}

console.log(
    "This will run because we've got a try/catch. Otherwise it would not"
);

Getting information about uncaught exceptions

If you can’t catch an error for any reason - for example if it’s a coming from a library - then Node.js provides a mechanism to see what the error was. By definition, you cannot catch an ‘uncaught’ exception.

To get access to the uncaught exception, add the handler before the code that might raise an exception.

import { readFile } from "fs/promises";

process.on("uncaughtException", (e) => {
    // This will log the error that was not caught.
    // The app will still crash
    console.error(e);
});

const result = await readFile(
    new URL("doesNotExist.js", import.meta.url),
    "utf-8"
);

console.log(
    `This will not run because the uncaughtException
    event does not allow you to catch errors. It simply
    allows you to log the error in some way
    `
);

Packages

Creating local packages

You can think of packages as a collection of modules (or sometimes even just one module). There are millions of packages on the Node Package Manager (NPM), which comes bundled with Node.js.

NPM allows us to search for, install and build packages.

To turn an ordinary app into a package is to create a package.json file. You can do this with NPM via npm init. This will guide you through a series of questions, some of which are listed below:

  • The package name has to be unique if you’re going to publish it on NPM. You can pay to namespace them.
  • The version (in SemVer)
  • The description will show up on a registry’s website (like NPM)
  • The entry point is the file which is the door into your module
  • The test command is the command to run

scripts in a package.json are powerful. You can make them do anything you want.

npm install —help will show a lot of different ways you can install packages.

Finding and installing packages

The ability to find what’s useful on the NPM registry (and knowing when to look) is an important skill.

When you click on a packages, you’ll see the README.md and a number of useful tabs showing dependencies, dependents, versions, GitHub links (which will allow you to see open issues and gauge how well maintained it is) etc. The npm info command will reveal a lot of this information too (i.e. npm info lodash)

If you use npm install lodash it will create a node_modules folder

The --save flag will add the package to the dependencies section of your package.json, so that they can be installed at a later date with npm install. The similar --save-dev flag differentiates those dependencies that are intended for development only (i.e. they are not needed for the app to run. A good example would be your testing framework). This distinction is important when publishing a package to a registry because those using it will not need the contents of devDependencies because it’s already been built. Similarly for when deploying via CI.

The package.lock file was introduced to lock dependencies to the exact version. This ensures everyone on the team has exactly the same version.

If you’ve got to go digging in the node_modules folder to see if something’s safe, it’s probably not worth it, in my opinion.

There is also a npm uninstall command which you can use to uninstall a package.

The difference between NPM and Yarn

Yarn was created by Facebook at a time when NPM was a bit under-loved. People then started using Yarn, and then NPM introduced the same features - if not better. They all do the same thing.

Both will allow you to point to different registries, rather than the NPM registry.

Using NPM packages

Once a file is declared as a dependency in your package.json and installed you can simply reference it in your code, either with a path to the local file or without a path (in which case it will look in node_modules to see if it’s there, then - if it’s not found - keep looking up the tree to see if it can other node_modules folders.

import _ from "lodash";

const arr = ["One", "Two", "Three", "Four"];
const shuffled = _.shuffle(arr);
console.log(shuffled);

Note here that the lodash page on NPM only shows common js examples of usage, but this works too. It’s worth trying the ES module syntax to see if it will work.

Global installs

You’ll often want CLIs to be installed globally so that they can be used from the terminal. The -g global flag installs a package globally. The uninstall command can also be passed -g to uninstall globally installed packages.

Sometimes you’ll want to run a package without installing it, and for this we have npx (which comes bundled with Node.js). It allows you to run CLIs without installing them. For example, npx yarn install lodash will install lodash via Yarn, without having installed Yarn.

Running scripts

To run scripts within the “scripts” section of your package.json you prefix it with npm run. The only exception of this is “test”, which doesn’t need to have run before it.

You can also easily pass flags or arguments (but not both) to scripts in your “scripts” section of package.json. For example, if you had a script called “remove” in your package.json which simply referenced rm, you could pass it the filename in the call. So that would be:

npm run remove delete_me.txt // passing an argument

When wishing to pass a flag, you’d end the script line with —.

But when you’ve got a lot to do in a “script” command, you can simply provide a filename with the code that you need.

There’s a misconception that everything needs to be built from scratch. In engineering, we’re always piggybacking off the work of others (through the use of NPM packages etc.). Somebody probably already solved it, just install it.

CLI

Any app that runs in a Terminal is a CLI. Node.js is a CLI, NPM is a CLI. There is no UI for a CLI.

The only difference with a CLI app is that you need to add a little bit more information, including which interpreter should be used (i.e. Node, rather than say Bash). This is achieved via a hashbang.

#! /usr/bin/env node

It’s worth bearing in mind that, if you’re using NVM it will create a symlink at the default location for Node, so you don’t need to have the full path to the Node version used by NVM.

You then add a new field to your package.json called “bin” referencing the file that we would like to act as the entry point. Doing so will instruct NPM to register the command in your “bin” folder.

"bin": {
"reddit": "./reddit.mjs"
}

You can think of the “main” entry in a package.json as being for those cases where a package has been imported, whereas the “bin” entry is for those apps which are run as a CLI. Packages tend to have one or the other.

It’s also worth noting that, where third party packages you install come with their own CLIs, you will find a .bin folder to have been added to node_modules.

You then need to install your package on your own computer, which can simply be achieved by running npm install -g while in the same directory as the package.json. Sometimes you might encounter errors when installing globally - due to conflicts etc - and you can resolve this by using npm uninstall -g to remove all your global dependencies.

At this point, you can just type “reddit” to run your CLI.

Using a CLI framework

Just like there are frontend frameworks, there are CLI frameworks that can help you create your own CLI. Some recommended ones are:

Servers

Node.js has access to OS-level networking tools, allowing us to build capable servers. The hard way to create a server is using the built in http module, which is an abstraction around OS level networking tools. It is low-level (for a Node module) and, while you can build a HTTP server from scratch using it, you should probably never do so over using something from the community.

HTTPie

HTTPie allows you to interact with APIs - a bit like curl, but easier to use.

Using Express to create an API

Here’s an example of a simple Express server that registers two middlewares:

  • morgan is a logging middleware (simply registering it results in logging)
  • body-parser allows you to parse the body of requests before they reach handlers.
import express from "express";
import bp from "body-parser";
import morgan from "morgan";

const app = express();

app.use(bp.urlencoded({ extended: true }));
app.use(bp.json());
app.use(morgan("dev"));

const db = [];

app.post("/todo", (req, res) => {
    const newTodo = {
        id: Date.now(),
        text: req.body.text,
    };

    db.push(newTodo);

    res.json(newTodo);
});

app.get("/todo", (req, res) => {
    res.json(db);
});

app.listen(8000, () => {
    console.log(`Server running on http://localhost:8000`);
});

Middleware

In Express, a middleware is code that sits between the request having been received and the response having been provided. They are registered and run in series, passing the req and res to the next. Middlewares can inspect, validate and manipulate the request and response objects.

Dynamic routes in Express

It’s easy to create dynamic routes in Express, as this example shows.

app.get("/todo/:id", (req, res) => {
    const todo = db.find((t) => {
        return t.id === +req.params.id;
    });

    res.json({ data: todo });
});

You can even use Regular Expressions in your dynamic routes.

Testing

Node.js has powerful tooling for testing both Node.js and frontend code. This can be achieved via the builtin assert module as well as via test specific libraries.

Basic unit testing using assert

The builtin assert module is pretty low-level, but allows you to perform tests without a third-party package.

import { add } from "./library.mjs";
import assert from "assert";

console.log(`add() should add two numbers`);

try {
    assert.strictEqual(add(1, 2), 3);
    console.log("Success");
} catch (e) {
    console.log("Fail");
    console.error(e);
}

Libraries

Many JavaScript testing libraries build upon Jasmine. Initially, Mocha followed Jasmine as a popular tool, but now people are converging on Jest. Pretty much every testing framework will work very much like Jest.

Jest

Jest needs a bit of configuration to enable ES Modules, so the examples here will use Common JS modules.

A basic test in Jest looks like this

// Jest requires some config to use ES Module syntax
const { getNewUser, mapObjectToArray } = require("./utils");

describe("mapObjectToArray()", () => {
    const obj = {
        age: 30,
        height: 170,
    };

    test("maps values to array using callback", () => {
        const result = mapObjectToArray(obj, (k, v) => {
            return v + 10;
        });
        expect(result).toEqual([40, 180]);
    });

    test("callback to have been called", () => {
        const mockFn = jest.fn();

        const result = mapObjectToArray(obj, mockFn);

        expect(mockFn.mock.calls.length).toBe(2);
    });
});

An asynchronous test looks like this (note the use of async await)

describe("getNewUser", () => {
    test("it gets user", async () => {
        const user = await getNewUser(1);

        expect(user).toBeTruthy();
        expect(user.id).toBe(1);
    });
});

A test to verify that an error was thrown looks like this

describe("getNewUser", () => {
    test("no user found", async () => {
        expect.assertions(1); // We expect one assertion 
        try {
            const user = await getNewUser(1000);
        } catch (e) {
            expect(e).toBeTruthy();
        }
    });
});

Jest also supports snapshot testing which allows you to compare the HTML string output of a test run with a reference version. This is a good way to get UIs.

A typical snapshot test case renders a UI component, takes a snapshot, then compares it to a reference snapshot file stored alongside the test. The test will fail if the two snapshots do not match: either the change is unexpected, or the reference snapshot needs to be updated to the new version of the UI component. Snapshot Testing · Jest

Debugging

There are several ways to debug in Node, including:

  • console.log
  • Interactive debuggers (built into tools like WebStorm and VS Code)

Debugging via the Chrome debugger

Because Node is built upon V8, you can use the Chrome debugger.

If you run something like

node --inspect-brk my-file.mjs

Which will output information including a URI where the debugger is listening and a link to the docs for Node inspector. You can then use the Chrome dev tools debugger by visiting chrome://inspect in Chrome and clicking ‘inspect’.

Deployment

Publishing packages

When you’re making an NPM package, you would publish this (on NPM, typically) rather than deploying it. This allows it to be added to the NPM registry and be installed by others.