Npm Tips and Tricks

Node.js developers use npm to install their favorite dependencies every day. However, far beyond the very famous npm install, npm offers a wide bunch of commands to ease our daily job. Most of them are unknown, so let's enlight them.

The Most Basic Command: install

The most famous command of npm is probably the install one. Here are two ways to install lodash:

npm install lodash
npm i lodash

Note the default alias at the second line. No need to write the whole install word, a single i is enough.

Yet, it won't save the dependency in our package.json file, and thus, our co-workers won't install lodash automatically at their next npm install. We can of course use the --save (or --save-dev) option, or be lazier and use their respective shortcuts: -S and -D:

npm install --save lodash
npm i -S lodash

npm install --save-dev lodash
npm i -D lodash

Five Types of Dependencies

npm has five types of different dependencies, each of them having a given purpose.

• Production dependencies (related to the dependencies property in package.json) are dependencies our project needs to work. For instance, we may find here express or koa.
• Development dependencies (attached to the devDependencies property) are dependencies which are useful only to developers. They are not required in production. Typically, we find here all testing related libraries (mocha, sinon, etc.) and module bundlers such as Webpack or Gulp.
• Peer dependencies are a special kind of production dependencies. Declared as peerDependencies in the package.json file, they are dependencies required by the project in production, but not embedded directly in current module. For instance, in ng-admin, we have declared angular as a peer dependency. Indeed, this is a module for Angular, and we assume our final users would already have Angular configured in their own project. No need to duplicate source code.
• Optional dependencies are another kind of dependencies stored in optionalDependencies property (we can install them using --save-optional flag). Imagine we want to give our users the ability to choose which rich text editor they can use. We can specify tinyMCE and Quill as optional dependencies. Then, in our code, just check which one is configured to embed features from one or the other:

const config = require("config");

if (config.rte === "tinymce") {
  const tinymce = require("tinymce");
  // configure TinyMCE field
} else if (config.rte === "quill") {
  const quill = require("quill");
  // configure Quill
}

• Finally, the Bundled Dependencies (bundledDependencies property) behave like the normal production dependencies, except they are not installed from npm. This may be useful if we want to retrieve a private module without setting up a whole private repository. But, as we are using Git for all our projects, we can just use Git repositories instead.

Using Git Repositories Instead of npm Modules

Sometimes, we don't want to use npm modules directly. Either we are maintaining a private repository, or we are waiting for a critical branch to be merged. We can simply use a Git repository URL directly in the package.json file:

{
  "dependencies": {
    "angular-ui-codemirror": "git+ssh://git@github.com:jpetitcolas/ui-codemirror.git#di"
  }
}

We chose to use the di branch in previous example (the part after the final #). We may also have used a commit hash or a tag name.

If we are using GitHub, the dependency URL can be simplified even more:

{
  "dependencies": {
    "angular-ui-codemirror": "jpetitcolas/ui-codemirror#di"
  }
}

Cleaning Up Not Declared Modules

Sometimes, we may forget to include the --save or --save-dev option when calling npm install, causing the "but it works on my machine" syndrom. To prevent it, just run the prune command from time to time:

npm prune

This command simply removes all dependencies that are not present in the package.json file. Trying to build your project after that is a good way to detect missing dependencies.

It may also be used in production. Imagine we are working on our deployment process, and that something broke. We are going to debug in production directly (even if we know it is bad). And we may sometimes make a standard npm install command instead of a npm install --production one. Not a big deal, but how can we clean up all unrequired development dependencies? Using the following command:

npm prune --production

Blazing Fast Documentation Browsing

npm is shipped with two useful (yet quite unknown) commands: npm home and npm repo. These commands open respectively the project home page and project repository in our favorite web browser. This way, no need to google cumbersome GitHub + [repository name] anymore.

These two URLs are extracted from the package.json file. For instance, considering the admin-on-rest repository:

{
  "homepage": "https://github.com/marmelab/admin-on-rest#readme",
  "repository": "marmelab/admin-on-rest",
}

Fixing Versions During Installation

As you may have noticed during a failed production deployment, npm doesn't strictly fix the versions of dependencies by default. It just fixes the major version, and accepts minor version upgrades. It sometimes leads to headaches, as the JavaScript ecosystem is not really rigorous about semver.

We noticed it again when letting jquery-ui upgrade itself on one of our customer project:

To better understand how to prevent this kind of issue, let's look under the hood of the npm versioning system.

Tildes and carets

There are three ways to define a dependency version in our package.json file:

• Caret (^) is the most relaxed. It matches all major versions and prefers the latest:

  ^2.2.3 means at least 2.2.3 up to 2.*.*. So it will install 2.2.12, 2.10, but not 3.0.

• Tilde (~) is stricter, matching the most recent minor version:

  ~2.2.3 means at least 2.2.3 up to 2.2.*. So it will install 2.2.12, but not 2.3.0.

• Finally, no prefix means the exact version (2.2.3). This may fix other dependent libraries. For instance, if you depend on react@15.3.0 and material-ui@~0.16.1, you may be surprised not to get the latest material-ui minor version (0.16.4) because it depends on react@15.4.0.

Using fixed versions avoids regressions in production, but you won't benefit from minor bugfixes automatically.

Fixing Versions for New Dependencies

If we add new dependencies to our project and we want to fix version directly during installation, we'll use the --save-exact flag:

npm install --save --save-exact <package_name>

Or, if we want to automatically save exact version without specifying this flag, we can configure our npm globally:

npm config set save-exact true

But how about existing dependencies?

Fixing Versions on an Existing Project

We may first think to simply remove carets and tildes from our package.json file. It may work, but it also may break. Indeed, as explained above, loose dependencies update themselves automatically without updating the package.json file. Hence, a dependency may be at version 1.2.7 while our package.json still refers to ~1.2.0 version.

We need a more precise method to use real installed version. Getting the version of an installed dependency is as simple as:

cat node_modules/<package_name>/package.json | grep version

Bash gurus may use esoteric command to retrieve version number. But, as I always forget how to write a simple for loop in Bash, let's use simple JavaScript instead:

const fs = require("fs");

// let's read our package.json file
const package = require("./package.json");
const dependencies = package.dependencies;

// do not mutate original dependencies
const updatedPackage = Object.assign({}, package);

["dependencies", "devDependencies"].forEach(depType => {
  Object.keys(updatedPackage[depType]).forEach(dep => {
    // read package.json file to extract current version
    const fileContent = fs.readFileSync(`./node_modules/${dep}/package.json`, {
      encoding: "utf-8",
    });
    const depPackage = JSON.parse(fileContent);

    // update version in memory dependencies tree
    updatedPackage[depType][dep] = depPackage.version;
  });
});

// update package.json content with updated versions
fs.writeFileSync(`./package.json`, JSON.stringify(updatedPackage, null, 4));

We take advantage of the JSON format of package.json to parse and manipulate it in pure JavaScript. The code should be self-explainatory. Note however the third argument of JSON.stringify() on the last line, which allows us to beautify the outputted JSON, using 4 spaces for indentation.

This script will transform your package.json from:

{
  "dependencies": {
    "babel-runtime": "~6.16.0",
    "inflection": "~1.9.0",
    "lodash.debounce": "^4.0.6",
    "lodash.get": "~4.4.2",
    "material-ui": "~0.16.4",
    "quill": "~1.1.5",
    "react": "^15.4.0",
    "react-dom": "~15.4.0",
    "react-redux": "~4.4.5",
    "react-router": "~2.8.1",
    "react-router-redux": "~4.0.6",
    "react-tap-event-plugin": "~2.0.0",
    "redux": "^3.6.0",
    "redux-form": "~6.2.0",
    "redux-saga": "~0.13.0"
  }
}

to:

{
  "dependencies": {
    "babel-runtime": "6.18.0",
    "inflection": "1.10.0",
    "lodash.debounce": "4.0.8",
    "lodash.get": "4.4.2",
    "material-ui": "0.16.4",
    "quill": "1.1.5",
    "react": "15.4.0",
    "react-dom": "15.4.0",
    "react-redux": "4.4.5",
    "react-router": "2.8.1",
    "react-router-redux": "4.0.6",
    "react-tap-event-plugin": "2.0.0",
    "redux": "3.6.0",
    "redux-form": "6.2.0",
    "redux-saga": "0.13.0"
  }
}

Shrinkwrapping Dependencies

We handled our dependencies the home-made way. However, there is a dedicated command to deal with them correctly:

npm shrinkwrap

This command creates a npm-shrinkwrap.json file, which is the equivalent of composer.lock in PHP. It contains a snapshot of currently installed dependencies. npm install takes this file in priority when installing dependencies.

It sounds like the perfect solution for our versionning issues. Yet, it has several drawbacks, making mostly unusable in production. The main issues are:

• the lack of Git dependencies support,
• the fact that you can't have one shrinkwrap for development and another for production - only one. If you fix dependencies for development and commit the shrinkwrap files, the development dependencies will be installed in production, too.
• it may be lossy and create not reproducible output.

We encountered several issues using shrinkwrap and so decided to not use it anymore.

I didn't take a look on Yarn, the npm alternative for now. But it promises to fix all these issues.

Checking Outdated Dependencies

Packages regularly update, fixing some bugs, improving performances, or adding great new features. Yet, following every package evolution is an inhumane task. That's why npm offers the outdated feature:

npm outdated

Package                 Current  Wanted  Latest  Location
babel-cli                6.14.0  6.16.0  6.18.0  admin-on-rest
babel-core              MISSING  6.17.0  6.18.2  admin-on-rest
babel-eslint              6.1.2   7.0.0   7.1.0  admin-on-rest
babel-preset-es2015      6.14.0  6.16.0  6.18.0  admin-on-rest
[...]

In a single glance, we have a map of all the efforts we need to bring to be up-to-date. Generally, minor version are painless to upgrade. Updating all our dependencies on a regular basis (twice a month?) is a good practice. It may cost a few hours per month, but it's far cheapest than being forced to update a major version because of a production blocking issue.

Automate Npm Init Parameters

Even if we don't initialize a new project every day, we can configure some of the parameters asked during npm init, using the following commands:

npm config set init.author.name "Jonathan Petitcolas"
npm config set init.author.url "https://www.jonathan-petitcolas.com"
npm config set init.license "MIT"

Updating Contributors List Automatically

We should always be grateful to all project contributors. They spend some time to contribute to our project, either by creating an issue or, even better, by submitting a pull request. On open-source projects, we can thank them at least by respectively solving their issue, or adding them in the contributors list. Automating the first case is far beyond the scope of this post, so let's focus on the second one.

The package.json file contains a contributors property as an array of developer names (and possibly email addresses). We may fill this field manually, gathering all commit authors from a git log list. But, we are developers, and prefer automatize cumbersome tasks. And so does doowb thanks to his update-contributors script (working only with GitHub).

Installing it is as simple as:

npm install --save-dev update-contributors

Finally, we just have to call this freshly installed script to update our package.json file:

./node_modules/.bin/update-contrib

And we are done! :)

More npm Tips

If you have other npm tips and tricks, please share them in the comments!