Create an NPM lock file for your packages with shrinkwrap

The other day I published a post on how the `–save-dev` flag in NPM is often used incorrectly. Hopefully some people will read it and somewhere a DevOps engineer will be able to lead a happy and peaceful life.

Alas though, it’s not quite all roses in DevOps NPM world. You see there’s still a problem with versioning. As with any good package manager, in NPM you can specify a version or range of versions that is acceptable to be installed for a package. For example:

“gulp-compass”: “~1.2.3” – will install the highest 1.2.x version available

“grunt-ruby-sass”: “^1.1.1” – will install the latest version 1.x.x version available.

Which is all fine and dandy. However,  as we know things change between versions, both major and minor, a lesson that was painfully reenforced yesterday. Developer A had added “gulp-“ruby-sass” at the start of the project and configured it as normal. Several days later Developer B jumped on the project and, ran `gulp` and the world imploded. 40 frustrating minutes later it turns out that gulp-ruby-sass had changed their input syntax and Developer B was running a slightly newer version.

Enter NPM Shrinkwrap

Any modern PHP developer uses Composer and will be able to tell you that when run it creates a composer.lock file. This lock file saves the exact version of each package and the packages’ packages installed so that whenever `composer install` is run in the future the exact same version of each package is installed. Of course if `composer update` is run the packages are resolved from the composer.json file and any new package versions (within the specified version range) are installed, in much the same way as NPM, and a new lock file is automatically created.

It turns out that NPM has the exact same functionality, they just keep very quiet about it and don’t do it automatically like composer or bundler (a Ruby Gem package manager). It’s called shrinkwrap. Once you’ve installed your NPM packages you can run `npm shrinkwrap` and it will create an npm-shrinkwrap.json file. Then in the future whenever `npm install` is run it will resolve the exact package versions (and your packages’ package versions etc etc) found in the shrinkwrap file, meaning no more discrepancies in versions used between developers. Just like composer, if you need to update just run `npm update && npm shrinkwrap`.

Bear in mind that shrinkwrap is intended for production use so won’t save devDependencies by default, if you want it to you can pass the `–dev` flag to it.


tldr: You can lock down your exact npm package versions by using ‘npm’shrinkwrap’

Stop using NPM wrong –save-dev

You see it all the time. There you are trying to install an innocent NPM plugin when BHAM, it happens, a DevOps engineer dies…

Ok so maybe I’m exaggerating a tiny bit but ‘gets very cross‘ is certainly accurate. What I’m talking about is the ‘–save-dev’ flag when adding a new NPM package to a project. For some reason this has now become the de facto standard and it’s, (mostly), wrong.

NPM install Ruby sass

The popular gulp-ruby-sass using –save-dev

NPM Install Grunt Compass

grunt-contrib-compass also recomending –save-dev

So what’s the problem with –save-dev?

Well nothing in theory. Like almost all other package managers NPM can distinguish between packages that should be installed in a production environment and packages that should be installed in a development environment. The obvious reason for this is that you’re probably not going to need all your javascript testing frameworks etc in production. However, unless you’re still FTPing CSS files up to the production server (bad), the chances are you’re also going to want gulp-ruby-sass or whatever else is in your default grunt task on the production server as well.

“But I run `git pull && npm install` on my server and it works fine, what are you on about?” you ask bewildered. Well, firstly congratulations for not using FTP, however what you should be doing is ‘npm install –production’. By default npm installs all packages including the dev ones, however, if you pass the ‘–production’ flag it will only install the production packages.

You can see where the problems start to come in. We now run nearly all of our deployments with Capistrano which we have setup to pass the ‘–production’ flag in by default. If a developer has installed a package with ‘–save-dev’ that needs to be run on the server the entire deployment fails, (gracefully, of course) and some poor DevOps engineer, (sometimes me), has to spend hours trying to work out why the default grunt task isn’t completing only to realise that it’s because the NPM package it’s trying to use isn’t on the server.

So next time you’re adding an NPM package take a moment and decide if it’s going to be required in production or not. If it is, don’t use ‘–save-dev’.

tldr: Don’t use ‘–save-dev’ to add NPM packages unless you’re absolutely sure that they’re not required in a production enviroment