Capistrano: Deploying multiple sites to the same server

I work for a digital agency that tends to deal with multi-market sites a lot. Most of the sites aren’t particularly high traffic and sit quite happily together on a large EC2 instance with nginx. However, a persistent problem for us was deployment. In our day to day work we generally use Capistrano to deploy sites to both staging and production (very few of our projects warrant build servers) and for the most part it works quite happily, except for when we need to deploy the same repository multiple times to a single server.

For example, one of the sites we manage currently exists in 10 markets and is planning on adding more. They are all served from different domains using separate databases but use the same repository and are hosted on the same server, (config files differ of course). Unfortunately, and after a lot of research, we discovered there is no easy way to do this in Capistrano, indeed it’s talked about quite a lot but there doesn’t seem to be an accepted solution,  so we put our thinking caps on and came up with one.

The goal

We decided that in an ideal world we’d like to be able to achieve the following:

  • Deploy all sites in an environment at the same time, (the most important).
  • Deploy a single site in an environment, (we may want to update just one site, no point in re-deploying all of them).

So not only can we deploy all of them but we still retain the ability to re-deploy an individual one should the need arise. Simple!!

The Solution

Firstly create your Capistrano setup as normal. I don’t really want to get into how to do that here but there are loads of guides available on google, this one for example.

Once Capistrano is installed, setup and configured ensure that the config/staging.rb and config/production.rb (or whatever environments you use) files exist. The content isn’t that important (note: it will be ignored) but they do have to be there. Now for each market/site/locale you wish to deploy for create new environment files for with the naming convention “{environment},{identifier}.rb”. So for example we have:

  • staging.rb
  • production.rb

In each file you set the stage, branch, deploy_to, role, server and anything else specific to that , anything you wish to be globally set can be put in your config/deploy.rb file.

set :stage, :staging

role :web, %w{}

ask :branch, 'develop'

set :deploy_to, "/var/www/"

server '', user: 'deploy', roles: %w{web}

Now at the bottom of your config/deploy.rb file add the following task:

namespace :deploy_all do
  task :deploy do
    on roles(:all) do

      files = Dir.glob("config/deploy/#{fetch(:stage)}.*.rb")
      files.each do | file |
        file = file.sub('config/deploy/', '').sub('.rb', '')
        info "Deploying #{file} to #{:stage}"
        system("cap #{file} deploy")


task :deploy_all => 'deploy_all:deploy'

This globs the deploy directory looking for config files depending on the environment variable passed, it then executes Capistrano for each file found. You can now deploy every site with a single command:

cap staging deploy_all

Or to deploy an individual site do:

cap deploy

Things to improve

This is were we’ve got to so far though we admit it isn’t perfect. Given time we’d quite like to add the following improvements:

  1. A slightly better way of managing the Capistrano settings so they’re a bit DRY-er not repeated in each config file, but I haven’t dug into it any further.
  2. This method is also annoyingly synchronous. I’m fairly sure that with a bit of time and Ruby knowledge the each loop could be re-written to make asynchronous calls, though this may convolute the terminal output somewhat.

Create an NPM lock file for your packages with shrinkwrap

The other day I published a post on how the `–save-dev` flag in NPM is often used incorrectly. Hopefully some people will read it and somewhere a DevOps engineer will be able to lead a happy and peaceful life.

Alas though, it’s not quite all roses in DevOps NPM world. You see there’s still a problem with versioning. As with any good package manager, in NPM you can specify a version or range of versions that is acceptable to be installed for a package. For example:

“gulp-compass”: “~1.2.3” – will install the highest 1.2.x version available

“grunt-ruby-sass”: “^1.1.1” – will install the latest version 1.x.x version available.

Which is all fine and dandy. However,  as we know things change between versions, both major and minor, a lesson that was painfully reenforced yesterday. Developer A had added “gulp-“ruby-sass” at the start of the project and configured it as normal. Several days later Developer B jumped on the project and, ran `gulp` and the world imploded. 40 frustrating minutes later it turns out that gulp-ruby-sass had changed their input syntax and Developer B was running a slightly newer version.

Enter NPM Shrinkwrap

Any modern PHP developer uses Composer and will be able to tell you that when run it creates a composer.lock file. This lock file saves the exact version of each package and the packages’ packages installed so that whenever `composer install` is run in the future the exact same version of each package is installed. Of course if `composer update` is run the packages are resolved from the composer.json file and any new package versions (within the specified version range) are installed, in much the same way as NPM, and a new lock file is automatically created.

It turns out that NPM has the exact same functionality, they just keep very quiet about it and don’t do it automatically like composer or bundler (a Ruby Gem package manager). It’s called shrinkwrap. Once you’ve installed your NPM packages you can run `npm shrinkwrap` and it will create an npm-shrinkwrap.json file. Then in the future whenever `npm install` is run it will resolve the exact package versions (and your packages’ package versions etc etc) found in the shrinkwrap file, meaning no more discrepancies in versions used between developers. Just like composer, if you need to update just run `npm update && npm shrinkwrap`.

Bear in mind that shrinkwrap is intended for production use so won’t save devDependencies by default, if you want it to you can pass the `–dev` flag to it.


tldr: You can lock down your exact npm package versions by using ‘npm’shrinkwrap’

Stop using NPM wrong –save-dev

You see it all the time. There you are trying to install an innocent NPM plugin when BHAM, it happens, a DevOps engineer dies…

Ok so maybe I’m exaggerating a tiny bit but ‘gets very cross‘ is certainly accurate. What I’m talking about is the ‘–save-dev’ flag when adding a new NPM package to a project. For some reason this has now become the de facto standard and it’s, (mostly), wrong.

NPM install Ruby sass

The popular gulp-ruby-sass using –save-dev

NPM Install Grunt Compass

grunt-contrib-compass also recomending –save-dev

So what’s the problem with –save-dev?

Well nothing in theory. Like almost all other package managers NPM can distinguish between packages that should be installed in a production environment and packages that should be installed in a development environment. The obvious reason for this is that you’re probably not going to need all your javascript testing frameworks etc in production. However, unless you’re still FTPing CSS files up to the production server (bad), the chances are you’re also going to want gulp-ruby-sass or whatever else is in your default grunt task on the production server as well.

“But I run `git pull && npm install` on my server and it works fine, what are you on about?” you ask bewildered. Well, firstly congratulations for not using FTP, however what you should be doing is ‘npm install –production’. By default npm installs all packages including the dev ones, however, if you pass the ‘–production’ flag it will only install the production packages.

You can see where the problems start to come in. We now run nearly all of our deployments with Capistrano which we have setup to pass the ‘–production’ flag in by default. If a developer has installed a package with ‘–save-dev’ that needs to be run on the server the entire deployment fails, (gracefully, of course) and some poor DevOps engineer, (sometimes me), has to spend hours trying to work out why the default grunt task isn’t completing only to realise that it’s because the NPM package it’s trying to use isn’t on the server.

So next time you’re adding an NPM package take a moment and decide if it’s going to be required in production or not. If it is, don’t use ‘–save-dev’.

tldr: Don’t use ‘–save-dev’ to add NPM packages unless you’re absolutely sure that they’re not required in a production enviroment