Speeding up composer on AWS OpsWorks
At EasyBib, we're heavy users of composer and AWS OpsWorks. Since we recently moved a lot of our applications to a continuous deployment model, the benefits of speeding up the deployment process (~4-5 minutes) became more obvious.
Composer install
Whenever we run composer install, there are a lot of rount-trips between the server, our satis and Github (or Amazon S3).
One of my first ideas was to to get around a continous reinstall by symlinking the vendor directory between releases. This doesn't work consistently for two reasons:
What's a release?
OpsWorks, or Chef in particular, calls deployed code releases.
A release is the checkout/clone/download of your application and lives in /srv/www:
srv/
└── www
└── my_app
├── current -> /srv/www/my_app/releases/20131008134950
├── releases
└── shared
The releases directory, contains your application code and the latest is always symlinked into place using current.
Atomic deploys
- When deploying code, deploys need to be atomic. We don't want to break whatever is currently online — not even for a second or a fraction of it.
- We have to be able to roll-back deployments.
Symlinking the vendor directory between releases doesn't work because it would break existing code (because who knows how long the composer install takes or a restart of the application server) and it would require an additional safety net in place to be able to rollback a failed deployed successfully.
Ruby & Chef to the rescue
Whenever a deployment is run, Chef allows us to hook into the process using deploy hooks. These hooks are documented for OpsWorks as well.
The available hooks are:
- before migrate
- before symlink (!)
- before restart
- after restart
In order to use them, create a deploy directory in your application and put a couple ruby files in there:
before_migrate.rbbefore_symlink.rbbefore_restart.rbafter_restart.rb
If you're a little in the know about Rails, these hooks will look familiar.
The migration hook is probably used to run database migrations — something we don't do and probably will never do. ;-) But be assured: at this point in time the checkout of your applications is complete: or in other words, the code is on the instance.
The symlink hook is what we use to run composer install to get the web app up to speed, we'll take a closer look in a second.
Before restart is a hook used to run commands before your application server reloads — for example something like purging cache directories, whatever you want to get in order before /etc/init.d/php-fpm reload is executed to revive APC.
And last but not least, after restart — used on our applications to send an annotation to NewRelic — that we successfully deployed a new release.
Before symlink
So up until now, the before_symlink.rb looked like this:
composer_command = "/usr/local/bin/php"
composer_command << " #{release_path}/composer.phar"
composer_command << " --no-dev"
composer_command << " --prefer-source"
composer_command << " --optimize-autoloader"
composer_command << " install"
run "cd #{release_path} && #{composer_command}"
Note: release_path is a variable automatically available/populated in the scope of this script. If you need more, your node attributes are available as well.
Anyway, after reading Scaling Symfony2 with AWS OpsWorks, it inspired me to attempt to copy my vendors around. But instead of doing it in a recipe, I decided to use one of the available deploy hooks for this:
app_current = ::File.expand_path("#{release_path}/../../current")
vendor_dir = "#{app_current}/vendor"
deploy_user = "www-data"
deploy_group = "www-data"
release_vendor = "#{release_path}/vendor"
::FileUtils.cp_r vendor_dir, release_vendor if ::File.exists?(vendor_dir)
::FileUtils.chown_R deploy_user, deploy_group, release_vendor if ::File.exists?(release_vendor)
composer_command = "/usr/local/bin/php"
composer_command << " #{release_path}/composer.phar"
composer_command << " --no-dev"
composer_command << " --prefer-source"
composer_command << " --optimize-autoloader"
composer_command << " install"
run "cd #{release_path} && #{composer_command}"
Step by step:
- copy the current release's vendor to the new release (if it exists)
- chown all files to the webserver (if the new vendor exists)
This allows the deploy hook to complete, even if we're on a fresh instance.
Benchmarks?
Effectively, this cut deployment from 4-5 minutes, to 2-3 minutes. With tailwind, a 50% improvement.
FIN
That's all. Happy deploying!