Disclaimer: I've been doing mostly PHP and Zend Framework based projects in the past two years, but the information from this article is general and should be applicable to most setups — even to non PHP-based projects (to a certain extent).
Inspired by Padraic's posting spree the other week, here's another attempt to provide you with some hands-on usefulness. I'm all open for all feedback, and sorry for the length!
What is deployment and how do you manage deployments?
Software deployment is all of the activities that make a software system available for use.
... and goes on:
The general deployment process consists of several interrelated activities with possible transitions between them. These activities can occur at the producer site or at the consumer site or both. Because every software system is unique, the precise processes or procedures within each activity can hardly be defined.
In general, there are different approaches to software deployment. Most people are probably not aware of a deployment process at all. They edit files and push it live. In most cases, live (sometimes referred to as production environment) is the webhosting account — for consistency, the environments setup for larger projects also includes a development and staging environment.
Taking the facts into account, we can summarize those efforts into three cases:
- Deploym..? I'm a skilled surgeon and shell ninja! I like to edit all my files online!
- I have a local WAMP, MAMP or LAMP and then FTP the files online.
- We have a defined process and use SVN, PEAR, phing or similar.
Why should I manage deployments?
There are a few reasons as to why it's good for youTM to come up with a release schedule to manage your deployments.
- Establishing a release schedule allows you to project the time necessary to implement and test new features and items.
- Planning in advance also helps to meet the plan.
- A schedule enables code testing (and other QA measures) before it's live. For example, assuming the schedule says to release a new version every two weeks, the net time (ten business days) could be divided up into eight business days for development, and two business days for testing. (Adjust as needed!) Take note — the schedule excludes weekends!
- A release schedule helps the development team to avoid all those extra last-minute changes which break things and cause grey hair without feeling bad or guilty. The established practice and process has to be used by all people involved, and developers are not to blame if someone else forgets that.
How do we achieve all of the above?
- In my humble opinion, the most important step is to use a system to keep track of all issues. When shopping for such a system, things to keep in mind are that not everyone involved is uber-hacker.
- Choosing a the right system includes compatibility with non-tech people because that is of greater advantage if they need to comply.
- The second thing to consider is to try to find a system that works for all teams/departments in your company so people don't have to use different tools for different problems. While management is an important part of day to day, we also like to get things done!
- If you can't get a system for the entire company, get one for your developers and while compatibility with non-tech personal is important, don't forget your development team, because they are likely to be using it primarily.
- Use to the system — always. No issue is too small for your system.
- Establish the rule that if it's not scheduled for a release, it's not getting done!
The choice couldn't be any wider, the first step is to decide on hosted (ASP, SAAS) or in-house (on your own server/webhosting account) and while hosted vs. in-house could be a blog post itself, the only thing to keep into account is that a system running on your own server is a critical piece of your day to day work.
If the resources to do maintainance are available, then it's fine to go for a free solution or two buy the often less expensive hosted version of a product. If the resources are not available, you might consider renting a system at a monthly expense.
In no particular order:
On a sidenote, I'm a huge fan of Fogbugz, and while it certainly is expensive, it's one of those system that everyone gets. It's not too web2-shiny and ajaxy to confuse people and so far everyone who worked with it required next to zero training in order to use it.
All people-issues aside, there will always be some effort required when you implement a new business process. Keep in mind to not just look for pretty-ware. Also think of features. For example one of the things that come to mind from a developer perspective are an integration with your SCM. An requirement which your sales department would probably never come up with.
Hands-on, so how do you deploy?
Because we can In different scenarios, we employ different strategies to deploy. One is the deployment using the PEAR installer, and the other is straight from Subversion.
A third solution which we employ (on Linux/Unix) is to use the package manager of your distribution. But due to the complexity and rather overwhelming choices (rpm/spm, deb, ports, ebuilds, ...) and their strict ties to a certain system (type of Linux/Unix distribution), I will skip on them in this blog post and focus on operating system independent strategies.
The PEAR installer is quiete a beast really. We use it to install various CLI tools (PHP-based and also bash, sh, etc. scripts). All that needs to be done is to create a
package.xml for each tool, create a release and distribute it using a private PEAR channel (pear.mycompany.com) to our servers.
The idea is the following:
pear install myChannel/My_Tool
pear upgrade myChannel/My_Tool
- Easy versioning.
- Easy install/update/delete (management) of code.
- Easy dependency tracking.
- The initial complexity of creating a package.xml file.
- More complicated hotfixing.
- A working installation of PEAR. :-)
Said complexity would be too much for this blog post. If requested, I'll do another post on
package.xml-creation next time.
Another issue is that a small code change technically requires you to re-generate the package (bump up the version), publish the package and run update on the server. I love PEAR and I know that the process can be semi-automated, but even if it is the so called high road, it's far more complex than the way you can do a hotfix using Subversion (see below).
Feel free to email or leave a comment in this case.
If you worked with a SCM, you know what tags are. Subversion's tagging couldn't be any easier, in order to create a tag for a version.
This is a standard directory layout in Subversion:
trunk/ tags/ branches/
This is how I create a new release:
cd /path/to/repo/ \ && svn cp ./trunk/ ./tags/2.0 \ && svn ci -m "* new release 2.0" ./tags/
In my case in order to finalize a release, I have to make a couple adjustments (load a different environment), which are hard-coded in my bootstrap and that's another change I'll commit, and that's basically it.
In order to deploy, I have a Capistrano task which executes the following:
sudo svn checkout --username=deploy \ http://svn.domain.com/repo/tags/2.0 \ /usr/www/domain.com/2.0
- Easy to implement.
- Allows hotfixes.
- Hotfixes. ;-)
Hotfixes are both an advantage, and a disadvantage. The advantage in this case is that all you need to do is an
svn update in the designated folder, to pull the change. The disadvantage is that this kind of thing makes your code sloppy. It's a phsycological thing when you know that you can fix stuff easy.
One should take into account that hotfixes need to be applied to trunk as well and the transition can get messy. Sometimes a hotfix, is really a new minor version.
Also, hotfixes should not be abused for new features.
Deploying the web application
So in the end — no matter if we deployed using PEAR or SVN — my files are located in a directory tree that looks like the following:
/usr/local/www/domain.com/1.0 /usr/local/www/domain.com/1.1 /usr/local/www/domain.com/1.2 /usr/local/www/domain.com/2.0 ...
2.0 designate directories for a specific version of the web application. To serve those pages using Apache, a
<VirtualHost /> would look like:
<VirtualHost 192.168.108.1:8080> ServerName domain.com ServerAlias www.domain.com ServerAdmin email@example.com DocumentRoot /usr/local/www/domain.com/www/public </VirtualHost>
For my Zend Framework-based web application, I pretty much follow the recommended directory layout:
app/ var/ library/ tests/ public/
So following my example from above, my
However, if you noticed — the
DocumentRoot in my example
<VirtualHost /> does not include one of the versions from above. The reason is that in order to avoid the added complexity of a shell script to alter the DocumentRoot (and to re-write the Apache configuration), we decided to symlink the www directory instead. A roll back or update to another release is as easy as overwriting the symlink.
ln -sf /usr/local/www/domain.com/2.0 /usr/local/www/domain.com/www \ && apachectl restart
(The Apache restart is necessary on our setup, in order to make APC work.)
I don't know what the performance implications of symlinks are — feel free to comment — but I know of some pretty heavy sites (for example the German Facebook, StudiVZ) which use the same technique.
Well, I feel much better with a master plan. Since most of us rarely ever find ourselves in a situation where a release engineer is employed to take care of these steps, we all need the knowledge to make things work.
Things I did not touch are general QA, for example running tests, enforcing a coding standard, documentation and so on — easily to be done by a continious integration server such as CruiseControl. Those items are a bit outside the scope of this article, which in turn does not suggest that QA is not a part of the process. But more details, next time!
Chuck gave me a couple ideas for a series, and I'll see if I can continue.
As always — any question, comments or suggestions, free to email or use the comments.