Till's blog

In my last blog entry, I demo'd how to get started with Vagrant and the ShellProvisioner.

To further illustrate how amazingly simple it is to get started on some Ruby, I'll convert the shell script from my last blog post to a little recipe for chef. Same objective, we install a PEAR package — but it could be anything really.

Follow me.

Shell

This is the shell script from before:

#!/bin/sh

apt-get update

apt-get install -y php5 php5-cli php-pear
hash -r

pear upgrade-all
pear install -f HTTP_Request2

Ruby

Create a cookbooks directory and create structure for your first cookbook in it:

$ mkdir -p my-cookbooks/first/recipes/

Create a default.rb file with the following content:

# my-cookbooks/first/recipes/default.rb
execute "apt-get update"

packages = ["php5", "php5-cli", "php-pear"]

packages.each|p| do
  package p
end

execute "pear upgrade-all"
execute "pear install -f HTTP_Request2"

The recipe is later referred to as first or first::default (name of the recipe directory, name of the .rb).

It's so simple it hurts. ;)

Step by step

1. I run apt-get update using Chef's execute resource.
2. I create an array of the packages (Arrays are ordered in Ruby, hashes are not (up until Ruby 1.9.x). Order is important here.)
3. I run pear upgrade-all using the execute resource.
4. I run pear install using the execute resource.

Vagrantfile

The Vagrantfile looks slightly different when you provision with chef-solo:

Vagrant::Config.run do |config|
  
  config.vm.define :web do |web_config|
    web_config.vm.box       = "lucid64"
    web_config.vm.host_name = "web"
  
    web_config.vm.provision :chef_solo do |chef|
      chef.cookbooks_path = "PATH-TO-YOUR-COOKBOOKS"
      chef.add_recipe "first"
      chef.log_level = :debug
    end
  end

end

The important bit: the path to the location of your cookbooks — could be an /absolute/path or ./../relative/path.

Because a Vagrantfile is essentially Ruby code, anything goes here.

Further reading

Getting started with Chef and Ruby can be intimidating or even frustrating at times. Google "chef cookbooks" and you know what I mean.

These links are what you need:

• Opscode's wiki with all available Chef resources
• A decent Ruby reference
• My blog ;)

Fin

Save, and enjoy — vagrant up.

This is something experimental I have been working on for our chef deployments. So the objective was/is to find a sane way to install PEAR packages and install dependencies with composer.

execute in chef recipes

In chef recipes, almost everything is a resource. In case you're just getting started with Chef, a list of current resources is available on the Opscode Wiki. It's a link I put in my browser bar since I frequently work on chef recipes.

Some examples for resources are:

• package (to install software)
• cron (setup a crontab)
• directory (create directories)
• template (install customized configuration files)
• user and group (to create users and groups)
• mdadm (to setup a RAID)

The above list are examples — so there is more. But if there isn't a designated resource, you can always use an execute block.

An example for an execute block is the following:

execute "discover a pear channel" do
  command "pear channel-discover easybib.github.com/pear"
end

This works pretty well, but it is also not very robust.

Fail hard

By default whenever a command fails, chef fails hard.

To illustrate what I'm talking about, let's test and execute the command from our execute block multiple times on the shell to see its exit status ($?):

till:~/ $ pear channel-discover easybib.github.com/pear
Adding Channel "easybib.github.com/pear" succeeded
Discovery of channel "easybib.github.com/pear" succeeded
till:~/ $ echo $?
0
till:~/ $ pear channel-discover easybib.github.com/pear
Channel "easybib.github.com/pear" is already initialized
till:~/ $ echo $?
1

So whenever a command returns not 0, chef will bail.

One solution is to brute-force your way through these things with ignore_failure true in your execute block. But that's usually not a great idea either because it hides other issues from you (and me) when we need to debug this later on.

For example, if this PEAR channel is unavailable during your next chef-run, it would be much, much harder to find the root cause as of why the install commands failed.

Another solution is using the not_if or only_if options with execute:

execute "discover a pear channel" do
  command "pear channel-discover easybib.github.com/pear"
  not_if do
    `pear channel-info easybib.github.com/pear`
  end
end

If the command wrapped in not_if succeeds (success is exit status), we would skip the execute block.

Optimize?

Since I discovered not_if and only_if, it allows me write recipes which work in most cases. More robust code, which allows me to re-execute recipes on already running instances. So for example when I update a recipe or configuration file which is distributed through a recipe I can re-run the entire recipe and it will not fail but instead complete successfully.

One problem remains with this approach I end up doing the same checks again and again.

Ever since I started playing around with Unix ~13 years ago, I've been a fan of automating things. What started out as writing little (maybe pointless) shell scripts slowly but surely morphed into infrastructure automation today.

As for my, or maybe anyone's, motivation to do these things, I see three main factors:

• I'm easily bored — because repeating things is dull.
• I'm easily distracted (when I'm bored).
• I'm German: Of course we strive for perfection and excellence. ;-)

Being on Unix (or Linux) it's fairly simple to automate things — add some script-fu to bash or csh (or even better zsh) and off you go wrapping things into a small shell script! Then execute again and again!

Before we decided to moved to AWS (and RightScale) in late 2009 we had half a rack of servers (in a Peer1's POP in NYC) and never did any or much infrastructure automation. We had an image and a set of commands to get a server up and running, but it was far from two mouse-clicks today.

At the time, I had read about cfengine a couple of times, but datacenter-grade infrastructure management along with a rather steep learning wall (at that time anyway) seemed overkill. Add to that, that there is not a lot of time for research Fridays when you work in a small company.

Moving to AWS and RightScale required us to write lots of small shell scripts using bash and Ruby. When we moved from RightScale to Scalarium in late 2010, we went from shell scripts to Chef.

Using Chef meant that we created so-called recipes which are used to bootstrap our servers. Recipes are little Ruby scripts which live in a cookbook — open source projects are sometimes so creative. Before this move I had very little, or next to no, experience with Chef, and Chef being Ruby didn't exactly make me want to try it either.

So what exactly is Chef?

A Chef recipe is a tiny bit of Ruby code — essentially a high(er)-level wrapper around calls such as installing certain packages, creating and parsing configuration files and many other things.

Chef offers a robust abstraction about everything you can do with shell and with a little effort it's also possible to write recipes which run on multiple OS'. Supported are Ubuntu, CentOS, FreeBSD and others. For an intro to Chef see the slides of a talk I gave a couple weeks ago; I briefly blogged about it too.

Our Chef recipes currently include things like installing and configuring PHP (from source and through a launchpad repository), nginx, MySQL, CouchDB, haproxy and many other things. The list was literally growing every day for the first few weeks.

In 2010, operations became an even more central part of my life. As I write this blog post (in early January, 2011), we have been running on Amazon AWS — and EC2 in particular — for over a year.

Previously we had used a service called RightScale but in Q3 of 2010, we moved on/away from RightScale and started using chef and a service called Scalarium.

Because Opscode's chef became such a big part of my work life, I gave a talk about chef, and chef-solo in particular, at last December's PHP Usergroup meeting in Berlin. My talk contains experience made and insights into the whole thing and demo'd a couple chef basics — enough to get started.

Here's a link to the slides: https://s3.amazonaws.com/chef-talk/chef-talk.html

Questions or general feedback — please use the comments (below).

Btw, the slides are using CSSS, a CSS-based slideshow system. I tried it for the first time and found working on my slides to be refreshingly simple compared to the things I had tried before. It's a pretty interesting and project.

P.S. Happy new year!