Ubuntu: setting up redis-server (reloaded)

Monday, November 28. 2011
Comments

I blogged about setting up redis before, here's a small update to the situation:

If you're running Ubuntu 9.10 or 10.04.x, the most recent release available through aptitude is 1.x. Version 1.x doesn't include many things (pubsub, etc.) which are available in 2.x. So this leaves you with two options!

Compile!

Compiling redis-server is simple (aka, make && sudo make install), a few things were missing (last time I checked):

Using a PPA!

The PPA (Personal Package Archive) concept is great. In a nutshell, it allows anyone to publish packages for various (Ubuntu) distributions. You upload a definition (sort of) and leverage Canonical's build farm, etc.. There are a few downsides to Launchpad: it's confusing, your PPA only holds the latest version of your build and overall publishing packages could be a lot easier. Which is why I'm so thankful for everyone who takes the time to publish.

Chris Lea seems to build most of the software I need. And he put all packages online in various Launchpad PPAs! Kudos to him!

Here's how you get setup!

Discover the Launchpad PPA:

server# sudo add-apt-repository ppa:chris-lea/redis-server
...

Update sources and install redis!

server# sudo apt-get update
...
server# sudo apt-get install redis-server
...

Done! ;-) All things (start script, configs) included!

Fin

In case that's too much type or you're looking for some automation, I also have a chef-recipe to get this done!

apt-repair-sources on Ubuntu

Wednesday, November 23. 2011
Comments

When I ran our setup on an instance the other day, I noticed how it failed with a "package not found" (or similar) error. After debugging this a bit, we discovered that Karmic moved from "archive.ubuntu.com" to "old-releases.ubuntu.com" (Probably diskspace or something — but who knows? :-)). And because the sources pointed to the former, it broke the bootstrap process on new and existing EC2 instances and Vagrant VMs for us. A truely consistent experience!

Whenever apt-get update is run in a chef-recipe and it exists with a non-zero status, the process is stopped. Of course there are ways to work around it (for example: ignore_failure true), but then again, most of these workarounds are hacks and not suitable for a production environment (IMHO, of course): we often discover new sources from launchpad PPAs and so on and it's paramount to want to know if discovery failed. You cannot assume that all went well

Scalarium fixed their AMI already and updated the sources to point to "old-releases". Running instances are of course still broken.

Enter apt-repair-sources

apt-repair-sources is a small (opinionated) tool written in Ruby.

It offers:

  • --dry-run (-d), which is the default
  • --fix-it-for-me (-f), which attempts to correct all problems

The reason why apt-repair-sources was written in Ruby is, that I wanted a tool to run with only the most basic setup (on Scalarium). Since Ruby comes installed by default, it was my weapon of choice (vs. Python or PHP). Another advantage was that I had an opportunity to check out more Ruby (aside from cooking with chef) and used this project to learn more anything about testing in Ruby (using Test::Unit).

Dry run

A dry run can be used to essentially debug the sources on a system.

Here's the output of a dry-run, and all is well:

[email protected]:~/apt-repair-sources/bin$ ./apt-repair-sources 
There are no errors in /etc/apt/sources.list
There are no errors in /etc/apt/sources.list.d/chris-lea-node.js-lucid.list
There are no errors in /etc/apt/sources.list.d/node.list
There are no errors in /etc/apt/sources.list.d/chris-lea-redis-server.list
There are no errors in /etc/apt/sources.list.d/silverline.list

Here's the output of a system, where sources are currently broken:

[email protected]:~/apt-repair-sources/bin$ ./apt-repair-sources 
/etc/apt/sources.list: http://us-east-1.ec2.archive.ubuntu.com/ubuntu/dists/karmic/main/binary-amd64/Packages.gz
/etc/apt/sources.list: http://us-east-1.ec2.archive.ubuntu.com/ubuntu/dists/karmic/main/source/Sources.gz
/etc/apt/sources.list: http://us-east-1.ec2.archive.ubuntu.com/ubuntu/dists/karmic-updates/main/binary-amd64/Packages.gz
/etc/apt/sources.list: http://us-east-1.ec2.archive.ubuntu.com/ubuntu/dists/karmic-updates/main/source/Sources.gz
/etc/apt/sources.list: http://security.ubuntu.com/ubuntu/dists/karmic-security/main/binary-amd64/Packages.gz
/etc/apt/sources.list: http://security.ubuntu.com/ubuntu/dists/karmic-security/main/source/Sources.gz
There are no errors in /etc/apt/sources.list.d/gearman-developers-ppa-karmic.list
/etc/apt/sources.list.d/karmic-multiverse.list: http://archive.ubuntu.com/ubuntu/dists/karmic/multiverse/binary-amd64/Packages.gz
/etc/apt/sources.list.d/karmic-multiverse.list: http://archive.ubuntu.com/ubuntu/dists/karmic/multiverse/source/Sources.gz
/etc/apt/sources.list.d/karmic-multiverse.list: http://archive.ubuntu.com/ubuntu/dists/karmic-updates/multiverse/binary-amd64/Packages.gz
/etc/apt/sources.list.d/karmic-multiverse.list: http://archive.ubuntu.com/ubuntu/dists/karmic-updates/multiverse/source/Sources.gz
/etc/apt/sources.list.d/karmic-multiverse.list: http://security.ubuntu.com/ubuntu/dists/karmic-security/multiverse/binary-amd64/Packages.gz
/etc/apt/sources.list.d/karmic-multiverse.list: http://security.ubuntu.com/ubuntu/dists/karmic-security/multiverse/source/Sources.gz

Problem?

Fix it for me

Fix it for me attempts to correct the sources like this:

  • sources with *.releases.ubuntu.com are moved to archive.ubuntu.com
  • sources with *.archive.ubuntu.com are moved to old-releases.ubuntu.com
  • sources with security.ubuntu.com are moved to old-releases.ubuntu.com

On top of these things, it will check Launchpad and third-party PPAs as well, if an issue is found, it'll just disable the entry in the sources file (by commenting it out: #).

Future releases will probably re-check commented out entries and also attempt to do some kind of sanity-checking of entries using the release name, etc.. These things are hard though and it might be the wrong approach to be opinionated here because e.g. Lucid packages sometimes also work on Karmic. Disabling these might break other things, etc..

Here's a run:

[email protected]:~/apt-repair-sources/bin$ sudo ./apt-repair-sources -f
[email protected]:~/apt-repair-sources/bin$ echo $?
0
[email protected]:~/apt-repair-sources/bin$ ./apt-repair-sources
There are no errors in /etc/apt/sources.list
There are no errors in /etc/apt/sources.list.d/gearman-developers-ppa-karmic.list
There are no errors in /etc/apt/sources.list.d/karmic-multiverse.list

Great success!

Automation

Both modes usually exit with zero (0), which makes it easy to include them for bootstrap processes, general trouble-shooting or periodic cronjobs etc..

Reason to not exit with 0:

  • attempt to run apt-repair-sources on another distro than Ubuntu
  • old-releases.ubuntu.com is down
  • you run with -d and -f (which of course makes no sense :-))
  • trollop (a rubygem i use for CLI option parsing is not found)

Setup

Gems!

# sudo gem install apt-repair-sources

Manually

  • install Ruby Enterprise Edition (steal Karmic here; this should be your default anyway)
  • sudo gem install trollop (don't use what is in apt)
  • clone my repo: git clone git://github.com/lagged/apt-repair-sources.git
  • cd ./apt-repair/sources/bin && ./apt-repair-sources

Todo

  • create a gem
  • add support for Debian
  • improve my Ruby

Fin

Sure hope it's useful for someone else out there.

The code is on github, and I take pull-requests: https://github.com/lagged/apt-repair-sources

Sound problems reloaded (Linux Ubuntu 10.04 and 10.10)

Thursday, August 11. 2011
Comments

So after updating 10.04 to 10.10 sound didn't work — again.

I've bitched and moaned about sound and Ubuntu countless times on Twitter and after literally wasting away countless hours on sound and Ubuntu in the recent months, here are some pointers so hopefully others won't waste as much time as I did.

Pulseaudio

There are countless of threads and blog posts online about how people despise pulseaudio and I have no idea if there's anything better, but I'm running Ubuntu because I don't care. And I don't want to care either.

So far I have tried OSS (Open Sound System) and esound. OSS was an especially painful experience for me and I tried it for about two weeks until I gave up, and esound just did not work at all. Pulseaudio seems to be the most stable of the three by far.

So when I discovered that sound did not work (again), I took the time to clean out all pulseaudio related packages using the Synaptic Package Manager.

I uninstalled everything pulseaudio-related before I continued.

Alsa

I haven't figured out this part yet. Not sure what it does — anyhow. Since I updated to 10.04 (the supposedly rock-solid and stable LTS) Alsa got constantly screwed up: the installed package linux-backports-modules-alsa-* often didn't match the kernel I had installed.

So before resetting alsa, I made sure the correct backport was installed.

I used the following to find my current kernel:

$ uname -r
2.6.35-30-generic

And then I installed linux-backports-modules-alsa-2.6.35-30-generic and nothing else.

Then I re-setted/-installed alsa with the following command:

$ sudo aptitude --purge reinstall linux-sound-base \
alsa-base alsa-utils linux-image-`uname -r` \
linux-ubuntu-modules-`uname -r` libasound2

Pulseaudio reloaded

Before re-installing it, I deleted .pulse* in my home directory:

$ cd
$ rm -rf .pulse*

Then I ran the following command to install pulseaudio (again):

sudo apt-get install pulseaudio

Fin

Last but not least — welcome to Windows, err... I had to reboot. And then it all worked.

I'm buying a mac within the next two weeks. This is just too much to bear.

vpnc (for Cisco VPNs with 2-factor authentication) on Ubuntu

Tuesday, July 19. 2011
Comments

In order to connect to firewall and DRACs of our new Rackspace cluster, I had to setup a VPN connection. Rackspace gave us a Cisco ASA device which is firewall and vpn in one. In order to connect to the VPN the client needs to do 2-factor authentication.

At first I tried to install the necessary add-ons for Ubuntu's network-manager, but after I spent an hour or so trying to make it work, I took an alternate route.

Requirements

Here's all you need to get started:

$ sudo aptitude install vpnc
...

Configuration

Create a configuration file — I called mine rackspace.conf:

$ gksudo gedit /etc/vpnc/rackspace.conf

I put in the following:

IPSec gateway X.X.X.X
IPSec ID GROUPNAME
IPSec secret GROUPPASSWORD
Xauth username USERNAME
Xauth password PASSWORD

Obviously, the capitalized values need to be replaced:

  • IPSec gateway: the VPN's gateway/address
  • IPSec ID: the group's name
  • IPSec secret: the group's password
  • Xauth username: the username
  • Xauth password: the password associated with the username

If you're with Rackspace, they'll tell you those exact things when the firewall is provisioned. I gather other ISPs call them similar.

Using your VPN

Once the file is saved, just start the vpn connection:

$ sudo vpnc rackspace
VPNC started in background (pid: 2691)...

Note: Just rackspace is plenty, not rackspace.conf or similar.

Testing:

$ ping 172.30.0.144
PING 172.30.0.144 (172.30.0.144) 56(84) bytes of data.
64 bytes from 172.30.0.144: icmp_seq=1 ttl=64 time=114 ms
64 bytes from 172.30.0.144: icmp_seq=2 ttl=64 time=113 ms
64 bytes from 172.30.0.144: icmp_seq=3 ttl=64 time=113 ms
^C
--- 172.30.0.144 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 113.766/114.199/114.866/0.552 ms

To disconnect it:

$ sudo vpnc-disconnect 
Terminating vpnc daemon (pid: 2691)

Fin

That's all.

VirtualBox Guest Additions and vagrant

Wednesday, May 25. 2011
Comments

If you followed my blog, you probably know about chef and vagrant.

So the other day I managed to upgrade to VirtualBox 4.0. The upgrade just happened by accident, so to speak. I noticed that Virtualbox 4.0 had moved from nonfree to contrib on Oracle's repository which is why I had previously missed it. With 4.0, I am now able to run the latest and greatest Vagrant — and with Vagrant being pre-1.0 it's safe to assume that newer is indeed better.

Anyway — the VirtualBox update fubar'd all of my boxes. They didn't outright refuse to start, but provisioning failed nine out of ten times and other random issues seemed to pile up.

I noticed this warning about the Guest Additions inside the box and how they are different from the VirtualBox installed on the host:

[default] The guest additions on this VM do not match the install version of
VirtualBox! This may cause things such as forwarded ports, shared
folders, and more to not work properly. If any of those things fail on
this machine, please update the guest additions and repackage the
box.

Guest Additions Version: 3.y.z
VirtualBox Version: 4.0.8

To cut to the chase — once I updated them and the box itself, they all continued to work.

So here's how!

Boot a blank VM

So first off, you take the image of your choice and boot a blank virtual machine. Blank because no chef-recipes should be configured, etc.. This VM should literally boot in a minute or so.

Now enter the VM and update the Guest Additions, and while we're at it, feel free to update the OS etc. as well.

Update the base

On my Ubuntu Karmic box, this is what I did (inside the VM) first:

vagrantup:~$ sudo aptitude update
...
vagrantup:~$ sudo aptitude upgrade
...
vagrantup:~$ sudo gem update --no-rdoc --no-test --no-ri
...
vagrantup:~$ sudo aptitude install portmap nfs-common
...

Pro-tip: The last step is necessary in case you run into the various issues with VirtualBox shared folders. The official website lists performance as the #1 one reason to use NFS shares instead of the VirtualBox shares, my #1 reason was issue #351.

Update Guest Additions

It took me a lot of Google, to actually find the download — regardless this is what seems to work. I extracted these steps from the blog post I found. For this to work, you are still inside the VM:

vagrantup:~$ wget -c \
http://download.virtualbox.org/virtualbox/4.0.8/VBoxGuestAdditions_4.0.8.iso \
-O VBoxGuestAdditions_4.0.8.iso
...
vagrantup:~$ sudo mount VBoxGuestAdditions_4.0.8.iso -o loop /mnt
vagrantup:~$ sudo sh /mnt/VBoxLinuxAdditions.run --nox11
...
vagrantup:~$ rm *.iso

There's an error about something failing to install (Windows-related), but I ignored it since I don't run the GUI anyway.

Pro-tip: Installing the latest vagrant along with Virtualbox 4.0.8 made the GUI mode fail consistently. Not that I really mind, but I feel compelled to share.

Repackage

Once these steps are completed, exit the VM (ctrl+d), halt the VM and up to test. If the VM starts up and no warnings show up, you got yourself a winner.

To repackage the VM, do the following:

~/karmic-test$ vagrant package

This creates a package.box file — I renamed mine to karmic-nfs.box. Import the box with:

~/karmic-test$ vagrant box add karmic-nfs karmic-nfs.box 
[vagrant] Downloading with Vagrant::Downloaders::File...
[vagrant] Copying box to temporary location...
[vagrant] Extracting box...
[vagrant] Verifying box...
[vagrant] Cleaning up downloaded box...

Done!

Fin

That's all, in summary the biggest obstacles are as follows:

  1. VirtualBox on the host and VirtualBox Guest Additions inside the box need to match (closely).
  2. The box needs to be kept up to date — more or less recent software inside the box makes bootstrapping and provisioning easier and faster.