Till's blog

One of my older hosting servers runs into space issues constantly. And most of the time it's a 24 GB log file (error_log) from a really old Joomla-based website.

Why it gets so huge? Well, when I tail the logfile, I see messages about functions being deprecated in PHP, warnings, notices and a whole lot more literally racing by. I'm not sure if Joomla changed in recent years, but most of the code-base is a great example of how PHP applications should not be build. Part of the problem in this particular case is that the system is hacked together and no one wants to update it. I'd argue another short-coming of the system since extensions actually require you to edit files and upgrade paths are not very clear.

Since it's a friend who's running the website, I haven't been very pressing on him to move to something more updated.

No space left on device

The usual quick fix is:

rm error_log && touch error_log && chown www:www error_log

Nice, and easy.

The only problem is that the system will usually take a while until recognizes that there is free space indeed.

Soft-updates?

FreeBSD recommends soft-updates — they have a lot of advantages. The only disadvantage is that e.g. the inodes will claim the address space until the write is completed. In case of a log file, this is usually when the process writing it is shut down.

Here's a system with soft-updates enabled:

server# mount
/dev/ad8s1a on / (ufs, local)
devfs on /dev (devfs, local)
/dev/ad8s1e on /tmp (ufs, local, soft-updates)
/dev/ad8s1g on /usr (ufs, local, soft-updates)
/dev/ad8s1d on /var (ufs, local, soft-updates)
/dev/ad8s1f on /web (ufs, local, soft-updates)
procfs on /proc (procfs, local)

The fix to making the system recognize the space is simple:

1. Restart your webserver, e.g.: /usr/local/etc/apache restart
2. Or run sync.

Fin

I hope I never ever forget this. Ever.

I noticed that archivers/pecl-phar vanished from the ports tree on one of my FreeBSD servers.

Problem?

Reasons to remove the port were:

• the port is unmaintained
• the port was based on the outdated phar extension from pecl
• phar (in pecl) contains open security issues

The simple solution is to create a new port which of course will use the phar which is bundled in PHP's core. And I will get to that (but feel free to beat me at it ;-)).

Solution

In the meantime, here's a simple solution to get phar on FreeBSD.

At first, make sure you're running PHP 5.3.8 (installed from lang/php5). Then fetch the PHP source and compile the phar extension only.

foo# fetch http://de2.php.net/get/php-5.3.8.tar.gz/from/de.php.net/mirror
foo# tar zxvf php-5.3.8.tar.gz
...
foo# cd php-5.3.8/
foo# ./configure --disable-all --enable-phar=shared
...
foo# make
...

If all goes well the modules/ directory contains a phar.so file (among other phar-things).

The remaining steps are to copy phar.so to your extension_dir and load it in extension.ini:

foo# cp modules/phar.so /usr/local/lib/php/20090626/

Feel free to double-check the location of directory:

foo# php -r 'echo ini_get("extension_dir");'

Finally, make sure PHP loads far by adding the following to /usr/local/etc/php/extensions.ini:

extension=phar.so

Fin

That's all. Once you restart php, or your webserver, phar should show up in phpinfo().

And as soon as a new php5-phar port is added (probably in archivers), you should make sure to delete phar.so and also remove the line added to the .ini.

So, every once in a while I find myself in need of trying out newer, maybe, not-yet-released features in PHP. For example, recently, I wanted to test RoundCube PHP6 — this is how I did it.

On a side note, the same setup would also work for testing code with previous versions of PHP.

Toolbox

I used nginx and the PHP source with a little bit of ./configure and make — for kicks!

My O.S. of choice is FreeBSD and therefor the installation steps covered are tailored to it. With a small amount of Linux/Unix-fu, anyone should make it work on another distribution regardless.

Install nginx

First off, install nginx. On FreeBSD, this should be all:

• cd /usr/ports/www/nginx-devel && make install distclean

On other systems, this maybe a:

• apt-get install nginx
• emerg nginx
• rpm -shoot-myself nginx

The next step includes the infamous spawn-fcgi which many people use to control the php-cgi processes. A lot of tutorials on the net suggest to install lighttpd because it's a part of it, but on FreeBSD, you may skip that and do the following instead:

• cd /usr/ports/www/spawn-cgi && make install distclean

Pretty cool, huh?

So once this is done, the usual tasks need to be completed — in no particular order:

• edit the nginx config and enable fastcgi in nginx (/usr/local/etc/nginx/nginx.conf)
• enable nginx itself in /etc/rc.conf (nginx_enable="YES")
• get another nifty start script (see Shell Script) to wrap spawn-cgi

... and done!

I'm a big fan of capistrano for my deployment and generally also prefer the command line (vs. nifty web interfaces).

Recently, I realized that it would be better to give up some responsibility to others on a project and it seems the easiest in terms of capistrano is webistrano — by the Berlin-based Peritor.

Gimme a port!

Because FreeBSD is and always will be my favorite operating system, I created a small port for webistrano on FreeBSD. The idea of a port is to have a command to install and remove software, generally, to keep your software installations manageable.

The 1.0 of my webistrano port is already on Github.

Usage

All required steps are listed in the README on Github, but in case you don't have ports on your FreeBSD, please follow these steps (assuming a more or less recent install, FreeBSD 6+):

• Get a port tree and install it to /usr/ports: portsnap fetch extract
• Install GIT: cd /usr/ports/devel/git && make install distclean
• Get the webistrano port:
  • cd (change to your home directory)
  • git clone git://github.com/till/webistrano-freebsd.git
  • cd webistrano-freebsd
  • ...

In case you want to skip on git, just download the webistrano-freebsd tar-ball from github:

• http://github.com/till/webistrano-freebsd/tarball/1.0

What's next?

I've got a small TODO (see README) and want to get that done ASAP, and send in a PR. I'd appreciate some feedback, if you got any.

UPDATE: #136108

Since part one and two were uber-successful, here's an update on my Zend Framework PHP performance situation. I've also had this post sitting around since beginning of May and I figured if I don't post it now, I never will.

Disclaimer: All numbers (aka pseudo benchmarks) were not taken on a full moon and are (of course) very relative to our server hardware (e.g. DELL 1950, 8 GB RAM) and environment. The application we run is Zend Framework-based and currently handles between 150,000 and 200,000 visitors per day.

Why switch at all?

In January of this year (2009), we started investigating the 2.2 branch of the Apache webserver. Because we used Apache 1.3 for forever, we never had the need to upgrade to Apache 2.0, or 2.2. After all, you're probably familiar with the don't fix it, if it's not broken-approach.

Late last year we ran into a couple (maybe) rather FreeBSD-specific issues with PHP and its opcode cache APC. I am by no means an expert on the entire situation, but from reading mailing lists and investigating on the server, this seemed to be expected behavior — in a nutshell: Apache 1.3 and a large opcode cache on a newer versions of FreeBSD (7) were bound to fail with larger amounts of traffic.

We tried bumping up a few settings (pv entries), but we just ran into the same issue again and again.

Because the architecture of Apache 2.2 and 1.3 is so different from one another (and upgrading to 2.2 was the proposed solution), I went on to explore this upgrade to Apache 2.2. And once I completed the switch to Apache 2.2, my issues went away.

So far, so good!

Performance?

On the performance side we experienced rather mediocre results.

While we benched that a static file could be read at around 300 requests per second (that is a pretty standard Apache 2.2 install, sans a couple unnecessary modules), PHP (mod_php) performed at a fraction of that, averaging between 20 and 23 requests per second.

Myth: Hardware is cheap(, developer time is not)!

Before some people yell at me for trying to optimize my web server, one needs to take the costs of scaling (to a 100 requests per seconds) into account.

One of those servers currently runs at 2,600.00 USD. The price tag adds up to an additional 10,400.00 USD in order to scale to a 100 (lousy) requests per seconds. Chances are of course, that the hardware is slightly less expensive since DELL gives great rebates — but the 8 GB of (server) RAM and the SAS disks by themselves are melting budgets away.

And on top of all hardware costs, you need add setup, maintenance and running costs (rack space, electricity) for an additional four servers — suddenly, developer time is cheap. ;-)

So what do we do? Nginx to the rescue?!