till's blog

I'm sure most of you noticed that with the recent upgrade of PHPUnit to version 3.6, a lot of breakage was introduced in various projects.

And for example Zend Framework 1.x won't update to the latest version either. When I ranted on twitter someone send me Christer Edvartsen's blog post on how to setup multiple versions of PHPUnit. It's really neat since it walks you through the setup step by step and you learn about things such as --installroot on the way. --installroot in particular is something I never ever saw before and I've been using PEAR for more than a few years now. So kudos to Christer for introducing myself to it.

The only thing to add from my side would be, Why are you guys not aggregated on planet-php?.

Cooking with Chef

Another reason why I decided to write this blog entry was that I created a chef-recipe based on Christer's blog entry.

If you follow my blog for a while, you might have noticed that I'm a huge fan of automation. I just moved one of our development servers the other day and had one of these moments where something just paid off. Taking for granted that I can spin up fully operational EC2 instances in minutes, I also had our development stack installed and configured in an instant.

My recipe basically follows Christer's instructions and because I distribute phpunit's command along with it, editing of the file is no longer required: when the chef run completes, phpunit34 is installed and ready to be used.

Get started

I'm doing the following commands as root — my setup is in /root/chef-setup.

Install chef(-solo) and clone my cookbooks

shell# gem install --no-ri --no-rdoc chef
... 
shell# git clone git://github.com/till/easybib-cookbooks.git
...

Chef configuration

Then setup a node.json file which chef-solo will need to run:

{
  "run_list": [
    "recipe[phpunit]"
  ]
}

Then create a solo.rb:

file_cache_path "/var/chef-solo"
cookbook_path ["/root/chef-setup/easybib-cookbooks"]

Chef run

Finally we start chef-solo with following command:

shell# chef-solo -c /root/chef-setup/solo.rb -j /root/chef-setup/node.json -l debug
...

The command runs chef-solo (which is part of the gem we installed) and reads the basic configuration from the solo.rb-file. This file contains the location of the cookbooks (remember git clone ...) and a path to cache files. You don't need to create anything, it should be all taken care of.

The node.json-part allows us to set node-specific values. The prime example is the run-list, but it allows you to set attributes as well. Attributes contain values for variables used in recipes, but are not used in this example.

Last but not least: -l debug — a lot of useful output, but we usually run with -l warn. And if this is interesting enough for you, I suggest the other blog entries I wrote on this topic.

Did it work?

Depending on the location of your pear setup — usually /usr/bin/pear or /usr/local/bin/pear — the phpunit34 script is created in the same path:

shell# which phpunit34
/usr/local/bin/phpunit34

Yay!

Fin

This feels like hitting two birds with one stone. Though just by figure of speech! I object to violence against birds.

It might be overkill to setup chef to just install phpunit 3.4 by itself, but I think this serves as a stellar example of how you can leverage the power of chef to get more done. Writing a couple more recipes to install and configure the rest of your stack shouldn't be too hard.

If you'd like to see anything in particular: I'll take requests via email, Twitter or in the comments.

Update, 2011-06-16, 12:15 AM Thanks for the comments.

(I swear I had something like that before and it didn't work!) Here's the solution:

$userId = 61382;
$docId  = 'CLD2_62e029fc-1dae-4f20-873e-69facb64a21a';
$body   = '{"error":"missing","reason":"problem?"}';

$client = new Zend_Http_Client;
$client->setAdapter(new Zend_Http_Client_Adapter_Test);

$couchdb = $this->getMock(
    'CouchDB',
    array('makeRequest',),
    array($userId, $this->config,)
);
$couchdb->expects($this->once())
    ->method('makeRequest')
    ->will($this->returnValue(new Zend_Http_Response(200, array(), $body)));
$couchdb->setHttpClient($client);
$couchdb->getDocument($docId);

--- Original blog entry ---

I wrote a couple tests for a small CouchDB access wrapper today. But when I wrote the implementation itself, I realized that my class setup depends on an actual CouchDB server being available and here my journey began.

Example code

Consider the following example:

My objective is not to be able to test any of the protected methods directly, but to be able to supply a fixture so we don't have to setup CouchDB to run our testsuite. My fixture would replace makeRequest() and return a JSON string instead.

Sometimes, I have to debug unit tests and usually this is a situation I'm trying to avoid.

If I have to spend too much time debugging a test it's usually a bad test. Which usually means that it's too complex. However, with Zend_Test_PHPUnit_ControllerTestCase, it's often not the actual test, but the framework. This is not just tedious for myself, it's also not the most supportive fact when I ask my developers to write tests.

An example

The unit test fails with something like:

Failed asserting last module used <"error"> was "default".

Translated, this means the following:

• The obvious: an error occurred.
• The error was caught by our ErrorController.
• Things I need to find out:
  • What error actually occurred?
  • Why did it occur?
  • Where did the error occur?

The last three questions are especially tricky and drive me nuts on a regular basis because a unit test should never withhold these things from you. After all, we use these tests to catch bugs to begin with. Why make it harder for the developer fix them?

In my example an error occurred, but debugging Zend_Test also kicks in when things supposedly go according to plan. Follow me to the real life example.

Real life example

I have an Api_IndexController where requests to my API are validated in its preDispatch().

Whenever a request is not validated, I will issue "HTTP/1.1 401 Unauthorized". For the sake of this example, this is exactly what happens.

class ApiController extends Zend_Controller_Action
{
    protected $authorized = false;

    public function preDispatch()
    {
        // authorize the request
        // ...
    }
    public function profileAction()
    {
        if ($this->authorized === false) {
            $this->getResponse()->setRawHeader('HTTP/1.1 401 Unauthorized');
        }
        // ...
    }
}

Here's the relevant test case:

class Api_IndexControllerTest ...

    public function testUnAuthorizedHeader()
    {
        $this->dispatch('/api/profile'); // unauthorized
        $this->assertResponseCode(401);
    }
}

The result:

1) Api_IndexControllerTest::testUnAuthorizedHeader
Failed asserting response code "401"

/project/library/Zend/Test/PHPUnit/Constraint/ResponseHeader.php:230
/project/library/Zend/Test/PHPUnit/ControllerTestCase.php:773
/project/tests/controllers/api/IndexControllerTest.php:58

Not very useful, eh?

Debugging

Before you step through your application with print, echo and an occasional var_dump, here's a much better way of see what went wrong.

I'm using a custom Listener for PHPUnit, which works sort of like an observer. This allows me to see where I made a mistake without hacking around in Zend_Test.

Here is how it works

Discover my PEAR channel:

sudo pear channel-discover till.pearfarm.org

Install:

till@till-laptop:~/ sudo pear install till.pearfarm.org/Lagged_Test_PHPUnit_ControllerTestCase_Listener-alpha
downloading Lagged_Test_PHPUnit_ControllerTestCase_Listener-0.1.0.tgz ...
Starting to download Lagged_Test_PHPUnit_ControllerTestCase_Listener-0.1.0.tgz (2,493 bytes)
....done: 2,493 bytes
install ok: channel://till.pearfarm.org/Lagged_Test_PHPUnit_ControllerTestCase_Listener-0.1.0

If you happen to not like PEAR (What's wrong with you? ;-)), the code is also on github.

Usage

This is my phpunit.xml:

<?xml version="1.0" encoding="utf-8"?>
<phpunit bootstrap="./TestInit.php" colors="true" syntaxCheck="true">
    <testsuites>
    ...
    </testsuites>
    <listeners>
        <listener class="Lagged_Test_PHPUnit_ControllerTestCase_Listener" file="Lagged/Test/PHPUnit/ControllerTestCase/Listener.php" />
    </listeners>
</phpunit>

Output

Whenever I run my test suite and a test fails, it will add something like this to the output of PHPUnit:

PHPUnit 3.4.15 by Sebastian Bergmann.

..Test 'testUnAuthorizedHeader' failed.
RESPONSE

Status Code: 200

Headers:

     Cache-Control - public, max-age=120 (replace: 1)
     Content-Type - application/json (replace: 1)
     X-Ohai - WADDAP (replace: false)

Body:

{"status":"error","msg":"Not authorized"}

F..

Time: 5 seconds, Memory: 20.50Mb

There was 1 failure:

1) Api_IndexControllerTest::testUnAuthorizedHeader
Failed asserting response code "401"

/project/library/Zend/Test/PHPUnit/Constraint/ResponseHeader.php:230
/project/library/Zend/Test/PHPUnit/ControllerTestCase.php:773
/project/tests/controllers/api/IndexControllerTest.php:58

FAILURES!
Tests: 5, Assertions: 12, Failures: 1.

Analyze

Analyzing the output, I realize that my status code was never set. Even though I used a setRawHeader() call to set it. Turns out setRawHeader() is not parsed so the status code in Zend_Controller_Response_Abstract is not updated.

IMHO this is also a bug and a limitation of the framework or Zend_Test.

The quickfix is to do the following in my action:

$this->getResponse()->setHttpResponseCode(401);

Fin

That's all. Quick, but not so dirty. If you noticed, I got away without hacking Zend_Test or PHPUnit.

The listener pattern provides us with very powerful methods to hook into our test suite. If you see the source code it also contains methods for skipped tests, errors, test suite start and end.

We're about to launch a new product, and this time it's pretty client-side-intense. The application is powered by a lot of JavaScript(-mvc) and jQuery, which do xhr calls to a ZF/CouchDB powered backend. While js-mvc has unit-testing sort of covetred, I was also looking for some integration testing, multiple browsers and all that.

Selenium vs. Saucelenium

I can't really say if you want one or the other. Revisiting Selenium in general, it's IMHO the only viable and suitable thing for a PHP shop. Primarily of course because all those nifty test cases will integrate into our existing suite of PHPUnit/phpt tests. And while I use Zend_Test already or find projects like SimpleTest's browser or even Perl's www::mechanize very appealing, neither of those executes JavaScript like a browser.

Selenium and Saucelenium have the same root — in fact Saucelenium is a Selenium fork. While the Selenium project seems to focus on 2.x currently, stable 1.x development seems to really happen at Saucelabs. That is if you call a commit from January 22nd of this year active development.

In the process of selecting one or the other, more people recommended that I'd use the Saucelabs distribution than the original one, and so I forked it on github.

Installation

Along with a script to start the damn thing, my fork also contains a README.md. Said README covers the installation part in detail, so I won't have to repeat myself here. All of this is pretty Ubuntu-centric and has been tested on Karmic Koala. I expect things to work just as well on Lucid, or on any other distribution if you get the installation right.

One thing that took me a while to figure out was the following error message:

till@testing:/usr/src/saucelenium$ sudo ./start.sh 
[...]
[config/dbus] couldn't take over org.x.config: org.freedesktop.DBus.Error.AccessDenied (Connection ":1.17" is not allowed to own the service "org.x.config.display0" due to security     policies in the configuration file)
(EE) config/hal: couldn't initialise context: unknown error (null)
13:17:23.166 INFO - Writing debug logs to /var/log/selenium.log
13:17:23.166 INFO - Java: Sun Microsystems Inc. 16.0-b13
13:17:23.166 INFO - OS: Linux 2.6.32.1-rscloud amd64
13:17:23.176 INFO - v1.0.1 [exported], with Core v@VERSION@ [@REVISION@]
13:17:23.296 INFO - Version Jetty/5.1.x
13:17:23.306 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
13:17:23.306 INFO - Started HttpContext[/selenium-server,/selenium-server]
13:17:23.316 INFO - Started HttpContext[/,/]
13:17:23.326 INFO - Started SocketListener on 0.0.0.0:4444
13:17:23.326 INFO - Started org.mortbay.jetty.Server@7526e85f
^C13:18:09.796 INFO - Shutting down...

After Google'ing for a bit, I came to the conclusion that the above means that I didn't have the xserver installed.

The fix was rather simple: aptitude install xserver-xorg.

Example test

The following is an example test case. It'll open http://www.php.net/ and make sure it finds "What is PHP?" somewhere on that page.

Then it will continue to /downloads.php (by clicking on that link) and will make sure it finds "Binaries for other systems" on that page.

To run this test, execute: phpunit ExampleTestCase.php.

class ExampleTestCase extends PHPUnit_Extensions_SeleniumTestCase
{
    protected function setUp()
    {
        $this->setHost('localhost');
        $this->setPort(4444);
        $this->setBrowser("*chrome");
        $this->setBrowserUrl("http://www.php.net/");
    }

    public function testPHP()
    {
        $this->open('/');
        $this->waitForPageToLoad('30000');
        $this->assertTextPresent('What is PHP?');

        $this->click('//a[@href="/downloads.php"]');
        $this->waitForPageToLoad('30000');
        $this->assertTextPresent('Binaries for other systems');

        // check this out, especially useful for debugging:
        $this->assertEquals('http://www.php.net/downloads.php', $this->drivers[0]->getLocation());
    }
}

That's all.

Thanks for reading, and until next time.

So, every once in a while I find myself in need of trying out newer, maybe, not-yet-released features in PHP. For example, recently, I wanted to test RoundCube PHP6 — this is how I did it.

On a side note, the same setup would also work for testing code with previous versions of PHP.

Toolbox

I used nginx and the PHP source with a little bit of ./configure and make — for kicks!

My O.S. of choice is FreeBSD and therefor the installation steps covered are tailored to it. With a small amount of Linux/Unix-fu, anyone should make it work on another distribution regardless.

Install nginx

First off, install nginx. On FreeBSD, this should be all:

• cd /usr/ports/www/nginx-devel && make install distclean

On other systems, this maybe a:

• apt-get install nginx
• emerg nginx
• rpm -shoot-myself nginx

The next step includes the infamous spawn-fcgi which many people use to control the php-cgi processes. A lot of tutorials on the net suggest to install lighttpd because it's a part of it, but on FreeBSD, you may skip that and do the following instead:

• cd /usr/ports/www/spawn-cgi && make install distclean

Pretty cool, huh?

So once this is done, the usual tasks need to be completed — in no particular order:

• edit the nginx config and enable fastcgi in nginx (/usr/local/etc/nginx/nginx.conf)
• enable nginx itself in /etc/rc.conf (nginx_enable="YES")
• get another nifty start script (see Shell Script) to wrap spawn-cgi

... and done!