Great success!

Monday, May 11. 2009
Comments

Even though I didn't like the Borat movie very much, I still like to use his line, Great success!. But on to more important things! ;-)

Last weekend, we — the Berlin PHP UserGroup (and Christian Weiske from Leipzig) — took part in what some people labeled the European PHP TestFest 2009. In a nutshell, we were writing tests to cover different PHP extensions.

Here's the outcome (numbers courtesy of Falko Menge):

Extension  Code Coverage    Line Coverage            New Tests
gettext    62.8 -> 98.8 %    54 ->  85 /  86 lines   19
intl/idn   11.6 -> 81.4 %     5 ->  35 /  43 lines    3
sockets    32.9 -> 70.5 %   282 -> 604 / 857 lines   23
posix      47.9 -> 70.0 %   147 -> 215 / 307 lines   16
xsl        67.3 -> 77.1 %   350 -> 401 / 520 lines   19

Our tests are available on a github repository and synced to the offical TestFest subversion repository.

I think we had a lot of fun, even recruited a couple newbies :-) to the whole testing thing. I want to extend my gratitude once more to everyone (Tim, Moritz, Robin, Falko, Christian, Jan and Florian) who showed up and helped us to improve PHP.

Great effort!

MySQL: Using indices correctly

Tuesday, May 5. 2009
Comments

The objective was to select sessions from a table, that are older than two days.

Table setup

This is the definition:

CREATE TABLE `session` (
  `id` varchar(32) NOT NULL DEFAULT '',
  `data` text NOT NULL,
  `user` int(11) DEFAULT NULL,
  `created` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  `updated` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  PRIMARY KEY (`id`),
  KEY `user_id` (`user`),
  KEY `rec_datemod` (`updated`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Right and wrong

Wrong query:

SELECT * FROM session
WHERE DATE_ADD(`updated`, INTERVAL 2 DAY)< NOW()

Correct query:

SELECT * FROM session
WHERE `updated` < DATE_SUB(NOW(), INTERVAL 2 DAY)

You wonder why?

Executing the first query, MySQL will scan the entire table and calculate the date from each row. Then it will continue and compare the value to NOW() and return the row if it matches. This is somewhat (Not really!) OK until a certain amount of traffic on the table. In my case, I have 500,000 (five-hundred-thousand) active sessions (aka rows) in the table, which makes it slower and slower and slower.

Because of the full table scan, this will also effectively lock the table (even though it's INNODB) and block it from further updates.

The second query (obviously) works around that and uses the KEY on updated.

Conclusion

The first lesson is to always use EXPLAIN!

Further more, I know some of you will shiver but phpMyAdmin is actually a pretty useful tool for these circumstances. The website stalled, you log into phpMyAdmin and figure out what's running ("Processes" tab, when you're logged in as a privileged account). If you're a shell-ninja, just execute SHOW PROCESSLIST (in mysql) and push whatever runs the longest to EXPLAIN.

The slow query-log is also something you should read up on.

Avoiding common pitfalls with Zend_Test

Saturday, March 28. 2009
Comments

Sometimes I think I'm particularly stupid when it comes to learning new things. Well, that, or maybe I'm just the only one complaining enough. ;-)

I feel like I've wasted a great deal of time last week with basics that were nowhere to be found or required debugging to no end. The following is the outcome, a rather random list of things to watch out for when you're starting on Zend_Test.

A general understanding of testing and PHPUnit is more than helpful.

How do you debug?

Did your test not redirect, or did your query assertions go wrong?

Generally, there are two ways:

<?php
// (...)
function testIfTheSunCameUp()
{
    var_dump($this->response, $this->request);
}

Not so pretty, right?

In general I don't understand why it will say, "failed asserting status code 404", and will not tell you what it got instead. It offers you a backtrace so you can go into the class and add var_dump(), but that's hardly useful.

One of the things I love about PHPT is that a failed test case is so descriptive. There's really no debugging a failed test because the process of debugging generally involves gathering information where PHPT hands it over to you right away. But maybe that's a PHPUnit thing.

And of course:

phpunit --verbose AllTests.php

My grudge here is that I'm a total PHPUnit newbie. Before Zend_Test I avoided PHPUnit where I could because I felt that it is a beast — one that's especially hard to tame.

Update: Someone else noticed too, please vote on ZF-6013.

Control your environment

We offer a REST-API to partners and that API is basically MVC and lots of context switching depending on GET, POST, PUT, HEAD, DELETE and a couple parameters. I like to think of it as true REST. ;-)

Since the unit tests are run on the cli, not even GET is set. So make sure to add this to your test:

<?php
// (...)
$this->request->setMethod('GET');

Keep in mind that $_SERVER is not set and if you really happen to rely on anything from that ($this->getServer()), you need to explicitly set it or figure out another smart way to deal with it. In my case, I'm overriding some of these $_SERVER variables in my bootstrap with a config setting when I'm using the "testing" environment.

Update: I've opend ZF-6162, please vote.

The basic AllTests.php

AllTests.php is used to tell phpunit what to do, a simple setup would look like this:

tests/AllTests.php

tests/controllers/AllTests.php

tests/controllers/IndexControllerTest.php

tests/controllers/AllTests.php:

<?php
require_once dirname(__FILE__) . '/IndexControllerTest.php';

class ControllersAllTests
{
    public static function main()
    {
        PHPUnit_TextUI_TestRunner::run(self::suite());
    }

    public static function suite()
    {
        $suite = new PHPUnit_Framework_TestSuite('Controllers');
        $suite->addSuiteCase('IndexControllerTestCase');
        return $suite;
    }
}

tests/AllTests.php:

<?php
(...)
class AllTests
{
    public static function suite()
    {
        $suite = new PHPUnit_Framework_TestSuite('AllTests');
        $suite->addSuite(ControllersAllTests::suite());
        return $suite;
    }
}

Zend_Session_Namespace

Avoid locking session namespaces, they will stab you in the eye. See ZF-6072.

Zend_Dom_Query

I'd suggest you remove all @-operators from Zend_Dom_Query since it doesn't really tell you what is wrong with the response otherwise. I've opened ZF-6142 so this gets fixed.

Slightly related to Zend_Dom_Query is a bug I noticed when you're response is empty. It's a rather trivial issue where null is not the same as an empty string. I've opened ZF-6143 and included a patch.

Bootstrapping the correct way

The most simple way to bootstrap is to create a small class to initialize your application. The class takes an environment parameter to __construct() which allows you to load a different config section or other nifty stuff. I called mine Lagged_App() and the code came originally from Andries Seuten's example ZF application but it evolved over the past years.

I wouldn't recommend his tutorial today (because things changed since 2007), but at the time it was pretty easy and straight forward. Today's bar is the quickstart section in the official manual. But since the quickstart suggests includes, I spent some time cleaning up my by going through said quickstart guide and I posted a quick and dirty example (which I labeled Lagged_Application)) in my subversion repository on Google Code.

A test case is a test case

I'm using the following "base" test case for all my controller tests:

<?php
class Lagged_PHPUnit_ControllerTestCase
    extends Zend_Test_PHPUnit_ControllerTestCase
{
    public function setUp()
    {
        Zend_Session::$_unitTestEnabled = true;

        $bootstrap       = new Bootstrap('testing');
        $this->bootstrap = array($bootstrap, 'start');
        parent::setUp();
    }
}

Clean up

One of the most fundamental things your mother tried to teach you.

This is in most of my tearDown()'s:

public function tearDown()
{
    $this->resetRequest()->resetResponse();
    $this->setQuery(array());
    $this->setPost(array());
}

Avoid direct PHP calls

I'd avoid the following:

  1. Skipping views with echo in the controller (bad, bad, bad ;-)).
  2. Use the Zend_Action_Helper redirector instead of header().
  3. Use Zend_Session etc. vs. session_*() (Don't forget Zend_Session::$_unitTestEnabled = true;).
  4. Don't dispatch always. ;-)

We are TestFest'ing!

Wednesday, March 25. 2009
Comments

English: Berlin's PHP usergroup is taking part in PHP's TestFest 2009, on the 9th and 10th May, 2009! The location will be Boxhagener Str. 119, Berlin (Friedrichshain), if you want to attend, please RSVP on our wiki!

Starting it off, I'll give an intro to PHPT-style testing at the monthly meeting of the usergroup in May!

Deutsch: Die PHP Usergroup Berlin nimmt am PHP TestFest 2009 teil. Wir treffen uns dazu am 9. und 10. Mai in Berlin Friedrichshain. Wenn Ihr kommen wollt, tragt Euch bitte in unserem Wiki ein! Wenn jmd. einen Schlafplatz braucht, bitte vorher melden. :-)

Eine Einführung zu PHPT werde ich beim monatlichen Treffen der Usergroup im Mai geben — mehr dazu später!

Managing software deployments of your PHP applications I

Saturday, January 31. 2009
Comments

Disclaimer: I've been doing mostly PHP and Zend Framework based projects in the past two years, but the information from this article is general and should be applicable to most setups — even to non PHP-based projects (to a certain extent).

Inspired by Padraic's posting spree the other week, here's another attempt to provide you with some hands-on usefulness. I'm all open for all feedback, and sorry for the length!

What is deployment and how do you manage deployments?

Wikipedia says:

Software deployment is all of the activities that make a software system available for use.

... and goes on:

The general deployment process consists of several interrelated activities with possible transitions between them. These activities can occur at the producer site or at the consumer site or both. Because every software system is unique, the precise processes or procedures within each activity can hardly be defined.

In general, there are different approaches to software deployment. Most people are probably not aware of a deployment process at all. They edit files and push it live. In most cases, live (sometimes referred to as production environment) is the webhosting account — for consistency, the environments setup for larger projects also includes a development and staging environment.

Taking the facts into account, we can summarize those efforts into three cases:

  1. Deploym..? I'm a skilled surgeon and shell ninja! I like to edit all my files online!
  2. I have a local WAMP, MAMP or LAMP and then FTP the files online.
  3. We have a defined process and use SVN, PEAR, phing or similar.

Why should I manage deployments?

There are a few reasons as to why it's good for youTM to come up with a release schedule to manage your deployments.

  1. Establishing a release schedule allows you to project the time necessary to implement and test new features and items.
  2. Planning in advance also helps to meet the plan.
  3. A schedule enables code testing (and other QA measures) before it's live. For example, assuming the schedule says to release a new version every two weeks, the net time (ten business days) could be divided up into eight business days for development, and two business days for testing. (Adjust as needed!) Take note — the schedule excludes weekends!
  4. A release schedule helps the development team to avoid all those extra last-minute changes which break things and cause grey hair without feeling bad or guilty. The established practice and process has to be used by all people involved, and developers are not to blame if someone else forgets that.

Continue reading "Managing software deployments of your PHP applications I"