till's blog

Playing with redis/Rediska and sessions, I wanted to get more numbers to compare this solution to a traditional MySQL-based approach which also made me revisit the idea of a CouchDB-based session handler for Zend_Session.

Implementing this handler, I ran into a weird issue:

Fatal error: Undefined class constant 'ALLOW_ALL' in /usr/home/till/foo/trunk/library/Zend/Uri/Http.php on line 447
Call Stack
#   Time    Memory  Function    Location
1   0.7357  3914816 Foo_Session_SaveHandler_Couchdb->write( )   ../Couchdb.php:0
2   0.7358  3916600 Foo_Couchdb->query( )   ../Couchdb.php:94
3   0.7361  3969464 Zend_Http_Client->__construct( )    ../Couchdb.php:368
4   0.7361  3969544 Zend_Http_Client->setUri( ) ../Client.php:250
5   0.7362  3976568 Zend_Uri::factory( )    ../Client.php:267
6   0.7365  4003352 Zend_Uri_Http->__construct( )   ../Uri.php:130
7   0.7367  4006216 Zend_Uri_Http->valid( ) ../Http.php:154
8   0.7368  4006216 Zend_Uri_Http->validateHost( )  ../Http.php:281

The funny thing is that that APC was added (for apc_store() and apc_fetch()) at the same time to the game (to cache the configuration) and when I disabled it, the error disappeared.

Talking to to one of the leads of APC — Gopal (Btw, cheers for helping!) — on IRC (#pecl@efnet) I thought at first that the issue was autoload related and we thought the order in which the extensions are loaded might make a difference. From Rasmus' comment, I later discovered bug #16745 with a proposed workaround to use session_write_close().

On a sidenote: I'm still not sure why the error is expected behavior for some people but yet it works with some PHP and APC versions and breaks with others. From what I gathered it broke for me with 5.2.6, 5.2.11 and 5.3.2. Tried all with the latest version of APC (3.1.3p1).

Here's how I fixed it for myself

I have a Lagged_Application class to bootstrap my application. Lagged_Application is kind of like Zend_Application sans a lot of safety nets and magic. Since it does a lot less, it's also quiet a bit faster. To get an idea, check out my Google Code repository (for an alas rather outdated version of it).

I added the following function to it:

<?php
// (...)
public function shutdown()
{
    session_write_close();
}

My index.php looks like the following:

<?php
include 'library/Lagged/Application.php';
$app = new Lagged_Application;
$app->setEnvironment('production');
$app->bootstrap();

register_shutdown_function(array($app, 'shutdown'));

Somewhat related — shutdown() could be a good start to tear down other objects as well, when needed.

More?

Now that this issue is fixed, I think also the infamous Fatal error: Exception thrown without a stack frame in Unknown on line 0 originates from the same issue. That is, when sessions and APC are around — but I should dig a little deeper to verify this.

All in all, it's a pretty weird issue and IM(very)HO, objects shouldn't be torn down or some sort of before hook should be executed to avoid this. But that's especially easy to say if you don't do C. :-)

Fin

That's all. I sure hope this saves someone else some time.

So I posted something on Twitter today, which wasn't exactly news to me. I was more or less pointing out the obvious.

From a couple follow-up tweets, I guess, I need to explain more.

The idea

My thesis is that there's a gain in page rendering time when I disable automatic view rendering and use explicit render calls ($this->render('foo');) inside my controllers. And to cut to the chase, there is. On our app, I measured a 12% improvement using Xdebug's profiler — simple before-after-style.

General setup

I've blogged about Zend Framework performance before (1, 2, 3). Our setup is not the average Zend Framework quickstart application. We utilize a custom (much faster) loader (my public open source work in progress), no Zend_Application and explicit (vs. implicit) view rendering. The framework code is at 1.10.2. On the server-side, the application servers are nginx+php(-cgi).

I don't feel like repeating myself and while a lot of issues were already addressed in new releases of the framework, or are going to be addressed in 2.0, the above links still hold a lot of truth or at least inside and pointers if you're interested in general PHP performance (in German).

Code

IMHO, it doesn't really matter how the rest of your application looks like. Of course all applications are different and that's why I didn't say, "OMG my page rendered in 100 ms", but instead I said something like, "we got a 10+% boost". The bottom line is that everyone wants to serve fast pages and get the most out of their hardware but since applications tend to carry different features there really is no holy grail or number to adhere to.

Proposal

I urge everyone to double-check my claim. After all, it's pretty simple:

1. Setup Xdebug
2. Profile the page
3. Restart PHP app server/processes (in case you use APC and/or php-cgi)
4. Disable automatic view rendering: $this->_helper->viewRenderer->setNoRender(true);
5. Add render() call: $this->render('foo');
6. Profile again

... simple as that.

Conclusion

All in all this thing doesn't require too much to follow.

Automatics — such as an automatic view renderer — add convenience which results in rapid development and hopefully shorter time to market. But they do so almost always (give it nine Erlang nines ;-)) at the expense of performance.

Update, 2010-03-20 21:37: As Rob pointed out, there's even more to gain by bypassing the helper entirely. Use the code snippet below, or consider something like the following:

Padraic also blogged extensively on Zend_Controller_Action_Helper_ViewRenderer, I recommend reading Having a bad ViewRenderer day in your ZF app?.

Update (2009-10-13): I posted part II!

This is the first part of a series. I'll start off by introducing CouchDB — from a PHP side, then I'll demo a couple basic use cases and I later on, I'll dive into migrations from MySQL.

My idea is to introduce CouchDB to a world where database-driven development generally refers to MySQL. By no means, this is meant to be disrespectful to MySQL, or SQL-databases in general. However, I'm a firm believer in using the right tool for the job.

First things first!

First off, before using CouchDB and maybe eventually replacing MySQL with it, we need to ask ourself the "Why?"-question.

And in order to be capable of more than a well-educated guess we need to familiarize ourselves with the CouchDB basics.

Basics

• Document-oriented and schema-less storage.
• Erlang (for rock-solid-scaling-goodness).
• RESTful HTTP-API (we'll get to that).
• Everything is JSON - request data, storage, response!

Document-oriented

In a document-oriented as to opposed to a relational store, the data is not stored in table, where data is usually broken down into fields. In a document-oriented store each record is stored along side and can have its own characteristics — properties of any kind.

As an example, consider these two records:

Till Klampaeckel, Berlin
Till Klampaeckel, till@php.net, Berlin, Germany

In a relational store, we would attempt to break down, or normalize, the data. Which means that we would probably create a table with the columns name, email, city and country.

Consider adding another record:

Till Klampaeckel, +49110, till@some.jabber

(Just fyi — this is not my real phone number!)

Looking for an intersection in the records, the name is the only thing this record has in common with the previous two. With a relational database, we would either have to add a column for phone number and chat, or we would start splitting off the data into multiple tables (e.g. a table called phone and one called chat) in order to get grip.

With a document-oriented database — such as CouchDB — this is not an issue.

We can store any data, constraints do not apply.

Erlang

Erlang was invented a while ago, by Ericsson, when it was still sans Sony. In a nutshell, Erlang's true strength is reliability and stability. It also manages to really utilize all the resources modern hardware has to offer since it's a master of parallelization.

CouchDB is written in Erlang, and also accepts view code written in Erlang. More on views later.

RESTful HTTP-API

For starters, a lot of HTTP-APIs claim to be RESTful, most of them are not. HTTP has so called request verbs (DELETE, GET, HEAD, POST, PUT among them) and a lot of APIs don't use them to the fullest extend, or rather not all.

Instead, most APIs are limited GET and maybe use a little POST. An example of such an API is the Flickr API.

Most of us are familiar with GET and POST already. For example, when you opened the web page to this blog entry, the browser made a GET-request. If you decide to post a comment later on — you guessed it, that's a POST-request.

Aside from its basic yet powerful nature, HTTP is interesting in particular because it is the least common multiple in many programming language. Whatever you use — C#, PHP, Python, Ruby — these languages know how to talk HTTP. And even better — most of them ship pretty comfortable wrappers.

JSON

JSON — it's godsend for those of us who never liked XML.

It's very lightweight, yet we able to represent lists and objects, integers, strings — most data types you would want to use. A clear disadvantage of JSON is that it lacks validation (think DTD), and of course comments — ha, ha!

Why, oh why?

So along with "Why?", we should consider the following:

• Does it make sense?
• Is CouchDB (really) the better fit for my application?
• What is my #1 problem in MySQL, and how does CouchDB solve it?

And if we are still convinced to migrate all of our data, we'll need to decide on an access wrapper.

It's all HTTP, right?

By now, everyone has heard that CouchDB has a RESTful HTTP-API. But what does that imply?

It means, that we won't need to build a new extension in PHP to be able to use it. There's already either ext/socket or ext/curl — often both — in 99% of all PHP installs out there. Which means that PHP is more than ready to talk to CouchDB — right out of the box.

Since I mentioned JSON before — today ext/json is available in most PHP installs as well. If however we happen to be one of the few unfortunates who don't have and cannot get this extension, we should use Services_JSON instead.

Install it!

CouchDB installations are available in most Linux and Unix distributions. On MacOSX, get CouchDBX — the one-click CouchDB package, and there's a work in process for Windows as well. Especially interesting for those who run Ubuntu 9.10 (which has been released a few days ago), there's already a CouchDB install included.

Ubuntu/Debian:

apt-get install couchdb

FreeBSD:

cd /usr/ports/databases/couchdb && make install clean

Every once in someone likes to argue that PEAR is all fugly PHP4 code and why you should not use it, and instead go and use another framework or component library. Most of those people also say that they looked at or used PEAR x years ago and then act all surprised when someone else disagrees.

In related (BC) news, most people probably read my blog because of Zend Framework, and I remember that one of the reasons I sold my clients on Zend Framework was a supposedly backward compatibility and clean API. Well, a couple years later one knows it's not all that and since another BC break was argued today on one of the mailing lists and the project lead said I spread fud, I felt l needed to write something on the topic.

Facts first.

A small history of PEAR.

I don't know how old exactly PEAR is, but the manual is copyrighted since 2001 and none of the other current frameworks have been around eight — almost nine — years.

Because PEAR has been around much longer, we also have more older code than any of those PHP5-only frameworks. In comparison, Zend Framework's first stable was release in June, 2007, almost six years later.

Major versus minor releases.

A package' version consists of x.y.z.

The rules are as follow:

• A BC break (see below) — increment x, and set y and z to 0.
• Adding a new feature — keep x, increment y and set z to 0.
• Fixing a bug — keep x and y, but increment z.

When someone refers to a major release in PEAR's context (and a lot of other projects such as Zend Framework, Solar and ezComponents follow this), it's one with a change in x. :-)

What is backward compatibility?

Backward compatibility, or BC, describes that a component, package or library preserves compatibility with an older versions.

Because programming itself and developers tend to evolve, PEAR tries to keep BC in all minor versions, but allows you to break compatibility to an earlier version with a new major release.

The exception to the rule is that you may break BC during alpha and beta releases before the package reaches a stable 1.0.0. Once a 1.0.0 is reached, BC may not be broken — for whatever reason.

PHP4?

Because PEAR aims to provide BC all the way, BC includes the PHP version when the package was first released. Which in turn means that of course you may make the code compatible to a later PHP release, but not without breaking compatibility to the initial release.

If you followed the above, you understand the reason why for example there is a Mail_Queue and (a soon to be) Mail_Queue2, or more importantly: why the Mail_Queue release in 2009 is still compatible to PHP4. Even though PHP4 EOL'd a while ago.

The first Mail_Queue package was released in September of 2002, the 1.0.0 stable release followed in December of the same year. Because its 1.0.0 was compatible with PHP4, we keep it backwards compatible with PHP4 until we release Mail_Queue2-0.1.0.

A principal

A lot of people argue that with the official end of life of PHP4, one should break BC anyway. But here is why you should not.

• Even though we all love to use PHP5, there is still a lot of PHP4 in the enterprise. And like it or not, many of those apps use PEAR, and not your funky PHP5 framework.

• How do you keep so-called necessary and unnecessary BC breaks apart? From another point of you (which is not your own), there is always a necessary BC break to fix implement something else.

• Because there is no such thing as small or acceptable BC breaks. There are BC breaks or there are none, it's one of those things that is black and white.

BC in other frameworks

I know for a fact that ezComponents is very strict on BC. I cannot comment on Solar or Symfony, but at least in Solar's case, I'm assuming that adhere to BC as well since a some former PEAR developers are active and they also follow the PEAR Coding Standard in many respects.

Zend Framework?

A friend of mine said that if Zend Framework really kept BC, we would at 10.0.0 and not on 1.9.3.

Reasons why Zend Framework likes to break BC, even though it advertises full BC:

• No versioning per components but per framework.

• Missing peer review and QA leads to unstable code in a so-called stable release. (Which in turn also fakes the stability of the entire framework since it suggests that a component that was added a couple weeks ago is as stable as a component added in 2007.)

• Because it fixes an "issue". (Biggest WTF of them all.)

The issue in question, I'm not even sure what they were trying to fix. Supposedly some developers found it too hard or did not understand how to write an adapter for Zend_Db and someone committed a fix in the 1.9 tree/branch and apparently it was OK to break BC because it was the easier way out.

I haven't updated some projects since late 1.8.x because of these BC breaks because no one can tell me what the issue is and I don't have a day to debug my application to figure out where and how it breaks. This is annoying as hell, especially since they are supposed to be tailored to the business.

On a side-note, I know of a couple components (e.g. Zend_Session) which really deserve (!) a BC break and don't get one until 2.0. And I totally get why, but why is it OK in some cases? All BC breaks fix issues!

(Btw, as I finish this post, I see an email to zf-contributor@ in my inbox where someone considers pulling the 1.9.3 release (because it obviously breaks BC). Guess I didn't spread that much FUD after all.)

I went to Hamburg last weekend to visit the PHP Unconference, which was probably my first conference ever. I've been to a couple barcamps and other smaller events, but anyway, this felt more like a real conference to me. That is, if I exclude ALA and the various ad:tech's I had to go to.

The reasons why I usually avoid tech conferences include foremost the price tag (working for myself, I can technically label it as an expense, but I still have to pay for it), doubts that it'll be worth it in terms of knowledge gained and probably time. I tend to catch up with people outside of conferences (when they are in Berlin :-)) and that has worked well for me.

I'm glad I set all these things aside for Hamburg (and it was all too easy). A lot of people expressed how much they liked their (often 3rd) PHP Unconference, and I can second, or third that — job really well done. Ulf Wendel took it one step further, blogged and asked, "Is perfect too boring?", because everything worked out so well. I guess I would say, "No, it's not boring", and I'm inclined to add, "Thanks, it really felt like having a weekend off, yet I still learned something and met a ton of nice people (or connected online nicknames to real faces)!".

I can definitely see why people visit the PHP Unconference each year, and I'll be one of them next year! ;-)

As I said, I had a great time, both my topics were accepted too. One was merged with another PHP performance talk which was overbooked with PHP VIPs which is why I decided to listen to Kore Nordmann's talk on CouchDB instead, and the other one ("Deployment") — I kind of overslept. And I'm sorry about that! I'll make sure to avoid party, party Hamburg next year.

Here are the slides for my Zend Framework (performance) talk, I hope you find them interesting:

The slides and speaker-notes contain…

• a small intro as of why I think it's worth while to get into the ring with it
• hints and pointers on general PHP optimization
• I detail on a couple components (e.g. things to look out for and how to overcome them)

Make sure to check the speaker notes (using this link) — I didn't put everything in there, but a lot.

(The deployment slides will be up later this weekend.)

This also reminds me to improve my presentation-fu. I need something as kickass as keynote, but for Windows (currently). If anyone has a pointer, let me know. ;-)