Deploying PHP applications: PEAR and composer resources for chef

Thursday, February 23. 2012
Comments

This is something experimental I have been working on for our chef deployments. So the objective was/is to find a sane way to install PEAR packages and install dependencies with composer.

execute in chef recipes

In chef recipes, almost everything is a resource. In case you're just getting started with Chef, a list of current resources is available on the Opscode Wiki. It's a link I put in my browser bar since I frequently work on chef recipes.

Some examples for resources are:

  • package (to install software)
  • cron (setup a crontab)
  • directory (create directories)
  • template (install customized configuration files)
  • user and group (to create users and groups)
  • mdadm (to setup a RAID)

The above list are examples — so there is more. But if there isn't a designated resource, you can always use an execute block.

An example for an execute block is the following:

execute "discover a pear channel" do
  command "pear channel-discover easybib.github.com/pear"
end

This works pretty well, but it is also not very robust.

Fail hard

By default whenever a command fails, chef fails hard.

To illustrate what I'm talking about, let's test and execute the command from our execute block multiple times on the shell to see its exit status ($?):

till:~/ $ pear channel-discover easybib.github.com/pear
Adding Channel "easybib.github.com/pear" succeeded
Discovery of channel "easybib.github.com/pear" succeeded
till:~/ $ echo $?
0
till:~/ $ pear channel-discover easybib.github.com/pear
Channel "easybib.github.com/pear" is already initialized
till:~/ $ echo $?
1

So whenever a command returns not 0, chef will bail.

One solution is to brute-force your way through these things with ignore_failure true in your execute block. But that's usually not a great idea either because it hides other issues from you (and me) when we need to debug this later on.

For example, if this PEAR channel is unavailable during your next chef-run, it would be much, much harder to find the root cause as of why the install commands failed.

Another solution is using the not_if or only_if options with execute:

execute "discover a pear channel" do
  command "pear channel-discover easybib.github.com/pear"
  not_if do
    `pear channel-info easybib.github.com/pear`
  end
end

If the command wrapped in not_if succeeds (success is exit status), we would skip the execute block.

Optimize?

Since I discovered not_if and only_if, it allows me write recipes which work in most cases. More robust code, which allows me to re-execute recipes on already running instances. So for example when I update a recipe or configuration file which is distributed through a recipe I can re-run the entire recipe and it will not fail but instead complete successfully.

One problem remains with this approach I end up doing the same checks again and again.


Continue reading "Deploying PHP applications: PEAR and composer resources for chef"

A case for PEAR and PHP4 (Or, why BC is important!)

Tuesday, September 22. 2009
Comments

Every once in someone likes to argue that PEAR is all fugly PHP4 code and why you should not use it, and instead go and use another framework or component library. Most of those people also say that they looked at or used PEAR x years ago and then act all surprised when someone else disagrees.

In related (BC) news, most people probably read my blog because of Zend Framework, and I remember that one of the reasons I sold my clients on Zend Framework was a supposedly backward compatibility and clean API. Well, a couple years later one knows it's not all that and since another BC break was argued today on one of the mailing lists and the project lead said I spread fud, I felt l needed to write something on the topic.

Facts first.

A small history of PEAR.

I don't know how old exactly PEAR is, but the manual is copyrighted since 2001 and none of the other current frameworks have been around eight — almost nine — years.

Because PEAR has been around much longer, we also have more older code than any of those PHP5-only frameworks. In comparison, Zend Framework's first stable was release in June, 2007, almost six years later.

Major versus minor releases.

A package' version consists of x.y.z.

The rules are as follow:

  • A BC break (see below) — increment x, and set y and z to 0.
  • Adding a new feature — keep x, increment y and set z to 0.
  • Fixing a bug — keep x and y, but increment z.

When someone refers to a major release in PEAR's context (and a lot of other projects such as Zend Framework, Solar and ezComponents follow this), it's one with a change in x. :-)

What is backward compatibility?

Backward compatibility, or BC, describes that a component, package or library preserves compatibility with an older versions.

Because programming itself and developers tend to evolve, PEAR tries to keep BC in all minor versions, but allows you to break compatibility to an earlier version with a new major release.

The exception to the rule is that you may break BC during alpha and beta releases before the package reaches a stable 1.0.0. Once a 1.0.0 is reached, BC may not be broken — for whatever reason.

PHP4?

Because PEAR aims to provide BC all the way, BC includes the PHP version when the package was first released. Which in turn means that of course you may make the code compatible to a later PHP release, but not without breaking compatibility to the initial release.

If you followed the above, you understand the reason why for example there is a Mail_Queue and (a soon to be) Mail_Queue2, or more importantly: why the Mail_Queue release in 2009 is still compatible to PHP4. Even though PHP4 EOL'd a while ago.

The first Mail_Queue package was released in September of 2002, the 1.0.0 stable release followed in December of the same year. Because its 1.0.0 was compatible with PHP4, we keep it backwards compatible with PHP4 until we release Mail_Queue2-0.1.0.

A principal

A lot of people argue that with the official end of life of PHP4, one should break BC anyway. But here is why you should not.

  • Even though we all love to use PHP5, there is still a lot of PHP4 in the enterprise. And like it or not, many of those apps use PEAR, and not your funky PHP5 framework.

  • How do you keep so-called necessary and unnecessary BC breaks apart? From another point of you (which is not your own), there is always a necessary BC break to fix implement something else.

  • Because there is no such thing as small or acceptable BC breaks. There are BC breaks or there are none, it's one of those things that is black and white.

BC in other frameworks

I know for a fact that ezComponents is very strict on BC. I cannot comment on Solar or Symfony, but at least in Solar's case, I'm assuming that adhere to BC as well since a some former PEAR developers are active and they also follow the PEAR Coding Standard in many respects.

Zend Framework?

A friend of mine said that if Zend Framework really kept BC, we would at 10.0.0 and not on 1.9.3.

Reasons why Zend Framework likes to break BC, even though it advertises full BC:

  • No versioning per components but per framework.

  • Missing peer review and QA leads to unstable code in a so-called stable release. (Which in turn also fakes the stability of the entire framework since it suggests that a component that was added a couple weeks ago is as stable as a component added in 2007.)

  • Because it fixes an "issue". (Biggest WTF of them all.)

The issue in question, I'm not even sure what they were trying to fix. Supposedly some developers found it too hard or did not understand how to write an adapter for Zend_Db and someone committed a fix in the 1.9 tree/branch and apparently it was OK to break BC because it was the easier way out.

I haven't updated some projects since late 1.8.x because of these BC breaks because no one can tell me what the issue is and I don't have a day to debug my application to figure out where and how it breaks. This is annoying as hell, especially since they are supposed to be tailored to the business.

On a side-note, I know of a couple components (e.g. Zend_Session) which really deserve (!) a BC break and don't get one until 2.0. And I totally get why, but why is it OK in some cases? All BC breaks fix issues!

(Btw, as I finish this post, I see an email to zf-contributor@ in my inbox where someone considers pulling the 1.9.3 release (because it obviously breaks BC). Guess I didn't spread that much FUD after all.)

Forced contribution

Monday, September 7. 2009
Comments

I'm not exactly neutral when it comes to anything remotely related to the GPL license. Personally, there's a bit of a GPL scare when I see code that's released under that license and I usually try to avoid it.

But (primarily) due to RoundCube being licensed under the GPL, I think I do know what it entails to release code using this license. In addition to that I have read a lot about the license, I even wasted spent three hours one night to listen to RMS.

In the end — the GPL just did not grow on me, or made me happy.

Open source

I am a firm believer in open source. I release code for free. Whenever I have the choice, my free contributions include the freedom to really do whatever you want with my code. Because if I did not meant it, I would either sell the code, or not share it at all.

I recently read Zed Shaw's reasoning why he believes in the [A,L]GPL and that same day I walked into yet another license discussion on PEAR's IRC channel (#pear@efnet), and I felt like I need to write it all out.

The GPLs

GPL in a nutshell

  • The GPL means that whenever I use code that is licensed under it in my code, my code automatically becomes GPL too.
  • The GPL requires me to release the source code of the software when I give it to others [them].
  • The GPL allows them to give it to other people as well, license still applies.
  • (Contrary to popular believe, ) The GPL also allows me to sell software. I'm not required to give it away for free.

If you still do not understand what the above means, here's an example: Wordpress. Wordpress is GPL and therefor all plugins written for Wordpress are GPL too. I realize some people may think that there might not be too much that you can do in a plugin, but if you wanted to make money of your work (plugin), the options are pretty limited.

LGPL in a nutshell

The LGPL is basically the same as the GPL, but if I use LGPL software inside my software, my software does not become GPL.

Affero (L)GPL

Because web software is often not distributed (think of SAAS), the GPL people came up with an Affero clause.

This clause requires you to open source your changes to a software/library even if it's only accessible through the network. In plain English — if you do SAAS (Remember, plain English!), or a simple website, and do not directly distribute your source code to your customer, you will still have to open source your changes because your customers can access it anyway.

The Affero clause is currently available as AGPL, and soon as ALGPL.

Non-restrictive licensing

When I speak of non-restrictive (or liberal :-)) licenses, I think of the (new) BSD, MIT and Apache licenses. In a nutshell, they all allow you to really do whatever you want. Whatever, of course except for removing the copyright on the source code.

  • They do not force you to open source your changes to the code.
  • They do not force a license on your own code just because you happen to use it.
  • The do not force you to release the source code to your customers.

Reading the above, one would see that these licenses are very compatible with typical business interests. They all come in handy for frameworks such as ezComponents or the Zend Framework — and many PEAR components use them as well. They really bring freedom of use the user, without imposing any duties on the user, and the best of all: the user can contribute anyway.

Impose

For a lot of people the world is black and white. You see other people benefiting from your own work hiding everywhere. And it's all too easy to find a license to impose your own beliefs on others. Because if they do not get it, there is a way to make them.

This rather simplified approach to a pretty complicated topic is easier to comprehend and therefor popular. And I can't blame anyone. Think further — religion. People who convert or find a thing for themselves easily become outspoken about and start preaching their new found happiness to others.

It's very human. If it makes you happy, you want to share.

This is of course meant with no offense to people like Zed Shaw who feel like they did not get their share of the cake (even though — Hey Zed! — a ton of people run your software (Mongrel)). I do understand Zed's point of view though. Visibility in the open source world does not pay your monthly bills. I'm not naive like that. On the other hand, there are more than a few examples from the the open source world where a company is build around an open source product and the company offers services — such as consulting and support — for said product.

The bottom line for myself is that I do not like to force people to do something. And no one does, right? (Except when the motives allow it!)

Conclusion

I believe that more people will contribute to open source because they believe in the cause. Not because some license forces them to do so. A lot of people get into open source because they use(d) open source software already and decided to contribute to the community. A lot of commercial entities fund open source development — heck, for whatever reason, even Microsoft does it.

If anything, the Affero clause will cause is to hinder the adoption rates of the software in question. And that is not just because all these licenses are written in English which requires a law degree, but because when you manage to understand them, they impose a threat on your own intellectual property.

Mail_Queue: 1.2.3

Friday, September 4. 2009
Comments

Despite Mail_Queue being a PHP4-compatible package, I still like to use it on current projects because it is so easy to implement and because it gets the job done. So, over the last weeks (and especially since PHP moved from CVS to SVN :-)), I put in a little time, and especially with the help of Ken, we managed to push out the 1.2.3 release today.

What's new?

changed license from PHP to the (New) BSD license

This is good news for two reasons. Numero uno, GPL-licensed (open source) projects may bundle Mail_Queue now since the (new) BSD license is compatible with the GPL. And secondly, because (re-)releasing the code using this well-known license also gets rid off another entry barrier for commercial entities and also provides them with enough flexibility to use this code inside commercial applications.

bug #7850

This one is a tough one, and only bites you if you really adhere to backwards compatibility (aka BC) and work with an older code base. ;-)

So the story is, that with PHP4, a constructor cannot return a PEAR_Error object. PHP4 also did not have exceptions. So what we implemented as a work around it, and of course to keep BC, was to introduce a new "factory" method (which can return a PEAR_Error object). In addition to the factory, we added helper methods (Mail_Queue::getErrors() and Mail_Queue::hasErrors()) so you can check the state of the class at any time.

bug #14626

This one's self-explanatory — and the only real bugfix in this release.

minor CS fixes

To make the PEAR test suite happy. :-)

request #15049

We added a method to count emails in queue. This is currently only implemented when you use the MDB2-based backend. (Side-note: DB, MDB and creole backends are deprecated!)

request #14921

Implemented optional support to sleep() in between the sending process — to not hammer mail servers.

request #6456

Additional parameter validation to deal with PHP's loosely typed variables.

requests #16064, #16068

These are Ken's contributions to the Mail_Queue release — callback support to run a custom function whenever an email was send (or queued on the SMTP side). This could be used to provide extended monitoring and metrics on the sending process. Yay!

Installation, or upgrading

Existing install:

pear upgrade Mail_Queue

New install:

pear install Mail_Queue

There is nothing else to do. All new features are optional, and nothing requires you to touch your code base. The beauty of BC!

What's next?

I added a roadmap for 1.2.4 last night, but I'm not sure if we will really get to it. I'm trying to channel all efforts to Mail_Queue2 (which is a PHP5-only port of Mail_Queue and already contains most (if not all) the features included in Mail_Queue's 1.2.3 release. If you would like to contribute, please check out the project on Google Code and let me know your thoughts.

Great expectations

Saturday, July 11. 2009
Comments

If you ever contributed your spare time to any project which gives away something at the price of zero, you may be familiar with a large variety of feedback.

It's always reassuring for those who spend their time to get feedback from happy users. But of course there's no way to please everyone at the same time. And aside from the usual "This doesn't work!!!11 WTF?!!", some people seem to be particularly ungrateful, or just way out of bounds when they email in.

I hereby classify feedback into three different types:

  • valuable
  • useless
  • mediocre — in between the above

By classifying feedback, I don't mean classify the content. Of course I have to admit that I prefer positive feedback over negative (Who doesn't?), but in the end it's more or less the attitude and effort which people bring to the table — in emails or bug reports.

Even if someone basically told me that something sucks, I can — given it's valuable feedback — work with them to improve.

Last night, a user had the urge to email us (RoundCube) to tell us how ridiculous RoundCube is. So according to him RoundCube does not provide enough features, whatever we provide is not up to his personal standards, he had issues installing it, using it and suggested that we should drop RoundCube and instead provide an ajax frontend to Squirrelmail.

He also managed to insult some ISP (pobox.com), which his "schoolmate from high school" founded.

No offense Squirrelmail (I'm using you as a backup from my non-JavaScript-friendly mobile), but I really don't see anything valuable in the email I read. I could have taken this a lot more personal when I took the liberty to reply. For transparency I copied our dev@ mailing list on my response.

This morning the user complained to me about sharing his email with the world, told me I attacked his family (Err, what?), then attacked me personally and demanded that I remove his personal information from the archives.

His response was of course emailed off-list and I didn't feel sharing it again to fuel the fire.

Anyway, I did reply on key and stayed away from personals. But I just received another email from him where he emailed the SourceForge legal team and CC'd myself (and most developers again), demanding that we remove his information, or that they make us because I violated SourceForge's terms of service (3A).

Err... it's certainly not going to ruin my weekend. But what the hell?