Till's blog

In my last blog entry, I shared some nodejs-code to read CouchDB's _changes feed and publish the data to a website. In order to update the page in a continous fashion, I used socket.io which provides a nifty abstraction across server- to client-side transports — for example, websockets and ajax longpoll.

Full-throttle

When we tested the code for a few days over the weekend, the largest issue we ran into was that the stream moved too fast. In fact it moved so fast, we couldn't read anything and were at risk of getting a seizure when we watched the page for too long.

Certainly awesome from one point of view — people are using the website — but it also led to the next objective: I had to find a way to throttle broadcasting to the client. Here's how!

I recently had the extreme pleasure to use node.js and socket.io on a project. Here are some insights.

Objective

So the objective of the project was to read data from the _changes feed of our CouchDB cluster (hosted by Cloudant) and publish the data to a widget which we can use to display a constant stream of "what are people doing right now".

The core of the problem we faced was not just taking this stream of data and feeding it on to a page, but since we'll deploy this widget to our homepage we needed to make sure that no matter how many clients see it, the impact on the database cluster is minimal; for example, it would be a single client (or down the road up to three for failover) who actually read data from the cluster.

After shopping around for a technology to use, it became obvious that we needed some sort of abstraction because of how the different technologies (e.g. comet, websockets, ajax longpolling, ...) are implemented in different browsers. We decided to build this project on top of socket.io — pretty much for the same reasons most people go to jQuery, prototype or dojo these days.

Update 2010-09-06: I turned my JavaScript code into a handy plugin for jQuery — let me introduce: jquery-simplerss.

So for whatever reason, on a lot of blogs (but not mine ;-)), the sidebar also contains the list of latest entries on said blog.

I recently edited a template for a client and he requested the same feature — which put me through three hours of nightmare.

Tumblr

Tumblr is a hosted blog service. It doesn't have as many (confusing) features as for example Wordpress or Serendipity, but it's doing really well at people usually do with blogs: posting stuff. Stuff includes text, excerpts from chats, photos, quotes, music, videos and maybe more.

The advantages of Tumblr versus any local blog install are:

• it's (completely ad-)free (a local blog requires a webhost or server of some kind)
• no installation
• no security updates

Tumblr also comes with a domain and custom theme feature where you can basically CNAME a domain name to their server and it looks like it's part of your domain — take my activity stream for an example.

While I realize that Tumblr is not free for forever, all of the above are plenty of good reasons why I'd rather recommend my clients to get a Tumblr account instead of installing Wordpress on their server, and getting hacked a week or so later.

Customized themes

For those of us with slightly less design skills, there are commercial themes available, for the rest, they may use customize any theme available.

Customized themes are simple and to be kept simple for the most part. So for example a sidebar with the latest posts is not something the theme allows you to do and while I value that my suggestion was taken to the development team (like others before), I couldn't really wait until someone gets around to making it happen.

JavaScriptjQuery to the rescue!

So I decided to use jQuery to parse the blogs RSS feed and display the three latest items. While I realize that there are plugins available, I thought I'd do it myself — quick and dirty — because jQuery offers with $.ajax() and .find() pretty much all you need to download the RSS feed and parse the bits from it. Since my blog runs on http://blog.example.org/ and the RSS feed is at http://blog.example.org/rss not even the cross-domain obstacles apply!

Here's how I did it!

Let's walk through the code, shall we?

• $.ajax() is used to GET the feed
• for Tumblr: dataType: 'xml'
• .find() is used to select the items (rss > channel > item)
• inside the loop:
  • find title, description and guid
  • create html (which is basically my template)
  • exit from the loop after we ran through the 3 latest items
• after the loop: append to a selector

For the above to work you'll need (to include jQuery and the following HTML):

<ul id="blogposts"></ul>

Gotchas

So even though Tumblr's feed is a valid RSS feed per feed-validator.org, there were a few things that didn't work right away.

At first I used dataType: feed in $.ajax() — which made sense at the time. But the problem is that Tumblr sends Content-Type: text/xml for their RSS feed. I haven't checked the internals of the dataType but apparently Google Chrome applies stricter rules to what it thinks are feeds.

This issues implies that for example the entities in <title> make Google Chrome drop the <title> all together when the XML feed is imported. There is an interesting thread on Stackoverflow about the issue. The quickfix is to use dataType: xml instead.

The second problem is that <link> is dropped/ignored because the link is wrapped in doublequotes. So I used <guid> instead.

Fin

I certainly hope that this example is complete and saves someone else some time.

Hey everyone,

it's my sincere pleasure to announce that we're looking to fill two positions for PHP developers (entry/junior) in NYC.

Expectations

This is what we look for from candidates:

• A strong and firm knowledge of PHP5
• First hand experience with the Zend Framework
• You've heard of PHPUnit and TDD
• An idea of what a HTTP request is and the different applications that take part in one
• You heard of CouchDB, MongoDB or Redis (generally "NoSQL") before

Last but absolutely not least:

We very, very, very much prefer people who contribute(d) to Open Source.

Playground

• A web start-up.
• The not-so-standard LAMP stack with: Linux, Nginx, PHP and mostly CouchDB.
• A lot time to play with Amazon Web Services.
• Size matters to you? Databases and indices in the 100 millions.
• Maybe Solr!
• Definitely Redis!

... generally, we always try to use the right tool for the job.

If you're interested, please email me your resume:

[email protected]

If you know someone else and we happen to hire this person my special referral bonus is a couple beers next time we meet. ;-) [Disclaimer: If you're 1821, or older.]

I know some of you out there are probably tired of jQuery and people raving about it's goodness, but bare with me! Because jQuery never ceases to amaze me — especially when I haven't looked at it — or client-side JavaScript code in general — in a good year or so.

Refactoring

I've been refactoring some of my old JavaScript libs on a project and I noticed that I had used evil eval() all over the place to parse the JSON from our API. Guess we all know that this is not just a security issue since it allows code execution, but also a performance hit. And here's how to get around it. :-)

A codesnippet to automatically parse a JSON response from a $.post-request:

    $.post(
      '/the-url-post-to',
      data,
      function(json) {
        // handle response
      },
      'json'
    );

Note: The 4th parameter 'json' in $.post(). Magic.

Fin

That's all.