I managed to get the time to do some research into Node.js last week. I have been meaning to check out the new coolness for hipster nerds like myself. I will try to explain some of the node concepts as I understand them.

Event based

Concepts borrowed from ruby’s event machine, python’s twisted and the event based closure syntax of javascript.

Often the plan when it comes to scaling request serving in web apps is to use threads. However there is an alternative method which yields better results. This is the single thread event loop and a good example of event based vs threading is that of a load test between apache and nginx.

Apache the worlds most popular webserver. Nginx is a wee russian webserver which punches beyond its weight.

Nginx versus Apache (with the worker-MPM) for serving a small static file.

Because Nginx is event-based it doesn’t need to spawn new processes or threads for each request, so its memory usage is very low.

These graphs are borrowed from this excellent post on the subject

Its all about the IO man

RAM is fast. Disk, DB and Network are very slow in comparison. This results in programs spending a lot of time waiting for IO operations to complete. Node approaches this as being completely the wrong way of doing things and everything should be non-blocking.

The problem being that most IO libraries are blocking, so there is a movement to upgrade these libraries to be non-blocking but there is also a c based thread pool which node uses to achieve non-blocking whilst using blocking IO Drivers.

Event based is hard

Coming from an OOP MVC background is a hard approach direction to Event based programming. A better approach is looking at javascript in the browser and a working relationship with a framework such as jquery which shows how events can be used in the javascript world to achieve unobstrusive RIA functionality to your web application.

Do I really want to build a website with this?

Currently it seems strange to me that you would want to build a website with server-side javascript. There are a lot of great frameworks out there for building web applications and most of them are a lot more mature and powerful than the javascript offerings.

However, I think that the power of node is in the access to low level functionality to build your own low level network applications.

Previously, these areas have been limited to people with black art C skills in memory management and other old school tools. Now, any joe blow with some javascript skills and create something which does some powerful things with good performance.

So essentially, programmers who have grown up in the internet age can now write ace applications in a way which was previously not possible. In my view, this is the main merit of node.

Simple Proxy in Node

To achieve an http proxy you would typically install something like squid which has a lot of configuration options

Obviously squid is very powerful and bursting full of features that you need, you should probably use it actually. However it is really easy to write a proxy in node.

  var http = require('http');
  var sys  = require('sys');
 
 
 
  function server_callback (request, response) {
    var proxy = http.createClient(80, request.headers['host']);
    var proxy_request = proxy.request(request.method, request.url, request.headers);
 
    proxy_request.addListener('response', function (proxy_response) {
      proxy_response.addListener('data', function(chunk) {
        response.write(chunk, 'binary');
      });
      proxy_response.addListener('end', function() {
        response.end();
      });
      response.writeHead(proxy_response.statusCode, proxy_response.headers);
    });
    request.addListener('data', function(chunk) {
      proxy_request.write(chunk, 'binary');
    });
    request.addListener('end', function() {
      proxy_request.end();
    });
  }
 
 
  http.createServer(server_callback).listen(8080);
  sys.puts("Server running on port 8080");

There is a proxy which a little bit more functionality which is available node proxy

How can I have a go?

The easiest installation path I have seen is to install homebrew then

sudo brew install node

You can then launch the proxy example with:

node proxy.js

I want to put it LIVE

Best plan is to deploy it with nginx as a frontend which proxies request to the node instance. One option is using it with upstart and monit.

This blog post describes the process really well. Heroku are in the process of starting of offer node support; but this is still in a closed beta.

Conclusion

I hope this has given a brief overview of node and it helps you to decide whether to have a go or not.

This is one of the best snowboard video teasers I have seen for a while.

BON VOYAGE TEASER from VIDEOGRASS on Vimeo.

A google server

Google storage was recently released as an open beta for developers to work with. All in all it seems like an Amazon S3 clone and whilst they have their REST api for integration they also support the Amazon S3 api.

Originally I was planning on writing a ruby wrapper of this api but decided to see if I could modify with existing aws-s3 gem to work with google storage.

After some in depth reading of both apis and understanding of the aws-s3 gem I have modified it to also work with google storage.

My github fork contains these modifications.

To use:

 
  require 'aws/s3'
  include AWS::S3
 
  Base.establish_connection!(
      :access_key_id     => 'abc',
      :secret_access_key => '123',
      :default_host => "commondatastorage.googleapis.com"
    )

Note the addition of the default_host to connect to google instead of AWS.

From then on you can use the gem as normal as is described in the Readme.

I have been using SVN a bit recently (euch after the power of git) and have been doing some merges using
FileMerge (opendiff) a utility which comes with the mac developer tools.

To make this play with SVN 1.5 or greater you have to use a wrapper to call the tool. There is an existing toolset written in shell script but they do not seem to work for me (1.6.9).

  #!/usr/bin/env ruby
 
  # A ruby wrapper
 
  unless ARGV.length == 5
    puts "Incorrect number of arguments"
    exit
  end
 
  left = ARGV[3]
  right = ARGV[4]
 
  `opendiff #{left} #{right} -merge #{right}`
  exit(0)

Slap this script somewhere and make it executable. Edit your .bash_profile to include the following:

  export SVN_MERGE='/path/to/svn_diff_wrapper.rb'

So when you get presented with the merge options press l and it should load filemerge for merging power.

In case you havent seen this already. Original promotional video for google’s web browser.

Check out this excellent self filming series of park runs!

Courtesy of whitelines

Nobody likes clutter!

I am getting used to pretty code. I have the ruby beautifier textmate bundle, the excellent jstools (a beautifier and jslint validation for javascript) and my home made wee php beautifier for my php code.

One thing that was missing was my css code, large css files can quickly get out of hand and especially when working with other developers it is nice to be able to make code look nice and clean with the minimal amount of effort.

This also fits in with the no broken window theory analogy coined by the pragmatic programmers

Andy Hunt said:

Researchers studying urban decay wanted to find out why some neighbourhoods escape the ravages of the inner city, and others right next door—with the same demographics and economic makeup—would become a hell hole where the cops were scared to go in. They wanted to figure out what made the difference.

The researchers did a test. They took a nice car, like a Jaguar, and parked it in the South Bronx in New York. They retreated back to a duck blind, and watched to see what would happen. They left the car parked there for something like four days, and nothing happened. It wasn’t touched. So they went up and broke a little window on the side, and went back to the blind. In something like four hours, the car was turned upside down, torched, and stripped—the whole works.

They did more studies and developed a “Broken Window Theory.” A window gets broken at an apartment building, but no one fixes it. It’s left broken. Then something else gets broken. Maybe it’s an accident, maybe not, but it isn’t fixed either. Graffiti starts to appear. More and more damage accumulates. Very quickly you get an exponential ramp. The whole building decays. Tenants move out. Crime moves in. And you’ve lost the game. It’s all over.

We use the broken window theory as a metaphor for managing technical debt on a project.

The bundle is hosted on github. It uses the excellent CSStidy project behind the scenes so make sure that you have php installed on your machine as CLI. > version 5.

Ctrl + alt + option + c will beautify your css file that you have open in the browser. It assumes that you are using 2 spaces for a tab instead of tab characters. So make sure you are using soft tabs in your code.

Install Instructions

• Download it
• Double click on the tmbundle to install it.

Contributing

The source is available on github. One of the big things which can be added is the addition of textmate options to control the behaviour of CSSTidy. At the moment it just makes the CSS look pretty in a way that I think is pretty. It might be ugly to the one line per definition kids.

Tiptip is a nice jquery plugin for displaying tooltips in a simple and elegant manner. All you need to do is set the title attribute with the tooltip content then give it a class name so it it picked up by the tooltip initializer.

However, it currently doesnt work in IE6 that well. Although users are falling I think you still need at least rudimentary support for IE6.

Create a conditional comment which includes a stylesheet load in your document header.

  <!--[if IE 6]>
     <link href="/stylesheets/ie6.css?1273083940" media="screen" rel="stylesheet" type="text/css" />
   <![endif]-->

And add the following rules to the stylesheet.

Always doing this on different ubuntu versions and always having to look it up.

Here is a guide for Ubuntu Intrepid

I have been working on a google base product search xml feed generator. It is in the form of a semi-customizable rake task. I have released this as a kind of rails plugin.

rake googlebase:generate generates an xml file of your products.

Configuration

To configure, add Rails.root/config/google_base.yml

Example google_base.yml file in files/google_base.yml

As the structure of the xml is dependant on your product models I have left it to the programmer to modify the google.rake file and set the correct instance methods for returning product information, along with the named_scope for returning live products.

TODO:

Instead of editing this rake file manually it would be good to add a plugin type class method which can take the configuration of your products and generate this automatically.

Its is available on github