Jaml updates

January 29, 2010 by Ed Spencer · Leave a Comment 

Jaml seems to have been getting a lot of interest lately. Here are a few quick updates on what’s been going on:

In addition Jaml was recently picked up by Ajaxian, and a couple of people have written up blog posts about Jaml in languages other than English, which is great to see.

Jaml is up on Github and has a number of forks already. If you like the library and have something to add, fork away and send me a pull request!

If you’ve never seen Jaml before or have forgotten what it does, it turns this:

div(
  h1("Some title"),
  p("Some exciting paragraph text"),
  br(),

  ul(
    li("First item"),
    li("Second item"),
    li("Third item")
  )
);

Into this:

<div>
  <h1>Some title</h1>
  <p>Some exciting paragraph text</p>
  <br />
  <ul>
    <li>First item</li>
    <li>Second item</li>
    <li>Third item</li>
  </ul>
</div>

See the original post for more details.

OSX Screensaver emulation with Canvas: That’s Bean

December 6, 2009 by Ed Spencer · 3 Comments 

OS X has a pretty little screensaver which takes a bunch of images and ‘drops’ them, spinning, onto the screen. Think of it like scattering photographs onto a table, one at a time.

Naturally, there’s a desperate need for a JavaScript/Canvas port of this functionality, resulting in the following:

I had to limit the video capture framerate a bit so the video makes it look less smooth than it actually is. Check it out running in your own browser here.

For obvious reasons I have called the code behind this Bean, and it’s all available up on Github.

For the curious, here’s a little explanation about how it works. Bean starts off with a blank canvas and a list of image urls, which it preloads before getting started. It then drops one image at a time, rotating it as it goes. Each falling image is called a Plunger, because it plunges.

Each Plunger gets a random position and rotation to end up in, and takes care of drawing itself to the canvas on each frame by calculating its current size and rotation as it falls away from you.

Drawing each Plunger image on every frame quickly starts to kill the CPU, so we take a frame snapshot every time a Plunger has finished its descent. This just entails drawing the completed Plunges first and then using Canvas’ getImageData API to grab the pixel data for the image.

This gives us a snapshot of all of the fallen Plungers, meaning we can just draw a single background image and the currently falling Plunger on each frame. This approach ensures the performance remains constant, as we are only ever drawing a maximum of 2 images per frame. Each time a Plunger finishes its descent a new snapshot is taken.

Bean attempts to draw a new frame roughly 25 times per second and modern browsers seem to handle this pretty well. Safari pulls around 60% of one core on my MacBook Pro, with Firefox somewhat less performant. Needless to say, I didn’t even bother trying to make this work with IE.

Here’s the code to set the Bean in motion. This is using a few bundled APOD images:

var bean = new Bean({
  imageUrls: [
    'images/DoubleCluster_cs_fleming.jpg',
    'images/NGC660Hagar0_c900.jpg',
    'images/filaments_iac.jpg',
    'images/m78wide_tvdavis900.jpg',
    'images/sunearthpanel_sts129.jpg',
    'images/NGC253_SSRO_900.jpg',
    'images/Ophcloud_spitzer_c800.jpg'
  ],
  canvasId : 'main',
  fillBody : true
});

bean.onReady(function(bean) {
  bean.start();
});

Jaml: beautiful HTML generation for JavaScript

November 4, 2009 by Ed Spencer · 30 Comments 

Generating HTML with JavaScript has always been ugly. Hella ugly. It usually involves writing streams of hard-to-maintain code which just concatenates a bunch of strings together and spits them out in an ugly mess.

Wouldn’t it be awesome if we could do something pretty like this:

div(
  h1("Some title"),
  p("Some exciting paragraph text"),
  br(),

  ul(
    li("First item"),
    li("Second item"),
    li("Third item")
  )
);

And have it output something beautiful like this:

<div>
  <h1>Some title</h1>
  <p>Some exciting paragraph text</p>
  <br />
  <ul>
    <li>First item</li>
    <li>Second item</li>
    <li>Third item</li>
  </ul>
</div>

With Jaml, we can do exactly that. Jaml is a simple library inspired by the excellent Haml library for Ruby. It works by first defining a template using an intuitive set of tag functions, and then rendering it to appear as pretty HTML. Here’s an example of how we’d do that with the template above:

Jaml.register('simple', function() {
  div(
    h1("Some title"),
    p("Some exciting paragraph text"),
    br(),

    ul(
      li("First item"),
      li("Second item"),
      li("Third item")
    )
  );
});

Jaml.render('simple');

All we need to do is call Jaml.register with a template name and the template source. Jaml then stores this for later use, allowing us to render it later using Jaml.render(). Rendering with Jaml gives us the nicely formatted, indented HTML displayed above.

So we’ve got a nice way of specifying reusable templates and then rendering them prettily, but we can do more. Usually we want to inject some data into our template before rendering it – like this:

Jaml.register('product', function(product) {
  div({cls: 'product'},
    h1(product.title),

    p(product.description),

    img({src: product.thumbUrl}),
    a({href: product.imageUrl}, 'View larger image'),

    form(
      label({'for': 'quantity'}, "Quantity"),
      input({type: 'text', name: 'quantity', id: 'quantity', value: 1}),

      input({type: 'submit', value: 'Add to Cart'})
    )
  );
});

In this example our template takes an argument, which we’ve called product. We could have called this anything, but in this case the template is for a product in an ecommerce store so product makes sense. Inside our template we have access to the product variable, and can output data from it.

Let’s render it with a Product from our database:

//this is the product we will be rendering
var bsg = {
  title      : 'Battlestar Galactica DVDs',
  thumbUrl   : 'thumbnail.png',
  imageUrl   : 'image.png',
  description: 'Best. Show. Evar.'
};

Jaml.render('product', bsg);

The output from rendering this template with the product looks like this:

<div class="product">
  <h1>Battlestar Galactica DVDs</h1>
  <p>Best. Show. Evar.</p>
  <img src="thumbnail.png" />
  <a href="image.png">View larger image</a>
  <form>
    <label for="quantity">Quantity</label>
    <input type="text" name="quantity" id="quantity" value="1"></input>
    <input type="submit" value="Add to Cart"></input>
  </form>
</div>

Cool – we’ve got an object oriented declaration of an HTML template which is cleanly separated from our data. How about we define another template, this time for a category which will contain our products:

Jaml.register('category', function(category) {
  div({cls: 'category'},
    h1(category.name),
    p(category.products.length + " products in this category:"),

    div({cls: 'products'},
      Jaml.render('product', category.products)
    )
  );
});

Our category template references our product template, achieving something rather like a partial in Ruby on Rails. This obviously allows us to keep our templates DRY and to easily render a hypothetical Category page like this:

//here's a second product
var snowWhite = {
  title      : 'Snow White',
  description: 'not so great actually',
  thumbUrl   : 'thumbnail.png',
  imageUrl   : 'image.png'
};

//and a category
var category = {
  name    : 'Doovde',
  products: [bsg, snowWhite]
}

Jaml.render('category', category);

All we’ve done is render the ‘category’ template with our ‘Doovde’ category, which contains an array of products. These were passed into the ‘product’ template to produce the following output:

<div class="category">
  <h1>Doovde</h1>
  <p>2 products in this category:</p>
  <div class="products"><div class="product">
  <h1>Battlestar Galactica DVDs</h1>
  <p>Best. Show. Evar.</p>
  <img src="thumbnail.png" />
  <a href="image.png">View larger image</a>
  <form>
    <label for="quantity">Quantity</label>
    <input type="text" name="quantity" id="quantity" value="1"></input>
    <input type="submit" value="Add to Cart"></input>
  </form>
</div>
<div class="product">
  <h1>Snow White</h1>
  <p>not so great actually</p>
  <img src="thumbnail.png" />
  <a href="image.png">View larger image</a>
  <form>
    <label for="quantity">Quantity</label>
    <input type="text" name="quantity" id="quantity" value="1"></input>
    <input type="submit" value="Add to Cart"></input>
  </form>
</div>
</div>
</div>

You can see live examples of all of the above at http://edspencer.github.com/jaml.

Jaml currently sports a few hacks and is not particularly efficient. It is presented as a proof of concept, though all the output above is true output from the library. As always, all of the code is up on Github, and contributions are welcome :)

Jaml would be suitable for emulating a Rails-style directory structure inside a server side JavaScript framework – each Jaml template could occupy its own file, with the template name coming from the file name. This is roughly how Rails and other MVC frameworks work currently, and it eliminates the need for the Jaml.register lines. Alternatively, the templates could still be stored server side and simply pulled down and evaluated for client side rendering.

Happy rendering!

Writing Better JavaScript – split up long methods

October 6, 2009 by Ed Spencer · Leave a Comment 

For the second time this week I’m going to pick on the usually delightful Ext JS library. Last time we discussed the overzealous use of the Module pattern; this time it’s the turn of bloated methods.

As before, I’m not really picking on Ext at all – this happens all over the place. But again, this is the library closest to my heart and the one I know the best.

The Problem

We’re going to take a look at Ext.data.XmlReader’s readRecords method. Before we get started though, I’ll repeat that this is intended as an example of an approach, not a whine at Ext in particular.

/**
 * Create a data block containing Ext.data.Records from an XML document.
 * @param {Object} doc A parsed XML document.
 * @return {Object} records A data block which is used by an {@link Ext.data.Store} as
 * a cache of Ext.data.Records.
 */
readRecords: function(doc) {
  /**
   * After any data loads/reads, the raw XML Document is available for further custom processing.
   * @type XMLDocument
   */
  this.xmlData = doc;
  var root = doc.documentElement || doc;
  var q = Ext.DomQuery;
  var recordType = this.recordType, fields = recordType.prototype.fields;
  var sid = this.meta.idPath || this.meta.id;
  var totalRecords = 0, success = true;
  if(this.meta.totalRecords){
    totalRecords = q.selectNumber(this.meta.totalRecords, root, 0);
  }

  if(this.meta.success){
    var sv = q.selectValue(this.meta.success, root, true);
    success = sv !== false && sv !== 'false';
  }
  var records = [];
  var ns = q.select(this.meta.record, root);
  for(var i = 0, len = ns.length; i < len; i++) {
    var n = ns[i];
    var values = {};
    var id = sid ? q.selectValue(sid, n) : undefined;
    for(var j = 0, jlen = fields.length; j < jlen; j++){
      var f = fields.items[j];
      var v = q.selectValue(Ext.value(f.mapping, f.name, true), n, f.defaultValue);
      v = f.convert(v, n);
      values[f.name] = v;
    }
    var record = new recordType(values, id);
    record.node = n;
    records[records.length] = record;
  }

  return {
    success : success,
    records : records,
    totalRecords : totalRecords || records.length
  };
}

Anyone care to tell me what this actually does? Personally, I have absolutely no idea. I recently found myself needing to implement an XmlReader subclass with a twist which required understanding how this works, and let’s just say it wasn’t easy!

So what is it that makes the above so terrifyingly hard to understand? Well, in no particular order:

  • It’s too long – you’d need to be a genius to easily understand what’s going on here
  • The variable names don’t make much sense – some of the oddest include ‘q’, ‘ns’, ‘v’, ‘f’ and ’sv’
  • There’s minimal commenting – we’re given a single-line clue at the very top as to what these 40-odd lines do

A Solution

Let’s see how the reworked code below addresses each of the concerns above:

  • Although we end up with more lines of code here, no single method is more than around 10 LOC
  • No single letter variable names – you no longer have to decode what ’sv’ means
  • Constructive commenting allows rapid comprehension by skimming the text

One additional and enormous benefit here comes directly from splitting logic into discrete methods. Previously if you’d wanted to implement your own logic to determine success, get the total number of records or even build a record from an XML node you’d be stuck. There was no way to selectively override that logic without redefining that entire monster method.

With our new approach this becomes trivial:

Ext.extend(Ext.data.XmlReader, Ext.data.DataReader, {
  readRecords: function(doc) {
    this.xmlData = doc;

    //get local references to frequently used variables
    var root    = doc.documentElement || doc,
        records = [],
        nodes   = Ext.DomQuery.select(this.meta.record, root);

    //build an Ext.data.Record instance for each node
    Ext.each(nodes, function(node) {
      records.push(this.buildRecordForNode(node));
    }, this);

    return {
      records     : records,
      success     : this.wasSuccessful(root),
      totalRecords: this.getTotalRecords(root) || records.length
    };
  },

  /**
   * Returns a new Ext.data.Record instance using data from a given XML node
   * @param {Element} node The XML node to extract Record values from
   * @return {Ext.data.Record} The record instance
   */
  buildRecordForNode: function(node) {
    var domQuery = Ext.DomQuery,
        idPath   = this.meta.idPath || this.meta.id,
        id       = idPath ? domQuery.selectValue(idPath, node) : undefined;

    var record  = new this.recordType({}, id);
    record.node = node;

    //iterate over each field in our record, find it in the XML node and convert it
    record.fields.each(function(field) {
      var mapping  = Ext.value(field.mapping, field.name, true),
          rawValue = domQuery.selectValue(mapping, node, field.defaultValue),
          value    = field.convert(rawValue, node);

      record.set(field.name, value);
    });

    return record;
  },

  /**
   * Returns the total number of records indicated by the server response
   * @param {XMLDocument} root The XML response root node
   * @return {Number} total records
   */
  getTotalRecords: function(root) {
    var metaTotal = this.meta.totalRecords;

    return metaTotal == undefined
                      ? 0
                      : Ext.DomQuery.selectNumber(metaTotal, root, 0);
  },

  /**
   * Returns true if the response document includes the expected success property
   * @param {XMLDocument} root The XML document root node
   * @return {Boolean} True if the XML response was successful
   */
  wasSuccessful: function(root) {
    var metaSuccess  = this.meta.success;

    //return true for any response except 'false'
    if (metaSuccess == undefined) {
      return true;
    } else {
      var successValue = Ext.DomQuery.selectValue(metaSuccess, root, true);
      return successValue !== false && successValue !== 'false';
    }
  }
});

(For brevity I have omitted the existing readRecords comment blocks from the above)

I suggest that you structure your code in this way at least 99% of the time. The one exception is if high performance is an issue. If you are in a situation where every millisecond counts (you probably aren’t), then taking the former route becomes more acceptable (though there’s still no excuse for not adding a few comments explaining what the code actually does).

My refactored code almost certainly runs slower than the original as it doesn’t take as much advantage of cached local variables as the monolithic version does. For library-level code this can make sense if the performance gain is significant, but for the everyday code you and I write it is rarely a good idea.

I’ll be watching.

JavaScript Module pattern – overused, dangerous and bloody annoying

October 5, 2009 by Ed Spencer · 10 Comments 

The Module Pattern is a way of using a closure in JavaScript to create private variables and functions. Here’s a brief recap:

var myObject = (function() {
  //these are only accessible internally
  var privateVar = 'this is private';
  var privateFunction = function() {
    return "this is also private";
  };

  return {
    //these can be accessed externally
    publicVar: 'this is public',

    publicFunction: function() {
      return "this is also public"
    },

    //this is a 'privileged' function - it can access the internal private vars
    myFunction: function() {
      return privateVar;
    }
  };
})();

myObject.privateVar; //returns null as private var is private
myObject.myFunction(); //return the private var as myFunction has access to private properties

Breaking this down, we create a function which is executed immediately (via the brackets at the end) and returns an object which gets assigned to myObject.

Because this object contains references to our private variable (privateVar is referenced inside myFunction), the JavaScript engine keeps privateVar available in memory which means myFunction can still access it using what is called a closure. This pattern as a whole is usually called the Module Pattern.

Why it’s bad

On the face of it, private variables sound like a good thing. We have them in other languages after all, so why not in JavaScript too?

The reason that you shouldn’t use the Module pattern 90% of the time you think you should is that it entirely negates the dynamic nature of the language. If a class does 99% of what you want and you (rightly) don’t want to directly modify the source code, you will be thwarted every time if the class uses this pattern.

Example

I’ll share a recent example of this using a class in the Ext JS library. Ext is by no means the only library guilty of this, but it’s the one I use on a daily basis, and this is not the only example of this problem in the library.

The Ext.DomQuery object is a helper which allows us to parse XML documents locally. Unfortunately, it suffers from a limitation which causes the text content of an XML node to be truncated if it is over a certain size limit (just 4kb in Firefox, though this differs by browser). This isn’t actually a problem of Ext’s making, though it can solve it using just 1 line of code.

Ideally, we’d just be able to do this:

Ext.apply(Ext.DomQuery, {
  selectValue : function(path, root, defaultValue){
    path = path.replace(trimRe, "");
    if(!valueCache[path]) valueCache[path] = Ext.DomQuery.compile(path, "select");

    var n = valueCache[path](root), v;
    n = n[0] ? n[0] : n;

    //this line is the only change
    if (typeof n.normalize == 'function') n.normalize();

    v = (n && n.firstChild ? n.firstChild.nodeValue : null);
    return ((v === null||v === undefined||v==='') ? defaultValue : v);
  }
});

All we’re doing in the above is making a call to ‘normalize’ – a single line change which fixes the 4kb node text limitation. Sadly though, we can’t actually do this because of the use of the Module pattern. In this example there are two private variables being accessed – ‘trimRe’ and ‘valueCache’.

We can’t get access to these private variables in our override, which means that our override here fails. In fact, the Module pattern means we can’t actually patch this at all.

The only way to do it is to modify the source code of Ext JS itself, which is a very dangerous practice as you need to remember every change you made to ext-all.js and copy them all over next time you upgrade.

Even if there are good reasons for enforcing the privacy of variables (in this case I don’t think there are), we could get around this by providing a privileged function which returns the private variable – essentially making it read-only:

Ext.DomQuery.getValueCache = function() {
  return valueCache;
};

Except again this needs to be defined inside the original closure – we just can’t add it later. Again we would have to modify the original source code, with all the problems that entails.

Ext.ComponentMgr does the same trick when registering xtypes. An xtype is just a string that Ext maps to a constructor to allow for easy lazy instantiation. The trouble is, Ext hides the xtype lookup object inside a private variable, meaning that if you have an xtype string it is impossible to get a reference to the constructor function for that xtype. Ext provides a function to instantiate an object using that constructor, but doesn’t let you get at the constructor itself. This is totally unnecessary.

Recommendations

  1. Think very carefully before using the Module pattern at all. Do you really need to enforce privacy of your variables? If so, why?
  2. If you absolutely have to use private variables, consider providing a getter function which provides read-only access to the variables
  3. Keep in mind that once defined, private variables defined this way cannot be overwritten at all. In other languages you can often overwrite a superclass’s private variables in a subclass – here you can’t

Either of the above would have solved both problems, but as neither was implemented we have to fall back to hackery.

And remember this about the Module pattern:

  • It’s overused – in the examples above (especially Ext.ComponentMgr) there is no benefit from employing the pattern
  • It’s dangerous – because of its inflexibility it forces us to modify external source code directly – changes you are almost guaranteed to forget about when it comes to updating the library in the future
  • It’s bloody annoying – because of both of the above.

Next Page »