Hawksworx

A little while ago, inspired by Simon Willison’s demo of Nodejs at the FullFrontal conference, I felt the urge to make something, anything, using Nodejs.

Nodejs gives lightening fast, event driven IO with Javascript running server-side in the rather nippy V8 Javascript engine. Until Nodejs, I hadn’t seen much need to bring Javascript to the server other than for testing and as a bit of a novelty.

Nodejs changed all of that. Built by Ryah Dahl with a strict philosophy that none of its procedures should ever perform a blocking operation, it has a single threaded architecture and relies heavily on the use of a single event loop. Anyone with a good knowledge of Javascript or even just a familiarity with jQuery will be familiar with the pattern of event handlers, callbacks and closures needed to build some really rather powerful things, quickly and easily with Nodejs.

For my part, I just wanted to make something to explore the ease in which Nodejs could support comet-style long polling for some real-time, collaborative task. The Web is already getting flooded with examples of chat rooms and IRC servers built with Nodejs, but I really wanted to make something that felt like a real web application, not an example.

To that end, I resurrected an old idea which I first built circa 1999 for randomly choosing someone from a list to make the tea. Not earth-shattering, but a bit of fun and simple enough for me to see through to completion. For this incarnation, dubbed Teafr.com, I piggybacked on the lists feature of Twitter to allow people to create and manage their lists of potential tea-makers somewhere they are already comfortable. My app would allow a user to recall a list of their choosing and then perform a lottery, selecting a winner (or loser) from the list at random. To add a little suspense (in the loosest possible sense) I added a basic animation which gradually got slower until the winner was revealed.

I captured a couple of videos showing teafr.com doing its stuff, one of them on an iPhone.

The fun part of this for me was in allowing this lottery and the animation to be visible to everyone viewing the same list from wherever they are. Nodejs makes it trivially simple to handle long-lived http requests so that all clients could listen out for the initiation of a new lottery event. Since this is all just lightweight HTTP, HTML, CSS and Javascript, it also runs pretty nicely on both my iPhone and Palm Pre. It is rather satisfying to stand next to someone who is viewing the site on a desktop machine, and then start their screen animating with a quick tap on your handset.

Cheap thrills, I know.

It seems to me though, that the cheapness of the thrills is what is so exciting. I’m no rocket scientist, and I managed to create this entire application, complete with real-time online views in a couple of evenings work. The short hop and low cognitive friction for a Javascript developer to be able to build these kind of things from end to end is incredibly liberating. I’m bursting at the seams with ideas for things I want to build, and am now capable of building with a new, powerful and yet familiar tool set.

The only detail which kept my new toy from the masses was having a suitable place to host it. I’ve been using Dreamhost for a while now and have to say that I’m pretty happy with them, but even with the great level of control and access to your server that they give you, you need a little bit more control to compile and run Nodejs. The easiest and cheapest way that I found to host Nodejs powered site was with a dedicated virtual server. There is no shortage of providers in this space and as Web developers get more sophisticated, the use of this kind of solution is getting more commonplace. I opted for Linode which is working out rather nicely for me, but equally, I could have plumped for a similar offering from Slicehost or Amazon Web Services. I’ll soon migrate all of my sites onto the single Linux instance that they host for me.

For now though, I need to resist the temptation to spend all of my spare time (ha!) noodling away with Nodejs, MongoDB, jQuery and various other fun things which ’speak Javascript’ that smart people on the Web keep on creating and selflessly sharing with the world for free.

The recent release of TiddlyWiki v2.5 included something rather exciting for me: JQuery, the popular Javascript library is now part of the TiddlyWiki core.

This is exciting for a number of reasons.

1. The TiddlyWiki core functions can now use jQuery to perform all manner of DOM inspection and DOM manipulation. We can refactor a ton of code to benefit from jQuery’s blistering Sizzle engine and pass the burden of maintaining lots of utility functions over to those clever jQuery bods. All of which will simplify the TiddlyWiki codebase and ultimately make it easier to read and easier to maintain.
2. TiddlyWiki plugin developers will now be able to make use of jQuery in their plugins. That’s great news for both hardcore plugin developers and people dabbling for the first time. jQuery is elegantly expressive, powerful, and superbly documented. All of this lowers the barriers to entry for a would be developer and smooths the way for exisiting developers.
3. There is a huge wealth of jQuery plugins which can now be utilised by TiddlyWiki. The quality of many of these plugins is tremendously high. Bring ‘em on!

But how does a TiddlyWiki plugin developer get started? How can we bring a jQuery plugin into TiddlyWiki and make it available via a Macro? Let’s take a look at an example.

Fred, my colleague at Osmosoft stumbled upon a plugin which creates a nice navigation structure from an HTML List element. The plugin has lots of examples and documentation and seemed like a good contender for bringing something useful into TiddlyWiki.

To include the plugin, all we need to do is copy it into a tiddler and then tag the tiddler with systemConfig. After saving the file and reloading, the plugin is available for us to use.

With the jQuery plugin availble, we can use it to provide a snazzy UI for any UL or LI elements on the page. The Javascirpt syntax for that is nice and simple:

// Turn the list element with an ID of 'myList' into a funky ListNav
$('#myList').listnav();

This is great, but we want people to be able to call this from TiddlyWiki without having to write Javascript. If we create a TiddlyWiki macro to call this for us, people can easily make a nice NavList anywhere. Let’s make a macro which turns the contents of a tiddler into a NavList like this:

<<listnav TiddlerTitle>>

To make that available, we should create a TiddlyWiki plugin which delivers this macro. We do this in a new tiddler. I created a tiddler called ‘ListNavPlugin’ and once again tagged it with systemConfig so that it becomes code that TiddlyWiki knows to interpret. To start with, let’s just create the macro and ensure that we can invoke it.

 
// create macro object
config.macros.listnav = { 
 
  // Add a handler function to be invoked by <<listnav TiddlerTitle>> 
  handler: function(place, macroName, params, wikifier, paramString, tiddler) {
 
    // do some magic...
    alert("I'm gonna make a funky listnav!");
 
  }
};

To see it working, I created two more new tiddlers. MyFruityList holds a list of items, and ExampleListNav holds a call to the new macro.

Saving the TiddlyWiki file and reloading will make the macro available to call. If we open the ExampleListNav tiddler, we should see our macro called like this:

All good, but not exciting yet. Let’s flesh out our TiddlyWiki macro a bit. We’ll use a TiddlyWiki function to get the text from our desired tiddler and then make a list from it. Where we might once have used some TiddlyWiki utility functions to help us create the HTML elements (like createTiddlyElement), we can now use jQuery’s helper functions to append elements to the document.

 
// create macro object
config.macros.listnav = {
 
  // Add a handler function to be invoked by <<listnav TiddlerTitle>> 
  handler: function(place, macroName, params, wikifier, paramString, tiddler) {
 
    // target tiddler passed in as macro parameter
    var title = params[0];
 
    // read list items from tiddler contents
    var text = store.getTiddlerText(title);
    if(text) {
 
      // generate list
      var items = text.split("\n");
      var list = jQuery("<ul />").attr("id", "listnav").appendTo(place);
      jQuery.each(items, function(i, itm) {
        jQuery("<li />").text(itm).appendTo(list);
      });
 
    }
  }
};

Now we’re getting somewhere. We’ve grabbed each line from our MyFruityList tiddler and used jQuery to turn them into an unordered list.

We can now call the jQuery plugin to render our list as a NavList. The plugin requires a little extra HTML so we can create that, and then call the listnav plugin.

 
// create macro object
config.macros.listnav = {
 
  // Add a handler function to be invoked by <<listnav TiddlerTitle>> 
  handler: function(place, macroName, params, wikifier, paramString, tiddler) {
 
    // target tiddler passed in as macro parameter
    var title = params[0];
 
    // read list items from tiddler contents
    var text = store.getTiddlerText(title);
    if(text) {
 
      // generate nav bar
      jQuery("<div />").attr("id", "listnav-nav").appendTo(place);
 
      // generate list
      var items = text.split("\n");
      var list = jQuery("<ul />").attr("id", "listnav").appendTo(place);
      jQuery.each(items, function(i, itm) {
        jQuery("<li />").text(itm).appendTo(list);
      });
 
      // apply listnav
      list.listnav();
    }
  }
};

When we save our plugin, reload the page and open up our ExampleListNav tiddler, we see the navlist. Cool! Hang on though, it looks a bit rubbish. Not like the examples we saw earlier. We have to include the CSS to style our new navlist. Easy enough, let’s use our TiddlyWiki plugin to create a sylesheet for our new HTML by adding this:

 
// add default styles (adapted from http://www.ihwy.com/labs/downloads/jquery-listnav/2.0/listnav.css)
config.shadowTiddlers.StyleSheetListNav = "/*{{{*/\n" +
  "#listnav-nav { margin: 20px 0 10px; }\n" +
  ".ln-letters { overflow: hidden; }\n" +
  ".ln-letters a { font-size: 0.9em; display: block; float: left; padding: 2px 6px; border: 1px solid #eee; border-right: none; text-decoration: none; }\n"+
  ".ln-letters a.ln-last { border-right: 1px solid #eee; }\n" +
  ".ln-letters a:hover, .ln-letters a.ln-selected { background-color: #eaeaea; }\n" +
  ".ln-letters a.ln-disabled { color: #ccc; }\n" +
  ".ln-letter-count { text-align: center; font-size: 0.8em; line-height: 1; margin-bottom: 3px; color: #336699; }\n" +
  "/*}}}*/";
store.addNotification("StyleSheetListNav", refreshStyles);

And we’re done. The jQuery plugin is turning our boring old list into a dynamic indexed list with navigation. Cool! For extra credit, we can make our jQuery a little more concise. Notice that we call jQuery by name rather than with the common idiom of $. We could simply replace each jQuery with $ in our code and that would work, but beware! There are other Javascript libraries that use the $ shortcut. we don’t want to introduce the possibility of a clash. Luckily, there is a simple way to get around that. All we need to do is wrap our TiddlyWiki plugin in a closure and pass jQuery in as an argument (as explained on the jQuery documentation site). Then we can use whatever shortcut for jQuery in our code that we like. The final code looks like this:

 
(function($) { // set up alias
 
  // create macro object
  config.macros.listnav = {
 
  // Add a handler function to be invoked by <<listnav TiddlerTitle>> 
  handler: function(place, macroName, params, wikifier, paramString, tiddler) {
 
    // target tiddler passed in as macro parameter
    var title = params[0];
 
    // read list items from tiddler contents
    var text = store.getTiddlerText(title);
    if(text) {
 
     // generate nav bar
     $("<div />").attr("id", "listnav-nav").appendTo(place);
 
     // generate list
     var items = text.split("\n");
     var list = $("<ul />").attr("id", "listnav").appendTo(place);
     $.each(items, function(i, itm) {
       $("<li />").text(itm).appendTo(list);
     });
 
     // apply listnav
     list.listnav();
    }
  }
};
 
  // add default styles (adapted from http://www.ihwy.com/labs/downloads/jquery-listnav/2.0/listnav.css)
  config.shadowTiddlers.StyleSheetListNav = "/*{{{*/\n" +
  "#listnav-nav { margin: 20px 0 10px; }\n" +
  ".ln-letters { overflow: hidden; }\n" +
  ".ln-letters a { font-size: 0.9em; display: block; float: left; padding: 2px 6px; border: 1px solid #eee; border-right: none; text-decoration: none; }\n"+
  ".ln-letters a.ln-last { border-right: 1px solid #eee; }\n" +
  ".ln-letters a:hover, .ln-letters a.ln-selected { background-color: #eaeaea; }\n" +
  ".ln-letters a.ln-disabled { color: #ccc; }\n" +
  ".ln-letter-count { text-align: center; font-size: 0.8em; line-height: 1; margin-bottom: 3px; color: #336699; }\n" +
  "/*}}}*/";
  store.addNotification("StyleSheetListNav", refreshStyles);	
 
})(jQuery);

You can explore the finished example in a TiddlyWiki.

Recently I saw Tim Stevens post on Twitter about a slide show he had built using his Liveloom application. His slide show grabs some photos from Flickr tagged with ‘Osmosoft’ and then renders them with all manner of visual effects using Flex.

While the visual effects available via Tim’s app are impressive, I’m a big advocate of open web standards and enjoy making things from HTML and Javascript, rather than using tools like Flex. My first encounter with the slide show made by Liveloom was on an iPhone where it couldn’t run. The temptation to cobble together a simple Flickr mashup which could operate in any Javascript capable browser was too much to resist.

Let’s not pretend, that there aren’t already slide show mashups galore, or that I was going to be able to rival some of the advanced visual effects that Liveloom can offer, but making a simple, attractive slide show which could run on lots of devices seemed like a fun and valuable exercise. I was also keen to see just how quickly and easily I could produce this using jQuery. I’m forever going on about how good jQuery is at manipulating the DOM, so this seemed like a good chance to demonstrate that.

I shared the result my efforts as the ‘Poloaroiderizer’ and after getting some great feedback, promoted it to its very own domain. Check it out over at polaroiderizer.com

As it turns out, the task was incredibly simple and took just a few hours one evening. The interface is rendered with HTML and CSS. As such, it will work in any modern browser. I used jQuery to request JSONP from the Flickr API. JSPONP gets you around the cross domain scripting restrictions and delivers JSON to the browser which is a snap to parse and render. There is no logic on the server. All of my code is run in the browser.

The animations are provided by CSS and DOM manipulation, made simple by jQuery’s animation functions. The display needs Javascript to render, so without it Polaroiderizer is a bit toothless, but that doesn’t mean we can forget about those without Javascript. If you submit a flickr tag while you don’t have Javascript capabilities, the form will submit your tag to the standard flickr interface and you’ll find yourself at the flickr search results page. This is another example of progressive enhancement. A topic close to my heart that I tried to demonstrate recently on Unobtrusify.

Another little bonus of building this slide show app this way, is that it is super-easy to share the results. I made sure that the tag entered was echoed into the address of the page as the page fragment identifier. Manipulating and interrogating the frag id can be useful in Ajax applications as my colleague Michael Mahemoff explained recently. When the page loads, it looks for a tag stored in the frag id as its starting point. That means you can bookmark slide shows and share them as easily as email a URI. Here are a few:

http://polaroiderizer.com/#PhilHawksworth http://polaroiderizer.com/#TheWebIsAgreement http://polaroiderizer.com/#TED2009

Recently Jon Lister, a colleague of mine at Osmosoft showed me a website made by his friend Joshua Bradley. The site, used some of the Javscript code from TiddlyWiki’s animation engine to create some nice visual effects. I loved the design, but could see some room for improvement in the implementation. I’m a big advocate of Unobtrusive Javascript and Progressive Enhancement and so I set about producing a quick demo of how a similar result could be achieved in the most Web-kind and accessible way available using jQuery for the behaviors.

The result has been published as Unobtrusify.com.

The aim.

• Create a similar effect to that on Josh’s site, but make sure that the page is readable without Javascript.
• Use images to make the headings look snazzy, but make sure that they are not required in order for the content to make sense.
• Use only unobtrusive Javascript and keep the HTML as clean as possible.
• Reduce the number of http requests required to as few as possible in order to improve performance.

The approach.

First of all, I wrote the text for the page. I chose a simple statement and tried to structure it such that it would make sense regardless of which sections were expanded.

Then I used the simplest HTML markup I could to logically represent the content with its various headings. This is how the page would look to text-only browsers search engines, web-crawlers and screen-readers.

I then used a well-known CSS technique to replace the text in the headings with images. This would ensure that the text would remain for non-human consumers of the site, while the images would be presented to those able to appreciate them. The technique is simple. You prevent the browser from scrolling the content of your element with overflow:hidden and then scoot the text out of the way with text-indent. Now that the way is clear, you can display an image with background-image. Be sure to specify the dimensions of your desired image as the background-image property will not resize your element to the correct size automatically. The CSS would look something like this:

#myHeading {
	text-indent: -9999px;
	overflow: hidden;
	background-image: url(myImage.gif);
	width: 380px;
	height: 123px;
}

My content had 6 headings to render in this way. I also wanted to have a mouseover effect to give some affordance for the click-ability of the headings so this would require another 6 images. Rather than having 12 images to download (which would require 12 separate HTTP requests), I combined all of these images into a single image. This would have a number of effects. Firstly, combining the 12 images into one meant that the total download would be a bit smaller due to the way that the file was compressed. (A tiny saving, but every little helps.) Secondly, there is an overhead with making HTTP requests so when it comes to performance, the fewer the better. This method cuts out 11 HTTP requests. Score! Thirdly, as the browser uses the same image for the original heading images and their associated mouseover images, there is no need to preload the alternate images to avoid that nasty pause when mousing over. The image is already downloaded and ready to display. A nice bonus.

In order to use this ‘image sprite’ for each and every heading, I just needed to specify the background-position for each one. Some attributes would be common to each one so I could save some code like this:

h1 {
	text-indent: -9999px;
	overflow: hidden;
	background-image: url(images/unobtrusive_sprite.gif);
	width: 380px;
}
 
h1#uj {
	background-position: 0 0;
	height: 123px;
}
 
h1#cmh {
	background-position: 0 -123px;
	height: 150px;
}
...

At this point our page looks like this. This is exactly how we want things to appear for those without Javascript. There is no ability to toggle the display of the various sections, the content is shown in full, and there is no mouseover behavior on the headings to suggest that they can be clicked (since they cannot). This is the essence of Progressive Enhancement. We have a perfectly serviceable web page (albeit a simple one) which we can now enhance for those with Javascript enabled.

Using jQuery to easily and unobtrusively add behavior to elements on the page, we can now hide all of the expanded sections. We do this with a simple jQuery statement like this:

$('#wrapper > div').hide();

This hides all of the div elements which are a direct descendent of the element with an ID of wrapper. (my chosen HTML structure).

Headings are not by default clickable so we can add some behavior to suggest that the clicking a heading will have a effect by changing the cursor for them like this:

$('#wrapper h1').addClass('clickable');

A CSS class of ‘clickable’ specifies the cursor with cursor: pointer;

We also use jQuery to show the hover image by just repositioning the background image when we hover with the mouse and also to show the hidden div element when we click a heading. Remember, none of this will happen unless Javascript is available.

I also use an additional trick to prevent a flash of unstyled content or FOUC (gratifyingly pronounced ‘FOOOOOOK’ by John Hicks) while the Javascript is being downloaded. This trick is very well explained by Karl Swedberg on the excellent learningjquery.com site.

For a better picture of exactly what is going on, why not swing by Unobtrusify.com and exercise your right as a citizen of the Web to view the source. Hitting View Source is so often the best way to learn how things are done. Go on! Go and get your hands dirty!

Once upon a time I was resistant to the idea of Javascript libraries. That was due to a couple of things. Firstly, I was comfortable with writing the Javascript for my projects myself and didn’t like the idea of relying on someone else’s code which I couldn’t easily inspect. Secondly, at the time there weren’t really any libraries. Then there were a few, but they were all, well, to be blunt, a bit pants.

That all changed for me when jQuery came along. jQuery is a lightweight, elegant but powerful Javascript library originally developed by John Resig. jQuery provides fast and efficient interrogation and manipulation of the DOM and borrows conventions from CSS and XPATH to provide concise and expressive queries to be constructed. It’s worth checking out if you haven’t already.

TiddlyWiki has been around since before the existence of Javascript libraries and long before jQuery came along, so it was never developed in a way to take advantage of such things. It could easily be argued that TiddlyWiki is itself, a Javascript library since it provides helper functions for many common Javascript tasks. The extent of this library though, is a little limited since it has evolved to serve a single purpose: To drive TiddlyWiki.

Recently I have been longing to experiment with replacing a lot of the TiddlyWiki core with code built to take advantage of jQuery. The idea of developing TiddlyWiki with a Javascript library turns out not to be a new one as similar discussions have occurred in the past and different libraries considered.

I then began to imagine other benefits from reengineering TiddlyWiki from first principles taking advantage of all of the lessons learned over its lifetime.

It became too hard to resist.

Over the course of 2 reasonably long train journeys, I set about building my own version of TiddlyWiki with jQuery at its core. I settled on a number of objectives:

1. To use jQuery to provide the core TiddlyWiki functions.
2. To make the code modular such that the core could be very small and additional functionality could easily be included via bespoke jQuery plugins.
3. To use unobtrusive Javascript.
4. For the document to be sensibly parsed by screen readers and web crawlers.
5. To allow navigation of the document even without Javascript
6. To use HTML and CSS which is valid according to the W3C.
7. To conform to the TiddlyWiki naming convention and adopt a suitably ridiculous working title for the project.

An important thing to note, is that I am not attempting to replace TiddlyWiki. I see JigglyWiki as an experimental prototype to explore ways that TiddlyWiki 3.0 might evolve.

I also hoped to keep this quiet for long enough to allow it to progress to the point that I was happy to reveal a working prototype for general discussion. That proved tricky thanks to my own excitement and the way that gossip spreads around the web and discussion groups!

In its current state, it provides some of the more basic TiddlyWiki functions in terms of displaying tiddlers and allowing editing. It also demonstrates how it might elegantly degrade when CSS or Javascript are not available. Below are a few different build which demonstrate those scenarios.

1. Just the HTML. I’ve not included CSS or any of the Javascript here. The data store is visible and you can navigate the document via the tiddler links.
2. The HTML and CSS. This will function just as the version above, only it will look a bit prettier. In environments where Javascript is not available or is slow to be initialised, this is how things look until the Javascript kicks in.
3. HTML, CSS and Javascript. Now the data store is hidden and the default content is displayed with additional, Javascript dependent functionality included.

I’d love to get comments on this approach. I’d also be very interested to get advise on TiddlyWiki issues that we might be able to avoid if the opportunity to develop this into TiddlyWiki 3.0 really did come about. I’m less interested in bug reports though. This is a very early proof of concept which will contain many bugs and glitches.

You can get to the source of this project via the TiddlyWiki subversion repository

More to come!