Stupid Nevow Tricks

Actually, this stupid Nevow trick is pretty damn clever.

So, you want to write a db-backed app with a web frontend. You plan to have, let's say, 60k users registered and you want them to be able to send one another a message via a webform by entering first and last real name as the recipient.

This could be a real pain. In the old days you'd need a separate form just to look up the user in the database first, then you can start the message sending bit. Not any more.

Check out this demo. This uses Divmod's Axiom as the database backend, and Nevow's new LivePage API called "athena" to drive the frontend. I'm very impressed with the power and performance of the Axiom database so far, and it holds up beautifully in this example. LivePage works wonderfully with it for a very compelling combination.

What does it do?

The demo database contains a single table with 10k names in it. (Locally I experimented with a 60k name database before trimming the fat for the checkin; performance did not suffer one iota.) The names are randomly generated from US Census data. When you start the demo and point your web browser at it (see the README.txt) you'll see a field. Type a name into this field; if part of what you typed has a database hit, you'll see the possible completions:



.. until you get a unique match, at which point it fills in your text field for you:


You can, of course, also click a name in the menu.

What's neat?

A lot of the interesting part is in JavaScript, so make sure you look at typeahead.html. In particular, selectRange handles selecting the untyped portion of the text field, and complete handles filling in the field and the options in the select element.

Nevow

The Nevow stuff is pretty straightforward. You create a LivePage instance as your page. Then you create elements capable of talking to the server as LiveFragments, and stuff them into the page. Here's the whole LiveFragment for the input field. This renders the input box and has a single method to handle a callRemote from JavaScript.

class TypeAheadFieldFragment(athena.LiveFragment):
  docFactory = loaders.stan(T.input(type="text", id="typehere", **athena.liveFragmentID))
  allowedMethods =  { 'loadCompletion' : True }

  def loadCompletion(self, typed):
      assert type(typed) is unicode
      if typed == u'':
          return None
      q = Person.name.like(typed, u'%')
      matches = theStore.count(Person, q)
      if matches <= 10:
          return [p.name for p in theStore.query(Person, q)]
      else:
          return None

The matching JavaScript is nice and simple:
var node = $('typehere');
var typed = node.value;
var d = Nevow.Athena.refByDOM(node).callRemote('loadCompletion', typed);
d.addCallback(complete);

(Again, be sure to read the rest of the JavaScript on the page.)

Axiom

Axiom is damn bloody simple to use. The demo doesn't really show off much in the way of Axiom features, but it does show off how simple your ORM code can be. A couple of things stood out for me.

One was that Axiom's latency is very very low indeed. At the place where I do the query I treat it as non-blocking, and for me on my one laptop it had better be. In fact, if this demo works at all it has to be non-blocking; a person typing would quickly get ahead of the database otherwise. The speed of Axiom at returning a list of 10 names from a list of 10k names is impressively subsecond. Now, I'll be the first to acknowledge that what works nice and fast for me on my laptop might not be so fast when you've got 10k live users banging away on their keyboards, but it establishes that this low latency is at least possible to achieve. I'll leave the macho scaling stuff to others.

The other was that Axiom's batch performance is pretty darn good. I left in the batchInsert function in namedb.py. The function I used to to import the original 60k names took about 7 seconds to run using Store's transact method. Be sure to use .transact() if you're going to be manipulating a bunch of rows at once though; before someone (I think JP?) pointed out that Axiom had transactions, I was trying to do the 60k inserts by creating 60k Items at one per second. It's worth highlighting that the transact method is so simple to use that as soon as I knew about it I was using it with no explanation whatsoever. Simplicity is beauty.

MochiKit

Another surprise winner here was MochiKit. I've heard so many good things about it, but until now only used it indirectly through LivePage. Even if I didn't already know who wrote it, it would have been obvious that this JS library was written by an artful Python hacker. I made use of MochiKit.DOM for constructing the items in the select widget, but I was particularly impressed with MochiKit's "bookmarklet debugging". Getting complex scripts to work in JavaScript is nothing short of horrifying for me, but having the log messaging and viewing facilities in such a nice format cut in half the time it would have taken me to get the JS right.