DataStore: A JavaScript ActiveRecord-style adapter for Google App Engine

I've been spending a lot of my spare time lately trying to tie some bows around server-side JavaScript and Google App Engine. My first attempt at something useful is a library that allows you to interact with the Google Datastore from within a Rhino/GAE application using an ActiveRecord-style API. It's largely exploratory at this point. Also, at a little over 200 lines of JavaScript, it demonstrates some of the cool features of newer JavaScript versions, notably properties. It also contains some nice uses of prototype delegation. Though it has dependencies on Java and GAE, and therefore must run on the server side, the idioms used will work in Firefox, Safari 3+, Opera 9.5, and IE8.

Enjoy!

http://github.com/sustainablecode/datastore

Testing JavaScript from Ruby

In the attempt to throw some unit tests around the javascript_named_routes plugin, I really wanted something simple and fast, that didn't require any significant setup like Rhino, Selenium, etc. In my searching, I happened to come across some examples using Applescript, and I couldn't believe how simple it was to script JavaScript from a test -

require 'appscript'
safari = Appscript.app('Safari')
window = safari.documents.get.first
window.do_JavaScript(@response.body)
{
  "Routes.users_path()" => "/users",
  "Routes.formatted_users_path({format: 'xml'})" => "/users.xml",
  "Routes.user_path(123)" => "/users/123",
  "Routes.user_path(123, {foo: 42})" => "/users/123?foo=42",
  # ...
}.each_pair do |expr, expected|
  assert_equal expected, window.do_JavaScript(expr) 
end

Of course, this test will only work on a Mac - but if you're in a Mac environment, and/or can handle having Mac-only tests, it's a pretty sweet solution with zero setup.

More Unobtrusive JavaScript With JavaScript Named Routes

It's no big secret that I have never been a fan of Rails JavaScript helpers. Even when two languages have a really low impedance mismatch - like JavaScript and Ruby - there is still only so much you can do through a translation layer like RJS. Sooner or later, you find yourself resorting to either 1) rolling and maintaining your own helpers, or 2) throwing your hands up in defeat and just putting inline JavaScript into your RJS templates. Maintaining your own helpers is arguably the better of the two choices; but stop and consider - why the additional layer of abstraction, when the behavior you're trying to describe can be expressed just as cleanly in JavaScript in the first place? Haven't we learned from the Big Corporate Java experiment that One Language To Rule Them All wasn't really a very good idea to begin with?

Ok, I've made my point. I want to write my Rails UI code in JavaScript. My view templates and styles - you're all using HAML and SASS now, right? - are the glue that binds my Rails controllers and JavaScript behaviors together. I expose appropriately named elements in my views, and then use JavaScript selector engines to locate them unobtrusively and apply relevant behaviors. Everything is delightful. Roses and poppies spring up around my desk.

Until something on the page needs to interact with the server - and I suddenly find myself exiled from the Rails information universe. How do I construct the proper URL to ask for an HTML fragment - or better, a RESTful resource? Sure, I can strategically embed application URL's in links, hidden divs and spans, or even in inline JavaScript blocks. But then I'm right back where I started - JavaScript helpers and RJS templates. There must be a better way.

Javascript_named_routes remedies precisely this deficiency.

Javascript_named_routes is a Rails plugin I created to expose the named routes in your application as JavaScript functions on the client side. Setting it up is a ridiculously easy 3-step process -

• script/plugin install git://github.com/sustainablecode/javascript_named_routes.git
• add map.javascript_named_routes to your routes.rb
• add <%= javascript_named_routes %> to your view layouts

That's it.

So, let's say your application has a standard User resource. That means you already have some URL helpers available to your application: users_path, user_path, edit_user_path, etc. With javascript_named_routes, you can now access these from JavaScript, like so -

• Routes.users_path() - no parameters
• Routes.user_path(123) - one parameter, the ID
• Routes.user_path({id: 123}) - or, as an object!

Routes with multiple placeholders work the same way - you can either pass them as positional parameters, where they'll be placed into the URL from left-to-right, or you can pass a JavaScript object containing the keys from the route string. For example, the named route:

• map.show_account, '/account/:last_name/:first_name', :controller => 'accounts', :action => 'show'

Can be accessed as either -

• Routes.show_account_path('Doe', 'John'), or
• Routes.show_account_path({first_name: 'John', last_name: 'Doe'})

Enjoy!

Checkers at Github

I have started implementing a checker player in Clojure. No AI yet, but will hopefully be coming soon. I put up a little Google site, and pushed the code to Github for your enjoyment. Feedback welcome.

• http://sites.google.com/a/polgardy.com/checkers/
• http://github.com/sustainablecode/checkers-clojure/tree/master

[Update: Project now has a player, based on a simple minimax search tree algorithm.]

Recursion lab

Recursion is great programming topic to get your head around - even more critical if you've taken the functional programming plunge. In a nutshell, a recursive function or procedure is one that, directly or indirectly, invokes itself in its own calculation. The textbook example of recursion is calculating the factorial of a number. The factorial function looks particularly elegant in Haskell, and can be expressed in a single line:

factorial n = if n > 0 then n * factorial (n-1) else 1

It's easy enough to see what's going on here. The factorial function takes a single argument n, and is defined such that, if n is greater than zero, the value shall be n times the factorial of (n - 1); otherwise it shall be 1.

---

As you can see from my previous post, I generally use the game of checkers to explore how to solve problems in new languages. If you were paying close enough attention, you noticed that I actually lied, or at best told a half-truth, about the Lisp code I wrote. I said the code generated the valid moves and jumps from a given position. My code, however, only accounts for single jumps. It turns out that the deceptively simple jumping rules of checkers make for a really nice focused programming exercise.

From a given position, the rules are as follows:

1. If there is a jump available, you must take it.
2. If there is more than one jump available, you may choose which one to take.

You needn't choose the path that results in the largest number of captures, but you must continue jumping along your chosen path until no jumps remain.

What is the simplest code you can come up with, that takes a two-dimensional vector of piece values (choose whatever representation you find most helpful) and returns a list of coordinate paths representing all the valid jumps?

---

For example, given the following position with red to move, it should generate the three jumps below:

8 . . . . . . . .
7 . . . . . . . .
6 . . . . . w . .
5 . . . . . . . .
4 . . . w . w . .
3 . . . . . . . .
2 . w . w . . . .
1 . . r . . . . .
. 1 2 3 4 5 6 7 8
  
[3,1] => [1,3]
[3,1] => [5,3] => [3,5]
[3,1] => [5,3] => [7,5] => [5,7]

Checkers Rules in Lisp

I've recently started trying to wrap my head around the beast that is Lisp. I have to say so far all the rumors are true -- just trying to program in it is enough to twist your brain into seeing problems in a whole new way.

So with that minor disclaimer, I offer you my first semi-useful chunk of Lisp code: a set of functions and macros to generate the valid checkers moves and jumps given a 2-dimensional grid representing the current position:

(defvar board
  (make-array
   '(8 8)
   :initial-contents
   '(( 1  0  1  0  1  0  1  0)
     ( 0  1  0  1  0  1  0  1)
     ( 1  0  1  0  1  0  1  0)
     ( 0 -1  0  0  0  0  0  0)  ;; piece is out of place
     ( 0  0  0  0  0  0  0  0)  ;; to demonstrate jump
     ( 0  0  0 -1  0 -1  0 -1)
     (-1  0 -1  0 -1  0 -1  0)
     ( 0 -1  0 -1  0 -1  0 -1))))

(defvar *size* 8)
(defvar *red* 1)
(defvar *white* -1)
(defvar *empty* 0)
(defvar *red-piece* 1)
(defvar *white-piece* -1)
(defvar *red-king* 2)
(defvar *white-king* -2)

(defvar *side* *red*)

(defun get-p (x y)
  (aref board y x))

(defun set-p (x y p)
  (setf (aref board y x) p))

(defun is-my-piece (p)
  (= p *side*))

(defun is-my-king (p)
  (= p (* 2 *side*)))

(defun is-mine (p)
  (or (is-my-piece p) (is-my-king p)))

(defun is-opp (p)
  (is-mine (- p)))

(defun is-empty (p)
  (= p 0))

;; (multi-loop ((for y from 0 to 7)
;;              (for x from 0 to 7))
;;             (print (list x y)))
(defmacro multi-loop (loop-clauses body)
  (if (null loop-clauses)
      body
      (let ((clause (car loop-clauses)))
        `(loop ,@clause do
            (multi-loop ,(cdr loop-clauses) ,body)))))

;; (each-square (x y p) body)
(defun %each-square (f)
  (multi-loop
   ((for y from 0 below *size*)
    (for x from 0 below *size*))
   (let ((p (get-p x y)))
     (if (= (mod (+ x y) 2) 0)
         (funcall f x y p)))))

(defmacro each-square ((x y p) body)
  `(%each-square (lambda (,x ,y ,p) ,body)))

;; (each-square-for-side (x y p) body)
(defun %each-square-for-side (f)
  (multi-loop
   ((for y from 0 below *size*)
    (for x from 0 below *size*))
   (let ((p (get-p x y)))
     (if (and (= (mod (+ x y) 2) 0)
              (is-mine p))
         (funcall f x y p)))))

(defmacro each-square-for-side ((x y p) body)
  `(%each-square-for-side (lambda (,x ,y ,p) ,body)))

;; (each-jump-from x y (mx my nx ny) body)
(defun %each-jump-from (x y f)
  (let ((p (get-p x y)))
    (multi-loop
     ((for dy in (if (is-my-king p)
                     (list *side* (- *side*))
                     (list *side*)))
      (for dx in (list *side* (- *side*))))
     (let ((mx (+ x dx))
           (my (+ y dy))
           (nx (+ x dx dx))
           (ny (+ y dy dy)))
       (if (and (>= nx 0)
                (< nx *size*)
                (>= ny 0)
                (< ny *size*)
                (is-opp (get-p mx my))
                (is-empty (get-p nx ny)))
           (funcall f mx my nx ny))))))

(defmacro each-jump-from (x y (mx my nx ny) body)
  `(%each-jump-from ,x ,y (lambda (,mx ,my ,nx ,ny) ,body)))

;; (each-move-from x y (nx ny) body)
(defun %each-move-from (x y f)
  (let ((p (get-p x y)))
    (multi-loop
     ((for dy in (if (is-my-king p)
                     (list *side* (- *side*))
                     (list *side*)))
      (for dx in (list *side* (- *side*))))
     (let ((nx (+ x dx))
           (ny (+ y dy)))
       (if (and (>= nx 0)
                (< nx *size*)
                (>= ny 0)
                (< ny *size*)
                (is-empty (get-p nx ny)))
           (funcall f nx ny))))))

(defmacro each-move-from (x y (nx ny) body)
  `(%each-move-from ,x ,y (lambda (,nx ,ny) ,body)))

;; display all moves from this position
(each-square-for-side
 (x y p)
 (each-move-from
  x y
  (nx ny)
  (print (list x y nx ny p))))

;; display all jumps from this position
(each-square-for-side
 (x y p)
 (each-jump-from
  x y
  (mx my nx ny)
  (print (list x y mx my nx ny p))))

Lightweight JavaScript Events

I've found it handy on a few projects to mix in minimalistic event support to certain JavaScript objects. The basic idea is that you listen (aka "subscribe") for some event on an object, and get notified by the object when it fires (aka "publishes") that event, along with any parameters it decides to attach. This is ridiculously easy to implement in JavaScript, yet it's a really powerful abstraction mechanism for UI programming. It's even easier when you use a library like Prototype to take advantage of its Enumerable and Function extensions. Here it is in under 20 lines of code:

EventManager = {
 notify: function() {
    if (this._listeners) {
      var args = $A(arguments), event = args.shift();
      this._listeners.each(function(listener) {
          listener[event] && function() {
            listener[event].apply(this, args);
          }.defer();
        }, this);
    }
  },
 
 listen: function(listener) {
    this._listeners = this._listeners || [];
    this._listeners.push(listener);
  }
};

To add event support to a class, just mixin the EventManager extension:

Foo = Class.create({
  setX: function(x) {
      this.x = x;
      this.notify("setX", this.x);
    }
  });
Foo.addMethods(EventManager);

And then get down to it:

var object = new Foo();
object.listen({
  setX: function(x) {
      alert("x was initialized to " + x);
    }
  });
object.setX(1);

That's about it. Let me know if it comes in handy.

Lisp, Agility, Readability

I recently had the opportunity to witness a curious dialogue on the mailing list for a local user group I participate in. The original poster, a public devotee and evangelist for Agile development practices, posted an announcement to the group for an upcoming Lisp workshop. A responder immediately threw down the gauntlet – How can you possibly be advocating Lisp as a development language, when Agile is so much about code readability? Doesn't Lisp encourage some of the most unreadable code on the planet? Doesn't Lisp stand for “Lots of Irritating Superfluous Parentheses?” Doesn't Lisp ... ?

I thought about this challenge for a minute, and had an immediate sense of both the rightness and wrongness of it. After all, haven't we all beaten our heads against the wall for days trying to debug some Rails code, only to realize that the answer was buried deep in some “magic” meta-programming hook? Or spent the afternoon hunting down the documentation for some obscure method that's throwing an annoying runtime exception, only to discover that our framework kindly generated it for us at compile time based on some esoteric configuration parameter? Lisp programs seem to represent the pinnacle of this kind of self-generating, self-manipulating metacode, and they can be damned hard to wrap your head around.

At the same time, isn't it hard to beat that sublime feeling of joy you experience when you write that minimal amount of declarative code, and everything Just Works? When you can alter some piece of metacode at the global level to effect an entire architectural change, without touching a single line of code throughout your entire application? Surely there is something deeply right about these kinds of abstractions that allow us to communicate at the level of meaning, as opposed to mechanics – to separate the why of a piece of software from the how.

It struck me then that we operate under two very different senses of the word “readability,” often slipping and sliding back and forth between the two without realizing it. To caricature them down to their essence, allow me to call them mechanical readability, and semantic or intentional readability. Mechanical readability is achieved when a piece of code clearly communicates what it does, whereas semantic or intentional readability is achieved when a piece of code clearly communicates what it means. And there's the rub – clearly these two concerns are constraining us constantly, tugging at each other from opposite directions, nearly always forcing us to sacrifice one for the other in any given instance. What's the answer? Can they be harmonized into some greater concern that preserves them both?

I don't think they can be harmonized exactly, but I do think there is a bit of sound earned wisdom that applies here – a jewel in the crown of the software developer's craft. Without trying to be too scientific or dogmatic about it, it might go something like this:

• When addressing lower layers of abstraction, strive for mechanical readability.
• When addressing higher layers of abstraction, strive for semantic readability.
• Within a given layer of abstraction, strive to decouple the mechanical and intentional concerns as much as possible.

So where does that leave us as far as Lisp's alleged agility? Well, in truth, Lisp actually does address mechanical concerns far better than you might think. But its ability to express the semantic level of software is where it clearly shines. The temptation is to become obsessed with the semantic approach, to the point of obfuscation of the plain, simple mechanical tasks your software needs to perform. This is possibly the greatest temptation programmers face when given powerful meta-programming facilities. There's nothing quite so frustrating as having to work within forced abstractions.

However, don't assume a more “straightforward” mechanical style will clarify your code either. The reason your DSL doesn't flow well enough, or map onto your domain well enough, might not be that it's too abstract. It might just be the wrong abstraction. The semantic realm is a very conceptual, philosophical sort of beast. You won't write better abstractions by learning more about technological wizardry, but by practicing the dark art of simple, clear thinking.

Ultimately, it's the right ideas that set really great software apart.

JS Library Roundup

In Part III of the JavaScript Renaissance, we spent another long evening exploring Prototype, Script.aculo.us, and jQuery, and getting a feel for the scope and power of these popular DHTML/Ajax libraries. A good chunk of that time was spent hacking at the Firebug console and experimenting with live HTML documents for immediate feedback.

You can download the presentation here.

Thanks again to the Wheaton ex-Perl Mongers for coming out to learn a little more JavaScript. Come out and see the Perl Mongers reborn as the first arm of the new Polyglot Programmers group!

Addendum: Some of you at the presentation wanted to know to use Greasemonkey to import libraries into web pages after they've already been loaded. Here's the skinny:

• Go to https://addons.mozilla.org/en-US/firefox/addon/748 and add the Greasemonkey Firefox extension
• After you restart Firefox you should see the telltale monkey face icon down in the lower right corner of your browser window
• Right click on the monkey face and select "New User Script..."
• Give the new script an identifiable name like "load_prototype" or "load_jquery", a namespace (can be anything, I'm not entirely sure what it's even used for), and a list of URLs to include or exclude from running the script
• Configure a text editor to edit Greasemonkey scripts
• Finally, enter and save the script

Here's the script I used to load prototype and most of script.aculo.us. Obviously, you'll need to change the paths to point to the files where they're installed on your system. Paste this into your new Greasemonkey document below the comments:

window.addEventListener('load', function() {
    var head = document.getElementsByTagName('head')[0];
    var libs =
      [
       'file:///..../prototype.js',
       'file:///..../effects.js',
       'file:///..../dragdrop.js',
       'file:///..../builder.js'
       ];
    for (var i in libs) {
      var script = document.createElement('script');
      script.type = "text/javascript";
      script.src = libs[i];
      head.appendChild(script);
    }
  }, false);

That should be enough to get you started! Please hit me up if you have any questions.

Function Arity and JavaScript

A commenter on my last post asked about variable arity methods in JavaScript. Arity, for those unfamiliar with that term, simply refers to the number of parameters a function takes. In dynamic languages, however, arity is a more, well, dynamic concept, since a function can often be called with any number of parameters, or even with a collection to be taken as the argument list. So there are actually two different things going on, which I'll call invocation arity and declaration arity.

There's not much to invocation arity in JavaScript; you've probably seen it hundreds of times before. To determine the arity of invocation from within a method, just use the length property of the array-like arguments object:

SomeType.prototype.foo = function () {
  if (arguments.length > 2) {
    // do something requiring > 2 arguments
  }
}

Declaration arity is similar: I can ask for the length property of the function itself. So for example, if I was passed a function into some higher-order operation, I might check the function's arity to determine how to invoke it:

function foreach(array, callback) {
  for (var i = 0; i < array.length; ++i) {
    if (callback.length === 1) {
      // yield the element
      callback(array[i]);
    } else {
      // yield the index and the element
      callback(i, array[i]);
    }
  }
}

Beware declaration arity though: if your function is trying to be smart about its own arity by foregoing formal parameters and just using its arguments object, it will advertise its arity as zero:

function sum() {
  var s = 0;
  for (var i = 0; i < arguments.length; ++i) {
    s += arguments[i];
  }
  return s;
}

print(sum.length); // prints 0