ClojureScript and Node.js – an experience report October 26, 2012

ClojureScript on Node.js is a (potentially) compelling story for writing scripting apps with Clojure.

I am just now finishing up the first of a handful of example apps to help tease out the approach, tooling, and ClojureScript patches needed to make Clojure a viable option in this regard.

git-ttt

git-ttt is a git-backed, text-based, ticket tracker written in ClojureScript, making use of EDN as the storage format, supporting datalog-like query driven by core.logic, running on Node.js.  I pushed an early glimpse of it to github and published an intro screencast to allow those that are curious to see it in action.

Two interesting pieces of the project are its use of protocols and the approach to avoiding constant callbacks.

Protocols

“Program to an interface, never to an implementation.” and “Favor composition over inheritance.”  – Gang of Four

Protocols enable developers to open their systems for extension, while also allowing others to compose targeted functionality ala carte.  Shoreleave’s pubsub abstractions and git-ttt’s search abstraction are two concrete examples to see these benefits.  In Shoreleave, one can extend the full functionality of the pubsub to any object in ClojureScript1 and extend the pubsub bus abstraction to any implementation.  Participation in this system is open to all, from all directions.

In TTT, the search protocol specifies a function to compare two attributes, while falling back on equality in the default case.  This allows plugins to extend search functionality and behavior.

Blocking Deref

One challenge when working with ClojureScript is staying true to Clojure style and programming with values, while also conforming to JavaScript’s callbacks.  One strategy in the browser is isolating those cases and placing them at the end of a threading macro.  Node rests its foundation on callbacks, so a strategy built around promises becomes useful.

Promises within JavaScript provide freedom from the constant callback concern, enable proper exception handling within the context of the exception, return the programmer to focusing on values, and allow for styling and composing ClojureScript like traditional Clojure code.  The pattern I’ve fallen back on is a combination of dynamic vars, promises, and a function called blocking-deref.

Scripting applications usually have some global constants (for example a file path string), which you might need to set from an external system call.  A more concrete example- TTT needs to know some of the user’s git settings, which are fetched with the git command via Node’s system exec call.  The result (a map of git config keys to their values) is bound to a dynamic var and the rest of code is written against that dynamic var.  To actually get that result though, we use an atom, a promise, and the threading macro.

First we write the system call to git in a standard callback style, except an atom will contain the eventual result.

1 2 3 4 5 6 7 8 9 10 11
(defn git-config-atom
"Call the `git config` command to grab the value of a given key.
Return an atom that will hold the return/output string in a map {k output-value}.
Optionally pass in an atom (allowing you to build up many returns)"
([k]
(git-config-atom k (atom {})))
([k ret]
(do (sys-exec (str "git config --get " (name k))
(fn [_ out _]
(swap! ret assoc (keyword k) (cstr/trim out))))
ret)))
view raw gitcore.cljs hosted with ❤ by GitHub

Multiple requests are threaded, passing the atom along and building up the data.

1 2 3 4 5 6 7 8 9
;; This is the direct Result object you can use to access
;; a git config map
(def git-config-res
(let [res (goog.result.SimpleResult.)]
(->>
(git-config-atom "user.email")
(git-config-atom "user.name")
2)))
res))
view raw configmap.cljs hosted with ❤ by GitHub

A promise awaits the values in the atom.  Blocking-deref just ticks away Node’s internal process until the promise is fulfilled.

1 2 3 4 5 6 7 8 9 10 11
(defn blocking-deref
"Given an atom and a Result object, attempt to cycle the process tick on derefing until `pred-fn` isn't true
By default, `pred-fn` is nil?
Once the condition doesn't hold, the Result will be set to @a, and @a is returned"
([a r]
(blocking-deref a r nil?))
([a r pred-fn]
(if (pred-fn @a)
(node/process.nextTick #(blocking-deref a r pred-fn))
(do (.setValue r @a)
@a))))
view raw blockingderef.cljs hosted with ❤ by GitHub

This overall approach is similar in notion to how transients work.  From a top-level, the value of the promises set the dynamic vars or are passed in before kicking off the main application loop.

1 2 3 4 5 6 7
(defn -main [& args]
(let [git-root core/git-root-res
git-conf core/git-config-res]
(result/wait git-root
#(binding [core/*repo-root* (.getValue git-root)]
(result/wait git-conf
(fn [] (main-control git-conf args)))))))
view raw main.cljs hosted with ❤ by GitHub

Scripting Clojure apps on Node.js is a real possibility and a complete pleasure by using the platform (Google Closure), the host (Node.js), and the language cooperatively.

 

 

  1. By default Shoreleave extends the abstraction to functions and atoms, allowing construction of reactive dataflows []
  2. fn [a] (blocking-deref a res #(< (count %) 2 []