predd
(predicate dispatch). I
believe it is Elisp’s very first complete multiple dispatch object
system! That is, methods are dispatched based on the dynamic,
run-time type of more than one of its arguments.
(Unfortunately I was unaware of the other Clojure-style multimethod library when I wrote mine. However, my version is much more complete, has better performance, and is public domain.)
As of version 23.2, Emacs includes a CLOS-like object system cleverly named EIEIO. While CLOS (Common Lisp Object System) is multiple dispatch, EIEIO is, like most object systems, only single dispatch. The predd package is also very different than my other Elisp object system, @, which was prototype based and, therefore, also single dispatch (and comically slow).
The Clojure multimethods documentation provides a good
introduction. The predd package works almost exactly the same way,
except that due to Elisp’s lack of namespacing the function names are
prefixed with predd-
. Also different is that the optional hierarchy
(h
) argument is handled by the dynamic variable predd-hierarchy
,
which holds the global hierarchy.
To define a multimethod, pick a name and give it a classifier function. The classifier function will look at the method’s arguments and return a dispatch value. This value is used to select a particular method. What makes predd a multiple dispatch system is the dispatch value can be derived from any number of methods arguments. Because the dispatch value is computed at run-time this is called a late binding.
Here I’m going to define a multimethod called combine
that takes two
arguments. It combines its arguments appropriately depending on their
dynamic run-time types.
(predd-defmulti combine (lambda (a b) (vector (type-of a) (type-of b)))
"Appropriately combine A and B.")
The classifier uses type-of
, an Elisp built-in, to examine its
argument types. It returns them as tuple in the form of a vector. The
classifier of a method can be accessed with predd-classifier
, which
I’ll use to demonstrate what these dispatch values will look like.
(funcall (predd-classifier 'combine) 1 2) ; => [integer integer]
(funcall (predd-classifier 'combine) 1 "2") ; => [integer string]
I chose a vector for the dispatch value because I like the bracket
style when defining methods (you’ll see below). The dispatch value can
be literally anything that equal
knows how to compare, not just
vectors. Note that it’s actually faster to create a list than a vector
up to a length of about 6, so this multimethod would be faster if the
classifier returned a list — or even better: a single cons.
Now define some methods for different dispatch values.
(predd-defmethod combine [integer integer] (a b)
(+ a b))
(predd-defmethod combine [string string] (a b)
(concat a b))
(predd-defmethod combine [cons cons] (a b)
(append a b))
Now try it out.
(combine 1 2) ; => 3
(combine "a" "b") ; =>"ab"
(combine '(1 2) '(3 4)) ; => (1 2 3 4)
(combine 1 '(3 4))
; error: "No method found in combine for [integer cons]"
Notice in the last case it didn’t know how to combine these two types,
so it threw an error. In this simple example where we’re only calling
a single function, so rather than use the predd-defmethod
macro
these methods can be added directly with the predd-add-method
function. This has the exact same result except that it has slightly
better performance (no wrapper functions).
(predd-add-method 'combine [integer integer] #'+)
(predd-add-method 'combine [string string] #'concat)
(predd-add-method 'combine [cons cons] #'append)
Hmmm, the +
function is already polymorphic. It seamlessly operates
on both floats and integers. So far it seems there’s no way to exploit
this with multimethods. Fortunately we can solve this by defining our
own ad hoc hierarchy using predd-derive
. Both integers and floats
are a kind of number. It’s important to note that type-of
never
returns number
. We’re introducing that name here ourselves.
(type-of 1.0) ; => float
(predd-derive 'integer 'number)
(predd-derive 'float 'number)
;; Types can derive from multiple parents, like multiple inheritance
(predd-derive 'integer 'exact)
(predd-derive 'float 'inexact)
This says that integer
and float
are each a kind of number
. Now
we can use number
in a dispatch value. When it sees something like
[float integer]
it knows that it matches [number number]
.
(predd-add-method 'combine [number number] #'+)
(combine 1.5 2) ; => 3.5
We can check the hierarchy explicitly with predd-isa-p
(like
Clojure’s isa?
). It compares two values just like equal
, but it
also accounts for all predd-derive
declarations. Because of this
extra concern, unlike equal
, predd-isa-p
is not commutative.
(predd-isa-p 'number 'number) ; => 0
(predd-isa-p 'float 'number) ; => 1
(predd-isa-p 'number 'float) ; => nil
(predd-isa-p [float float] [number number]) ; => 2
(Remember that 0
is truthy in Elisp.) The integer returned is a
distance metric used by method dispatch to determine which values are
“closer” so that the most appropriate method is selected.
You might be worried that introducing number
will make the
multimethod slower. Examining the hierarchy will definitely have a
cost after all. Fortunately predd has a dispatch cache, so
introducing this indirection will have no additional performance
penalty after the first call with a particular dispatch value.
Something that really sets these multimethods apart from other object
systems is a lack of concern about encapsulation — or really about
object data in general. That’s the classifier’s concern. So here’s an
example of how to combine predd with defstruct
from cl/cl-lib.
Imagine we’re making some kind of game where each of the creatures is
represented by an actor
struct. Each actor has a name, hit points,
and active status effects.
(defstruct actor
(name "Unknown")
(hp 100)
(statuses ()))
The defstruct
macro has a useful inheritance feature that we can
exploit for our game to create subtypes. The parent accessors will
work on these subtypes, immediately providing some (efficient)
polymorphism even before multimethods are involved.
(defstruct (player (:include actor))
control-scheme)
(defstruct (stinkmonster (:include actor))
(type 'sewage))
(actor-hp (make-stinkmonster)) ; => 100
As a side note: this isn’t necessarily the best way to go about modeling a game. We probably shouldn’t be relying on inheritance too much, but bear with me for this example.
Say we want an attack
method for handling attacks between different
types of monsters. Elisp structs have a very useful property by
default: they’re simply vectors whose first element is a symbol
denoting its type. We can use this in a multimethod classifier.
(make-player)
;; => [cl-struct-player "Unknown" 100 nil nil]
(predd-defmulti attack
(lambda (attacker victim)
(vector (aref attacker 0) (aref victim 0)))
"Perform an attack from ATTACKER on VICTIM.")
Let’s define a base case. This will be overridden by more specific methods (determined by that distance metric).
(predd-defmethod attack [cl-struct-actor cl-struct-actor] (a v)
(decf (actor-hp v) 10))
We could have instead used :default
for the dispatch value, which is
a special catch-all value. The actor-hp
function will signal an
error for any victim non-actors anyway. However, not using :default
will force both argument types to be checked. It will also demonstrate
specialization for the example.
However, before we can make use of this we need to teach predd about
the relationship between these structs. It doesn’t check defstruct
hierarchies. This step is what makes combining defstruct
and predd
a little unwieldy. A wrapper macro is probably due for this.
(predd-derive 'cl-struct-player 'cl-struct-actor)
(predd-derive 'cl-struct-stinkmonster 'cl-struct-actor)
(let ((player (make-player))
(monster (make-stinkmonster)))
(attack player monster)
(actor-hp monster))
;; => 90
When the stinkmonster attacks players it doesn’t do damage. Instead it applies a status effect.
(predd-defmethod attack [cl-struct-stinkmonster cl-struct-player] (a v)
(pushnew (stinkmonster-type a) (actor-statuses v)))
(let ((player (make-player))
(monster (make-stinkmonster)))
(attack monster player)
(actor-statuses player))
;; => (sewage)
If the monster applied a status effect in addition to the default attack behavior then CLOS-style method combination would be far more appropriate here (if only it was available in Elisp). The method would instead be defined as an “after” method and it would automatically run in addition to the default behavior.
If I was actually building a system combing structs and predd, I would be using this helper function for building classifiers. It returns a dispatch value for selected arguments.
;;; -*- lexical-binding: t; -*-
(defun struct-classifier (&rest pattern)
(lambda (&rest args)
(loop for select-p in pattern and arg in args
when select-p collect (elt arg 0))))
;; Takes 3 arguments, dispatches on the first 2 argument types.
(predd-defmulti speak (struct-classifier t t nil))
;; Messages sent to the player are displayed.
(predd-defmethod speak '(cl-struct-actor cl-struct-player) (from to message)
(message "%s says %s." (actor-name from) message))
As of this writing there isn’t yet a prefer-method
for
disambiguating equally preferred dispatch values. I will add it in the
future. I think prefer-method
gets unwieldy quickly as the type
hierarchy grows, so it should be avoided anyway.
I haven’t put predd in MELPA or otherwise published it yet. That’s what this post is for. But I think it’s ready for prime time, so feel free to try it out.
]]>Before I dive in, this is what the user configuration now looks like,
(javadoc-add-artifacts [org.lwjgl.lwjg lwjgl "2.8.2"]
[com.nullprogram native-guide "0.2"]
[org.apache.commons commons-math3 "3.0"])
That’s right: it knows how to find, fetch, and index documentation on its own. Keep reading if this sounds useful to you.
The problem was that java-mode-plus was doing two unrelated things:
Supporting Ant-oriented Java projects. Not being a fan of Maven, I’ve used Ant for all of my own personal projects. (However, I really do like the Maven infrastructure, so I use Apache Ivy.) It seems Maven is a lot more popular, so this part isn’t useful for many people.
Quick Javadoc referencing, which I was calling java-docs. I think this is generally useful for anyone writing Java in Emacs, even if they’re using another suite like JDEE or writing in another JVM language. It would be nice for people to be able to use this without pulling in all of java-mode-plus — which was somewhat intrusive.
I also didn’t like the names I had picked. java-mode-plus wasn’t even a mode until recently and its name isn’t conventional. And “java-docs” is just stupid. I recently solved all this by splitting the java-mode-plus into two new packages,
ant-project-mode — A minor mode that performs the duties of the first task above. Since I’ve phased Java out from my own personal projects and no longer intend to write Java anymore, this part isn’t very useful to me personally at the moment. If I do need to write Java for work again I’ll probably dust this off. It’s by no means un-maintained, it’s just in maintenance mode for now. Because of this, this is not in any Emacs package archive
javadoc-lookup — This is java-docs renamed and with some new goodies! I also put this on MELPA, where it’s easy for anyone to use. This is continues to be useful for me as I use Clojure.
This is used like java-docs before it, just under a different
name. The function javadoc-lookup
asks for a Java class for
documentation. I like to bind this to C-h j
.
The function javadoc-add-roots
provides filesystem paths to be
indexed for lookup.
(javadoc-add-roots "/usr/share/doc/openjdk-6-jdk/api"
"~/src/project/doc")
Also, as before, if you don’t provide a root for the core Java API, it will automatically load an index of the official Javadoc hosted online. This means it can be installed from MELPA and used immediately without any configuration. Good defaults and minimal required configuration is something I highly value.
Back in the java-docs days, when I started using a new library I’d track down the Javadoc jar, unzip it somewhere on my machine, and add it to be indexed. I regularly do development on four different computers, so this gets tedious fast. Since the Javadoc jars are easily available from the Maven repository, I maintained a small Ant project within my .emacs.d for awhile just to do this fetching, but it was a dirty hack.
Here’s the cool new part: I built this functionality into
javadoc-lookup. It can fetch all your documentation for you!
Instead of providing a path on your filesystem, you name an artifact
that Maven can find. javadoc-lookup will call Maven to fetch the
Javadoc jar, unzip it into a cache directory, and index it for
lookups. You will need Maven installed either on your $PATH
or at
maven-program-name
(Elisp variable).
Here’s a sample configuration. It’s group, artifact, version provided as a sequence. I say “sequence” because it can be either a list or a vector and those names can be either strings or symbols. I prefer the vector/symbol method because it requires the least quoting, plus it looks Clojure-ish.
(javadoc-add-artifacts [org.lwjgl.lwjg lwjgl "2.8.2"]
[com.nullprogram native-guide "0.2"]
[org.apache.commons commons-math3 "3.0"])
Put that in your initialization and all this documentation will appear in the lookup index. It only needs to fetch from Maven once per artifact per system — a very very slow process. After that it operates entirely from its own cache which is very fast, so it won’t slow down your startup.
This has been extremely convenient for me so I hope other people find it useful, too.
As a final note, javadoc-lookup also exploits structural sharing in its tables, using a lot less memory than java-docs. Not that it was a problem before; it’s a feel-good feature.
]]>Here some some examples. This function, foo
, has three required
parameters, a
, b
, and c
.
(defun foo (a b c)
...)
To make b
and c
optional, place them after the symbol &optional
,
(defun foo (a &optional b c)
...)
If second and third arguments are not provided, b
and c
will be
bound to nil
. To provide a default argument, put that parameter
inside a list. Below, when a third argument is not provided, c
will
be bound to "bar"
.
(defun foo (a &optional b (c "bar"))
...)
To write a function that accepts any number of arguments, use &rest
followed by the parameter to hold the list of the remaining
arguments. Below, args
will be a list of all arguments after the
third. Note how this can be combined with &optional
.
(defun foo (a &optional b c &rest args)
...)
Often, the position of a parameter may be hard to remember or read,
especially if there are many parameters. It may be more convenient to
name them with &key
. Below, the function has three named parameters,
specified at the call site using keywords — special symbols from
the keyword package that always evaluate to themselves.
(defun foo (&key a b c)
...)
(foo :b "world" :a "hello")
Like optional parameters, when a parameter is not provided it is bound
to nil
. In the same way, it can be given a default argument.
(defun foo (&key (a "hello") (b "world") c)
...)
&key
can be combined with &optional
and &rest
. However, the
&rest
argument will be filled with all of key-value pairs, so it’s
generally not useful to use them together.
Lambda lists are not exclusive to defun
and can be used in any place
that needs to receive values in parameters, such as flet
(function
let), defmethod
, and so on.
Clojure forgoes these complex lambda lists in preference for
overloading by arity. When a function is being defined, multiple
functions of different arities can be defined at once. This makes for
optional parameters. Note how this leaves no room for a default
argument of nil
for unspecified optional arguments.
Here, b
is an optional parameter for foo
, defaulting to "bar"
when not provided by the caller. The first definition has an arity of
one and it calls the second definition with the optional argument
filled in.
(defn foo
([a] (foo a "bar"))
([a b] ...))
Variadic functions are specified with &
, similar to &rest
in
Common Lisp. Below, xs
is a sequence of all of the arguments
provided after the first.
(defn foo [x & xs]
...)
As far as parameters are concerned, this is all Clojure
has. However, Clojure’s parameter specification is actually more
flexible than Common Lisp’s lambda lists in two important ways. One is
that parameter position can vary with the number of provided
arguments. The Clojure core functions use this a lot (ex.
reduce
).
The following in Common Lisp would require manually parsing the
parameters on some level. The last
parameter can be either second or
third depending on whether a middle name was provided.
(defn make-name
([first last]
(make-name first "Q" last))
([first middle last]
{:first first, :middle middle, :last last}))
(make-name "John" "Public")
;; => {:first "John", :middle "Q", :last "Public"}
That covers optional parameters with default arguments and variadic functions. What about keyword parameters? Well, to cover that we need to talk about destructuring, which is another way that Clojure parameters are more powerful than lambda lists.
A powerful Lisp idiom is destructuring bindings. Variables can be
bound to values in a structure by position in the structure. In Common
Lisp there are three macros for making destructuring bindings,
destructuring-bind
, loop
and with-slots
(CLOS).
Below, in the body of the form, a
, b
, and c
are bound to 1, 2,
and 3 respectively. The form (a (b c))
is mapped into the quoted
structure of the same shape to the right.
(destructuring-bind (a (b c)) '(1 (2 3))
(+ a (* b c)))
;; => 7
Because of Common Lisp’s concept of cons cells, the cdr of a cell
can be bound to a variable if that variable appears in the cdr
position. This is similar to the &rest
parameter (and is how Scheme
does variadic functions). I like using this to match the head and tail
of a list,
(destructuring-bind (x . xs) '(1 2 3 4 5)
(list x xs))
;; => (1 (2 3 4 5))
Perhaps the neatest use of destructuring is in the loop
macro. This
loop walks over a list two at a time, binding a variable to each side
of the pair,
(loop for (keyword value) on '(:a 1 :b 2 :c 3) by #'cddr
collect keyword into keywords
collect value into values
finally (return (values keywords values)))
;; => (:A :B :C), (1 2 3)
Unfortunately destructuring in Common Lisp is limited to these few cases, or where ever else you write your own destructuring macros.
Clojure takes destructuring to its logical conclusion: destructuring can be used any place bindings are established! This includes parameter lists. It works on any core data structure, not just lists.
Below, I’m doing destructuring inside of a standard let
form.
(defn greet-dr [fullname]
(let [[first last] (clojure.string/split fullname #" +")]
(str "Hello, Dr. " last ". "
"It's good to see you again, " first ".")))
(greet-dr "John Doe")
;; "Hello, Dr. Doe. It's good to see you again, John."
Similarly, I could destructure an argument into my parameters. (Note the double square brackets.)
(defn greet-dr-2 [[first last]]
...)
(greet-dr-2 ["John" "Doe"])
Because hashmaps are a core language feature in Clojure, they can also be destructured. The syntax is a bit like flipping the hashmap inside out. The variable is specified, then the key it’s mapped to.
(let [{a :a, b :b} {:a 1 :b 2}]
(list a b))
;; => (1 2)
When variables and keys have the same name, there’s a shorthand with
:keys
.
(let [{:keys [a b]} {:a 1 :b 2}]
...)
Variables default to nil
when the corresponding key is not in the
map. They can be given default values with :or
.
(let [{a :a, b :b :or {a 0 b 0}} {}]
(list a b))
;; => (0 0)
Now, here’s where it gets really neat. In Common Lisp, the &key
part
of a lambda list is a special case. In Clojure it comes for free as
part of destructuring. Just destructure the rest argument!
(defn height-opinion [name & {height :height}]
(if-not height
(str "I have no opinion on " name ".")
(if (< height 6)
(str name " is short.")
(str name " is tall."))))
(height-opinion "Chris" :height 6.25)
;; => "Chris is tall."
We can still access the entire rest argument at the same time, using
:as
, so it covers everything Common Lisp covers.
(defn foo [& {a :a, b :b :as args}]
args)
(foo :b 10)
;; => {:b 10}
(A side note while we’re making comparisons: keywords in Clojure are not symbols, but rather a whole type of their own.)
Clojure parameter lists are simpler than Common Lisp’s lambda lists and, thanks to destructuring anywhere, they end up being more powerful at the same time. It’s a full super set of lambda lists, so there’s no practical trade-off.
]]>Personally, there’s a high barrier in place to learn new programming languages. It’s entirely my own fault. I’m really picky about my development environment. If I’m going to write code in a language I need Emacs to support a comfortable workflow around it. Otherwise progress feels agonizingly sluggish. If at all possible this means live interaction with the runtime (Lisp, JavaScript). If not, then I need to be able to invoke builds and run tests from within Emacs (C, Java). Basically, I want to leave the Emacs window as infrequently possible.
I also need a major mode with decent indentation support. This tends to be the hardest part to create. Automatic indentation in Emacs is considered a black magic. Fortunately, it’s unusual to come across a language that doesn’t already have a major mode written for it. It’s only happened once for me and that’s because it was a custom language for a computer languages course. To remedy this, I ended up writing my own major mode, including in-Emacs evaluation.
Unsatisfied with JDEE, I did the same for Java, growing my own extensions to support my development for the couple of years when Java was my primary programming language. The dread of having to switch back and forth between Emacs and my browser kept me away from web development for years. That changed this past October when I wrote skewer-mode to support interactive JavaScript development. JavaScript is now one of my favorite programming languages.
I’ve wasted enough time in my life configuring and installing software. I hate sinking time into doing so without capturing that work in source control, so that I never need to spend time on that particular thing again. I don’t mean the installation itself but the configuration — the difference from the defaults. (And the better the defaults, the smaller my configuration needs to be.) With my dotfiles repository and Debian, I can go from a computer with no operating system to a fully productive development environment inside of about one hour. Almost all of that time is just waiting on Debian to install all its packages. Any new language development workflow needs to be compatible with this.
Until last year sometime the standard way to interact with Clojure from Emacs was through swank-clojure with SLIME. Well, installing SLIME itself can be a pain. Quicklisp now makes this part trivial but it’s specific to Common Lisp. This is also a conflict with Common Lisp, so I’d basically need to choose one language or the other.
SLIME doesn’t have any official stable releases. On top of this, the SWANK protocol is undocumented and subject to change at any time. As a result, SWANK backends are generally tied to a very specific version of SLIME and it’s not unusual for something to break when upgrading one or the other. I know because I wrote my own SWANK backend for BrianScheme. Thanks to Quicklisp, today this isn’t an issue for Common Lisp users, but it’s not as much help for Clojure.
The good news is that swank-clojure is now depreciated. The replacement is a similar, but entirely independent, library called nREPL. (I’d link to it but there doesn’t seem to be a website.) Additionally, there’s an excellent Emacs interface to it: nrepl.el. It’s available on MELPA, so installation is trivial.
There’s also a clojure-mode package on MELPA, so install that, too.
That covers the Emacs side of things, so what about Clojure itself? The Clojure community is a fast-moving target and the Debian packages can’t quite keep up. At the time of this writing they’re too old to use nREPL. The good news is that there’s an alternative that’s just as good, if not better: Leiningen.
Leiningen is the standard Clojure build tool and dependency
manager. Here, “dependencies” includes Clojure itself. If you have
Leiningen you have Clojure. Installing Leiningen is as simple as
placing a single shell script in your $PATH
. Since I always have
~/bin
in my $PATH
, all I need to do is wget/curl the script there
and chmod +x
it. The first time it runs it pulls down all of its own
dependencies automatically. Right now the biggest downside seems to be
that it’s really slow to start. I think the JVM warmup time is to
blame.
Let’s review. To install a working Emacs live-interaction Clojure development environment,
Install the nrepl.el package in Emacs. For me this happens automatically by the configuration in my .emacs.d repository. I only had to do this step once.
Install the clojure-mode package. Same deal.
Install a JDK. OpenJDK is probably in your system’s package manager, so this is trivial.
Put the lein
shell script in the $PATH
. This takes about
five seconds. If even this was too much for my precious
sensibilities I could put this script in my dotfiles repository.
With this all in place, do M-x nrepl-jack-in
in Emacs and any
clojure-mode buffer will be ready to evaluate code as expected. It’s
wonderful.
I made some tweaks to further increase my comfort. Perhaps nREPL’s
biggest annoyance is not focusing the error buffer, like all the other
interactive modes. Once I’m done glancing at it I’ll dismiss it with
q
. This advice fixes that.
(defadvice nrepl-default-err-handler (after nrepl-focus-errors activate)
"Focus the error buffer after errors, like Emacs normally does."
(select-window (get-buffer-window "*nrepl-error*")))
I also like having expressions flash when I evaluate them. Both SLIME
and Skewer do this. This uses slime-flash-region
to do so when
available.
(defadvice nrepl-eval-last-expression (after nrepl-flash-last activate)
(if (fboundp 'slime-flash-region)
(slime-flash-region (save-excursion (backward-sexp) (point)) (point))))
(defadvice nrepl-eval-expression-at-point (after nrepl-flash-at activate)
(if (fboundp 'slime-flash-region)
(apply #'slime-flash-region (nrepl-region-for-expression-at-point))))
For Lisp modes I use parenface to de-emphasize parenthesis. Reading
Lisp is more about indentation than parenthesis. Clojure uses square
brackets ([]
) and curly braces ({}
) heavily, so these now also get
special highlighting. See my .emacs.d for that. Here’s
what it looks like,
The next step is actually learning Clojure. I already know Common Lisp very well. It has a lot in common with Clojure so I didn’t want to start from a pure introductory text. More importantly, I needed to know upfront which of my pre-conceptions were wrong. This was an issue I had, and still have, with JavaScript. Nearly all the introductory texts for JavaScript are aimed at beginner programmers. It’s a lot of text for very little new information.
More good news! There’s a very thorough Clojure introductory guide that starts at a reasonable level of knowledge.
A few hours going through that while experimenting in a *clojure*
scratch buffer and I was already feeling pretty comfortable. With a
few months of studying the API, learning the idioms, and
practicing, I expect to be a fluent speaker.
I think it’s ultimately a good thing I didn’t get into Clojure a couple of years ago. That gave me time to build up — as a sort of rite of passage — needed knowledge and experience with Java, which deliberately, through the interop, plays a significant role in Clojure.
]]>