Nil Punning (Or Null Pointers Considered Not So Bad)

Ready to take your Clojure to the next level?

Deepen your Clojure skills with my Advanced Clojure Signature Course.

• 8 advanced modules
• 84 profound lessons
• 30 hours of video

buy now

Null pointers are considered by their inventor to be a huge mistake. Clojure inherits its null pointer, called nil in Clojure, from the JVM. In contrast to Java^1, Clojure seems to embrace the null pointer. In this post, I'd like to explore how Clojure uses the null pointer in what is often called nil-punning.

Nil-punning has its roots in the very first Lisps, where nil was both false (the boolean value) and the end of a list (the empty list). It was also often used to represent "no answer", as in what is the first element of an empty list. It is called punning because you can use it to mean different things in different contexts.^2

In Clojure, nil, as a value, is nearly void of meaning. And it is all pervasive, because it can be returned from any Clojure function or Java method.

Let's go through that last part bit by bit.

1. It is a value. Java made the mistake of making null a lack of object even though it was pointed to by an object pointer. You can't call methods on it. It is not an object. It has a weird nameless type. Clojure did not make this mistake. It is a first-class value and type^3, meaning it can be compared to other values, it can implement protocols, and be used as the key or value of a map, etc. Using nil where it doesn't make sense in Clojure code is usually a type error, not a NullPointerException, just as using a number as a function is a type error.

2. It is nearly void of meaning. It means "no answer", but not much more. Because of this, it can take whatever form fits the context. With proper wisdom in choosing what form it takes, nil can become an asset instead of a liability. Clojure takes nil-punning to an extreme.

nil can be many things. To name but a few, nil plays false as a boolean. It plays the empty seq as a seq^4. It plays the empty map as a map. Because nil has a role to play in most of the major abstractions of core Clojure, it rarely leads you into an error situation. An unexpected nil can surprise a good programmer, just as much as an unexpected Nothing from a Haskell function can bewilder even the most experienced Haskeller. ^5 Finding out where a nil came from is the hardest problem that nils present.

3. It is all-pervasive. nils are normal parts of Clojure programs. They are not anomalous as in Java, where you often have to check it everywhere. This means it is always on the experienced programmer's mind. nils flow like water through s-expressions.

first has nothing to return if the seq is empty, and so it returns nil. Because nil is a seq, first works on nil, and returns nil. ^6. rest works on nil as well, because nil is a seq.

These examples show the best of nil-punning. When nil-punning works right, nils are expected and they give the expected results.

nil is everywhere, but it can be used mostly everywhere as well---without error and often with exactly the desired result. There are many abstractions that nil does not participate in (for instance IFn, which is Clojure's interface for things that can be called like functions). In these places, nil can present a problem---a problem of type, the same as if you tried to call a number as a function.

The best thing to do, in my experience, is simply to wrap the expression in a (when ) to catch the nil cases, if appropriate, while also preserving it. Otherwise, perhaps letting the Exception bubble up is the best answer. If you got a nil where you couldn't use one, the stack trace is probably your best clue to where it came from.

After a bit of experience with Clojure, I rarely have difficult problems with out-of-place nils in pure Clojure code. However, there is often some Java interop---namely, calling Java methods directly---that will cause a NullPointerException if the object of the method call is nil. In these cases, wrapping a Java method call in a (when ) is often appropriate. But sometimes not, and the NullPointerException is welcome.

There are some decisions in Clojure that I think make poor use of nil-punning. These places actually make working with Clojure more difficult than they need to be. For instance, (str nil) is the empty string. Printing this out prints nothing---a form of silence, which is rarely what you want so you have to check for nils in these cases. But nil is not the empty string, like it is the empty seq. (clojure.string/trim nil) throws a NullPointerException. This is inconsistent behavior. When nil acts inconsistently, nil-punning does not work right. nils need to be checked. In the worst cases, nils fail silently. While I have learned to deal with these situations, they are a wart on the language. The fact that nils are so common does help surface the bugs sooner. A small consolation.

Let me make it clear: null pointers are still a costly problem in Clojure. But I can make a claim similar to what Haskellers claim about the type system: nil-punning eliminates a certain class of errors. A fortuitous set of decisions in Clojure has reduced the magnitude of the problem. And some decisions have made the problem worse by hiding it. In general, I find that by embracing nil-punning, my code gets better.

1. I don't mean to pick on Java alone. I just wanted to be specific.
2. Note that this is very different from weak typing as you find in Javascript or Python. Nil-punning is more like polymorphism.
3. The type of nil is nil
4. Clojure's core is built on several small, powerful abstractions. The most prominent abstraction is seq, which stands for sequence. seq basically has two operations, first and rest. The most obvious use for them is to iterate through items of a collection. There are built-in implementations for lists, vectors, sets, hashmaps, and even strings. But anything that has a notion of sequential values can implement seq, including Java Iterators. I would also like to posit that the most important and often overlooked implementation of seq is for nil itself.
5. Even the best Haskellers complain about not knowing where a Nothing came from.
6. You might look at this as nil-preserving behavior---much like the Nothing-preserving behavior of the Maybe Monad