What is Nil Punning?

Nil punning is a feature of Lisp. It began when nil was both the empty list and the false value. Two meanings for the same thing led to the idea of punning. Lispers liked it. And now, Clojure carries the tradition forward. In Clojure, nil has different meanings in different contexts. Is it good? Is it bad? We explore it in this episode.

Transcript

Eric Normand: What is nil punning? In this episode we're going to go over the benefits, some of the disadvantages of nil punning and dive into what this is all about — its origins and things like that.

My name is Eric Normand and I help people thrive with functional programming. The folks at Functional Design in Clojure recently, a couple days ago, put out an episode called, "What is nil punning?" Same as my episode.

It's a great episode, you should go listen to it. I wanted to throw in my two cents into this discussion. Think podcasts could be a lot more back and forth between different podcasts, just like blogs used to be back in the day. This is my addition, my response to their episode.

Why is it called nil punning? I think that's important to talk about. When you have a word that has two meanings, that word can be used in some sentences, some wordplay where the sentence now has these two meanings. It's a fun thing that the same sentence can be read twice in two different ways.

That's what nil punning is, that nil has different meanings in different contexts. You can treat it like different things in different contexts, and have the same wordplay going, same kind of pun. That's why it's called punning.

The original nil punning was in Lisp, as far as I can tell. The idea in Lisp was that nil represented the empty list. Nil was the empty list. When you had these cont cells that would point to the next list, the last one would point to nil to represent, "OK, that's it. We're done. You can stop, you know, looping through these cont cells."

Then nil was also false. If you had an if statement and one of the things would turn nil, it would go to the next line. It would go to the else branch. You could pun it. It was this thing that represented both the empty list and falsehood.

Then, another thing that happened was, well if nil is false, you have an if statement that the test is a nil expression, returns nil, that makes everything else must truthy. Everything else must be a true value. Then, it would do the then branch.

You had this cool thing where you could say, "If tale, or if cutter of this list, and if the cutter was nil," you didn't have to say, "equals nil, cutter of the list." You didn't have to do the equals nil or not equals nil because it was a Boolean also. If you had it nil represented no answer. It's a third meaning for it. Not only is it, there's the empty list, it's also no answer.

Now you could say "Do I have an answer?" just by naming the thing. People liked this in Lisp. It felt really nice and natural. There's, in fact, a whole poem about, what would happen if you systematically got rid of the nil punning in Lisp, and how much of your code would grow because you're constantly having to check if this is nil?

You compare it to nil, instead of just, this nice, elegant if x, meaning if I have an x, if x is not nil, if x, do this. Otherwise, do that. It explodes. Nil punning has been there for a long time. Fast forward 50 years to a Clojure and there's a lot more than cont cells and atoms, symbols, the few things that the original Lisp had.

There's a lot more, and especially, there's a lot more data structures. Things are based on interfaces, abstractions. You have the sequence abstraction. You have the associative abstraction. There's more. Now, it's like, what does nil mean in each of those contexts?

In Clojure, if you use a nil value in an associative context, let's say you associate a new key value pair onto a nil, it will promote it into a HashMap. If you conj onto a nil, you treat it like just a generic collection that can add new items to it, that nil will be promoted into a list. That's the round braces list, etc. Every abstraction has a meaning for what you do with a nil.

This is another of punning and now nil has all these different meanings in different context. It's fun, you pawn, you pun. It did change stuff though. Clojure got rid of nil being the empty list and nil being the falsey value. Nil is not the empty list. They're not the same.

There's an empty list which is when you print it out, it's two parens. You can just type two parens and it'll make the empty list. The empty list is a truthy value, while nil is a falsey value.

Also, because Clojure's on the JVM, Clojure has Booleans. It has a true and false value, a type that is true and false. Nil is not equal to false. They're different. Nil is still falsey. Nil and false are the two values that will fail a test in an if statement, but they're not equal.

That's one of the objections that people have to nil punning. Is that they say, "Isn't this just another version of weak typing?" I would argue no, it is not a version of weak typing. Nil is just a value that is not equal to anything else.

When people are talking about weak typing, they mean stuff like how in JavaScript, you will do stuff like zero and you use the double equals so 0 == 0. Obviously, that's true. It should be true. 0 == the array with zero in it. The singleton array with zero in it. That's true.

Then you do 0 == false, then that's true. How is zero equal to false? Does that mean that false is equal to the array with zero in it? It just doesn't compute. In Clojure, that doesn't happen. We don't compare across types in that way with equals.

What nil is, it's a value that has a meaning in this different context, just like a HashMap has a different meaning from a vector in the same context. This nil value in that context has a meaning. It's like a type. It has its own implementation of those different interfaces.

They believe me on the weak typing. Let's assume that they believe me, hypothetically. It's not weak typing.

There's another issue which they bring up and I bring it up too. This is a real issue. You don't know where the nil came from. You have some nil and just some background.

In Clojure, since nil has a meaning in every context, almost every context, what happens is a nil value is perfectly valid. Nothing's going to throw an exemption because it's there and it keeps working like any other value that's valid. In fact, nil is valid in way more context than any other value. At some point, will throw an exemption.

You try to do something with it. You call a method on it and now you get a null pointer exemption. Where did that come from? This value started way up here and got changed here and then this other thing replaced it. Where in that chain did the nil come from?

You could say that this is the same problem that you have in Haskell with maybe. In my experience with Haskell, when you have a maybe and you're using it monastically so that a nothing, so the version that's the analogy with nil in Clojure, you have nothing.

Once you have a nothing, that whole maybe is going to be a nothing. The whole chain, so you're chaining all these computations. Once you have a nothing somewhere in that chain, the rest of it is going to be a nothing. You just get a nothing at the end and you don't know which link in the chain caused it to be nothing.

You have this problem that you don't know where it came from, but that's not really what they're saying. That is a minor inconvenience, but that's not really what they're saying. What they're saying is in Clojure, you don't even know that nil was possible in that chain. Whereas in Haskell, you knew that nothing was possible because you were using a maybe type. The type checker was telling you, you can have a nothing.

Whereas in Clojure, you called some Java methods and you did some stuff, at some point somewhere, you got a nil and you didn't even know. The semantics of Java is that you can return nil from anything that returns an object. It just appears out of nowhere, then it flows through your program, and you don't know where that nowhere was.

It appears out of somewhere. [laughs] It's not out of nowhere. It's out of somewhere, but you don't know where it was, and it's very hard to track down. It would be much nicer if there was some kind of error semantics for nil somewhere so that it would have caught it sooner, at development time.

You would have been like, "Oh, boom. I called this method, it returned nil, it explodes, and tells you exactly where that nil came from." It doesn't do that. The nil will flow through and have different meanings in all the contexts it goes through. Then at some point, you get the answer. You're like, "Where did this nil come from?" That is a problem in Clojure.

Why do we like nil punning in Clojure? We have to take a look at the JVM. The JVM has nulls. It also has this problem of methods returning null that are not part of the typed signature. Nothing says, "This method's not going to return null." In fact, a lot of methods do return null when you wouldn't expect it to.

In fact — I know it's not a best practice, but it happens — Something will say, "Well, I return a list of answers, but if there's no answers, I return null." It's unfortunate. This happens in JavaScript, too. It returned null. In ClojureScript, we have the same semantics of nil punning.

What's annoying is that when you get the empty list, you can run a method on it. If you got a list, you can run a method on it. You can map over it, or whatever you can do in JavaScript, you can run a map on it. In Java, you can do other things. You can get an iterator from it or something. Because they'll have the possibility of returning null, you have to check before you run a method on it. You say, "If this is not null, then iterate over it, and do this thing with every element."

Where in Clojure, we treat the nil like an empty collection. It's the same code. We just say, "Iterate over this." Whether you get an empty collection, a collection with elements in it, or you get a nil, the code is the same. In a lot of cases, this is better.

Considering the constraints of the JVM, this does solve a lot of the problems. Not every problem has that problem I just talked about, but it solves so many problems. I run my site on WordPress, so it's PHP. This problem is everywhere.

Some functions return false. Some functions return null. Some functions return an empty array. You have to check before you iterate over a thing every single time. The thing is, you're going to forget one time. If you forget that one time, then, your software crashes. The page doesn't load.

It becomes an idiom. Is this thing empty? No. OK. Now, I'm going to iterate over it. Why don't you just iterate over the empty thing? In Clojure, it's built into the language. You could put some wrapper around it that returns it like turns a null into an empty array. Something like that would be really helpful.

In Clojure, it's built in that nil because it has a context in this as an iterable thing. We call it the sequence abstraction because you can iterate over it like you can iterate over a collection. We don't have that same trouble. Like I said, this solves a ton of problems. Not all of them.

Sometimes it can get awkward because nils and false, they're both falsey and you want the false values, you don't want the nils. You have to be very choosy about how you pick out values from a collection. If you just OR them all together, you lose all the nils and falseys in the same.

There's problems and you got to be careful with them, but it does solve a lot of problems at the same time. I think I left something open that I can't remember now. I'm just going to close this out and hope I'll think of it before I finish the episode.

Nil punning basically means nil has different meanings in different contexts. You can play with the different contexts, same value in different contexts, and treat, for instance, if you're in a Boolean context, if you're doing an if statement, you can treat it like a Boolean. If you're doing an iteration, you can treat it like a collection, etc.

It makes things nice. We like it. It makes the idioms that we use very concise. The original nil punning was just the two things is that nil was the empty list and it was a falsey. I guess the third thing is that it represented no value, if you had no answer.

They kind of work together. Falsey is like no truth. [laughs] The empty list is no rest of a list, so there's no answer to the question, what's the rest of the list? They kind of work together and it was a very tight little thing in the original Lisp.

Clojure has a lot more abstractions, a lot more context, basically, for nil to have a meaning. Nil takes on meanings in all those contexts. Nil becomes a thing that can be passed around and be meaningful.

Unlike in Java, for instance, where the only thing you can really do with a value that might be no is check if it's no. Is this thing no before you move on to calling methods on it?

It's not week typing because we're not saying that nil is equal to false. We're not saying that nil is equal to the empty list. It's simply that like different types, it has different meanings in different contexts.

Just like a HashMap has a different meaning from a vector. Even though you can iterate over both of them, it has different meanings. The nil also has a meaning in that context.

There is a problem which is that you don't know where the nil came from. Some function, some method that you called, somewhere in your code produced a nil and you weren't expecting it.

Often it will get threaded through till the end of the calculation, and you have it there. You don't know what to do with it. You don't know where it came from. You didn't expect it. Haskell does not have nulls, but it does have maybes, and the nothing is the most analogous thing to nil or a null in Clojure.

At least you know that nothing is possible in Haskell because it's part of the type, and the types are consistent. I didn't think of the thing. Maybe I'll have to do another episode.

If you like this episode, you should check out the Functional Design in Clojure episode that came out. It's episode 47, came out September 20th. It's called, "What is nil punning?" It's a good podcast. You should subscribe to it if you're into Clojure.

They have a real knack for explaining a lot of stuff in Clojure but over voice, which is not the easiest thing to do. If you like this episode, you like this podcast, you can find the past episodes at lispcast.com/podcast. There you'll find audio, video, and text versions of all of the past episodes. You can consume it however you like.

You'll find some links to subscribe. If you want to subscribe to the podcast, or to the videos, it's on YouTube. You can also just get the text as RSS, whatever you like. You'll also find links to social media where I love to get into these discussions. Nil punning is a touchy subject. I try to be careful. I don't think it solves all problems.

I think it solves a lot of problems. Some people are so fed up with it. They're like, "I want types. Get me out of here." Let me know what side of the fence you are on in that. Do you like types? Do you think types solve this problem once and for all? I think that this is a big topic, so I'd love to hear your opinions on it.

Awesome. I'm going to close out now. I really thought I would think of it. This has been my thought on functional programming. My name is Eric Normand. Thank you for listening. As always, rock on.