Where does the power of nil punning come from?

Nil punning does give power to Lispers. But where does the power come from? Is it that nil really is a great value? Or is it more about the design choices made? In this episode, we explore this question.

Transcript

Eric Normand: Where does the power of nil-punning come from? In this episode, we're going to explore the power of nil-punning that we went through in the last episode. I'm going to give my recommendations for how to make the best use of nil-punning.

My name is Eric Normand, and I help people thrive with functional programming. This episode is going to be like the last one, very focused on Clojure and Lisp, in general.

I think it's still a good talk about like programming language semantics and I hope you learn something even if you're not into Clojure. We're going to touch on JavaScript and Haskell as well.

In the last episode, I talked about what nil-punning, why it's in Clojure and Lisp, its origins, and why we like it. In this one, I want to talk about something that I've...It just slipped my mind at the end of the episode that I wanted to come back to, and now I'm coming back to it.

That is, where does the power come from? Is nil-punning by itself a powerful idea? I don't think it is. Before we get into that, what is nil-punning? Nil-punning is simply that nil has different meanings in different contexts. You can do a pun where you are using a value that could be nil in all these different contexts, and relying on the different meanings in those contexts.

For example, nil means false in a Boolean context. It's not equal to false, but it means false. If you have an if statement or an if expression, if you have a nil for your test, then it will use the else branch, just like a false would. What that means is if you get a return value from a function, you can say, "Did I get a value?" Nil represents no value. "Did I get a value?"

Everything will be truth-y in that context, except nil and false. If you're not expecting a Boolean, you can play with that, that difference of context. Whereas here it means no value, but here it means false. That's nil-punning.

That in itself, the fact that it has different meanings and different context, is not that interesting. It doesn't give you a ton of power. What does give you the power is that every time — Let's imagine you're designing this language — Every time you make a new context, in the semantics of your language, you have a decision to make. What does nil act like in this context?

You can make a good decision, meaning one that's aligned with design goals of your language. You can make a bad decision, meaning it's inconsistent design goals, and it makes the language wordy, it makes it harder to use. That's where the power comes from. It comes from all these decisions. Did you make a good decision here?

I would argue that in Clojure, a lot of the decisions were good, enough where it feels very nice to use nil-punning. Enough of a decision, not all of them, but enough. There are a lot of places where it causes problems. For instance, one of those is — I don't know if you'd call this nil-punning, but it has the same effect, same consequences. If you call the keyword function...Here, I'm going to do this right now at a REPL — If you call the keyword function with nil, it returns nil. What's happening?

Keyword, basically, it takes a string, and it returns a keyword with that string. Keyword is just a wrapped up string. If you give it something that's not a string, or isn't easily turned into a string, like a symbol, what does it even mean to call keyword on it? If you pass in a number, a Boolean, or a list, what does it mean to call keyword on this? It doesn't have a meaning.

What happens in Clojure is nil is returned. If you pass in a nil, same thing, nil is returned. It is just about how keyword is implemented, and the Clojure's design decision not to check arguments.

A lot of dynamically type languages, one thing they do is they check the arguments as they come into a function. They say, "Is this a string?" No. Throw an exception. Clojure doesn't do that. It lets the expression...It's like, "Is it a string? Is it a symbol? Is it another keyword?" then nothing else.

The defaults for the nothing else case is returned nil. It falls off the can't and a nil comes out of the function. It's not even like a designed thing, but it does have the same effect, which is that, in the keyword context, nil gives you no keyword which is represented as a nil.

Nil that comes in, that comes in some random place gets passed through a keyword, doesn't throw an exception, it comes right out, goes to some other function, gets called there. At some point, it's going to cause a problem, but you don't know where the nil came from because it flowed right through that keyword function.

You would expect it to blow up or something. Some languages, it would. This is one of those problems, where I assert that it is a nil-punning or whatever you want to call this thing that a keyword does. It's the wrong decision. In general, like I said, I think Clojure did a really good job.

Rich Hickey did a really good job of designing what nil means in each context so that most of the time, you feel like nil-punning is a good thing. Now I want to talk about my recommendations for nil. It's just one overarching big recommendation because when you're coding, you're building out a domain model.

You're building out systems. You're making all these contexts. You're making new contexts, and you have to decide what every kind of value means in that context. Sometimes nil makes perfect sense in the context. Usually it's like a low level context. Nil is a low-level value.

It has no meaning except, "I don't have a value for you." It's a pointer that points to nothing. You have this nil. Sometimes that makes sense in your context like, "I don't know what to do," or "I do know what to do with a...like if it's an empty collection." Yes.

If you're talking about a collection context, nil being an empty collection, that makes a lot of sense, but it doesn't always make sense.

My recommendation is, one, if it doesn't make sense to give it a meaning, don't give it a meaning or give it an error exceptional meaning where it throws an exception so that that nil doesn't get bubbled through. Don't try to be too clever and say, "Nil means this, so I'm going to use it like that." No. Do not do that.

Make it clear that nil is too little...It doesn't have enough meaning attached to it to be useful in this scenario. Check and throw an exception. Don't let it through. You're not writing the low-level Clojure code. You can make different design decisions for your application.

The second part of that is, what do you do? When you have something that is like no answer or whatever, I say use an explicit value. Don't use nil to represent, "I didn't get an answer from the server." Use a keyword or some other thing that's not nil, that encodes meaning because as soon as you say, "I didn't get an answer from the server," meaning there was a time out when I made the request. "I didn't get an answer. The server was down. I didn't get an answer from the server."

It's a meaningful answer because as soon as you do that, you're going to then say, "What if we do get an answer from the server?" but the answer is, "I don't know the answer to that question." You got the response, and the server said, "Don't know." That could happen, too. You need to encode that in some other way.

Too often, we overload nil because it's usually convenient, and it seems to make sense. When you look at each one in isolation, if there's a time out, of course, no answer, so nil. If the server says, "nil," then, yeah of course, nil.

When you put them together, you've lost information because very often in the domain, you're working in a positive answer from the server that says, "I don't have an answer," is very different from, "I couldn't even reach the server. I don't know what the server will say." That's different. That's a question mark.

Where's the other one is a positive not no. You might want to encode that in your domain. In fact, you probably do want to encode that in your domain. This is very much like the null object pattern in object-oriented programming.

Null object would be to instead of using a null, you make a new class that implements the same interfaces that represents no person. The classic example is, you need to display employee information on a screen.

Someone types in...They have a box, they can type in the ID, and then boom, it shows the employee information for that employee ID. What do you do if they type in the ID, and there is no employee to show? You return null.

Now everywhere, you have to check is it no? Yes? Then print this string in the first name box like, "No person" or whatever even empty string, but you have to have in if else everywhere to check if it's no, put this string. Otherwise, call the first name method on the object, and then put that there.

You have this if statement that's like everywhere, and it's the same check. This is no. This is no. This is no. What they recommend is make an object that represents, "No person found." All the responses to the methods are indicative of that. First name would be like NA. Last name, NA. Salary, zero or whatever it makes sense for your UI.

You make that thing. You don't have all the checks in your UI. You can just display whatever the answer is. You just call the method in. It never returns a null, so you're good. It's also similar to something that happens in Haskell. For very local things, sure, use some maybe number, it's fine.

At some point, you're going to want to encode more meaning than nothing can encode because you have just the number or nothing, and nothing isn't meaningful enough. It just means, "I don't have a number." You make a new type, and it has the number, in those cases when you have the number.

It has other constructors that indicate some other meaningful information, like same example I had before. You couldn't get in touch with the server versus the server answered that it doesn't have any answer. Those are two different cases and you might want to incorporate them into your domain instead of collapsing everything to the meaningless, no.

That's my recommendation is, use nil only if it really truly makes sense. Otherwise, you'll want a more meaningful value. If it doesn't make sense, check it as soon as it doesn't make sense and throw. Just don't let it in. Don't let the nil in, and then use these other values, could be a keyword.

It's Clojure, you're pretty flexible with what you use, but something with more meaning like a keyword because it's a string. You can read it.

We talked about the power of nil-punning. It comes from the good design decisions. If you don't have good design decisions, nil-punning can be way worse.

In fact, we see that in JavaScript where null is equal to zero, and it's equal to false. It's just a mess. Instead of saying nil is its own value, that is not equal to false, but in a Boolean context, we're going to treat it the same as we treat false. JavaScript equates them. It just causes trouble everywhere.

They're not the same. They're different values that happen in this context to have the same meaning. It's a very different thing. JavaScript got that wrong. We all recognize that. That's why everyone recommends using the triple equals because that double equals does this like cross-type equality, which is just weird and causes a lot of trouble.

If you like this episode, please listen to the last one which was about nil-punning. I go deeper into that. You can go to lispcast.com/podcast. Find links to all the old episodes including audio, video, and text transcripts of all of them. You'll also find links to subscribe. You can subscribe to any of those three media types.

Also, you can find me on social media so there'll be links to my Twitter, my email, also LinkedIn. Please get in touch with me. I'd love to hear your horror stories with nil-punning in Clojure or your success stories, if you have some of those. This has been my thought on Functional Programming.

My name is Eric Normand. Thank you for listening, and as always, rock on.