Is your layer of indirection actually useful?

There's a cliche: any problem can be solved with another layer of indirection. That's true, but does your brilliant idea for a new layer of indirection actually solve a problem? In this episode, we explore this question and develop a rule of thumb for evaluating layers of indirection.

Transcript

Eric Normand: Is your layer of indirection actually useful? Do you really need that one more layer of indirection?

In this episode I want to incite, inspire, engender a deeper appreciation of indirection and specifically what it's useful for and when you should avoid it. My name is Eric Normand and I help people thrive with functional programming.

This is an important topic, indirection is one of our most useful tools for having a place where a problem is solved. It doesn't actually solve it, but it gives you a place to put the solution. It also helps us separate concerns that have a little piece of indirection, a little trick that lets you separate two things.

Also for encapsulating modules from each other so that they can change independently. That's what interfaces are. They're like a piece of indirection. Instead of just like, "Oh, I know how that thing works," and reaching into its memory and just changing a thing. You say, "We're gonna go through this interface."

Even if it's a tiny bit less efficient in terms of runtime, it's better because I can always change how it's implemented behind the interface without having to figure out everywhere. That's a layer of indirection.

It's useful. I want to start the discussion with the cliché. It's cliché at this point. That all problems, any problem can be solved with another layer of indirection. I think it's true. I've tried to think of problems that indirection, like another layer of indirection couldn't solve. Sometimes it's not indirection itself that solves it but the layer gives a place to solve it.

It gives you a place where it can be solved. Make you go through another not one more little decision, one little if statement more, and I'll have a little layer where that if statement happens. It'll just keep everything clean, but it's got still another layer in there.

That's important and it's very useful. The problem is that indirection has costs.

Let's not even talk about the runtime costs of indirection which are real but in terms of making your code harder to understand, it's indirection is one of the big deals that makes things hard to understand.

Because if you're debugging something and you need to know how did this value get calculated and it went through 10, 20 layers of indirection to get calculated, it's going to be hard for you as a programmer with your...OK, I'll talk about myself. I have limited mental capacity. I'm already overtaxed with all the stuff I have to think about.

Now I have to trace how this value went through all these different layers, what interactions there were. How did this get decided at each step? It's a difficult task to do.

If it was all laid out in one place like I said only first to do this and you decide that then you decide that then you decide that and then there you have answer, that would be much easier. Instead, indirection tends to — because it's separating concerns — spread that flow of control of the data into different parts of your code.

That makes it hard. It also tends to hide problems. One thing that my friend always talks about is how caching, which is the kind of indirection. It hides problems. If you're getting the wrong answer or something is not looking right on your Web page, I do front end stuff all the time.

Sometimes I'm like this is not the right answer that I expect, given that I just changed the server or what do I do. I'm going to do a hard refresh and say no more caching, just give me everything new. That solves the problem a lot of times, so much so that I think I always hard refresh. I never do like a cached refresh, because I don't want to deal with that.

If you've got caching on, it could be that you're hiding that problem. You don't see the problem.

In general, it makes it harder to figure out where the work is happening. In fact, I remember someone talked about in small talk, because they have factored things out into such small units, such small classes that it always felt that the work is always done somewhere else.

Where is the work actually done? It's always somewhere else. Whatever code you're looking at, it's going to send four messages out [laughs] to get the work done. Then you're like, "OK, let me go look at the code for those methods and you go look at the methods." It's like, "Oh, they're sending four messages out."

Where does it actually get done? Where does the step-by-step algorithm...where does the work happen? It's impossible to find. It always seemed like there was another layer of indirection going on. That's frustrating. That can be tough.

My experience tells me after so many years working in software, working in the industry, is that it is true that any problem can be solved in other layer of indirection. That we need to amend that, which is to say if you don't have a real problem you should not be adding a layer of indirection.

If you cannot name the problem you are solving with that layer, you don't need it. I've seen people add queues between things for no reason. They build an interface around something for no reason. Those things all add up and make it harder and harder to understand. This is one of the things that happens a lot with the YAGNI principle, "You Aren't Gonna Need It."

Very often, us programmers, we feel "I might need this" so let me build it now while I'm here. Usually, you don't need it. You can imagine all the many situation you might need it or imagine situations where you will need it. They're all imaginary and they all seem really important.

You're probably not going to need it and you should wait on it before you add the indirection unless it has zero cost. That's another, I guess, caveat to that appétit. As an example, in Java, the common practice is you make a class and then you put all of these fields and you make them all private. Then of course, you need to change them so you make getter methods.

You need to get the value from the getter methods. All they do is return the private fields. It's silly. You just made a private so that you can open it up with a getter.

When you ask a Java programmer why they do that, why do they do all of this work when no one is even calling these methods yet. This is the first time you're writing the class. You've already built all this indirection. What they will say is, one day, maybe, we will need to replace this Get X, which just returns X with some calculation.

Instead of just returning the value directly, we will want to calculate it from two other fields. We don't want to have to change all of the code that gets the X. We want to keep this level of indirection in this layer of indirection just in case one day we want to change it.

The question is, does the cost of changing the code later justify the cost of adding in that interaction now? Because that's a lot of code. It's a lot of code to make all these getters. Each one is three lines of code. That's an interesting question.

I don't know Ruby, but I've seen that they have a very good solution to this, which is on one line they can list all of their fields and say make automatic getters for these. One line. It's the same amount of code it is as it is to just make them all private and not have getters. It's basically zero cost.

They've taken this thing that could be a tough thing to calculate how much will it cost to change our code versus how likely is it to need to be changed versus how much code am I generating here? They've turned it into like a non-question. It's the non-issue. It costs no more code to make all the getters than it does to not make them and to access the fields directly.

That's an interesting trade off. It's like in terms of the amount of code and the effort it takes. That is a free level of indirection, which approve. That's great.

In the Java side, I'm thinking what problem is this solving? It's a problem you don't have, you don't have a problem of what's the way to put it? What problem is that trying to solve is trying to make these...What is the problem that you want to give access to the values of those fields, but you don't want to let someone change them.

I said, usually people make getters and setters anyway so it's just unclear. It's just unclear what the problem is that is being solved. I really look at all these things with a skeptical eye because I've done it myself. I've over complicated stuff with tons of indirection.

When a simple step-by-step do this, then that, then that was clearly a superior solution after someone showed me. I've seen other people do it where I've been reviewing their code and I'm like, "What is this?" It's a queue.

You're putting something on a queue and then there's a worker that's taking the thing off the queue and doing it. Why don't you just do it instead of putting it on a queue? Do it right there. Why does it go in a queue? The answer is always, "Well, what if we want to add some other kind of work that the worker can do?"

It's like, "OK, great. Let's look into the future. What kind of work will we want to do?" They can't name anything. It's like, "Well, if you can't name it now, who's to say you're going to be able to name it in the future?" If we will have something in the future that needs this queue, we'll just add it instead of adding it now we'll just add it in the future.

What's the difference? The difference is today we save all of this work and this indirection and maybe we won't even need it. There, that's my long rant.

Let me recap. Any problem can be solved with another layer of indirection. It's true. Indirection can make things harder to understand and hide problems. The problem could be hidden in this huge multilayer cake of indirection.

My recommendation is whenever you're thinking about adding a new layer of indirection or you're evaluating an existing layer, you ask, "What is the problem that this is actually solving?" It could be very neat. It could be a nice pattern, a very common pattern, a design pattern, if you will.

But, what is it actually giving us? What problem is it solving? Because if it's not, get rid of it. Reduce it down.

Let me tell you about lispcast.com/podcast. If you go there, you'll find this podcast, all the past episodes and in the future, you will find the future episodes. Each episode has a page that includes audio, video, and a text transcript. You can listen, watch, or read, if that's how you like to do it, whatever floats your boat at that time.

There are subscribe links for the podcasts so you can get the new ones as they come out. There's also links to my email and other social media that, if you want to get in touch with me, you want to follow, you want to send me a message, say hello, say goodbye, whatever, my email is out there.

I can't say I'll see you later because this is a recording and you are watching me. I will say, I hope you liked this episode and that you will watch others. Thank you. Bye-bye.