How does making something first class give you power?

Often, functionality starts off as code. It's if statements and imperative ideas. But, over time, you notice patterns. Those patterns can be reified as data. And that gives us tremendous power. How so? We explore it in this episode.

Transcript

Eric Normand: How does making something first-class give you power? By the end of this episode, you'll have a better understanding of why making things first-class is the most common way to gain power in your language. My name is Eric Normand. I help people thrive with functional programming.

There's this idea of making ideas first-class. In this episode, I thought it'd be nice to talk about representing things as data as a way of making things first-class. We've all heard about first-class functions and things. Often, we make stuff first-class as data, and we don't even think about it as such.

I was listening to a conference talk about Pixar. Ed Catmull has been with Pixar since the very beginning. He was doing his PhD in the '60s or something, maybe in the early '70s, on 3D graphics, 3D rendering.

His PhD project was to model a hand. It's something very difficult. It's got a lot of bones, a lot of complex shapes, a lot of muscles and flesh that has to bend with the model.

I think he was working with a partner, too, I don't remember his name. He coded up the thing by hand. There were no 3D modeling packages, rendering packages. He had to write everything by hand.

It was just hard coded. It was like, this is what needs to happen when we deformed this mesh. When we bend the joints, the mesh has to deform.

Eventually, he recognized the patterns and the concepts, and started building them into data structures that could hold the data necessary for the algorithms in it. This, I thought, was a perfect example to illustrate why moving stuff from code into data is so powerful.

Because now, 60 years later, 50 years later maybe, we have all these data formats for 3D models. The data structures have, let's say, stabilized. They've found the sweet spots. We don't think of it this way. When you had something that was just code — back when he was writing this it's just instructions to the computer — is very hard to work with.

If you need to change the shape of the hand because you got the joint length wrong or something, you had to go into the code, modify it, maybe even recompile it. You're modifying code.

Now, if you had to change the shape of the hand, you would just open up your editor, drag a visual representation of that point, that vertex, and the shape, and then save it. It would save to a file. Now, wherever you use it, will load the new thing. You don't have to touch code.

We've got multiple ways of using that data structure. The two easy ones to see are the editor is using the data structure. Your rendering is using the data structure. Already, we see two possible types of uses for that same data.

There's probably more. There are probably like shape optimizers or analyzers that will tell you things about how you could get rid of this triangle, or maybe they could get rid of that triangle for you. All sorts of things like that that are added on top of this data structure.

Let's talk about some characteristics of this data. One, it's more manipulable in those multiple ways I was talking about, more manipulable than code.

Two, it's serializable data. You can write out to disk, you can send it over the network. It's got some format to it.

Third, it's less powerful than code. Code is turn-in complete. You could do anything in his routines for drawing the hand, anything, including going into an infinite loop.

These data structures are engineered in a way that they can't do everything. Basically, all they can do is render [laughs] some polygons or something. That limitation is the constraints that make it powerful. Because you know when I load this shape file, it's going to be safe.

I'm not saying that all the data formats for 3D objects are safe. What I'm saying is that, if they're constrained correctly, they should be safe. They should be able to be, avoid infinite loops, and things like that.

The problem is that, often, we want to add in some loopholes. We say, "OK, then you can have JavaScript here." You've opened the door up, again, to code. Data itself, totally safe. You can load it, and work with it without any halting problem.

I feel like this is something that we take for granted. When we start programming, there's all these interesting data formats already available. We forget that they started off as code and gradually evolved to become data.

This is one of the most common ways that we have of redefining an idea. Redefine, I have a whole episode on it. Redefine basically means making it real. Making it a concept that you can manipulate in your language. Most languages don't have a way to manipulate the code that you have.

The code, you write a function, and all it does is compiles the code into bytecode. You can't really get at that anymore. Most languages do have a way to take a data structure and pass it to other functions, and walk the data structure, all those things.

It is first-class. You could potentially make nested data structure, and a nested 3D object that contains more 3D objects inside. That's a way of composing them up. This is why it's one of the reasons why data is a very important concept, and something that we should keep in mind when we're coding.

Like, is this regular enough to represent as data? Will that give me the power I need instead of just having it as a function? Not everything can be data, but much more than you would expect it first.

Awesome. My name is Eric Normand. This has been one of my thoughts on functional programming. If you want to see this episode or other episodes, you can find them all at lispcast.com/podcast.

I've got audio, video and text transcripts of all the past episodes there. You'll also find links to subscribe, and how to find me on social media. If you would like to discuss this or any other topic, just hit me up. You'll find my email there. You'll find Twitter.

Emails, probably the best thing for longer-term, longer-form discussions, but Twitter is nice, too. I'm also trying to get into LinkedIn so you can find that, too. This has been my thought. Take care and rock on.