Why is data so powerful?

In this episode, we explore why Clojure's stance of not wrapping data much is so powerful in the world we live in.

Transcript

Eric Normand: Why is data so powerful? Hello, my name is Eric Normand and this is my podcast. Welcome. In the last episode, I went through a debate, argument, disagreement between Alan Kay, who was the creator of Smalltalk, legendary Turing Award winner and everything.

Created the GUI, Object Oriented Programming, and Rich Hickey, who is the creator of Clojure. The debate was about data. Alan Kay was asking whether data is a good idea, and Rich Hickey said, "It's undoubtedly a good idea. It's an old idea. You can't do anything without data." He missed the point of the question.

It was a research question that was trying to free your mind from sticking to the ideas that we've become accustomed to. Nevertheless, there is an interesting question in there about why...Given that data does have all the problems that came up in the discussion with Alan Kay, meaning it has to be interpreted.

If you got some signal, you got scribbles on a piece of paper, it's just ink on paper. You don't know what it means unless you know how to decode it. You need a lot of context. When I'm programming in Clojure and dealing with just data, why does it feel so natural, so right, so powerful? Where does the power come from?

I was in a discussion after the last episode with a lovely listener named Benoit, and I realized that Clojure has made a very pragmatic decision. Yes, Smalltalk is a research language. It was created to explore ideas, and Alan Kay's question was about exploring a new idea. Clojure is very practical.

It was made to solve problems that Rich Hickey was encountering in software development, and he couldn't find a language that solved those problems well, so he had to create his own.

One of those problems is that we live in a world where one of the most reliable things that we have is data. We're talking about files on a Unix File System, super reliable, JSON from an API that you can always get more data. You can always get it in a certain standard format, even XML files. We're not talking about the format here.

We're talking about the reliability of it, that it can be transmitted. It can be read. It can be processed. Databases, you can always get rows from a database. This is a world we live in today. It's the practical reality. We don't live in a world where we can assume that someone can run our code, that something we write here is going to be able to run over there.

We can't send the ambassador, for example. We also live in a world where a lot of languages add complexity by adding layers. They do object relational mapping, trying to fit this thing that's data into an object-oriented framework. There's all sorts of modeling issues in there.

Because data is so reliable, Clojure is taking a pretty controversial stance in saying, "Well, we're going to operate there. We're going to do as little adding on top of that. We're not going to force some different model. We'll just use data, and we'll have a bunch of powerful tools for operating there."

That's where Clojure gets its power from, is a very pragmatic choice of today in the world we live in. With the systems that are available, data is a reliable thing. It's a known thing. It has a long history.

By sticking there and making sure that we can process it easily, efficiently and expressively, then we got a big advantage over other languages that try to turn it into something else, try to adapt it poorly to a different model of computation. I just wanted to bring that up, that this research idea of Alan Kay is brilliant, and it should be explored.

I often go off and think about what it might look like to do something like that. I'd love to spend 10 years noodling on that, and coming up with prototypes and things. In the world where we have to make money, where our software has to work and get a job done, and the fact that there's data all over, then that's one of the most reliable things we have today.

In terms of ability to store it, ability to transfer it to another system, read it on different operating systems in different environments, it is a very practical choice. I can imagine a day where perhaps that isn't the practical choice anymore, but today it seems very reasonable to have that stance of we're not going to put too many layers on top of data.

We're going to make it easy to get the data in, make it easy to transform it into some other shape, process it, do some aggregation or what have you, and send it on out. It's a practical reality. I don't know what else to say. I enjoy it because it does feel powerful. It does feel like it's the right choice for a lot of problems.

That's where I'm happy when I have to do practical, pragmatic things that can't wait for a research agenda to pan out. All right, this has been a short one. My name is Eric Normand. Thank you for listening, and as always, rock on.