Eric Normand Newsletter 474: Data-Oriented Programming
From OO to Clojure Workshop!
Watch my free workshop to help you learn Clojure faster and shift your paradigm to functional.
I just finished Data-Oriented Programming by Yehonathan Sharvit. It's a good book. You should read it. I had followed along during the early access, but I dropped off at some point. Since Yehonathan and I are planning to stream a discussion about our books, I needed to finally finish the book. Luckily, my print copy arrived.
I want to focus on three points that I really like.
- It nails the principles of Data-Oriented Programming (DOP).
- It presents a comprehensive and practical programming system.
- It justifies a very different programming style to people unfamiliar with it.
Nailing the principles of DOP
Yehonathan lists four principles. When I first heard them, I thought, "Nah, that can't be all there is." But as I thought about it, I realized he did find the crucial aspects of what we do. Discovering principles as he did is very difficult. People summarize DOP as "just use maps." As Clojure programmers, we know what they mean. But there's more going on. And what is going on? Well, Yehonathan lists them:
- Separate code from data. (Normally, maps just have data, no functions.)
- Represent data with generic data structures. (Like a map.)
- Data is immutable. ("just use an immutable map.")
- Separate data schema from data representation. (If you want to validate the map's contents, that's done elsewhere.)
Presenting a comprehensive and practical programming system
When I read the first couple of chapters months ago, I was happy that the book existed. But I thought it would be an explanation of the four principles. And I continued to believe that until I read the rest.
At first, I thought, "why are you talking about testing?" and "why is there a chapter on polymorphism?" As I read those chapters, I tried to put myself in the readers' shoes. I didn't learn much myself (since this is how we test and do polymorphism in Clojure). But the non-Clojure programmer is probably somewhat skeptical about "just using maps." They need to know that this is a comprehensive system: that they can write unit tests with no problems, that they can manage state, and that they won't give up polymorphism when they ditch classes. While the book couldn't address every concern, it addressed enough that you could get an idea about what programming with DOP would look like.
Justifying the style of programming to those unfamiliar
Besides showing that the style is practical to write real programs in, the book justifies the style using realistic yet simple examples. The book is divided into three parts, each dedicated to a different concern: flexibility, scalability, and maintainability. Even though I've used DOP for a while, I had never thought about why I prefer working with data over building many classes. The book makes a good case that it is better on several critical dimensions without dismissing other styles.
What he didn't get into
There are a couple of things I wish he had had time to explore. The three big things he did were vital to establish the concept, and these wishes are not essential. But, as someone who is already bought into the idea, I would love to have these questions answered.
1. What kinds of data structures can we work with?
Principle #2 says to use generic data structures. It does not specify what
those data structures need to be. The examples in the book use maps, arrays, and
is to come up with a relational model, where data is represented as sets of
tuples, and the data are accessed using a relational algebra. How about a graph
model of entities? Or what about a generic
Entity class that you could
subclass for specific entity types. The
Entity superclass would give you all
sorts of generic accessors. These ideas interest me, but they should not be in
2. What are the common DOP practices that make a system more scalable?
The book deals with the very basics of testing, state management, and polymorphism. But there are many other things the Clojure community has learned about writing good software with DOP. I'm thinking about returning a modified version of a map instead of making a new map with just the keys you know about. It's much better to pass along keys you don't understand. Or the idea of chaining sequence operations together. How do you do that most effectively? These are all questions I would like addressed, but I think breadth was more critical for this book.
Anyway, I am looking forward to our livestream. Watch this newsletter for a time and place. It hasn't been scheduled yet. And read Data-Oriented Programming in the meantime.
Is programming getting worse? This talk makes a case that it is.
What? This classic of Lisp programming is now available for free in a number of formats. This book is from the time when Norvig would say "artificial intelligence just means good programming." The title is about AI, but the subject is Lisp.
A new podcast episode about, you guessed it, high-level languages. What does it mean?
For some reason I really like to read about how computer games get made. I don't even play games!
Grokking Simplicity 📘
You can order my book on Amazon. Please leave a rating and review. Reviews are a primary signal that Amazon uses to promote the book. They help others learn whether the book is for them.
You can order the print and eBook versions on Manning.com (use TSSIMPLICITY for 50% off).
Clojure Challenge 🤔
This week's challenge
No challenge this week. It's late!