Eric Normand Newsletter 474: Data-Oriented Programming
Reflections 🤔
Data-Oriented Programming
I just finished Data-Oriented Programming by Yehonathan Sharvit. It's a good book. You should read it. I had followed along during the early access, but I dropped off at some point. Since Yehonathan and I are planning to stream a discussion about our books, I needed to finally finish the book. Luckily, my print copy arrived.
I want to focus on three points that I really like.
- It nails the principles of Data-Oriented Programming (DOP).
- It presents a comprehensive and practical programming system.
- It justifies a very different programming style to people unfamiliar with it.
Nailing the principles of DOP
Yehonathan lists four principles. When I first heard them, I thought, "Nah, that can't be all there is." But as I thought about it, I realized he did find the crucial aspects of what we do. Discovering principles as he did is very difficult. People summarize DOP as "just use maps." As Clojure programmers, we know what they mean. But there's more going on. And what is going on? Well, Yehonathan lists them:
- Separate code from data. (Normally, maps just have data, no functions.)
- Represent data with generic data structures. (Like a map.)
- Data is immutable. ("just use an immutable map.")
- Separate data schema from data representation. (If you want to validate the map's contents, that's done elsewhere.)
Presenting a comprehensive and practical programming system
When I read the first couple of chapters months ago, I was happy that the book existed. But I thought it would be an explanation of the four principles. And I continued to believe that until I read the rest.
At first, I thought, "why are you talking about testing?" and "why is there a chapter on polymorphism?" As I read those chapters, I tried to put myself in the readers' shoes. I didn't learn much myself (since this is how we test and do polymorphism in Clojure). But the non-Clojure programmer is probably somewhat skeptical about "just using maps." They need to know that this is a comprehensive system: that they can write unit tests with no problems, that they can manage state, and that they won't give up polymorphism when they ditch classes. While the book couldn't address every concern, it addressed enough that you could get an idea about what programming with DOP would look like.
Justifying the style of programming to those unfamiliar
Besides showing that the style is practical to write real programs in, the book justifies the style using realistic yet simple examples. The book is divided into three parts, each dedicated to a different concern: flexibility, scalability, and maintainability. Even though I've used DOP for a while, I had never thought about why I prefer working with data over building many classes. The book makes a good case that it is better on several critical dimensions without dismissing other styles.
What he didn't get into
There are a couple of things I wish he had had time to explore. The three big things he did were vital to establish the concept, and these wishes are not essential. But, as someone who is already bought into the idea, I would love to have these questions answered.
1. What kinds of data structures can we work with?
Principle #2 says to use generic data structures. It does not specify
what those data structures need to be. The examples in the book use
maps, arrays, and sets in JavaScript. But what else is possible? One
thing I thought would be cool is to come up with a relational model,
where data is represented as sets of tuples, and the data are accessed
using a relational algebra. How about a graph model of entities? Or what
about a generic Entity class that you could subclass for specific
entity types. The Entity superclass would give you all sorts of
generic accessors. These ideas interest me, but they should not be in
the book.
2. What are the common DOP practices that make a system more scalable?
The book deals with the very basics of testing, state management, and polymorphism. But there are many other things the Clojure community has learned about writing good software with DOP. I'm thinking about returning a modified version of a map instead of making a new map with just the keys you know about. It's much better to pass along keys you don't understand. Or the idea of chaining sequence operations together. How do you do that most effectively? These are all questions I would like addressed, but I think breadth was more critical for this book.
Anyway, I am looking forward to our livestream. Watch this newsletter for a time and place. It hasn't been scheduled yet. And read Data-Oriented Programming in the meantime.
Preventing the Collapse of Civilization
Is programming getting worse? This talk makes a case that it is.
Paradigms of Artificial Intelligence Programming
What? This classic of Lisp programming is now available for free in a number of formats. This book is from the time when Norvig would say "artificial intelligence just means good programming." The title is about AI, but the subject is Lisp.
High-Level Language 🎙
A new podcast episode about, you guessed it, high-level languages. What does it mean?
Efficient partitioning
For some reason I really like to read about how computer games get made. I don't even play games!
Grokking Simplicity 📘
You can order my book on Amazon. Please leave a rating and review. Reviews are a primary signal that Amazon uses to promote the book. They help others learn whether the book is for them.
You can order the print and eBook versions on Manning.com (use TSSIMPLICITY for 50% off).
Clojure Challenge 🤔
Last challenge
Issue 473 - Simplifying fractions - Submissions
This week's challenge
No challenge this week. It's late!
Rock on!
Eric Normand
