Clojure Gazette 171: The Structure of Structure

The Structure of Structure

Issue 171 - May 02, 2016

Read more about this week's sponsor, Arachne , at the end of this letter.

Hi Clojurnators,

One of my heroes is Buckminster Fuller. I learned, through him, that in physical structures, you can separate out two types of forces, tensile and compressive. If you select materials according to their tensile and compressive strengths, you can build very strong structures with very little matter.

Things have gotten stronger and lighter for thousands of years. You can see the trend from the Egyptian pyramids through modern suspension bridges. These bridges are much stronger, higher, and useful than the pyramids. And they are made of less stuff.

What's the important difference between a Greek post and lintel structure and a Roman arch? What's the difference between a Roman arch bridge and a suspension bridge? What's the difference between a wagon wheel and a bicycle wheel?

The answer to all of these questions is structure. A small rearrangement of the parts results in a better, stronger whole. The discovery of each type of structure opened the field of architecture to new possibilities.

Alan Kay compares the discovery of Object Oriented Programming (message-passing and stateful, encapsulated objects) to the discovery of the arch. It let the Smalltalk system be programmed more effectively than languages that just gave you procedures and arrays of bytes. It was a better structure for exploring new ideas in computing. But just like the Roman arch, it has limits.

For instance, while messages are a way to decouple the behaviors of objects, the memory layout inside of the object is still a flat array. That's not much structure. Each object creates bespoke data management patterns. Could there be some common, underlying structures of data that help us build without reinventing the buttress every time?

I believe Clojure has an answer to that question. Clojure's data structures capture some very useful arrangements of data. I'm not quite sure whether these are fundamental (in the mathematical sense). But used together they feel like they cover my needs.

Lists give you sequential, one-at-a-time arrangement. They allow for efficient iteration, lazy generation, and stack discipline.

Vectors give you sequential, all-at-once, indexed arrangement.

Maps give you unordered, arbitrary index arrangement. Access is iteration of key/value pairs or getting a value for a given key.

Sets give you unordered, unique constraint arrangement. Access is iteration of elements or checking containment.

Those are the four data structures I use every day. I typically compose more intricate structures from those. I can do that because their properties feel constrained enough to rely on.

I don't think of them as collections. That would mean they're like bags to hold things for later use. I think of them more like constraints. A ball and socket joint constrains motion to certain angles. Clojure's data structures constrain data to a certain arrangement and hence access patterns. When I'm programming in Clojure, I'm composing constraints.

Clojure's data structures have their problems. We're hearing all the time about how hard it is to keep track of and manipulate deeply nested structures. It's certainly something I've experienced before. It reminds me a bit about the gothic architects complaining that they can't build higher vaulted roofs. Eventually, someone will discover a new structure that supercedes this limit.

I purposefully did not go deep into the data structures themselves. Looking back at their explanations, I can't help but think how obtuse the descriptions are. Each of Clojure's data structures deserves more exposition than I could give here, but I plan to go into more detail in future issues.

Rock on!

PS Please check out the sponsor. The Arachne Kickstarter is doing well and can use your support.
PPS Want to get this in your email? Subscribe !
PPPS Want to advertise to smart and talented Clojure devs ?

Sponsor: Arachne

For a hobby project for learning Clojure, piecing together a web app from libraries works fine. But important concerns like security, logging, and authorization often get left behind because who has time for the boilerplate? Luke VanderHart has a new Kickstarter to build a modular app framework for Clojure. With it we will still piece together our app from parts but the boilerplate will be taken care of by the modules. It's the best attempt at a framework I've seen that leverages Clojure's unique strengths. Please back the project if you'd like to build production apps in minutes, not months.