Reification
“Grokking Simplicity is by far the gentlest introduction to FP I've encountered and I've been recommending it to people that want an on-ramp to FP ever since.”
Summary: Reification means making an abstraction into a concrete value that can be manipulated at runtime. Reification is the core of what makes a language dynamic. Three types of reification in Clojure are discussed.
What made Object Oriented programming (in the Smalltalk sense) so powerful? So powerful, in fact, that the GUI, WYSIWYG editing, overlapping windows, MVC, and more, were invented using it, not to mention an entire programming paradigm.
What made Lisp so powerful? So powerful that we still see new Lisps popping up and its legend looms over every serious programmer's mind.
Now, I don't want to boil it down to one thing. But one thing that was important, that you see in both of these languages, was the idea of reification. In fact, reification is possibly the essence of what makes a language "dynamic".
Reification, as I define it here in this article, means to make an abstraction available at runtime. Many languages have a text-based syntax that is read in by a compiler and compiled to machine code. But Smalltalk made each line of code into an object that you could manipulate, either with the input devices or with other code. And it let programmers bootstrap an IDE that was unparalleled at the time.
But Smalltalk's coup de grace reification was the class, which is an object which represents the behavior of another object. Instead of some static switch statement that dispatched the methods, the methods were stored in a data structure that could be inspected and added to at runtime. Dynamic dispatch! You see this in multimethods in Clojure.
Clojure's namespaces are reified. Some languages have a linking step in the compiler where modules are brought together to form a binary. But in Clojure, the namespace is accessible at runtime. Does your code want to know what Vars are defined? Easy. How about add a new Var. Done. These serve to support interactive programming---a hallmark of dynamic languages.
Lisps have always, from the very beginning, supported homoiconicity, which is a silly way of saying that programs are reified into the language as data. This means you can write functions that write code---also known as macros. Macros serve a very useful bootstrapping function because you can gradually add to the language instead of having to design it up front. And sometimes you get a huge win, like core.async, which adds a totally new semantic.
The next level of reification in Lisps is the higher-order function. Functions are not just things to call, but things to pass as arguments, save in collections, etc. They are real values, just like numbers and strings. There was a time in the history of programming when you could not refer to a function except to call it. Now, we take it for granted. Being able to reify an abstraction into a thing to pass around is amazing, and we should all just take a moment to ponder just how awesome it is.
Lisps have traditionally gone to the next level, which is to reify a problem into a data-driven solution. Nowadays, people call this type of programming "DSL". But it's just a type of reification. Instead of writing code to solve the problem, let's encode the solution as data. The problem domain is encoded in the interpreter for that data. Now it's accessible at runtime in a way simple code never could be.
Prismatic's Schema is a great example of this. You define a validator for a piece of data using existing data types: maps, vectors, strings, classes, regexes, etc. Then the library can interpret that data structure and tell you if a piece of data is described by that validator. If schemas were merely a static construct, this would not be possible. You would have to wait for the language to "support" it, which is a terrible form of tyranny.
Here's the secret to compounding the power: that data structure can be interpreted in many ways. Take Prismatic's Schema again. You can generate schemas at runtime. You can print them out. You can use them to build test.check generators. When things are reified and use the same interface as everything else, you can see synergy between libraries. You use one reification to enhance another.
Data-driven solutions are superior to macro-driven ones or even higher-order function solutions. Data can be stored. It can go over the wire. It can be meaningful in different contexts. A macro is useful at compile time, which happens once. Functions are black boxes and can really only do one thing (apply to arguments). But data is just data, ready to be interpreted.
Many Clojure libraries are considered "language features" in other languages. You don't have to mess with the internals of the language. Dynamic languages can do this, but Clojure (and most Lisps) has it at enough levels that interesting things happen.
Here's another reification: Haskell reifies side effects into values that can be composed. That's cool.
If you're into this whole reification thing, where language features that require major releases in other languages are just libraries; if you think you should learn functional programming; if you are curious about what everyone is talking about, check out the LispCast Introduction to Clojure video series.
The series is 1.5 hours taking you from zero Clojure knowledge through data-driven programming, one of the coolest types of reification. You'll help a robot who always wanted to be a baker learn to make bread :) Help him learn the recipes and convert them from static code to dynamic data.