PurelyFunctional.tv Newsletter 339: Design is about pulling things apart

Issue 339 - August 12, 2019 · Archives · Subscribe

Clojure Tip 💡

Design is about pulling things apart

I think it was Rich Hickey who said this. It's a big part of the philosophy of Clojure and its community. Pulling things apart is why we have reducers. It pulls apart folding from order so it can be parallelized. Pulling things apart is why we have transducers. It pulls apart sequence operations (like map: do f to every element) from where you find the items (sequences or core.async channels). It's also what lets Datomic scale reads separate from writes. And of course, there's the term decomplect.

I don't now if design is only about pulling things apart, but it's a big part of it. Pulling things apart lets you get really granular when you're controlling things. So much complexity comes from not having the precise control you need. Instead, you're doing workarounds, which add complexity. If you could just get down a layer, where things are controlled separately, you could do just what was needed.

But it's not easy. The first step is to recognize the problem. The second step is to see that maybe things could be separated. Third is figuring out how to separate them. This third step is usually a messy, iterative process with many experiments. You won't know if you can succeed until you do.

Even if you don't succeed, it's worth the effort. You'll learn a lot.

Just finished reading 📖

At home in the universe by Stuart Kauffman.

Well, what a great book. It came out in 1995. Complexity theory was so hot then, and this book had a lot to do with it. By working with computational models, Kauffman explains a lot about our universe and why there is life. After reading a few chapters, I was very convinced that life is inevitable. The details are random, but it's quite easy to reach a critical mass of proteins that self-catalyze. Once you're over the threshold, you get a mess of new proteins, and those new proteins make it even more likely to catalyze.

It made me think a lot about computation. What can we learn from life to design distributed systems that are vastly larger than we currently do? How can you guarantee that it will converge on an answer?

This book has not changed my life, but I think it could have if I had read it when I was younger. In High School, I was getting into Complexity Theory and Artificial Life, but I never came across this book. If I had read it then, it may have altered my path substantially.

Book status 📖

Well, I am confident now that the book will be published very soon. It was in process internally at the publisher for a while, but it looks like it could emerge this week. Yes, it has been stressful. I want my book out there, and if the messages I get are any indication, many people also want it. But the publisher is making it as good as they can before it comes out.

I'll keep posting about it weekly in the newsletter. Or you can sign up for the book-specific launch list here.

Currently recording 🎥

I am now recording a course called Property-Based Testing with test.check. Property-based testing (PBT) is a powerful tool for generating tests instead of writing them. You'll love the way PBT makes you think about your system. And you can buy it now in Early Access. Of course, PF.tv members have had access since the beginning, along with all of the other courses.

New lessons this week include:

1. Strategies for properties: generate the output --- in which test things that are not easy to generate.
2. Strategies for properties: metamorphic properties --- in which we test things even if we don't know exactly what they do.
3. Behind the scenes: size --- in which we peer behind the curtain at how random values are.

Last week I said that there were three more lessons for properties, and here I am showing you only two. I reviewed the notes for the third one and I had already covered everything. Soon we're going to get into real testing examples. But before that, we have to talk about shrinkage.

I've made the first and fifth lessons free to watch. Go check them out.

Members already have access to the lessons. The Early Access Program is open. If you buy now, you will get the already published material and everything else that comes out at a serious discount. There are already 6 hours of video, and looking at my plan, this one might be 9-10 hours. But that's just an estimate. It could be more or less. The uncertainty about it is why there's such a discount for the Early Access Program.

Clojure Challenge 🤔

Last week's challenge

The puzzle in Issue 338 was to write a function that created the derivative of a function using the limit definition of the derivative.

You can check out the submissions here.

This week's challenge

symbolic differentiation

I got excited last week by the numeric differentiator challenge. When I originally read that section of SICP so many years ago, I was elated. Do you know how hard that would be to do in Java?

Now for something that's even harder to do in Java: symbolic differentiation.

Here's the relevant link into SICP, for reference.

The challenge this week is to use four differentiation formulas to make a symbolic differentiator. It has been a while since I've done any calculus, so I had to look these up:

1. dc/dx = 0 (for a constant c, or variable c different from x)
2. dx/dx = 1
3. d(u+v)/dx = du/dx + dv/dx (addition rule)
4. d(uv)/dx = u(dv/dx) + v(du/dx) (product rule)

This won't differentiate every formula, but it will differentiate sums and products of variables and numbers. You should make a function that takes an expression and a variable to differentiate by, like so:

(deriv '(+ x 5) 'x) ;=> (+ 1 0)

Don't worry about simplifying the final expression. Just make sure it's right.

As usual, please send me your implementations. I'll share them all in next week's issue. If you send me one, but you don't want me to share it publicly, please let me know.

Rock on! Eric Normand