Clojure Gazette 179: Trade user stories for hammock time

Trade user stories for hammock time

Issue 179 - June 27, 2016

I want to run a training in your area. See the message at the end of the letter.

Hi Friends,

On my best days programming, I enter into a prolonged and deep state of flow. I experiment with different expressions of the same functionality, looking for one that seems optimal. And I experiment with new, higher-order abstractions.

Good higher-order abstractions are rare. They're hard to find. They're elusive. Sometimes I feel like I've got hold of one, only to realize it was nothing. A lot of search and thinking and experimentation is in order. And even after lots of work, results aren't guaranteed. But when you do fond one, it's a thing of beauty. A thing of wonder. It's an inflection point that enables new things. It renders whole classes of problem irrelevant. The sublime feeling of deleting unnecessary code. These experiences motivate me and restore me---like good, hard work should. A long day of work dealing with this rare and wonderful possibility is worth every frustration and uncertainty that led up to it.

At the jobs I've had, the occurrence of this kind of experience has varied. At some jobs I could have that feeling every week. And others it just never happened. Some it would even require working on the weekend, off of official product work, to find the time to explore. What I tried to explore last week was why that variance happened. I went down a line of reasoning and made a guess at the problem and suggested some solutions. I don't think I was totally wrong, but I buried the lead.

As far I can tell, it looks like team size at my jobs was the biggest culprit for reducing exploration. Once there is a ticket system to manage work and once people are counting on you to finish before they can start, in my experience, exploration slows down. The big picture and ownership you need to really make big leaps of abstraction are hard to find.

At least, that's my experience. I haven't worked on any really big projects. Projects with thousands of programmers probably need some top-down division of responsibilities that require ticket systems. And there's always dirty work (code shoveling) that needs to be assigned. I know it's not all roses. But if that human potential to find powerful abstractions is not used, that is a waste. You can always find more people to dig a hole faster. But if they've got excavators sitting there unused, it's a waste.

I think this is a common attitude in the Clojure world, and probably the FP world in general. Paul deGrandis explained how he and a small team designed an abstraction for weeks before coding. The abstraction let him create many web apps quickly. It's quicker to design and write the abstraction first , then implement several web apps, than it is to hand-code them all without the abstraction!

Conal Elliott practices what he calls Denotational Design. He develops simple and robust abstractions, guided by category theory and types, without regard for their practical implementation. When thinking about the practical implementation, he misses opportunities for better abstractions. The resulting abstraction enables better implementations and opens up entire new paradigms like Functional Reactive Programming.

There's always the classic Hammock Driven Development talk by Rich Hickey ( transcript ). He talks about how deep thought over time, punctuated by experiment, applied to hard problems can yield exceptional results. He even explains how Agile doesn't allow for this thought process. That's what I was trying to get at last week. There's something about user stories and sprints that favors digging a hole by hand . The excavator lies dormant and starts thinking about side projects

The line of thinking in this essay was sparked by a recent essay by Ashton Kemerling about his frustrations with Clojure. His sentiments fall into to categories: frustrations with the communication from the Clojure team and frustrations with progress on outstanding bugs and improvements. Both types of complaint reminded me a lot of past managers I've had.

I won't address the communication issues here. Communication is important, but it's beyond the scope of this essay. Now for the other frustrations Ashton expressed, I really get it. There are bugs and choices in Clojure I wish were fixed. If there are so many open bugs, many with patches just waiting to be merged, why were they working on some crazy new library for checking data that nobody expected? They seemed to go quiet for a while then bring up these new features while there are high priority bugs open!

Well, the answer is they're practicing what they preach. They're seeing the backlog of bugs not as a stream of things to fix one at a time, but as a signal of more general problems that require more general solutions. They've been discussing and thinking and experimenting on how to solve a class of problem. It's how Clojure was made in the first place. To pack up the hammock and start fixing all of the bugs one at a time would fundamentally change the language for the worse.

The Clojure team's approach is working. clojure.spec can squash many more bugs than any of the existing data checking tools on their own. It replaces Schema, Herbert, Sequence Expressions, and other validation libraries. It will help you test your code, document your code, give existing and new functions and macros better error messages, and let you write efficient functions that don't need to check their input at runtime. One set of functions can do all of that if you put down the ticket tracker, grab a notebook and pen, and lie down in a hammock.

Rock on!

PS Want to get this in your email? Subscribe !
PPS Want to advertise to smart and talented Clojure devs ?

Clojure Training

I benefit so much from live, in-person trainings. It helps me refine my material and get essential information about what people need to learn. So, I'd love to do a training in your area, if there are enough people to fill it up. Imagine two days of intensive Clojure! You'll learn so much. Let me know where you'd like the training by filling out this easy survey . I need to fill the trainings to justify the cost of travel, so get your friends to complete the survey, too!