How do Clojure Programmers Deal with Long Startup Times
// TODO: Add Babashka and Sci
Last time I discovered that the JVM startup times aren't that bad. Clojure and Leiningen are much slower.
Clojure startup times suck. Let's just be honest. Starting lein repl
in a typical project takes about eight seconds on my machine. Running
lein test takes over twelve seconds. And I don't even have any tests.
How do Clojure programmers live with this?
Further, Clojure is all about interactive development. It's all about fast feedback, incremental compiling, exploration. How can you have fast feedback when it takes 12 seconds to run tests? How do you do exploration when things take so long?
The startup times are annoying! When I want to start coding, it takes a lot of time before I can even type the first paren. I have to wait and remember all of the stuff that's going on in my head that I just want to type out. It's very stressful. I love fast feedback. I can't stand programming in a system that forces me to type stuff, then takes a while to show me the result of typing that stuff. But let me tell you, Clojure is not one of those systems.
The (Not so) Secret
Here's the key to fast feedback in Clojure: I rarely restart my application. I have REPLs that have been open for days. When I'm actively working on a single application, I will keep it running for weeks sometimes. It's not crazy. So I am willing to wait 20-30 seconds for my application to start. 20-30 seconds is nothing compared to weeks of development with super fast feedback.
Most workflows get it all wrong
Let me tell you a little story that Alan Kay has told about Smalltalk back in the day. People writing C++ would show all of these microbenchmarks showing how C++ way outperforms Smalltalk, especially when doing integer arithmetic. But when they'd write a GUI in C++, they'd waste all of the performance gained. Mouse clicks would take a long time to have their effect. Windows wouldn't update their contents quickly. The actual goal of showing the answer to the user quickly was ignored. Smalltalk let you do the math, see the answer, and change the calculation faster than the C++-based GUIs could.
Smalltalk focused on giving users the whole cycle. C++ focused on fast math, one little piece. When people focus on startup times, they are missing the big picture. It's not about startup times, it's about fast feedback.
What I see in Clojure (and some other communities) is a consistent high-level concern with the speed of feedback. That includes the entire cycle, from thinking of some code, to typing it in, to compiling it and running it and seeing the result.
It's why Figwheel exists. People complain about ClojureScript compile times. In most frontend systems, there's almost no compilation time. But they are not counting the time it takes to refresh their browser and click buttons to test out what they just coded. With Figwheel, you literally type code, save it, and less than a second later see the result in your browser. The compilation time is only a part of the equation.
When people complain about the startup times, I wonder about their workflow. I've even considered offering a consulting service to help people set up their workflow for fast feedback. If you want me to get annoyed and stressed and quit my job, give me long feedback loops. The absolutely most important thing to me in any system is making it fast to make changes and see the results. I'll spend a lot of time speeding up my workflow instead of working on features.
Cider (the Emacs development environment for Clojure), Cursive, and proto-repl (the Atom development environment for Clojure) all focus on this entire feedback loop.
My workflow
With the obligatory caveats that my workflow is certainly not ideal for everyone, let me try to describe how I work in Clojure.
I open up Emacs, browse to a random file in my project, then run
cider-jack-in, which starts a connected REPL. That takes about 20-30
seconds. I twiddle my thumbs.
Now that I'm in, I compile the current namespace with a keystroke
(C-c-k). Then I code. Then I compile everything. Then I code. Then I
compile everything. Over and over.
If I'm doing TDD, after I compile my code, I run the tests with a
keystroke (C-c C-t n). All of this stuff is so ingrained in my muscle
memory that I don't think about it. I do it without realizing it.
Some other resources
Okay, startup times is a very common complaint and people have worked hard on it. There are some interesting resources.
One approach is to keep a JVM around that's already booted up and ready
with the classpath you want. Then when you want to run some code, you
just connect to the JVM and add the new code and run it. That's the
approach of drip and it should be a
drop-in replacement for the java executable.
The Leiningen wiki has a page about making Leiningen faster.
Alan Dipert wrote a guide to avoid restarting the JVM when developing in Boot. Boot is cool because it totally gets the fast-feedback mindset. Boot lets you add dependencies as you go, instead of having to restart to add a new JAR.
But startup time of Clojure is still bad for shell scripting. Planck and Lumo are ClojureScript REPLs that start up fast and let you code ClojureScript. It's not JVM Clojure, but it's a way to run Clojure for quick scripts. They can handle command line arguments, input/output, and shelling out to other programs, among other things. These are under active development and they get new features all the time.
Conclusions
Startup times are still important for a lot of applications, like running shell scripts. However, when developing applications, the Clojure community focuses on fast fee dback more than it focuses on the startup time. Get your workflow set up so that you can see the result of your changes instantly.
If you are coming to Clojure from another language (or it's your first language), I don't want you to get stuck on the intricacies of the JVM. It can be a huge stumbling block. That's why I created a course called JVM Fundamentals for Clojure. It explains all sorts of stuff that will make you more effective when using the JVM, configuring it, doing interop, and understanding what's going on.
We're going to take a little turn now and next time we'll be exploring the wide variety of JVM deployment options.