Why Clojure starts up slowly — is it really the JVM?

One of the most common complaints about the JVM is the long startup time. People running their apps can sometimes wait up to a couple of minutes just to be able to start their servers. People complain regardless of language. Java programmers complain, JRuby programmers complain, and Clojure programmers complain.

Let's do a careful study of where that time is going. We're going to test the JVM itself, the JVM with Clojure, and the JVM with Clojure and Leiningen.

Table of Contents

• Why is JVM startup so slow?
• What does the JVM do when it starts up?
• But is it really slow?
• Running Clojure
• Running Leiningen
• Running code
• Conclusions

Why is JVM startup so slow?

The easiest answer is that startup time has never really been a big focus. Most JVMs run on servers for a long time. So most installations don't care if it takes a few more seconds.

What does the JVM do when it starts up?

There are three things the JVM does when it starts.

1. It grabs a bunch of memory for the initial heap.
2. It loads in the classfiles (bytecode) it needs to run.
3. It initializes needed classes. Some classes do quite a lot.

Then it can run your code. Now, this is complicated (or simplified?) a little because the JVM will lazily load and initialize classes as they are needed. But, still, the classes can't be used until they're initialized. And very often you have to initialize quite a lot of classes even for the simplest programs.

But is it really slow?

I'm all for experimentation for questions like this. Just like we can work at the REPL to understand our Clojure code's behavior, we can similarly do some benchmarking in our terminals.

Let's start with a super simple Java program that does nothing, in the file Nothing.java.

class Nothing {
    public static void main(String[] args) {
    }
}

I'll compile it with javac Nothing.java then run it:

> time java Nothing

real    0m0.101s
user    0m0.083s
sys     0m0.021s

I've run it several times and it's always very close to this answer.

That's actually pretty fast. Starting the JVM, grabbing the memory for the heap, and then exiting takes one tenth of a second.

When I try to change the heap size by passing command line arguments, it seems to take longer by a few milliseconds. Perhaps just parsing the arguments adds time. It doesn't seem to matter how big I make the heap. Even a 4GB heap takes the same amount of time.

I can count the number of classes being loaded like this:

java -verbose:class Nothing | grep Loaded | wc -l

I get:

429

So just to do nothing, we're loading 429 classes! That's good to know. So that's probably a lower limit.

Running Clojure

Let's do the same test with a minimal Clojure example. Luckily, Clojure lets us pass in a program on the command line. So we don't even need a file. This simplest program is just nil.

> time java -cp clojure-1.8.0.jar clojure.main -e nil

real    0m0.780s
user    0m1.340s
sys     0m0.093s

This one took about a second. Where is the extra time going?

java -verbose:class -cp clojure-1.8.0.jar clojure.main -e "nil"|grep Loaded|wc -l

It loads 1988 classes! Just reading the output of the -verbose:class option yourself will be instructive. Here's an excerpt:

...
[Loaded clojure.main$repl from file:clojure-1.8.0.jar]
[Loaded clojure.main$repl$fn__7404 from file:clojure-1.8.0.jar]
[Loaded clojure.main$repl$fn__7406 from file:clojure-1.8.0.jar]
[Loaded clojure.main$load_script from file:clojure-1.8.0.jar]
[Loaded clojure.main$init_opt from file:clojure-1.8.0.jar]
[Loaded clojure.main$eval_opt from file:clojure-1.8.0.jar]
[Loaded clojure.main$init_dispatch from file:clojure-1.8.0.jar]
[Loaded clojure.main$initialize from file:clojure-1.8.0.jar]
[Loaded clojure.main$main_opt from file:clojure-1.8.0.jar]
...
[Loaded clojure.core.server$start_server from file:clojure-1.8.0.jar]
[Loaded clojure.core.server$stop_server from file:clojure-1.8.0.jar]
[Loaded clojure.core.server$stop_servers from file:clojure-1.8.0.jar]
[Loaded clojure.core.server$parse_props from file:clojure-1.8.0.jar]
[Loaded clojure.core.server$start_servers from file:clojure-1.8.0.jar]
...

Those "classes" are actually functions defined in Clojure.

Running Leiningen

Let's testing Leiningen. I created a new Leiningen project depending on Clojure 1.8.

> time lein run -m clojure.main -e nil

real    0m2.506s
user    0m2.554s
sys     0m0.267s

Wow! That's a lot of time. Two-and-a-half seconds.

First, we can try to AOT compile it and build an uberjar.

In project.clj:

:main clj-test.core
:profiles {:uberjar {:aot :all}}

In clj-test/core.clj:

(ns clj-test.core
  (:gen-class))

(defn -main [& args])

Then:

> lein uberjar
> time java -jar target/clj-test-0.1.0-SNAPSHOT-standalone.jar

real    0m0.913s
user    0m1.709s
sys     0m0.120s

That's not bad.

Let's try a lein trampoline (which starts two JVMs):

> time lein trampoline run -m clojure.main -e nil

real    0m2.416s
user    0m2.931s
sys     0m0.302s

Well, that didn't help. But I remember reading about an environment variable called LEIN_FAST_TRAMPOLINE. Let's try setting that:

> LEIN_FAST_TRAMPOLINE=y time lein trampoline run -m clojure.main -e nil

real    0m0.964s
user    0m0.913s
sys     0m0.143s

Wow! So that is fast. Just under a second. It's back down to around the same time as not using Leiningen. I am going to set that flag in my .bash_profile!

Running code

So far, we're not running any interesting code. But it's clear that startup times will only go up if you've got code in your system. And real applications do a lot of stuff at startup: initializing logging, setting up the server, reading in configuration files, etc.

If you're running Clojure files, they'll have to be parsed and compiled. Any required namespaces will have to be read in, parsed, and compiled as well. AOTing your code will definitely help by eliminating the parsing and compilation.

Conclusions

Running Clojure takes a lot of time above the JVM startup time. It has to load a lot of classes. Different setups take more and less time. But it looks like the minimum on my machine is about one second when using Clojure.

And that brings up a good point: it tooke me about 30 minutes to test all of these variations and figure out which commands were faster and which slower. You can do this, too!

In fact, others have done similar things. Alex Miller analyzed a lot of different variations. Nicholas Kariniemi analyzed where time went when booting Clojure. Joe Kutner analyzed JVM startup times for Spring. And the JRuby Wiki has some information about making JRuby boot faster.

Now, I hope this shows that the JVM is a fine platform. I hope you want to learn more about it, and how to make use of it with Clojure. I've got a course teaching all the JVM stuff I learned from ten years of experience that I still use as a Clojure programmer. You can also get it as part of a membership, along with all of the other courses.

What is clear is that Clojure (or JRuby or Spring) are what take a long time to load, not the JVM. Clojure can take 7-9x the time the entire JVM takes. So how do Clojure p rogrammers deal with long startup times? We'll touch on that next time!