tools.analyzer.js: An Interview with Nicola Mometto

Clojure Gazette

Issue 1.81June 22, 2014

Hi Clojure lovers,

This week I have a great interview with Nicola Mometto (Bronsa), who created and develops the tools.analyzer, tools.analyzer.jvm, and tools.emitter.jvm projects. This summer he's working on a Google Summer of Code project.

Enjoy!
Eric

Clojure Gazette: Where are you from? Where and what are you studying?

Nicola Mometto: I'm from a small town in northern Italy, not too far from Venice. Currently I'm studying for a Bachelor's degree in Computer Science at the University of Padua.

CG: How did you get interested in Clojure?

NM: Back in high school while I was trying to find a good tiling windows manager to use, I stumbled upon Stumpwm which is written in Common Lisp and obviously uses it as its configuration (extension?) language. I had never heard of Lisp until then but the weird syntax intrigued me, so I went on to reading ANSI Common Lisp and Practical Common Lisp and fell in love with the concept of Lisps.

I played with Common Lisp for a while, discovered SICP and Scheme and immediately liked its simplicity and elegance but I was then left wishing for a frankestein language that included the features I liked most of both lisps.

Almost by accident I watched one of the first videos by Rich Hickey (I believe it was "Clojure for Lisp Programmers") and I was hooked: that was the lisp I had been looking for! So I bought and read Programming Clojure and The Joy of Clojure, started interacting with the Clojure community on IRC and have been using Clojure since then.

CG: What project are you working on?

NM: For the past year or so I've been working almost entirely on a set of contrib libraries grouped under the umbrella name of "CinC": tools.reader, tools.analyzer, tools.analyzer.jvm, tools.emitter.jvm and since a couple of days tools.analyzer.js.

tools.reader is a Clojure/edn reader implemented in Clojure and has been adopted by ClojureScript as its official reader since it allowed for much better source maps support with its enhanced source info annotations. tools.analyzer is arguably the most important library I'm working on, and the one I've spent the most amount of time developing: it's an analyzer for Clojure code that produces an AST similar to that of ClojureScript's analyzer, but that takes things a bit further: it's been designed from the start to be pluggable and host-agnostic and the produced AST has been designed to be traversable using some provided generic walkers which are used extensivly in the library itself to implement a multi-pass architecture.

tools.analyzer.jvm and tools.analyzer.js are host-specific implementations on top of tools.analyzer. All they do is set up a number of extension points and combine host-generic passes with some host specific ones.

Finally tools.emitter.jvm is a code emitter that takes a tools.analyzer.jvm produced AST and emits JVM bytecode.

CG: Will the JS analyzer be used in ClojureScript?

NM: Hopefully.

I have yet to talk about it with David but once the functionalities of the library are on par with those of the CLJS analyzer I'll start making the appropiate changes to ClojureScript and propose the switch. Since I just started working on tools.analyzer.js, I wouldn't expect anything to happen on that front for the next couple of months at least, then we'll see.

Switching frontends on a compiler is always a risky move and it might take some time to make sure there's no accidental breakage, but I believe there's value in switching to a tools.analyzer based analyzer, if not for the multi-pass design simply for having a unified AST between Clojure and ClojureScript.

CG: core.async recently was converted to use tools.analyzer. Do you know of any other projects using it?

NM: There are at least two other "big" projects that I know of that are using tools.analyzer right now: Eastwood (a clojure linter) and core.typed. Eastwood has been the first library to be ported from Ambrose [Bonnaire-Sergeant]'s jvm.tools.analyzer to tools.analyzer.jvm and the couple of months I've spent helping Andy Fingerhut and Jonas Enlund work on the porting have been incredibly useful.

In particular Andy set up an automated system that run the analyzer through a big amount of popular Clojure libraries that proved immensely helpful in finding and fixing some rough edges of tools.analyzer which would have been really difficult for me to find otherwise.

On the other hand, I wasn't much involved with porting either core.async or core.typed to tools.analyzer: by the time Timothy [Baldridge] and Ambrose started working on porting their libraries, tools.analyzer had been stress-tested enough times by Andy's system that it worked as expected right away.

CG: Clojure already compiles to JVM bytecode. What is the purpose of the JVM bytecode emitter? Are there plans to eventually use that instead?

NM: Replacing the original Clojure compiler has never been a goal for me while writing the bytecode emitter, mainly for two reasons:

1. Because of the multi-pass and immutable data driven approach shared by both the analyzer and the emitter, tools.emitter.jvm is and will always be slower than Clojure's compiler; right now the difference in speed is around 15x, there's plently of room for improvements but because of the way it's designed I doubt I'll ever be able to lower the slowdown beyond 5x.

2. The current compiler works, and it works fine, it has years of testing under its belt and replacing it would take a big amount of time to make sure everything works as expected; this aspect should not be taken lightly.

So what were my reasons for writing tools.emitter.jvm if replacing the original compiler was never the goal? To put it simply, hacking the Clojure compiler is a horrible experience.

My goal when writing the emitter, then, was to provide an easily hackable platform in order to be able to quickly experiment with different compilation approaches/optimizations, instead of wasting hours trying to make the same change in the Clojure compiler only to, maybe, find out later that it's not going to work.

I've already seen people using it for this purpose: there's a project for this year's GSoC by Reid McKenzie with the goal of making an optimizing compiler/tree shaker based on t.e.j; I've also talked on several occasions with Ghadi Shayban and he expressed interest in trying to introduce support for invokedynamic using t.e.j before diving into the Clojure compiler to make the required changes.

CG: Where can people follow your progress?

NM: I don't maintain a blog so the best way to keep up to date with my progress at the moment is by watching the activity on the various repositories I mentioned before under the Clojure Github organization. I'm also fairly active on the #clojure IRC channel on Freenode and I'm always open to answer questions there.

CG: Is there anything else you'd like to say before we close?

NM: One thing I didn't mention yet is that none of the stuff I've built until now (and that I'm still working on) would have been where it is now if it wasn't for Aaron Cohen, Ambrose Bonnaier-Sergeant, and Timothy Baldridge.

Aaron was my mentor in last year's GSoC when I started working on the CinC project, Ambrose supported me with a funding as part of his typed-clojure campaign and Timothy acted as mentor for the sponsorship that Cognitect offered me.

Now all of them are mentors for my 2014 GSoC project enabling me to keep working on tools.analyzer and writing tools.analyzer.js so I wanted to say: thank all of you, really.

Thanks also to all the amazing folks in the Clojure community for all the support and guidance, and finally, thank you for this interview and for giving me a chance to talk about what I've been building for the past year.

CG: Thanks, Nicola, for a great interview.

NM: You're welcome, Eric, it was a nice experience for me and I hope your readers will find it interesting.