Pre-Conj Interview: Colin Fleming

Introduction

The next interview is with Colin Fleming. He is the developer of Cursive Clojure. His talk at Clojure/conj will be about that. Read the background to his talk.

Interview

LispCast: How did you get into Clojure?

Colin Fleming: I got into Clojure bit by bit, starting about four years ago. I was working in Java on highly concurrent transaction processing systems and the promise of a language that would make that easier was very appealing. I was also playing with Scheme at the time so it seemed like a good fit since I already knew the JVM really well.

I've used IntelliJ for Java for years now and I wanted to use it for Clojure too, so I installed La Clojure, which was the only option at the time. But I quickly got frustrated with its limitations and started fixing it up, porting the parts I worked on to Clojure as I went. That was really my first real Clojure project, which in retrospect wasn't a great choice - it was a tough project to get started on. But it evolved into Cursive, which I'm very happy with, so it worked out well in the end.

LC: Developing an IDE is quite a big task. And the web site indicates that you will charge for it. What has the response been? How many users do you have in the Early Access Program?

CF: Right, it's definitely a big task. Initially Cursive started as a hobby, just scratching my own itch. But I quickly realised that if I was going to develop all the functionality I wanted to add it was at least a full-time job, probably more. I had just taken some time off after leaving my previous job at Google and I thought I would see if there was appetite in the Clojure community for a professionally-supported development environment, and the initial response on the Clojure mailing list was pretty positive.

I released the first public beta version almost a year ago now, and the response has really been fantastic. Quite a few people have asked me when they'll be able to pay for it so they can continue to support development, which is really nice to hear - I'm really happy people like Cursive so much. It's hard to tell exactly how many people are using it, but each version tends to get around 10,000 downloads. There's quite a long tail so it depends how long each version is out for, but generally there are around 2,000 downloads within 24 hours of making a release so I guess that's roughly the number of people using it daily, or at least have it installed in the copy of IntelliJ that they use daily.

LC: I use Emacs but I can't really recommend it to someone who just wants to try Clojure. Does Cursive fit that bill and if so, how?

CF: Sure, I think Cursive is definitely easier to get started with than Emacs. In general IntelliJ's interface is more like a standard application than Emacs, that familiarity helps a lot. And I try to make sure that Cursive is easy to get started with and to minimise the yak shaving required to open a Leiningen project and a REPL. There are still some outstanding things to do there, a big one has been that Cursive didn't ship with default keybindings assigned - that's fixed in the next release which should be out shortly. Cursive also has paredit turned on by default for Clojure which can confuse newbies, but it's easy to toggle. And Cursive still doesn't have a great story for a Clojurescript REPL, which is always a big pain point for people getting started - I'm hoping to fix this soon. Light Table really did a great job on that one.

Cursive has a user guide with a lot of screenshots that should hopefully make it easier for people to get started. It's not 100% up to date with all the latest features, but it should help people to get going. I think having one definitive source of friendly documentation really helps at the beginning. I'd actually like to have an interactive tutorial in the IDE that people could run that would show them how to use the editor and possibly the basics of Clojure too, but that's looking like a v2 feature at this point.

LC: I've always liked IDE's for their ability to smooth over the rough parts of a language. And tooling often trumps terseness in a language with less tooling. What are some of the trouble-spots you see in Clojure, from a beginner's perspective, that haven't been addressed with tools?

CF: The classic one that a lot of people seem to struggle with is the stack traces - I've never found them to be too difficult, but I worked with Java for years so I know what they look like. It's easy to forget how difficult things like that are for people new to the JVM. Leiningen has done a great job of hiding things like the class path and dependency management as well, which can be really tough.

The other thing that I'd really love to see improved is the error messages. Since Clojure macro forms are essentially parsed by hand some of the error messages are really impenetrable. For example, something that might be a common newbie error like putting an (:import) clause ahead of the docstring in an ns form will give you a "java.lang.Character cannot be cast to clojure.lang.Named" error, and will also frequently attribute it to the wrong namespace - these sorts of errors can be really discouraging for someone trying to get started. Cursive internally uses something similar to the seqex library (from Chris Houser and Jonathan Clagget's great Illuminated Macros talk) for its parsing, I'd love to see libraries like that being used in Clojure core.

Cursive doesn't actually mark a lot of these errors in the editor yet, but I'd like to mark as many as possible. In particular, I'll be able to mark things like function invocations with the wrong number of args, non-gensym'ed variables inside syntax-quote and many of the problems detected by tools like Kibit and Eastwood. I hope that'll make Clojure less frustrating for newbies and more experienced folk alike.

LC: There's a bunch of stuff I'd like to ask you about. Let's talk about the practical side of developing an IDE. You say it's a plugin for IntelliJ. How much of the IntelliJ system are you able to leverage, how much is custom, and how much is a hack to fit the Clojure peg in the Java hole?

CF: Right, it's currently only available as a plugin to IntelliJ. JetBrains actually put a lot of work into making their base platform reusable so I'll be able to create a standalone IDE as well, as JetBrains have done with Redmine for Ruby, or PyCharm for Python. But it's a reasonable amount of work and probably adds some support load with cross-platform issues so it hasn't been a focus for me yet.

The IntelliJ infrastructure is really great - it's certainly saved me a huge amount of work being able to build on top of it. It's very impressive how well abstracted it is, I've encountered very few places where I've had serious roadblocks from IntelliJ itself due to the nature of Clojure. Most of the real difficulties are just intrinsic to how flexible the syntax of Clojure is, they're not problems with IntelliJ. Obviously all the base IDE functionality comes for free (project management, VCS support, indexing infrastructure, debugger and so forth) and the Java support is very easy to hook into which is important for Clojure. The biggest part that's totally custom is the symbol resolution, which generally has to be written for each language anyway.

Cursive also supports Clojurescript, and that's been more challenging since the Javascript support in IntelliJ is only available in the paid Ultimate Edition and is closed source. Cursive currently implements fairly primitive Javascript support itself which is just enough for Clojurescript, but it would be nice to be able to integrate that better with the existing support - I'm hoping to in the future.

The part that is very different in Cursive from a more standard IntelliJ language plugin is that it all has to be extensible since everything in Clojure is based on macros, even core language features. People will eventually have to be able to add support for the libraries they use or for the forms they define in internal projects so Cursive is internally based around an extension API that I'll open up at some point soon. More conventional languages generally have a fixed syntax, even if it's very flexible like Ruby for example, so this isn't an issue for them but it's essential for Clojure.

LC: In the talk description, you mention Rename Refactoring. It's not something I can say I miss in Emacs, but perhaps I don't know what I'm missing. Do people make use of that and other refactorings?

CF: Oh yes, I'd definitely argue that you don't know what you're missing out on! The two operations I use a lot are rename and find usages - I'd struggle to live without them now, especially find usages. I also recently added refactorings to extract and inline let bindings, which are really useful - people definitely use them a lot. I'm planning to add support soon for extracting and inlining functions, as well as introducing parameters which I use a lot in Java. Refactoring in Clojure will never be as accurate as in Java, but it's still very useful. Things like marking unused local variables is also something which sounds trivial but really catches a lot of bugs while you're editing.

I think this is the main difference between something like IntelliJ and something like Emacs, actually. When I'm working on Java code in IntelliJ the refactorings and all the small operations they provide as intentions are so good that I really feel like I'm working directly on my program, not on text. We've all experienced that flow state when programming when you're totally absorbed in what you're doing and you lose track of time, whether you've eaten or need to go to the bathroom or whatever. IntelliJ makes it much easier for me to achieve that state because it provides a huge number of code manipulations that directly reflect what I want to do to my program, without me then having to muck around with text. I've seen comments online along the lines of "IntelliJ is great but I type so much faster in Emacs/vi/whatever". I'd argue that approach is totally wrong - I don't want to avoid typing because I'm lazy, I want to avoid typing because text manipulation is the wrong level of abstraction. Cursive is a long way from providing that experience, partly because of Clojure's language characteristics and partly because it took teams of dozens of smart people at JetBrains years to achieve it for Java, but that's always my ultimate aim whether or not I ever actually get there.

LC: What resources would you recommend people make use of before your talk in order to get the most out of it? Anything useful about refactorings or IntelliJ?

CF: I don't think so, no - hopefully it'll be pretty self-contained. I'll be showing the features I'm talking about and explaining some of the implementation, so hopefully it'll be understandable to everyone. And hopefully it'll be interesting for everyone too!

LC: Where can people follow your adventures online?

CF: There's a twitter feed for Cursive and there are also some mailing lists for general Cursive discussion. I've also been hanging out more in the #clojure channel on IRC as cfleming in case anyone has questions. I've been meaning to blog more about Cursive but so far that's just one to-do item among many!

LC: Nice. One more question: what is Clojure's favorite breakfast?

CF: Cursive is most frequently powered by Tropical Mango and Nut Crunch, but the true breakfast of the gods is scrambled eggs with little chunks of Venison Pepperoni salami cooked through it.

LC: Cool.

Thanks for the great interview. It was fun.

CF: Thanks Eric, that was fun! I'm looking forward to seeing it, and to catching up at the conj too!