PurelyFunctional.tv Newsletter 215: Clojure and Generative Testing
Issue 215 - March 06, 2017
Hi Clojurers,
I'm not sure when I first heard about Generative Testing. I do remember being excited when we got into it in Haskell at the company I worked for. Although it was more work than just writing a few tests by hand, it also gave us way more confidence that we were testing more of the system. And, surprise, it forced us to think deeper about how the code was supposed to work.
Then I learned that Clojure was getting serious about Generative Testing. Could you believe I was even more excited to play with it? I was working on a Clojure system at the time and I knew a few parts that could use a little more testing. We were developing browser extensions for three browsers and we wanted to share code. So I wrote some tests for the API wrappers we needed to make sure they worked as expected when we ported them to the other browsers.
A lot of work went into testing that system. test.check was meant for testing pure functions, really. So I had to research and develop techniques to test the stateful APIs I was wrapping. It was mostly stuff from John Hughes' papers and talks, but as far as I could tell, it had never been done in Clojure.
My code wasn't of the kind that would go into a library. But the techniques would be useful. I never had a chance to share them. But now I do! I'm giving a talk at Clojure/West in a few weeks about how to test stateful systems with test.check.
I've been doing a lot of background research for the talk, and thought I'd share some of my findings in this week's issue. One thing that is very clear: Generative Testing is given a lot of attention from the Clojure stewards. I recommend you find a way to use it if you haven't already.
Please enjoy the issue.
Rock on!
PS Want to get this in your email? Subscribe!
Testing the Hard Stuff and Staying Sane YouTube
John Hughes gave the keynote at Clojure/West 2014. Great stuff. It starts from the very basic abilities of Generative Testing all the way to how Generative Testing can find bugs in highly concurrent systems.
Testing Asynchronous Behavior in ejabberd with QuickCheck
John Hughes presented three problems testing stateful systems with the Erlang QuickCheck and how they worked through them. The solutions to the first two are easily explained, but for the last one I recommend reading the paper. Slides from the talk are available.
clojure.test.check YouTube
At Clojure/West 2014, Reid Draper introduced clojure.test.check to the world. It was a port of QuickCheck, the generative testing library for Haskell. While it started as a library called simple-check, it was later adopted under the Clojure umbrella.
QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs
Back in 2000, John Hughes introduced QuickCheck (and I believe generative testing, though please correct me if I'm wrong) to the world. It was a Haskell library (about 300 lines originally) that let you build generators and properties.
Generative Integration Tests YouTube
Ashton Kemerling presented at Clojure/conj 2014. He talks about using generative tests to test a large JavaScript frontend application.
Generating Generators YouTube
Steve Miner shows how you can use his Herbert library to generate test.check generators from declarative descriptions of the data.
A Deep Specification for Dropbox
Benjamin Pierce presented research he did to generatively test Dropbox. But before he gets into that, he surveys the field of formal specification in general, and how Generative Testing can be used to exercise a formal specification.
Purely Random YouTube
Gary Fredericks explains how a technique called Splittable Random Number Generators can make random processes truly repeatable even when run in parallel. This repeatability is essential for Generative Testing.
Genetic Programming with clojure.spec YouTube
Carin Meier, code explorer extraordinaire, shows how clojure.spec's Generat ive Testing features can be used to do genetic programming and self-healing code.