Eric Normand
PodcastClojureCoursesBook

Mike Rodriguez Clojure/conj 2016 Speaker Interview

Written by Eric Normand. Published: November 16, 2016. Updated: August 9, 2018.

Mike Rodriguez will be giving a talk at Clojure/conj 2016. He will be talking about Rules Engines.

Follow him on GitHub and Twitter.

PurelyFunctional.tv: How did you get into functional programming?

Mike Rodriguez: I didn't really learn about functional programming until four years ago. I started working on a new team that was experimenting with using Clojure. When learning Clojure, my eyes were open to the world of functional programming. It made me become more curious of the "functional" ideas in general. I then read about and explored other languages like Haskell and Scala. However, I've mostly focused on Clojure.

Coming from a background using languages such as Java, Python, Perl, and C++, functional programming was a change of mindset. I've found the change to be a good thing though. I've definitely been converted to the ways of functional programming. I think mutability is used way more than it needs to be and is only needed in control contexts. Basically, I think the default should be immutability, not the other way around. The idea of programming with values, writing mostly pure functions, and modeling data as simple data structures have resonated with me. I can see the power in this approach. I am more productive programming in Clojure than I've been with non-functional languages I've used before.

PF.tv: What is your talk about?

MR: My talk is about forward-chaining rules engines. Specifically, I will focus on those that are based around the Rete algorithm which was first introduced in 1982. Most mainstream rules engines in use today are based on some variant of this algorithm. I want to discuss what rules engines are, what they are good for, and how they work. The goal is to expose people to what rules engines are and how they can be a powerful tool to be aware of. Rules engines give a way to express complex conditional logic. This logic is often called the "business" logic, which in this context, just means logic that involves asking a lot of domain-specific questions about some data to derive domain-specific conclusions. Rules engines allow the logic to be expressed declaratively. Rules represent the logic in a flexible and extensible way.

I plan to introduce and use an open-source rules engine called Clara rules throughout the talk. This is a rules engine written in Clojure. I've been using it with large-scale production workloads for about two years now. Clara rules has performed competitively with Drools rules that I was originally working with. Drools is one of the most popular mainstream JVM-based rules engines, so it serves as a good mark to compare against.

Clara rules embraces the major concepts of Clojure. For example, most mainstream rules engines have a stateful, mutable memory model. Clara's memory model is an immutable, persistent data structure. Also, many rules engines used in industry attempt to be much more than a rules engine. They have an entire rule management system intertwined with the engine itself. Clara rules is a light-weight library. It is just a rules engine. This allows it to be used in a much more "a la carte" manner, which is popular among Clojure libraries. Rules in Clara are expressed with a well-defined data model. This data model gives leverage for programmatic manipulation and tooling. Also, Clara provides a DSL to write rules. Importantly, this DSL sits on top of a solid data model though. You do not need the DSL to effectively make rules, e.g. they can be generated programmatically or via a different DSL. This is often not true for other rules engines commonly used in enterprise applications.

PF.tv: What problems are people using rules engines like Clara to solve?

MR: At my current company, we have been using Clara rules in the healthcare space over the last two years. We are evaluating thousands of rules at a time against healthcare datasets containing up to hundreds of thousands of facts. We perform a lot of this work in a big data ecosystem. Our input data is typically Avro Java POJOs and our output models are also typically Avro. Internally we express our rules and values via Clojure records. Time and space performance has been a big concern for us in the rules we write. Clara rules has undergone extensive optimizations over the last few years and it has been able to meet our demanding performance goals.

Based on some of the Clara rules mailing list activity I've seen, I believe others are using Clara for various other domains. The common trend has just been that people are needing to express complex conditional, "business" logic in an appropriately expressive way. Clara rules also has ClojureScript support and it seems that others have used it to some degree on the client-side.

PF.tv: What do you hope people will take away from the talk?

MR: I think it is important to be able to understand what rules engines are in case you end up with a problem that a rules engine could help you with. I want to provide some insight into how rules engines work so that they do not seem to be mysterious black boxes. Having background with how they work can help to understand their performance characteristics and general execution model. This has always been important to me when evaluating if a tool is a good fit to solve a particular problem.

PF.tv: What concepts do you recommend people be familiar with to maximize their experience with the talk?

MR: I'll expect people are familiar with Clojure, but not necessarily with rules engines. It would be advantageous, but not required, to take a look at Clara rules, the Rete algorithm, and forward-chaining rules engines. Wikipedia would be a sufficient place to look over concerning forward-chaining rules engines and Rete.

PF.tv: What resources are available for people who want to study up before the talk?

MR: For Clara rules:

  • https://github.com/cerner/clara-rules
  • https://github.com/cerner/clara-examples

For a good background of the Rete algorithm:

  • The 1995 paper by Robert B. Doorenbos, "Production Matching for Large Learning Systems"
    • This paper is lengthy and discusses a lot more than necessary for this talk. The important part is chapter 2 "The Basic Rete Algorithm".
    • This is an older paper and modern day implementations have extended on these ideas since then. However, I think this paper has one of the best conceptual descriptions of Rete algorithm that I've been able to find. Clara rules' implementation drew inspiration from this paper's description as well.

Jess rules is another rules engine. I think it is outdated in comparison to Clara now, but it runs on the JVM using Java and has a DSL expressed via Lisp. So some similarity to Clara in Clojure can be seen. The important part is that Jess rules has some good documentation that is generally applicable to rules engines, including Clara rules.

This includes:

  • http://www.jessrules.com/jess/guidelines.shtml - on when to consider using a rules engine
  • http://www.jessrules.com/jess/docs/71/rete.html - on the Rete algorithm using Jess as an example

PF.tv: Where can people follow you online?

  • GitHub
  • Twitter - [@MrMikeRodriguez]Twitter
  • I'm in the clojurians Slack as mikerod. I'm in the #clara and #clojure channels, among others.
  • I'm also in the #clojure IRC on freenode as mikerod

PF.tv: Are there any projects you'd like people to be aware of? How can people help out?

MR: Clara rules can always benefit from feedback and contributions. To help out, the important places to look are:

Co ntributing to Clara in the form of reporting issues and fixing issues with pull requests has always been welcome.

PF.tv: Where do you see the state of functional programming in 10 years?

MR: The history of functional programming goes back a long time now. I think there were some fairly straightforward reasons why imperative style programming paradigms were used initially. These reasons were mostly about performance concerns due to hardware limitations in the earlier days of software development. However, the world is much different now as far as hardware limitations go.

Functional programming ideas seem to be getting a lot more attention now and coming into the mainstream. It looks like functional programming languages are taking off more and more, such as Clojure, Haskell, Scala, F#, etc. Even Java, in Java 8, added first-class support for lambdas and high-order logic for collection operations (aka streams). I expect this trend to continue.

The functional approach is proving itself to be a fundamentally simpler way to reason about complex systems. Nowadays, our software problems are less about squeezing every bit of space and time efficiency as we can from a single thread. We now have much to benefit from our powerful CPUs with strong multithread support. Also, now that hardware is cheaper, we can embrace the capabilities of distributed systems. Functional programming is well suited for these approaches.

PF.tv: If rules engines were a superhero, what superpower would it have?

MR: As a superhero, rules engines have the power of being omniscient. The rules engine can remember all facts ever known in working memory. It can see how all facts match with each other and how facts are derived from one another.