Re-frame Database Best Practices

The Re-frame Database is the central store for data in your Re-frame application. It will be read from and written to by many different parts of your app. And the Database will evolve quickly as your application evolves. How do you make sure your database can evolve? How do you avoid building yourself into a corner? This guide collects best practices from my own experience and the experiences of others for naming, structuring, and accessing your Re-frame Database.

Table of Contents

• Introduction
  • What is Re-frame Database?
  • Your Database Will Evolve Over Time
• Naming Recommendations
  • Events
  • Subscriptions
  • Other considerations
  • Event and Subscription Naming Summary
• Structure Recommendations
  • Use a map
  • Initialization
  • Liberal subdivision with nested keys
• Database Access Recommendations
  • Use assoc-in and update-in
  • Handle empty case
  • Get deeply nested stuff out of the database
  • Keeping Events and Subscriptions together
  • Calculate stuff in Subscriptions
  • Recombining Subscriptions
• Indexed Entities Pattern
  • Indexed entities
  • Dealing with order
  • Nesting the refactoring
• Conclusions

More posts in this Re-frame Series

• State in Re-frame
• The Re-frame Building Blocks Guide
• Guide to Reagent
• React Lifecycle for Re-frame
• Re-frame Database Best Practices ← you are here
• Re-frame, a Visual Explanation
• Optimistic Update in Re-frame
• Timeout Effect in Re-frame
• Why Re-frame instead of Om Next
• 6 things Reacters do that Re-framers avoid

Introduction

What is Re-frame Database?

The Re-frame Database is a central, global store of your application's state. The Database is essentially a single Reagent Atom, but you can't access it directly. It is modified through Event handlers, and read through Subscription.

Evolving a database over time

The challenge with the Database is the same challenge any central, global store has to deal with. When you're first starting your app, you don't know exactly what data you will need, nor will you know how to structure the data you have so that it can be used easily. You basically make guesses, and they often turn out to be wrong.

Meanwhile, your application code is going to be reading and writing that data in the Database. It's going to make assumption about what is available and how it is structured. For instance, a list view component might assume that the data it needs to show is in an ordered sequence like a vector. This means that as your application grows, it could ossify the initial mistakes in your Database structure.

Luckily, Re-frame's Database solves exactly this problem. The Re-frame Database is not accessed directly. It is accessed through a layer of indirection. For reading from the Database, the indirection layer is built with Subscriptions. For writing to the Database, the indirection layer is Events. Both of these help separate our application code from the structure of the Database.

The consequence is that the Re-frame Database, and its structure, is not a source of coupling in Re-frame apps. If you're doing Re-frame correctly, you're naming your Events and Subscriptions. Events and Subscriptions form a single layer of indirection around your Database. That indirection isolates your Database's structure from the rest of your application. In short, you can change the Database structure without affecting your Components. And you can change your Components without affecting your Database. Re-frame gives us a way to make evolving the Database relatively straightforward.

However, this means that we've promoted good naming to a first-class problem to allow us to evolve things. So let's talk about naming.

Naming Recommendations

There are two things we have to name, Events and Subscriptions. Naming is hard. But it has tremendous value. It means your Database can evolve over time without totally rewriting your app. So you can relax and be a little exploratory with how you build your Database. Here are are some recommendations for naming.

Events

Events capture a fact about something that happened. When the User clicks a button, we want to react to that, so we capture the information as an Event. And if we get a response from the Server, we capture that as an Event. The role of the name of the Event is to capture that information so we can reconstruct what happened, but not in a way that is tied to the details of the UI or what Effects the Event will have.

Let me give an example:

Let's say we have an Add-to-Cart Button on an e-commerce site. When the User clicks that button, we update the Database to add the correct item to the cart. How do we name the Event? We could call it :click-add-to-cart. But that is tied up with a UI concept---clicking. What if on mobile phones we decide to make it a drag action instead of a button click? We want to isolate our Event names from any particular UI decisions.

We could name it :store-item-in-cart-db, which is essentially what will happen. But that, too, is tied to the Effect it will have. If we change the way our app works (for instance, we have it backed by a server so we're doing Ajax instead of using the local application Database), our name no longer represents reality.

The key here is to think about the user's intent. What are they intending to do when they click that button? They want to add the item to the cart. So we should call it something like :add-to-cart. The event could take the item's ID as a parameter.

Now, let's say we send an Ajax POST to the server when the item is added to the cart. This happens as an Effect returned from the Event Handler. The Re-frame way to handle the response is to fire a new Event---actually, one Event for a success and one for a failure. How do we name these Events?

We could call it something like :cache-cart-from-server. This is accurate, but it's too tied into the Effect---namely, caching. We could call it :cart-ajax-response-success, which is also accurate. But it's too much detail about where it came from. What if we switch from Ajax to WebSockets?

Again, we need to look at the intent. What is the intent from the server? In the case of a success, we could say the server is confirming the item is now in the cart. So we could call it :confirm-add-to-cart. And for the failure case, the intent is to deny that it was added to the cart. :deny-add-to-cart.

We see something interesting here: we want the Events to be named in terms of domain concepts, not technical concepts. Shopping cart actions are part of the domain of e-commerce. But clicking and Ajax responses are technical choices.

Subscriptions

Subscriptions have a bit of an easier naming problem, but it's basically the same recommendations. Subscriptions should name the value they will return, in domain terms, not UI terms or other technical terms.

Here's an example: let's say you want to have a little icon in the header that has a little shopping cart picture with the number of items in the cart. You'll need a Sub scription to get the number of items so you can show it in the icon Component.

You could call the Subscription :icon-badge-number. But that ties it way too much to the particular Component you're expecting to use it in. It's about the UI, not the domain concept. You could call it :get-cart-items-and-count-them. That's technically accurate, because that's how you'd implement it. But you're trying to isolate things from the implementation. You should call it :cart-items-count.

Other considerations

There's a lot of general naming recommendations in the software engineering world. One source of ideas is Command-Query Responsibility Separation (CQRS) community. They tend to name their Events in the past tense to denote that it is something that happened. They'll say :item-added-to-cart instead of :add-to-cart. In turn, they name their "commands" in the imperative. You can adopt this if you want, but I don't find it to be too useful, since we're not terribly interested in distinguishing events from commands. Effects in Re-frame are not given semantic names. But Events in Re-frame usually do have a verb.

Another thing you might notice is that Subscriptions are often named with nouns. They denote a thing, not an action. You might find this helpful to make things more clear.

Event and Subscription Naming Summary

• Name so you can reconstruct what happened later
• Names should reflect the domain concepts not the technical concepts
  • Events names should capture the intent, not their effects and not the UI action
  • Subscription names should describe the data, not the implementation and not the Component

Structure Recommendations

Use a map

The Database, at the top level, should be a hashmap.

Initialization

The first Event your app dispatches should be one to initialize the database. I like to call mine :page-load or something like that. It captures the fact that the app was loaded in the page. It should set the Database to some known value and it can also do other stuff like start fetching data from the server. That's up to you. But you should initialize the Database.

Liberal subdivision with nested keys

I like using sub-maps liberally to subdivide my state into smaller maps. For instance, I could have a section just for user/account information under the :user key. And everything to do with product information under :products.

Database Access Recommendations

Use assoc-in and update-in

The reason we can nest as deeply as we want is that we have two really great functions that let us modify values at arbitrary paths in the nested map. Those two functions are assoc-in and update-in.

If you want to set or reset a value at a given key path, assoc-in is the tool you should grab. The nice thing is it will create intermediate maps if they are missing. You can use it with vectors, too, but you should provide integer keys.

(assoc-in db [:user :address :zip] "72773")
;;           ^ key path            ^ value

If you need to modify a value at a key path, meaning you want to take the existing value and apply a function to it, use update-in:

(update-in db [:user :score] inc)
;;            ^ key path     ^ function

It also will create intermediate maps.

Handle empty case

There's one catch: if you use update-in and there's no value there, how do you handle that? Like with any map, a missing value will give you a nil. So the function you pass will get nil as an argument. That's not good for a whole lot of functions. In the example above, inc will get a nil and that will throw an error. It's expecting a number. How do you handle it?

The idiomatic way to handle it is to use a function called fnil to set the default. Here's how you should really write the example above:

(update-in db [:user :score] (fnil inc  0))
;;            ^ key path           ^ fn ^ default

In this case, if the score has never been set, inc will be called on 0. So the first time we increment the score, it will be set to 1. Try it at the REPL!

So check out what happens when we do update-in on an empty map:

(update-in {} [:user :game :score] (fnil inc 0))

  ;; => {:user {:game {:score 1}}}

Notice that it created all of the intermediate maps we need. It means we can start with an empty Database and build it up as we need it.

Get deeply nested stuff out of the database

Okay, so we've started building up a deeply nested Database. How do we get it back out? The answer is to use get-in in Subscriptions. If we've got the score at the path [:user :game :score], we can get that score using this Subscription:

(rf/reg-sub
  :user-score
  (fn [db]
    (get-in db [:user :game :score])))

get-in is the complement of update-in and assoc-in. It lets us get stuff out from deeply nested Databases. It can even give us a default value, if we so choose, in case the path leads nowhere.

(get-in db [:user :game :score] 0)
;;                              ^ default value

Keeping Events and Subscriptions together

Now I hope it's clear that Events and Subscriptions are tied together. They are coupled because they both need intimate knowledge of the structure of the Database. They both need to change when the Database changes. That's why I like to keep Events and Subscriptions that touch the same part of the Database together. For instance, all of the Events and Subscriptions dealing with the shopping cart could go near each other or at least in the same file. When I want to change where in the Database things are stored, I can see everything that needs to change within close proximity.

This recommendation actually contradicts the structure given to you by the Re-frame Template. That template has separate namespaces for Events and Subscriptions. The template is wrong in that way. I would much rather have a shopping-cart namespace with Events and Subscriptions mixed together. I will put everything that touches something under the :cart key in that namespace. Those things will change together, so they belong together.

Calculate stuff in Subscriptions

Now, you may be faced with a choice. Let me set up the example. If we have a shopping cart, you'll probably want to display the number of items somewhere. Should you store that count in the Database?

In my opinion, no, you shouldn't. You should calculate it in a Subscription. Let's say we have a Subscription that returns all of the details of the cart item:

(rf/reg-sub
  :shopping-cart-items
  (fn [db]
    (get-in db [:cart :items])))

Now, we can do something cool with that original Subscription. We can chain another one off of it.

(rf/reg-sub
  :shopping-cart-count
  (fn []
    (rf/subscribe [:shopping-cart-items]))
  (fn [cart-items]
    (count cart-items)))

Notice, this time we're making a Subscription with three arguments. The first one is familiar: it's the name. The second one is a function that returns a Subscription. Notice the Subscription returns the items from the cart. Then in the second function (the third argument), we are calculating the count of those items.

It's much easier to calculate these things reactively, as they change, than to try to keep the Database up to date.

Recombining Subscriptions

When you've got so many Subscriptions, it may seem like you'll never get relevant data back together again. But don't worry, you can recombine Subscriptions into new Subscriptions. You can t ake two or more and calculate a new value from their values.

Just for fun let's say we've got coupon codes on our site. The user loads one up, and so it should affect the prices of all items in their cart. We have a Subscription that returns the current coupon code and its % discount. And we have one that shows the items in the cart. We can make one that combines those two into one that shows the discounted items.

(rf/reg-sub
  :discounted-cart-items
  (fn []
    [(rf/subscribe :shopping-cart-items)
     (rf/subscribe :coupon)])
  (fn [[items coupon]]
    (mapv #(apply-discount % (:discount coupon)) items)))

In this combined Subscription, we don't even care where the data for the other two Subscriptions is in the Database. This Subscription will get recalculated each time either of the other two change.

You should be moving as much of this kind of calculation as possible out of your Components and into Subscriptions.

Indexed Entities Pattern

Indexed entities

Let's say I've got a Database structure that has evolved to look like the following. There's a section for the product listings and a section for stuff I've added to my cart:

{:products [{:id 123
             :name "Bag of holding"
             :price 40
             :description "..."}
            ...}]
 :cart [{:quantity 2
         :item {:id 123
                :name "Bag of holding"
                :price 40
                :description "..."}
       ...]
 ...}

It works fine until you get an update from the server changing the description of the item. You have to update it in two places! Every Event has to know to look into each of those places. Plus, there's another problem. How do you write the update? The products are in vectors. So you have to iterate through the vector, looking for a product who's :id matches. Inefficient and hard to get right.

The solution is to normalize the data in this Database. We'r e going to do this by applying the Indexed Entity pattern, which is a way to solve both problems. It removes duplication and makes reads and updates much easier. At the same time, we can keep the same data in our Subscriptions. That's what evolving our Database should look like.

In this pattern, we will treat products as entities, we will pick a place for them to live, and we index them by id. The refactoring is easy: we want to make maps with the keys being the id. Here's what it will look like:

{:products {123 {:id 123
                 :name "Bag of holding"
                 :price 40
                 :description "..."}
           ...}
 :cart [{:quantity 2 :item 123} ...]
 ...}

Notice that we've gotten rid of the duplication. We can write our update product Event as:

(rf/reg-event-db
  :update-product
  (fn [db [_ product-info]]
    (assoc-in db [:products (:id product-info)] product-info)))

And we can write a Subscription to get any product by id:

(rf/reg-sub
  :product-info
  (fn [db [_ product-id]]
    (get-in db [:products product-id])))

Dealing with order

Okay, but we've lost something. Before we had the products in a vector, so we know what order to show them on the products page. Now we have them in a map, so we can't guarantee order.

That's okay! Because we can re-introduce a vector whose sole purpose is order.

{:products {123 {:id 123
                 :name "Bag of holding"
                 :price 40
                 :description "..."}
           ...}
 :product-list [123 ...]
 :cart [{:quantity 2 :item 123} ...]
 ...}

Then we have a Subscription to return the complete list of products.

(rf/reg-sub
  :product-list
  (fn [db]
    (mapv (fn [id] (get-in db [:products id])) (:product-list db))))

I like to think of this as a join, like you see in traditional relational databases. You're joining the products map with the ordering vector to produce one collection that has all of the information.

Nesting the refactoring

We have another problem: how do we update the quantity of an item in the cart? We have the same problem as before. They're in a vector and we'll have to iterate through the vector, looking for a matching id. Inefficient and inconvenient.

The same pattern can be applied here. We'll make an index and add an order.

{:products {123 {:id 123
                 :name "Bag of holding"
                 :price 40
                 :description "..."}
           ...}
 :product-list [123 ...]
 :cart {:quantities {123 2 ...}
        :order [123 ...]}
 ...}

Adding an item to the cart becomes:

(rf/reg-event-db
 :add-to-cart
 (fn [db [_ product-id quantity]]
   ;; check if it's already in the cart
   (if (nil? (get-in db [:cart :quantities product-id]))
     ;; not in the cart, add to quantities and order
     (-> db
       (assoc-in [:cart :quantities product-id] quantity)
       (update-in [:cart :order] (fnil conj []) product-id))
     ;; in the cart, add to existing quantity
     (update-in db [:cart :quantities product-id] + quantity))))

It's easy to access the cart items using a Subscription (doing another join):

(rf/reg-sub
  :cart-items
  (fn [db]
    (mapv (fn [id]
            {:quantity (get-in db [:cart :quantities id])
             :item (get-in db [:products id])})
      (get-in db [:cart :order]))))

We're joining three things: the order, the quantities, and the product information. The Subscriptions are returning the same data they had before, so your components won't need to change. However, the Database is much easier to work with. It's a win!

Conclusions

Applications evolve over time. We start out with a simple Database and access the data directly. But as our our requirements change and our understanding of the problem deepen, we increasingly need to be able to modify our code. However, as our codebase grows, what seemed like simple and direct access now seem like a simple and direct path to tightly coupled spaghetti.

Re-frame, as an application framework, gives you tools to truly separate out your views (Components) from your state (Database). It does that with a simple encapsulation layer made of Events and Subscriptions. They may seem like boilerplate at first (they did to me!) but as your application evolves, you'll come to find that they are the bedrock of your application. Components change and your Database structure will change, but the names of your Events and Subscriptions will lay a semantic foundation that you can build on.

What Re-frame gives you is not magic. It's just some tools to help you isolate different concerns in your application. It will depend on you, with hard work, to maintain the separation. I don't know if any framework could do that work for you. However, Re-frame does have something going for it---the guidelines are very clear and easy to follow.

Name your Events and Subscriptions well, make Components about HTML generation, make your Database convenient to read and update, and do calculations and joins in your Subscriptions.

More posts in this Re-frame Series

• State in Re-frame
• The Re-frame Building Blocks Guide
• Guide to Reagent
• React Lifecycle for Re-frame
• Re-frame Database Best Practices ← you are here
• Re-frame, a Visual Explanation
• Optimistic Update in Re-frame
• Timeout Effect in Re-frame
• Why Re-frame instead of Om Next
• 6 things Reacters do that Re-framers avoid