Clojure Gazette 119: Continuous Integration Special

Written by Eric Normand. Published: March 22, 2015

Continuous Integration

Clojure Gazette

Issue 119 March 22, 2015

Editorial

Hi Clojurists,

We don't know how lucky we have it as developers today. There are stories on the internet about a phase of development called "integration". What happened was after planning, there was development. And after development, you would spend a huge amount of time integrating everybody's work together for the first time.

Integration would take months. Sometimes it would take longer than development. Engineers considered integration to be so painful that they would put it off till the last possible moment. Some thoughtful people realized that the longer they waited, the harder the integration would be. So they advocated doing the painful thing more often. By doing it often, developers were forced to communicate sooner and more frequently. Integration fixes became small.

Then Extreme Programming (XP) came along, with its idea of "If something is good to do often, do it all the time". So XP advocated for integration several times per day. The extreme ideal would be to integrate continuously: every keystroke that resulted in a good build would go straight to the master branch. This is still impractical, but tooling gets us close (once every commit). Git came along with much faster merging and improved diffing. Automated testing has come a long way. And Continuous Integration servers, both open source and hosted services, are easy to set up.

CircleCI asked me to write an issue exploring the workflows, practices, and tools that help achieve continuous integration. Continuous Integration is more than a service. It's a practice that aims to make a release candidate on each commit. That means that some discipline with branching, merging, and testing is important.

Please enjoy the issue and thank my sponsor, CircleCI.

Rock on!
Eric Normand <eric@lispcast.com> @ericnormand

PS Please tell your friends about the Gazette! It's a great way to show support.

PPS Learn more about the Clojure Gazette and subscribe. Learn about advertising in the Gazette.

Sponsor: CircleCI

I am so proud to have CircleCI as a sponsor. Every now and then you find a programming tool that doesn't just make you do your job better, but it makes your life better. CircleCI is one of those tools. It makes software development more relaxing by doing the tedious work of building, testing, and (potentially) deploying every commit you push. It's free to sign up and use. And you can pay for more concurrent builds when you need them. Please support the Gazette by signing up for CircleCI and trying it on one of your GitHub projects. Getting set up took me less than 20 seconds.

Continuous Integration (Martin Fowler)

The most prolific writer about Continuous Integration and related practices is Martin Fowler. Martin Fowler has written the canonical reference to Continuous Integration. This text is a little dated (he recommends Subversion) but the principles are clear and solid. He gives an outline of the practices and principles of CI, along with an detailed explanation of the benefits.

Understanding the GitHub Flow

A short guide to the recommended use of branches, pull requests, and merging using GitHub.

Feature Branch (Martin Fowler)

Martin Fowler discusses the dangers of using feature branches (creating a new branch for each feature). If the feature takes more than a day to be pushed to master, continuous integration is not achieved.

However this kind of refactoring on a feature branch is awkward because it makes the Big Scary Merge much worse. The result we see is that teams using feature branches shy away from refactoring which leads to uglier code bases.

Indeed I see this as the decisive reason why Feature Branching is a bad idea. Once a team is afraid to refactor to keep their code healthy they are on downward spiral with no pretty end.

Feature Toggle (Martin Fowler)

Martin Fowler explains one method for dealing with features that take longer than one day to write. Instead of developing them in a separate branch and merging at the end, you wrap the code for that feature in a toggle (essentially a conditional) and merge it immediately.

Release toggles are a useful technique and lots of teams use them. However they should be your last choice when you're dealing with putting features into production.

Your first choice should be to break the feature down so you can safely introduce parts of the feature into the product. The advantages of doing this are the same ones as any strategy based on small, frequent releases. You reduce the risk of things going wrong and you get valuable feedback on how users actually use the feature that will improve the enhancements you make later.

Decoupling Deployment and Release- Feature Toggles

Frankly, Feature Toggles scare me. But lots of companies use them. This article explains some of the discipline required to use them effectively.

In a new world most of web-scale companies have decoupled the deployment from release process. Decoupling the deployment from the release helps to achieve the zero-downtime release. With Zero-downtime release, switching user from one release to another happens instantaneously. It also helps to instantaneously rollback to previous release when required. This decoupling also enables dev and ops teams to deploy new versions of application continuously, completely independent of the decision as to which features are available to which users.

Conclusion

Continuous Integration is a useful practice for getting all of the developers communicating about our changing code. Underlying it all is the severe cost of unused code. Code rots. The longer it is fermenting off of production, the most costly it is. The first cost is writing the code. The second cost is maintaining that code in the face of a changing codebase. And the third is integrating it back into master. All of these costs are accrued before the code ever adds value to the product. Regardless of the practices and services we choose, we should be aware of these costs and work to minimize them.