Why do we use Web Frameworks?

Summary: While contemplating a Clojure web framework, I explore the reasons we use web frameworks in general. I conclude that the framework should support a learnable development process.

People often lament the lack of a Clojure web framework, despite several alternatives existing. The choices must not be meeting their needs. I've been thinking about what a Clojure web framework should look like. That got me thinking about what the purpose of a web framework is, and what the needs are that are not met.

Why do we use web frameworks in general?

The web is complicated. You've got to know HTML, CSS, JavaScript, your backend language, probably SQL. And there are security holes everywhere. The HTTP spec is huge and full of corner cases. We would like something to reduce this complexity by handling it for us. Clojure has HTTP libraries but they require a good understanding of how the web works. Further, they only add to the number of things you need to learn (learn HTTP and Ring). A framework can turn 1000 things to learn into 30.

There are many expected features not provided by the spec. Many apps need user signup, password reset, authorization, API's, logging, database persistence, sending emails, payment processing, metrics gathering, deployment, input validation, ... We would like to implement them once and reuse them if possible. A framework means even the integration of those implementations is trivial.

The web is full of security holes. Rolling your own security is costly and dangerous. A framework can integrate security features into its operation and aggregate bug fixing efforts to make something more robust. A framework, at least in theory, can be secure by default.

Writing an app requires many small decisions. Many, if not most, of those decisions are arbitrary. For example, how to name your tables, the specific routes you choose, what file to put your code in, etc. If the framework made those decisions for you, then you would reduce the cognitive burden on the programmer and leave more brainpower for the important decisions.

The #1 real reason to use a framework

These are all good arguments and solving these things would really help web development in Clojure. However, I have come to believe that there is an even more important reason, which comes from discussions with Jen Eliuk and others.

framework : a: a basic conceptional structure (as of ideas) : b: a skeletal, openwork, or structural frame

A framework is primarily cognitive and secondarily physical. The "conceptional structure" is more important than the code. The framework gives you a process for turning an idea into code. On the other hand, a bunch of libraries gives you code out of which to build your ideas, but no process. You must understand each library enough to get them to work together. A framework inverts that and says "understand this process and we'll take care of the rest".

As an example, in Rails the process is about breaking up your feature into three parts. Your Controller, your Model, and your View. The View is your HTML with no logic. The Model is the "business rules". The Controller is the code that converts a web request into actions on the Model, then displays the View.

As experts, we tend to want control of the entire system. We ask "What if I don't want to use MVC?" We understand so much, from years of experience, and know how we want things to work. So we trade ease for control. We say "I don't mind if it's hard because I know how it should work."

Beginners, on the other hand, would much rather have something be easy. They're willing to learn, but they have limits. They will accept something less than perfect as long as they can solve their problems. They much prefer ease over control. Even the choice of which file to put some code in can be a big decision for a beginner. The easier you can make it on them, the more forward motion they can have.

Am I saying that a framework caters to beginners? Partly, yes. Since the number of professional programmers is doubling every five years, at any point in time, half of programmers have less than five years of experience. They need a process to follow at first which will let them gain experience. On the other hand, ease benefits everyone, including experts.

Conclusion

People use a framework primarily to lessen the mental burden of developing on the web. By giving people a learnable process, the framework should allow beginners to make progress and experts to progress quickly. The process should be the primary focus; the framework should exist only to support it. What could this process look like in the Clojure world? How can we keep it learnable?