What a Clojure Web Framework might look like

One of the questions that came up a couple times was what is a Web framework? What does that even mean? I don't want to get into philosophical discussion about what a framework is. Is it a library or framework or any of those kinds of questions. I'll tell you what I meant, what I still mean by this assertion that we need one.

Transcript

Hey, my name is Eric Normand and these are my thoughts on Functional Programming.

I mentioned, a couple episodes ago, that I think that Clojure needs a boring Web framework and so I got a bunch of people replying. Thank you. That's awesome. It's awesome to know that people are listening.

One of the questions that came up a couple times was what is a Web framework? What does that even mean? I don't want to get into philosophical discussion about what a framework is. Is it a library or framework or any of those kinds of questions. I'll tell you what I meant, what I still mean by this assertion that we need one.

There's a reason why Rails was successful. There's a reason why when we think of Web framework, like that's the big one in the room.

It's more than it was just convenient and made it easy for beginners and stuff like that, but you know, it did. It made Web development really fast, but more than all that, is it's a mental framework for how to code for the Web.

What Rails did was said, "Hey, if you think about your app as MVC, everything else is taken care of for you." If you just think of it as a CRUD app and then you can figure out how to break stuff up in to MVC, you're 90 percent on the way there.

Before that, if you wanted to make a Web app you were thinking about Web parameters and methods and the HTTP methods and the status codes and security and getting your HTML forms right and all that stuff. It was actually quite a lot to keep in your head especially for a beginner. By moving it out of your head and to this configuration over, what is it called?

Default over configuration, something like that. Where it was all set up for you to work right out of the box like even all the routes and everything are just done for you. I think that was what made it really work.

The thing is, I don't think CRUD is the best thing for a Clojure app, especially an app or a Web framework, where you're pushing a similar idea, but I think that's what we need.

It's something where you can give someone a very easy mental model that if they just adopt this mental model that the rest of the job of building the app becomes about incidentals. It becomes more about thinking about the domain and how you would break up your app or your problem into the pieces that it gives you. That's what I mean.

I also said that we need something like WordPress. When I say that, I mean something specific because I don't think we should just re-implement all of WordPress or anything like that. What I mean is something where it's a plug-in system.

The unit of reuse is a plug-in. Currently, in the Clojure Web space, the unit of reuse is the library and integration with your current app is up to you, as the owner of the app. If you want to put buddy in or a friend because you want to use the authentication library, there are several integration points that you have to get right.

You have to add something to your middleware stack and you have to wrap the handlers and is it safe by default and there's all sorts of issues with that. Especially dealing with something like security, it is a big deal, but I think that if you built it as a plug-in system where people will write a plug-in that hooks into the core at known hooking points, known hooks, known integration points, then you won't have this problem.

I say in theory because I haven't written the thing yet and this doesn't exist, but in theory, you can install the plug-in and activate it and it just works and there's no other integration required because it's all done through these known hooks, these published hooks that are part of the core.

In that way, you're actually trying to build an ecosystem, an ecosystem of plug-ins that provide a lot of the functionality that you need. You still get that nice compositionality that we like in the Clojure Web world, but you get it at a different level, because what the Clojure Web world gives you is ring as the de facto platform, which gives you certain integration points like the middleware stack or the handler or the adapter.

Really, that is not...Those are great, and I think it's a wonderful platform, but it doesn't really capture the Web. It doesn't break it. It doesn't really model the Web. It doesn't provide a theory of the Web. If software is all about theory building, Rails, the theory of Rails is that Web apps are CRUD apps.

I'm caricaturing it, but just bear with me on this, OK.

The framework that I'm proposing needs to do is propose a different theory from the ring theory, which is that. Web requests are data to go into functions. Which sounds great because then you're like, "Oh, I'm just in the land of functions. This is what I'm familiar with," but what it doesn't do is ask hard questions or something that you would quickly run into, which is, "Well, I know that the title has to go in a certain place in the HTML."

Like we don't deal with that, that's someone else's problem. I've got a user that I need to login and how do I store a little bit of information about that user, such a common problem, you're going to run into it a lot and ring simply says, "Well, you just have to make it part of the request that gets passed to the function, to the handlers."

Where do you do that? Probably in a middleware, but that's not...It's forcing all the knowledge of how the Web works and how your Web app should work back onto you. I'm going to turn because I'm hitting some wind.

What the Web framework needs to do is take a much more opinionated stance on the Web because we all deal with the same issues all the time. A plug-in could be the user system and it could come in by default. One of the things I've really appreciated about WordPress, running it for a few years now, what I really appreciate is that there's so much built-in, and the integration or the ability to add a little bit of stuff on top is so easy.

Adding a medium amount is a little bit more work, but not much, because now you have to think about making your own database table and how do you install that when the plug-in is activated and stuff like that, but you have this very easy integration point because you can hook metadata onto the users.

You can just say, "Oh, well, I know this little key value thing about the user, I'm just going to store it." Those kinds of things where the user system is standard, it's just the plug-in, and you just want to add one little piece of information about the users, you shouldn't have to do a whole bunch of work because we know that that's very common.

We meaning the Web development industry, we know that that's common, so we should just build that in to the plug-in. Of course, you would have a CMS plug-in that would store posts, their content, their title, and store their revisions and other metadata on it.

It would be a plug-in. In WordPress, that comes with it because it's basically a blogging platform that they've added and added to. Posting and CMS, CMS is content management system. It means a blogging system, but you could add that in as a plug-in in case you wanted it. It's very common to have that.

Then, of course, the CMS could come with an editing interface. I haven't talked about that. This is just an idea, but that plug-ins have both the server-side code that runs and the interface side code that runs to have a single page app for the admin interface.

It could be as simple as, here's some JavaScript that you should provide in the admin interface and it probably creates a react class or something, so it would easily connect up with reframe and etc., and then that would be it. That would be your integration into the admin interface. You could have an API plug-in that provided like a rest style API.

Really, the possibilities are endless, but the key thing is to give the base that has a really strong foundation in...Anything that you could say you're probably going to run into this within the first few weeks of writing your app, we're just going to save you all of that time. That's what I mean by framework. What I don't want is something where...

I respect Luminus, and the work that went into it and all the goals of Luminus even. I think that they're valiant goals, and even I think as a thing itself, it's pretty good. I don't think it's a good solution for beginners because what it basically is a standard ring app that is x percent already done. They just let you code it, edit the code, just add to this.

What that doesn't let you have is that plug-in system where...Like I said before, plug-in as the unit of sharing, as the unit of composition. I can't write a system, a plug-in for Luminus, and then people can just install it.

I could write a thing that is made to be used in Luminus, and then provide documentation for how to integrate it in, but I really think we need something where the ecosystem is based more around these reusable components that just drop into place. In Rails, the way a lot of those things work is, there's like a standard way the request is processed and it provides hooks, and a lot of those hooks are just [inaudible 15:55] monkey patch, something.

That's fine, but Rails [inaudible 16:06] ...Sorry, the wind, you probably cannot hear me. I don't think that that's very safe and a lot of people move away from Rails because of that. It's not the Clojure way we do stuff like just redefining functions that don't do exactly what we want, we don't do that. What we do is provide a lot of introspection.

Once you have this hook system, where you can have plug-ins that declare like, "Well, before you pass the request on, please process it through this function."

That kind of hook. If you have that, you can actually introspect into that. You can say, "What are all the plug-ins that use this hook?" or "What are the available hooks?" You can have online documentation about what hooks are available.

Of course, plug-ins can provide new hooks, so you can see what other hooks are available once you install this plug-in.

You can see where did this hook come from, what plug-in put this in, and what line of code was it. I just feel like that is the level of added features that Clojure should be providing is that once you have a dynamic run-time like Clojure where you boot up the server, it is just running and things are shared in memory like all the hooks and stuff and what's registered already, then you can start introspecting into that.

It's a totally boring system at the bottom but with great tooling on top. A lot of the attempts at Web frameworks so far have been, "Hey, let's do something and let's focus on ease. Let's focus on quickness to write a handler or let's just make it very neat." I think that we need something that's maybe a little bit more robust at first.

Start with the robustness, the boring and then to spend a lot of time and effort, lot of work building the plug-ins out of the parts that we already have. Just doing the grunt work of saying, "Oh, this library is really cool. We are going to build it into a plug-in so that this integration point in the library you're supposed to put this thing in the middle of a stack."

"We're actually going to rewrite or adapt it to fit into the hook system." Boom, now you have that as a plug-in and have a way to a repository to list the plug-ins that are available and make them easy to install really quickly. You see all this is boring. It's stuff that we already have in other libraries or other ecosystems. WordPress has a plug-in directory. Rails has the gems.

I feel we need some boring stuff like that. Just build out the thing. Spend the time in the work instead of trying to bootstrap a thing. That's the curse of LISP where things are so powerful. We have such a powerful language that we are so used to having super high leverage.

Where it's, "Look, I wrote this macro. It generates at the same time a route. It outputs HTML. It does all your Web stuff for you. It's 20 lines of macro. That's my framework." This is a lot of leverage in 20 lines of code but that's not enough. That's not what people want.

People want full framework experience of everything that you would ever need is available. It's something very common. It already exists. This idea of a quick win it's destructive to the effort. It's going to take a lot of work, a lot of time, a lot of collaboration, a lot of fights, meaning disagreements over how the thing should actually work.

We're not going to get a framework that's widely used without all of that. Those are my thoughts, a little bit of ranting. Thanks so much. I like these. I like doing these things. Please subscribe wherever you found this. I'll see you next time. Bye.