Haskell and Clojure in Production

I was delighted to have a conversation about types with Richard Feldman on the Software Unscripted podcast. We were answering the question: Why doesn't Eric get more type errors in Clojure?

Transcript

[00:00:00] Richard Feldman: You're listening to Software Unscripted. I'm your host, Richard Feldman. On today's episode, I'm talking with Eric Normand, the author of the book, rocking Simplicity from Manning Publications and the host of the Eric Normand podcast. He's one of the few people I know who's had production experience writing both Haskell and Clojure, and today we get into a gigantic food fight about static and dynamic types.

[00:00:19] Just kidding. We talk about his experiences with both and actually get into a lot of specific details about the differences between the two. Not just from a typing perspective, but also from the perspective of ecosystem culture and more. And now Haskell and Clojure in production.

[00:00:36] Eric, thanks so much for joining me.

[00:00:38] Eric Normand: Hey, thanks for having me. It's great to be here.

[00:00:40] Richard Feldman: All right, so I wanted to start off by bringing up again a little like conversation we had over like Twitter DM a while back, and this is based on somebody else's tweet that's now been deleted. So I can't, I dunno what the original wording was, but I think I'm gonna try and recreate it.

[00:00:53] I think what they were asking. Is there anybody out there who agrees? Both of the following statements. Number one, I have used Haskell professionally before. And number two, I also think that most of the bugs that I run into, like a type checker, would not have helped or, or would not have caught them because a lot of people seem to agree with one of those two things.

[00:01:14] Like they'll say, I don't think a type checker's that helpful because the bugs that I run into are, are not type errors. And then a lot of people say, I've used Haskell, but I disagree with that professionally as opposed to like, you know, just like a Weekend Pro project, something. So I've used it in anger and it seems like pretty hard to find somebody who agrees with both of those, at least to some extent.

[00:01:32] And obviously there's nuance there, right? Right. It's not like, you know, anyone's like a hard line, you know, one way or the other. But it seemed like you, you were the first person I talked to who actually seemed like you've had Haskell experience professionally. I think you said like two years, something like that.

[00:01:44] Eric Normand: Yeah, about two years. Mm-hmm.

[00:01:45] Richard Feldman: And also like your experience is that it's. The type checking is not that essential as far as like it doesn't catch as many bugs as maybe like others would say. Is that a fair, I don't know, assessment of your perspective on it?

[00:01:58] Eric Normand: Yeah, so man, this is a really complex answer, so I have to like be a little caveat.

[00:02:05] So I worked in Haskell. From 2010 to 2013 about. Okay, so this was almost 10, I mean, yeah, more than 10 years ago. Sure. So I haven't done it professionally since then, and it might have changed, you know, I haven't been keeping up so much with it

[00:02:24] at that point. It was like, Well, like 20 plus years old. Yeah. Um, so I like the basics of the language are probably the same.

[00:02:31] Richard Feldman: Yeah.

[00:02:31] Eric Normand: Haven't changed. But the ecosystem has probably changed a lot. And, you know, I worked on one code base and so I had to figure out like what was the language, what was the code base I was working on, and what was. Difficulty with understanding it. Right. So I had to really separate these out because I left that job with a kind of negative view of Haskell.

[00:03:00] Interesting.

[00:03:00] And I went back to Clojure, which I preferred. Now I know the, the first thing people are gonna think of is like, Types versus un typed. No, no one. No one would go there. So let's dive right in. So my overall feeling about the whole discussion is that we're not listening to each other, is that we're not empathizing with the person and we put them in a box and then we don't have to listen to them.

[00:03:29] I experience it myself when I say something about types. People are like, oh, you're a Clojure guy. That's why you don't like types. Like I didn't say I didn't like types. I said one specific factual thing. And we can discuss that whether it's true or not. But don't just throw it out because I don't you think I don't like types.

[00:03:49] Right, right. I do like types. I've found that there was a lot of benefit from the static type discipline of Haskell. Okay. So I wanna be really clear about that. I find that I'm, and my, and the people I work with in Clojure are not making buggy or software mm-hmm.

[00:04:10] Compared to like, people writing it or like your experience with With Haskell.

[00:04:14] With Haskell, yeah. So like, I'm trying to be very specific, like Clojure versus Haskell. Mm-hmm. Yeah, because I know it's so easy to go abstract, especially when you get on the defensive and you're like, oh, but type systems can do that. Yeah. But can Haskell's type system do that? I, I can't attack types in general, cuz there's always some type system somewhere that can.

[00:04:38] Handle this thing. And I think that that's one of the things I don't like about the discussion is, you know, it often goes too abstract. Like you're anti types. No, I'm anti this particular type. Like I don't, oh, and that's like a call back to a discussion you had about rich hickey's talk. Maybe, maybe not, maybe not.

[00:04:56] Yeah.

[00:04:56] Richard Feldman: Yeah. Yeah. Like he talks about like the distinction between some type systems will let you do sort of like, or types, like you can say, right, I have this or that, and he's a big fan of that. And yeah, I mean it's a great point. Like type systems are, yes, you can technically, broadly categorize them as one or the other.

[00:05:14] And there are certainly trends having to do with one or the other. But at the end of the day, if you're gonna compare them, it's really much more useful to compare specific type systems.

[00:05:23] Eric Normand: But it's often not what people do, cuz I'll say, oh, but I couldn't do this in Haskell. Right? And they're like, oh, what you wanted was row typing.

[00:05:32] And like, well, yeah, but it doesn't have it, so like I can't do that. Or someone will say, well that's a type error. And I'm like, yeah, but not in Haskell. It couldn't detect that type error. So like it doesn't, it's like calling it a type error. It's like a category or like tell me what language that's practical to use that would've done that and like maybe I would've chosen to switch, but probably not.

[00:05:53] Yeah. Okay. So I'm trying to, Come to this. This is a really complex thing. Yeah. I find that having worked in Haskell for two years, I could internalize a whole bunch. Not all of it, of course. It's a big, you know, there's too much, but a whole bunch of the type discipline into my own being, my own way of thinking.

[00:06:18] And I can use that in enClojure, even though there's no checks. Right. I know when I'm writing a function that I wanna return a string no matter what. I also know Clojure programmers who will say, oh, I'll return a string or nil, or maybe a number, and then I'm like, well, you don't know what you've got anymore and you're gonna need an if statement somewhere else where the return value gets, you know, read to figure out.

[00:06:48] What you got and what it means because now you're like totally outside of the context. Right. And I tried to explain this to them and they don't understand so well.

[00:06:56] Richard Feldman: That's a really interesting perspective. It's familiar to me because at some point in my career, like the, the first professional functional programming that I did was basically what you just described.

[00:07:05] It wasn't Clojure, but it was coffee script and. Essentially what I was doing was like sort of pretend I'm following haskell's rules. That was kind of what, what I thought about it as in my head. So I'm like, okay, let's pretend I had to pick a specific type for this, and also let's pretend that I wasn't allowed to do side effects, except at some point I had to because you, you don't have the same runtime.

[00:07:24] Even though I didn't have a compiler enforcing it, I would still try to follow those rules. Now that's interesting though. I, I haven't heard many people talk. Doing that specifically en Clojure. So I guess it makes sense that a lot of people would, not everybody would be on board with your way of doing that.

[00:07:38] Eric Normand: Yeah, yeah, exactly. And I just found it to be one of the, the best things for keeping the bugs down, right Uhhuh. So like there's something there where like I learned that by writing in Haskell and it forced me because I couldn't compile my code if I didn't think ahead and like, will this compile like, well I'm gonna.

[00:08:00] Correctly as close to correctly as I can the first time. So having gone through that, I find that I don't need it. Right? Yeah. But at the same time, I have to work on teams and you know, maybe I would want it if I was on a team, because I wanna make sure everybody else, like if, you know, if I was the lead of the team, right?

[00:08:26] I would think these people are not following a type discipl. So I would want that to force them to do it. And so it's sort of like, you know me saying, yeah, types are really. It's useful to at least think about the types and have a strict discipline with them, but you don't need it to be part of the language, but it can be very helpful to be part of the language.

[00:08:48] So I'm not coming down either way on this. And I also think that there's more practical type systems out there now. You know, like type script I find is pretty good for the kinds of stuff I wanna be doing. And so where I've gotten after all this like. You know, introspection is, I think that what both sides need.

[00:09:13] So what Clojure needs and what Haskell needs. Let's talk specifics. Clojure needs a very strong type system that's optional, so you can apply it to certain parts of your code because Clojure really thrives. In the place where you have a lot of unknown unknowns, you don't know what data you're gonna get back.

[00:09:34] And so you don't wanna commit to a type yet. And also, it's maybe not even typable, right? You don't know. And, and for instance, here's an example. Dealing with a WordPress api, like this is a PHP program generating J S O N. There's no type discipl. And so sometimes you would get, you know, a J S O N object.

[00:09:59] Sometimes you get a null, sometimes you get false or zero, because in PHP, zero is equal to false. Right? And so you can't say like, oh, here's the type of this, it's an unknown unknown. At any point you might get a zero that you weren't expecting, right? And I know people say, oh, you can deal with that in Haskell.

[00:10:19] Yes you can. But would you arrive? Is basically Clojure's data model. If you push it to the limit, right, and then Haskell with its strict types, it is better when you do have a lot of known knowns. Right Or known unknowns. The known unknowns. Any, any of the known stuff, it's really useful to deal with because you can just put it in there.

[00:10:41] You can make a flexible type if you need it, cuz you have known unknowns. And if you happen to know the type and you trust it, you can put it in there. And so what I think is both of them need the other side. Like in Clojure sometimes I'm like, I have a lot of no knowns, I wanna write it in the code so that it's.

[00:11:00] For me, biologic engine, like gimme a type system just for those situations. Likewise, I think Haskell needs to have some way of dealing with these unknown unknowns by getting the data into the system. So you have a J S O type that's just like, I know it's J S O, it parces, but that's it. And then a bunch of routines that are probably partial function.

[00:11:25] For dealing with that. So when I say a partial function, I mean you can have a map function that can operate on a raise and it just errors on everything else, right? Yeah. And it deals with that J S O array, but everything else it fails on.

[00:11:38] Richard Feldman: So this is interesting. So the approach that I've used in Elm, and I know you can do this in Haskell, but there is a sort of a question of.

[00:11:45] How much it's done. Like it's certainly quite popular to auto derive your J S O N sterilization in Haskell and basically say like, here's my data structure. I want you to put something in here and if it doesn't look exactly like that, then give me an error. And I totally get how. If you're getting something where, let's say you don't have a well-defined schema, you're like, okay, this thing, it says it's gonna give me back a string.

[00:12:09] But I've seen in practice that, although that's what it says, sometimes it gives me back a null there, or sometimes it gives me back a zero instead of a false or, you know, stuff like that. You know that there exists, hypothetically, some schema conceptually in the world that can describe this thing, but you don't know what it is and so you need to be resilient to that.

[00:12:27] Some. But I have seen a middle ground, as I would say, in like what Elm does. So in Elm U specify these J s O decoders. And one of the things you can say is you can say basically this field can be one of these several different alternatives. And I don't know which one I'm gonna get, but here's how I want to handle each of these.

[00:12:45] So you could say it's could be zero, or it could be false, or it could be null. And if it's any of those, I wanna handle all those by turning it into a false or something like that. Right. And so in my mind that's a way of handling that. So then once it has been decoded from then on, you have your normal elm type and everything.

[00:13:01] You know, the type checker is y'all, you're all in the known knowns land, and the type checker can be very helpful. Well,

[00:13:06] Eric Normand: you're in a known unknown, right? So you know it's one of these, but you don't know which one until you do the pattern match.

[00:13:13] Richard Feldman: Right? Okay. So in this case, I'm assuming, We don't care about preserving the original Js o Uhhuh, we're just like, I just wanna translate it into something I'm gonna work with.

[00:13:22] Yeah. I don't care. After I translated it, whether it started out as false or zero or whatever, I just need it to be in the format that I wanna work with. Yes. Right. And then on. Right, right, right, right. So in that world, once you've done that translation, it becomes, everything is known. Right, right, right. You know, when it's coming over the wire, you don't know what it's gonna be, but if you've written your decoder to be flexible to those different possibilities.

[00:13:43] Yeah, and I would even go as far as to say, if you really wanted to, you could, you could get kind of extreme with it and just write some incredibly flexible decoder that's like, I will treat empty string as false and I will treat, you know, empty array as false. Like for some reason, I, I probably wouldn't go that far, but you could do it.

[00:13:58] And from my perspective, that is how I really like to deal. That sort of like uncertainty is, is right at the border as opposed to like, you know, after I've turned it into my own data types, I'm like, let me resolve that. And if I, uh, just centralize all my ambiguity, resolution, logic right in that decoder.

[00:14:16] And then once I've got into my types, I'm like, cool. Now the rest of my system works the way I like it to, and if I can't figure out a way to deal with whatever, you know, nonsense they've given me, then I'm gonna fail right there. Sure. That's kind of where I've come at

[00:14:28] Eric Normand: it. Yeah. Yeah. And I, and I totally understand that.

[00:14:31] I think that. Another aspect of Clojure that's like orthogonal to the type versus UN type, which is the, the ripple driven development, right? We have a much more interactive view of, you know, interfacing with the outside world, and so we wanna get it into the system as quickly as possible. And basically with as little translation as possible, right?

[00:14:56] So we take that raw json and convert it into the data structures that maps and vectors and strings that we know how to deal with, and then look at it. What is it? What did it give me? Right. Let me make, let me try to write a decoder. Right. And I think that there's a mindset that I didn't see in the Haskell world of sort of trying to capture things like just one step too soon.

[00:15:22] Capture it in types one step too soon. Like let me get the raw thing. I saw this a lot with like c SV parsers and even stuff like the lib curl binding. That we were using. Now again, this is more than 10 years ago now, so like it could all have changed. But like the CSV parser enClojure, all it does is gives you a list of vectors of strengths.

[00:15:49] That's all the parser does because that's kind of all you can guarantee about any given C S V in general. Yeah, just like A J S O parser should just turn it into data structures that you're familiar with in this philosophy. Right. It should not try to turn it into something else and give you like coercion to some other type or things like that because you want the raw.

[00:16:12] You want to look at it in its pure form before you start making assumptions about it, because you really don't know. And so I do agree that like at some point you need to check the stuff at the border, but there's a step in there where you're in development, you're programming the system, and you're discovering what you.

[00:16:34] At that point, if you have already closed the gates to anything that doesn't match, you're gonna have this loop of, okay, oh, it print out to JSON and try to pars it and like, oh no, it doesn't like that. Zero. My parser doesn't like the zero. So like, let me make it, let me convert the zero into a empty list, whatever it is.

[00:16:54] And then, oh no, I, this time I gotta null. And like that could be hours in. To a run. You know, you could be like processing every post on this WordPress blog. Right. Two hours in and it's like, oh, I failed. I found something I haven't seen before. And like, what do you do with that? Whereas you can, I don't know, kind of handle it a little better.

[00:17:16] If you have this raw thing, you know, you can put it in a dead letter list, you can, you know, do all sorts of stuff. I find this attitude of typing it and putting a guard at the border is like missing. Like, but, but maybe you wanna let them in like you need a. You know, you need a holding room for that person who's trying to get in.

[00:17:38] Cause like you, you need that data. Like you can't skip the post and you can't, and then failing on a two hour run is not an option either, especially if there's side effects like you, you see, do you see what I'm saying? Uh, I

[00:17:50] Richard Feldman: wanna break it down further, so. Okay. Okay. So you mentioned like, let's say you get some data in and, and you're.

[00:17:56] Partway through a two hour run and like one of these things comes in and it's not what you expected. So one of the things I'm curious about is like, so you mentioned like, hey, let's, let's handle it later. Do you mean like you get an error somewhere? You literally like pause execution, you're like, I'm gonna go in and like, wrap em in and like write some code?

[00:18:13] No. Okay.

[00:18:13] Eric Normand: I'm, I'm not, yeah, I mean some languages can do that, but Clojure cannot, like Okay. You know, throws an exception and it's like, what do you wanna. Like to the user. Right, right, right. If the exception comes all the way up the stack and it's like, what do you wanna do? Like some reasons, right? I heard there

[00:18:28] Richard Feldman: there are some lips that do this.

[00:18:29] Yeah, yeah, yeah. Okay. But no Clojure. Can't do that. So when you say handle it later, do you mean like, Let's use the example of like, you're expecting a bull, but zero comes through. So in the decoder style, you would say, handle that at the border, translate it into false right away, and then from then on, you know, you have either true or false, right?

[00:18:46] Eric Normand: But if you didn't, if you hadn't done that, that's what I'm talking about. Right? So if you hadn't expected the zero,

[00:18:51] Richard Feldman: so what does handle it later look like specifically? Is it like at some point in your code you have a conditional, like you're like, if you know this thing, True or false or zero, do something different.

[00:19:01] Like, I guess I wanna understand what does the handling it later look like? Handling

[00:19:05] Eric Normand: it later looks like, let's say I tried to parse a string as an int and it doesn't parse. Okay. My code doesn't know what to do. You know, whatever loop is going through all the posts, I'd have a try catch, block, catch that and be like, okay, save this json to a collection mutable thing and keep going.

[00:19:28] Keep trying and, you know, probably record what you were trying to do, like the exception, right? Couldn't parse the int and then, uh, keep going and then I'll come back to. To my repel and print out whatever's in that collection. Like how many things were in there and like what? Show me the first one. Oh, I see.

[00:19:49] I didn't pars, I thought it would be an end, but sometimes it's empty string, you know? Okay, I

[00:19:55] Richard Feldman: see. Yeah, so I'm, I'm thinking like, what would I do in l and I think in a similar vein, my typical workflow for the handle it at the border approach would be whatever. String is that I expect to be an int like let's say it's a, it's a coming out of j s O string, I would put in my decoder, okay, I expect this thing to be an int and if it's not, then I'll get a decoding error and I usually wouldn't do it later.

[00:20:16] Like c, come back to it later cue. I would usually just send it to an error reporting service like Bugsnag or something. Ah. Granted, but there's no reason I couldn't do it that way. There's

[00:20:24] Eric Normand: nothing innate to, you could write it to a file or you know, something like that. Right. To, to process later.

[00:20:28] Richard Feldman: Yeah.

[00:20:28] There's nothing innate to the border decoding. I think that would prevent me from doing that. Right. I just like historically

[00:20:33] Eric Normand: haven't, and I think that there's maybe a little bit more, that's not exactly types, but we do a lot of nil punning. Right. EnClojure. What nil punning is, is that nil, which is basically a null pointer.

[00:20:51] Has meaning in the different contexts. So abstractly, if you just ask, what does nil mean? It means no answer. When you ask a question and you get back nil, it means I, I couldn't do, I couldn't answer this, but if you like, use nil as a boo, it would be false. And if you use nil as a collection, it would be an empty.

[00:21:16] Right. And so it has these meanings already. I mean this is really subtle stuff cuz you never write exactly the same program in different languages. Right. It's not like Haskell EnClojure have the same semantics but one has a type system and one doesn't like it's not right. Yeah. Haskell Haskell's

[00:21:32] Richard Feldman: not typed Clojure

[00:21:34] Eric Normand: for sure.

[00:21:34] Exactly. Exactly. So there's something going on where we are always aware that cuz we're also, we're in Java land, right? We're on the jvm. We're always aware. Anything could return nil at any point. Right? And so we're always thinking about that case and often it's handled. By the, the standard library. Like if you

[00:21:58] Richard Feldman: give it a a nil, it, it sort of like, does what you would have it do anywhere very often if you were defensively checking?

[00:22:04] Yes. Like

[00:22:05] Eric Normand: if you can cat nil to a list, it'll just give you the original list, right? Like that kind of thing. And so you don't have to, you're like, okay, N is handled. I don't, you know, that's fine. If I have a nil, it means they don't have an answer. It means empty list. Basically, right? So you just do that kind of calculus, and then sometimes you're like, okay, I wanted a string, but I got a nil.

[00:22:25] Well, that's just an empty string. Sometimes you want another string, you're like, no name, right? If you ask for the first name, you want something else to display. But very often, like if you're concatenating strings, you're like, yeah, nil is an empty

[00:22:38] Richard Feldman: strength. It's a good point about the differences between the languages, but also the cultural differences in how things are done.

[00:22:44] Like, for example, enClojure, it sounds like people are more aware of defensive nil checking, but also are aware of the fact that in a lot of cases you don't need to bother because the standard libraries are gonna do what you would've written by hand anyway. Exactly. I think this idea of when do you. Do your checking is really interesting because one of the trade offs that I think about is, so you gave the example of like, yes, technically you can do the translated into Haskell data types and then mess with it later.

[00:23:11] But yeah, it's not ergonomic at all because what you end up with is you basically can say, here is some arbitrary j o data structure. Basically what you're translating into is like an abstract syntax tree for json. That's like, okay, we have either an object or a string or a this or that and like, and.

[00:23:27] Working with that abstract syntax tree is not fun. It's not awesome. Exactly. And

[00:23:31] Eric Normand: that's what I mean. I'm just saying like take that J SSON thing and make it fun. Just like write a map, A J S O N map that is partial. If you print it out like as a human, you can see, oh, that's an array. I can do this. It'll be safe.

[00:23:46] That's what we do in the un typed world, and it's fun. Yeah, sometimes you get an exception, like, who cares? Just try some, you know, look at it and be like, oh yeah, I, I thought it was an array, but it was a string. The thing is though, is you need a whole library, a whole suite to make it fun because otherwise you're constantly writing your own.

[00:24:08] Like, well, if it's a, you know, pattern match, is it an array? If it's an array, then map over it. Otherwise error. And like you're writing that all the time and you wanna just be like, of course it should just error. Like, that's what I'm gonna want it to do. Cuz it, there's no, it doesn't make sense to do anything else.

[00:24:24] So. I really think that I would've been happy to have that because I mean, I felt like I was writing stuff like that. Anyway,

[00:24:33] Richard Feldman: that's where the ecosystem kind of starts to play a role. I'm, lemme try and like summarize some of the trade-offs between like the Haskell approach, the ELM approach, and the Clojure approach with the caveat that like, It's not like these are the only ways you can do it in each of them, but I'm gonna say these are the most popular ways of doing it.

[00:24:50] Yeah, yeah. Yeah. So in Haskell, the most popular way is to say, I want to derive my j s o Derealization and just say, it's gonna be exactly the shape that I expect it to be. And if it's not, give me an error. And so the, the upside of that is, and

[00:25:03] Eric Normand: sometimes I want that too, just to be clear enClojure. Yeah.

[00:25:06] Richard Feldman: Like, yeah. If you're in control of both sides Yes. You're like, I know. Yeah. That is great.

[00:25:10] Eric Normand: Yes. Or if you. In control of the protocol. So like some client is gonna use it and you're like, I want you to know that you've sent me the wrong thing. Sure, yeah. I'm not gonna be lenient. And then you make all these, you know, sloppy mistakes in your J S O, right?

[00:25:27] Richard Feldman: So it's quite concise. And then once you've gotten it validated, you now have. It's something that you can really rely on because the type checkers, you know, sort of got your back from there. The elm approach of writing a decoder is more of a boas than that, but it's also significantly more flexible, where you can say, okay, here I actually will accept either a string or a number or this or that, and here's how I'm gonna translate all those into the type that I expect.

[00:25:51] And then from there, you have everything that you need. And then the Clojure approach is, The most flexible because it's not trying to do any upfront validation. It's just like I'm gonna translate these reliably a hundred percent of the time into data types that I can work with. But from then on you, the trade off is you don't get guarantees about that.

[00:26:09] You now have,

[00:26:10] Eric Normand: like if those, you don't get static guarantees. Right. And we will do the en coder thing. Oh, enClojure, really? Yeah, like you get this WordPress thing and it's like, wow, there's too much junk in here that I don't need and I wanna normalize it to something that I can rely on later. And so yeah, like as you learn the crazy data values you could get back, you start to say, okay, I just want everything.

[00:26:35] To either be true or false for this value. I don't wanna have to, yeah. Have an if statement everywhere. Like, oh, if it's an empty list or if it's a zero or a null. Oh my goodness. It could be anything,

[00:26:47] Richard Feldman: right? Yeah. I think the reason that I gravitate towards the ELM approach there is that a thing that I've been bitten by in the past is not validating.

[00:26:56] Really strictly early on. And then rather than getting an error report, what I instead get is a bug. It's like the zero or the string or whatever, like passes through and it ends up getting used in a way where maybe it crashes. And I, I figure out what the problem was that there was traced back to the J s O.

[00:27:12] But oftentimes it's gone through so many pipes and tunnels and Sure passageways. I'm like, why could it possibly be this here? And it ultimately turns out, oh, it was because the JSON gave me something I wasn't expecting. I didn't make that connection or like, it took me a while to track that. I wonder

[00:27:26] Eric Normand: about that.

[00:27:27] That's one of the things I still think about. So here's the thing, because I'm not doing strict typing that's on my mind all the time. I could have a weird value here that I'm not expecting. Whereas in Haskell, it's impossible. So you just, you, you don't think about that. Yeah.

[00:27:45] Richard Feldman: Which to be fair, I, I like that.

[00:27:47] Peace

[00:27:47] Eric Normand: of mind. No, and it's, and it's very freeing, you know, you're like, yeah, I have a type called non-zero integer. I don't know where I got it from. I don't know how it got made, but I am pretty sure I can divide by this number. Right. Yeah. And I'm just gonna pretend like I can, right. And you have a pretty good guarantee, but enClojure, We're constantly facing, I guess you can call them bugs, but we're constantly interactively at the repel.

[00:28:19] Being surprised by the data that we're getting. And so we're, I guess, coding very defensively all the time. And of course those are things you have to like learn over time and it takes a while to see the tricks. Right? For sure. Cause I don't even know the tricks I do that work. I hear other people say like, oh man, I got this bug and I, you know, where did it come from?

[00:28:42] And I'm like, well, Look at your code, like if, if you just did it this way, there would be no bug, right? And so that person is like pining for a type system, but I'm saying like your code is really not idiomatic. There's a lot of habits that Clojure programmers have that avoid those bugs. I'm not trying to say it's better, but.

[00:29:05] It might be one of the reasons why I'm not encountering as many bugs. I've got the type discipline from Haskell. I've got all these idioms from Clojure and they're together avoiding the bugs. I definitely

[00:29:19] Richard Feldman: know what you mean with that. I had, you know, speaking of using like functional style coffee script, I had a past job where we were using coffee script on the front end.

[00:29:25] This. More than a decade ago now. So I mean, it was, you didn't have all the tools that you had today as far as like typed front end stuff, but I do remember very distinctly that we had one person on the team who had come in from a Haskell background, and even though we were using the like basically pretend it's Haskell, but it's just actually just coffee script technique.

[00:29:44] He was like really struggling with it because for exactly that reason. Because he didn't have the habits of like checking defensively for things. And so for me, I was like, oh, this is fine. And he was just like really struggling with it. He's like, man, This is like really hard for me. I feel like I'm, I'm not doing a good job at this.

[00:29:59] I'm like bad at my job suddenly, because even though we're using this style, there's all these extra things that I just don't, you have to keep in your head. I'm not familiar with this. I'm used to just like being able to rely on these things and I don't have that anymore. I mean, to me, that says two things.

[00:30:14] One is that. One, it definitely makes a difference if you have that mindset or not. I know that it's like difficult to find research that you can draw really good conclusions on this from, but it does seem like there has not been research successfully demonstrating that like static type systems really prevent good bugs.

[00:30:32] And before anyone listening says, I've got a study for you that proves this, please go read Dan Lu's article about all of these studies. Yes. Where he basically points out that like all these studies that have come out, like unless the study came out in the past like year, maybe like, like 2020 plus. Cuz I, I forget when he wrote that article, but like Dan Lu's article about like, research into like papers on, on and research into static type systems and like their efficacy and it's like he just tears 'em apart.

[00:30:59] They're, they're, they're really not saying things like

[00:31:02] Eric Normand: reliable. And then again, even if one might show a small thing, cuz some of them. Right, but they're always specific type systems. You know, they're not comparing. I don't know. Sometimes it's like, well, it's Java with an i d e that knows the type system, right?

[00:31:20] Like it's stuff like that. It's like that's not fair. You're giving them a better tool. No,

[00:31:25] Richard Feldman: to be fair, anecdotally, like without having any research to back it up, I do feel that like if nothing else, I can compare. Like at work at at Nore Inc. We have a big legacy Rails code base and a Haskell code base that are coexisting and.

[00:31:40] It's like the same team of people wrote most of the code in in OR or all the code in like both of those systems and most of the people on the team started out with a Ruby background and then learned Haskell. So it's not like they were unfamiliar with the how to like defensively. Check for Neil and I'll grant you that.

[00:31:54] Ruby is not Clojure. And Ruby also I know does not have like the nil punting stuff that you just talked about. It's not, I, I don't think Ruby's at the same level of like gracefully automatically handling nil that Clojure is, but I will say, I mean we, both of them are hooked up to error reporting services and in terms of like how many errors we get and like how much they crash per line of code.

[00:32:15] It's just like way different like Ruby is. All the time crashing and like, and the HASKILL code just isn't,

[00:32:22] Eric Normand: and how much is the type system and how much is the, like the api?

[00:32:25] Richard Feldman: It's a fair point, right? It has to be said that it's also not like Haskill is typed Ruby.

[00:32:30] Eric Normand: Right, exactly. And the ecosystem, Ruby is huge.

[00:32:35] And so there's a wider variance of people in it. And so the libraries that they're writing also have that wider variance of. Bug proofness and design.

[00:32:46] Richard Feldman: We started using sorbet types, which is like type Ruby Uhhuh Uhhuh for some of that. I'd be actually interested to check out, I, I don't think anyone's, we, we've liked it enough to keep using it.

[00:32:56] So that's, that's an endorsement right there. I actually would be kind of curious if we, like, I don't think anyone's ever done this analysis is going back and looking at the end points that we've added sorbet to in comparing the error rates to the ones where we didn't. I don't think anyone's done that, but I mean, we have the logs.

[00:33:09] We could do that. But to your point, Like, yeah, you're using sorbet, but then you use this third party dependency that's not using sorbet, and so how trustworthy are those types, et cetera, et cetera. There's a lot of considerations

[00:33:22] Eric Normand: there. I think there's a lot of opportunity now because of these type systems for dynamically type languages to do research.

[00:33:30] That's kind of apples for apples, right? So you can do. And Ruby with sorbet, we can do That's a great point. Java script and JavaScript with type script and Right. You know, you're not gonna be able to generalize it to type systems are better or worse of worse of course. Yeah. Right. Or the same or Right.

[00:33:45] Waste of time

[00:33:46] Richard Feldman: High is not typed Ruby, but typed Ruby actually is typed

[00:33:48] Eric Normand: Ruby. Exactly. Yeah, exactly. And so maybe you could have a pretty good research thing there.

[00:33:55] Richard Feldman: Even still though, like, I mean, like you said, first of all, those claims wouldn't generalize to other languages, I don't think or Or like it would be really hard to do that.

[00:34:01] You'd have to be really careful. But also I. They don't generalize it. The, the ecosystem right, is still a big factor there, where it's like, yeah, you have type ruby in your code base, but you're building that on top of a huge ecosystem of probably un typed, right? But you could

[00:34:14] Eric Normand: at least give it back the scientifically backed advice.

[00:34:18] Of whether you as a company should adopt sorbet. Sure, yeah. For your

[00:34:23] Richard Feldman: code base, if you wanted to try and do an experiment like that, I think probably the best thing you could do would be a lot of libraries are now type script first in the JavaScript ecosystem. Mm-hmm. Mm-hmm. So I bet if you took. Type script.

[00:34:34] And then you tried to have like a team use type script and all TypeScript ecosystem libraries. And then another team do the same thing with the same libraries except only use them without the type checking, right? Like strip out all the type annotations and then just have it just be straight JavaScript, but using actual type script libraries.

[00:34:51] So then it's like as apples to apples as it could be. But then again, like you. In that world, those libraries and you know, all of the code that you've written will be written, assuming there's no need for any defensive checks because it's type script. So it's really hard to do an apples to apples comparison.

[00:35:06] I think that's right. That's right. Well, even in that world,

[00:35:08] Eric Normand: but see, that's part, that's kind of part of it, right? If the type system is there. If you're expecting to have to eventually. Type check, like you're gonna think differently, right? Yeah. And that might be part of the advantage of it. Yeah. It's not just like, oh, it caught a bug.

[00:35:25] It's like, no, I didn't even write the bug because I knew. That I had to like be stricter with the types.

[00:35:32] Richard Feldman: Another aspect that's underrated here and like speaking about differences between like Haskell and Ruby and Clojure, is that like there's a different, even if you took away the type systems, there's definitely a different style that people use to write each of these things.

[00:35:46] And actually even like Haskell and Elm is a good example. Like for lack of a better term, I, I think Elm tends very much towards. Simplicity, both from the implementation perspective as well as from the types perspective. If you look at like a Haskell package that does a thing and an ELM package that does the same thing, you're not gonna see as many like type variables in the Elm version.

[00:36:08] Right? And you're not gonna see like higher kind of types and stuff like that, right? But also similarly, if you compare like Ruby EnClojure, Ruby does a lot more meta programming. Uh, like culturally, there's a lot more sort of magic, for lack of a better term. It would be not only harder. Put types on that because things can be kind of shifting, but also I think it would be harder to adopt the discipline that you mentioned using when you write Clojure, which is to think about what type is this returning?

[00:36:33] Because again, that could also vary based on like meta programming things. And maybe it's hard to figure out, like to answer the question like what actual type is this method returning compared to something like Clojure? That's like sort of intentionally more

[00:36:45] Eric Normand: straightforward. Right, right. Interesting. I have a project in the back of my mind.

[00:36:50] Wanna work on one day, which. To write a type system for not Clojure or a subset of Clojure. Yeah. I feel like the attitude of like, let's try to type how, like all these existing Clojure programs, I think that that's like a fool's errand. In the end, it's gonna be so complex because you have to deal with all the dynamic stuff that people do, that it's not gonna be valuable anymore.

[00:37:15] It's just gonna tell you like, yeah, your program works as expected because it's Clojure. But what I do want is a subset that is easy to type. That once I understand exactly the algorithm I want and like what are its inputs and outputs and all, all this stuff, I can lock it down and say like, okay, it's gonna be this very strict subset of Clojure.

[00:37:40] That's easy to type, that I can make sure it's still Clojure. It'll still run without the type system, but it. Also type check. Yeah. And so you turn it on like, let's say for one module, right? And you say like, you know, you annotate it and everything. Give it the types and it'll say, ah, you can't really do this dynamic thing where you're like, I don't know, parsing a string in the, you know, string lead typed stuff.

[00:38:04] No, stop doing that. You can't do that. And things like that.

[00:38:07] Richard Feldman: That's an interesting approach. I mean, it seems like most type checkers try to just like do their best with, with all that stuff. And that seems like it's better for adoption, but. Less good in terms of like how much you can trust, what it, what comes out.

[00:38:21] The other side, which is kind of the whole thing that I like about type checkers is like you said earlier, all this stuff that I can just get out of my brain and stop thinking about, right? Because I'm like, oh yeah, this has been taken care of for me,

[00:38:32] Eric Normand: and I think it might have some adoption in the Clojure world we're.

[00:38:36] Very into like a library being done. People are like, oh, it hasn't been committed to for four years. It's like, yeah, but everyone's using it. Like if it's, it's done like there's no new features, what else would

[00:38:48] Richard Feldman: it need? We have like implicitly that, that same kind of thing in the ELM community, but, but it's not consistently celebrated, I would say.

[00:38:55] You'll see people say like, Are people gonna be worried that like no one's committed to this library in like, more than like a couple years. And then other people are like, yeah, but like, what should we change about it? Like every, everybody loves it. Like it's, we're doing great. Like, what, what's the problem?

[00:39:09] It's

[00:39:09] Eric Normand: a JSON par. So JSONs the same. Like, what, what could be better? And like, actually that's a interesting example cuz recently someone took on the task of making our J S O parser faster. So it's much faster now. So yeah, that was a change, right? Sure. Yeah. But like it's probably gonna stay like it is for another five

[00:39:28] Richard Feldman: years.

[00:39:28] I'm a big fan of like, once something gets sufficiently matured, just make it faster and that's it. Yeah. Don't change the a p I like you don't like, I mean, if there's some really compelling reason to, okay, fine. But like hopefully over time the percentage of your time spent performance optimizing would just approach one.

[00:39:45] That's why, right? That's

[00:39:46] Eric Normand: my preference. Yeah, well actually it's interesting in the Clojure actually core code base, there's even, uh, reticence to do that. There's like a reticence to change the code, kind of like, oh, let's refactor, it looks ugly. You know, people have that kind of attitude, a lot about code that they didn't write, and that's old and Reiki the creator and cu, you know, maintainer of Clojure.

[00:40:13] He's like, no, like, just leave it. It works. Don't touch it. And what's interesting is this attitude, you can see it someone made, uh, this graph, Of lines of code as they change over time and. The Clojure one is very layered. It's like there's these old lines of code that just have not changed ever. Once they stabilized at the beginning, they're just like straight across.

[00:40:37] And then there's another layer with the, you know, version 1.2 came out and there's like these other layer that comes in. It just hasn't changed. And then you compare it to something like Scala. It's just constantly shifting these, it looks like geological strata, you know, geological layers, but like there was a lot of seismic activity, a lot of earthquakes and you know, plates going on top of each other, just like constant churn.

[00:41:03] I think it shows in the stability of Clojure just always works. It's one of the reasons why you can just leave a library to just be and just work, because the language doesn't change that much and it's only adding new stuff. It's not like going back and saying, oh, we made a mistake here. If we made a mistake.

[00:41:24] If there's a mistake enClojure, you leave it. But you might make a new thing like

[00:41:29] Richard Feldman: deprecate it. Say like say, Hey, don't use this anymore. Use this new thing. But yeah, but it's still there in, yeah, so So your old stuff still works. Exactly.

[00:41:37] Eric Normand: Exactly. Which is interesting cuz now there's all this new stuff and Rich Iki says all the time, like if I had made this new stuff first, all the other stuff would be based on this new stuff.

[00:41:47] But. I didn't do it in that order. And so this is what you get as, as, as it happens. Yeah. I mean it's similar to like in, in Haskell where they came up with, you know, map and other stuff before they had the func type classes. Yeah. So they didn't have func yet. And of course map over a list should be fmap.

[00:42:09] Like it should be the same. Right. It was written first and they haven't gone back to change.

[00:42:15] Richard Feldman: I don't think that's cuz of a deep commitment to backwards compatibility. I think that's like, because there's other stuff, well, whatever. Anyway, I actually think they should just make that one consistent Also, I think they should all be mapped.

[00:42:24] Not fmap, but that's neither. Sure. Yeah.

[00:42:26] Eric Normand: F Map is a a hack, but it was an addition. The same, same kind of

[00:42:30] Richard Feldman: thing. I get it like at the outset, but it's. I don't know, like they make breaking language changes. Why not include that in one of them? Like That's true. That's true. It's not like it would be a big difficult find replace job anyway.

[00:42:45] Yeah, we talked about a bunch of different stuff. Anything else we should make sure to talk about? I do

[00:42:48] Eric Normand: wanna talk about my book. Oh sure. And I don't want to be pegged as like the type versus dynamic type guy. So my book is called Rocking Simplicity. It's all about functional programming for beginners.

[00:43:04] Beginners to functional programming. So I know your audience is probably like pretty advanced already in functional programming, but No, I don't think that's the safe assumption

[00:43:12] Richard Feldman: actually. No. No. Okay. I mean, I think most people who listen to this podcast are familiar with functional programming, but I think that you'd be surprised there's, there's lots of different variety in like, How much experience people have.

[00:43:23] Eric Normand: Nice. Okay, good. Well, I had like a functional programming meetup that I ran in town and I would always get like the first three months, it was always new people coming in, being like, what is functional programming? Why should I, so we had this discussion over and over and over again, and. There would always be like, well, what should I read?

[00:43:43] What's a good resource? And for the people who were like curious or like they had seen, you know, they look it up on Wikipedia and they're like, that doesn't really seem helpful to have no side effects. Like, I need side effects. Like, this is not practical. I wanted. Write a book for them that like helped guide them into like making sense of the Wikipedia article basically.

[00:44:09] And I feel like we as a community have kind of neglected those people. I don't wanna blame anybody, I'm not doing that. But like functional programming, a lot of it is academic, certainly until very recently. So a lot of it is very inaccessible. Like even. Academic writing is not made to be like, oh, I'm a PhD student.

[00:44:30] Like I have this intro papers and then this. No, it's like they just throw the hard stuff right at you and you have to get it from your professor or your o other grad students around. Anyway, I wrote this book. It tries to break it down the journey, much like start the journey much. Like a lot of places will say like, oh, let's learn, map, filter and reduce, and then just as a, like a throwaway sentence, they're like, and you should pass it.

[00:44:59] A pure function. A pure function doesn't have side effects. Yeah. Yeah. Most people get lost there. The whole idea of pure function is not obvious to people who aren't used to it. And so I have my first eight chapters. First part of the book is Basical. Explaining pure functions and why they're useful and how you get to a pure function if you haven't.

[00:45:20] And like how do you refactor something into pure functions? And it's taking like actually saying like actually functional programming is basically just making a distinction between impure and pure functions. That's like the first step in functional pro and probably the most useful one. And I don't call 'em pure and UR functions, I call them.

[00:45:39] Calculations are the pure functions and actions are the impure functions. And then there's also data, which is. Just inert stuff that, you know, it doesn't run. The whole first part is about actions, calculations, and data. Then we get into, you know, map, filter, reduce, and higher order

[00:45:56] Richard Feldman: functions. So it sounds like someone could use this book as a way to learn the sort of like functional programming style and it's not necessarily coupled to one particular

[00:46:07] Eric Normand: language.

[00:46:07] That's right. All the examples are in JavaScript. Oh, wow. Yeah, because. I found when I was looking around the functional programming books would really heavily lean on a functional language, and so it always seemed like, well, these are features of this language that I'm learning, not a style.

[00:46:29] Richard Feldman: Or, I mean, I've also seen books that are like functional programming in JavaScript, but then I would guess that not having read any of those books, but I would assume that those tend to get into like pretty JavaScript specific stuff too.

[00:46:41] Like That's right. These libraries and you know.

[00:46:43] Eric Normand: Yeah. Right. And this is language agnostic. I use JavaScript because it was. Readable by most people. Yeah.

[00:46:53] Richard Feldman: It's probably the most widely known

[00:46:55] Eric Normand: language. Exactly. Yeah. It's right. And like if you don't know JavaScript, but do you know Java or C? Like you can read it, right?

[00:47:00] Yeah. It's got the curly braces and the Yeah, the if and the four. You know, it's got everything. It's not JavaScript specific. I emphasized that several times in the book and people are still like, like, why do you, you know, this is a JavaScript functional programming book. Like, no, it's not about JavaScript.

[00:47:16] I also found a lot of the books and like, here I am critiquing the books. This is a reason why I wrote it, that I couldn't recommend these other books. A lot of the books were just doing. Like Lambda calculus, gymnastics. It's like, look, all the currying, isn't this fun? And you can write this and like, yeah.

[00:47:32] That's not, I mean, it's fun. I like that stuff too. But it's not the practical industrial stuff that we, that is useful. What's useful, right?

[00:47:41] Richard Feldman: Your audience is like people who want to get stuff done and they think that functional programming might be a way to help them get stuff done more effectively, but they're not sure or they, they.

[00:47:51] They, they don't know how to like, get started with it in a practice sense.

[00:47:54] Eric Normand: Exactly. Exactly. And it's not just about like, you know, first class functions, anonymous functions, that kind of stuff. We get to that, but really the core of it is identifying that side effecting functions are hard, they cause bugs and problems, and so you wanna move as much of your code as possible into pure function.

[00:48:17] And then how do you do that? Because that's not obvious either. Like a lot of people think about their code, it's like, well, I have to take this step and then this step and this step. What else is there? Like, and so I give some examples like, you know, you might be doing a for loop. Reading from the database.

[00:48:34] So that's a side effect. You're reading from the database and then you loop through each one and you send an email to that customer, right? So you're doing all of this is all action. It's like, where's the calculation? Well, how do you produce the text of that email? That's a calculation. It has inputs and it has outputs.

[00:48:52] You could actually read in the whole database, the whole table. One, go and generate all the emails in one go, and then have this really tiny loop that just sends 'em one at a time, right? So you have this big calculation, and so most of your code is in calculation, and then your action code is really simple and clear, and you can test this calculation much more easily.

[00:49:18] Oh. And so we go into, you know, all sorts of stuff like that. That's like, these are things that non-functional program, Don't get from, you know, reading a random article on the internet about functional programming that a lot of the stuff that you're doing could actually be pure and then, then it's easily testable.

[00:49:37] You don't have to set up a database, you don't have to fake an email server and all that stuff.

[00:49:43] Richard Feldman: Very cool. All right, so rocking simplicity, uh,

[00:49:46] Eric Normand: working people. Anywhere. Amazon, it's on Manning. Oh, don't we have a code for your listeners? Yes. So we have a, what is this, a 35% discount

[00:49:57] Richard Feldman: code. Yeah. Check out the, the description for, uh, 35% off.

[00:50:00] Rocky Simplicity. Yeah. Eric, thank you so much. Really appreciate your taking the time to talk to me. Yeah,

[00:50:05] Eric Normand: of course. It was fun.