Four More Domain Modeling Lenses

I was honored to speak at HFPUG again about the topic of my book. This talk is a sequel to this one. Watch it first because there is some intro material I don't repeat in this talk.

Download slides

Transcript

Okay, so I want to welcome everyone to this month's meeting of the Houston functional programming user group. Today we have joining us yet again is Eric Normand. I think this is your third time presenting with us. So you're you're a regular now. And so we always enjoy when you present. And we always have very, very good spirited discussions. So we're looking forward to what you're going to present today. And I think that is it. I will turn it things over to you. Okay, everybody, Eric Normand. Thank you so much, Claude. I'm I'm Eric. And I've got a talk today that is a continuation of another talk because I'm working on the contents of my book. And so I find that making the slides helps me develop the content. And last year I was here about this time. And the discussion was really good because I was getting questions that I wasn't expecting. And really told me that I'm presenting it in the wrong way. And so after after that presentation, I kind of went back to the drawing board and thought a lot about what I was trying to say and how to say it best. And I think I got I got something really good now. So this isn't going to be like a rehash of the stuff we did last year. This is just a total new reorganization and a new way of presenting the stuff I have. So I want to say thank you. And I am very open to discussion. I really want to I would love to have that experience again, even though I would dread having to reorganize all of this again. All right. But with that preface, I'm going to share my screen. Do we all see that? Yes. Awesome. Okay. So this one is titled for more domain modeling lenses. So this is a talk about software design. And software design is subtle. It's something I have to keep reiterating to myself. And I want to make sure that it's it's very clear. So here it is. Here's the slide about that. And if I could summarize what I'm trying to say very succinctly, it's that good information leads to good design decisions, which leads to good design. So design is all about making decisions. And you need good information to make those decisions. So I've organized my my ideas on on domain modeling as it pertains to design terms of lenses. So each lens gives you like a little people view of your software, your domain. And if you get enough of them, you start to hopefully be able to give a bigger picture of what's going on. Currently in the organization, this this keeps changing, especially the order keeps changing. But I've got nine different lenses. The ones on the left, the data operations composition and time are kind of like basic building blocks. And then the ones on the right are more like orthogonal ways of viewing your software to just get more information that informs how you use the ones on the left. And this didn't this talk is like part two in a series. I gave another talk called better software design with domain modeling at Funk Prague, Sweden. And the folks here on this call were asked to preview it before this talk. And I just wanted to make sure that was part of the recording. That one goes over these lenses, these three data operations and volatility. And in this talk, we're going to go over four different ones, composition, scope, platform, and runnable specifications. Okay, now a quick quick preview or review of our domain. So we're doing a coffee shop. There are three sizes, super mega galactic. There are three roasts, raw burnt and charcoal. And then there's a bunch of different add-ins like soy shots and espresso shots that you can add in. And to the right, we see the JSON that I'm going to be referring to. Hopefully you've seen the talk. We like go in depth of how we choose all this stuff. But basically, there's a key for size, a key for roast. And then the add-ins are a map of the add-in to the number of shots or pumps of the syrup that go in. All right, so let's get to the composition lights. So when I say composition, I mean something very simple. It's two or more calls to the operations in your domain working together. I want to jump in here. Can you just for people who might not have seen the talk, can you just explain the lenses? Oh, sure. Yeah, so like I was saying, at the beginning, software design is really hard. And it's like multi-dimensional. And it's like a changing landscape. Like as you move in one dimension, the dimensions change. Everything's interconnected. And a lot of the design, the software design advice out there is based on rules of thumb or principles. And I've always found them to be lacking. And you can tell they're lacking because they have so many exceptions to them. This rule applies, except don't do it here because that's this thing. And then don't do it over here because of that thing. And you're like, is it a rule or not? Is it a principle or not? And my attempt to avoid doing rules is to say, look, we're in this very difficult space, problem space. The best we can do is arm the programmer or the designer with a bunch of ways of looking at their software, give them some tools, give them perspectives on that they can use while they're in the field, while they're in their code base, making decisions, and pat them on the back and say, like, good luck. And so another way to look at it is they're kind of like figure ground exercises. So if you're Oh, wait, I was reading this book after I gave this talk last year, here at Houston functional programmer group, I was disappointed with how I did. And so I was reading this book, this like totally unrelated, it's about drawing. And it's called drawing on the right side of the brain. The author has taught a lot of people how to draw. And she's broken it down into, I think it's like five different skills. Because the thing about drawing is when you when you look at like a plate or a table, you say, Oh, that's a circle, or that's a rectangle, but you're not seeing a circle. There's like, it's very unlikely that you're looking at it straight on so that it's a perfect circle. It's always foreshortened in some way. And things don't often have like easy edges to draw. They're like very irregular. And so people will like draw an eye, like they think an eye should look instead of drawing the lines that they actually see. So a lot of what you have to do when you're learning to draw is learning to not is to perceive more directly. And so she has an exercise that is trying to help you draw the outline of a of a shape, like of a figure. And it's draw the background. Don't draw the figure draw the background. And she has like four other exercises like this. And it's trying to trick your brain into perceiving more directly. And I thought that that was perfect for for what I'm trying to do is try to get people to see more clearly past their biases, past their habits of how they they're they've been taught to to see a problem and converted into code. So each of these lenses is trying to be one of those things like look at it from this perspective and it gives you some information. You can use that information to make better design decisions. Awesome. Okay. All right. So this lens is called composition. This is a lens about how the operations work together. And even like an operation could be working with itself. Right. So rarely do we see one operation doing all the work, right? You've got a bunch of methods or functions, and they're going, you're going to compose a solution with with all of these together. So here's an example of something that it's kind of a scenario of what might happen when a barista is typing in your order. You know, you're speaking your order and she's typing it in. So you set the size to galactic. Well, then you change your mind. You want a mega. And then you set the rows to add an add in. And then they accidentally press the remove almond button. And so this is an example of composition. Right. It's it's a simple example. It's a thing happening in sequence, but it's it's a good example of what might happen. Uh, so we could test each of our operations in isolation. So we could test the semantics of set size. So we have some example based tests where we assert that after we set it to galactic, it should be galactic. And after we said it, the rose to charcoal, the rose should be charcoal. But frankly, these are just these are boring and they don't capture the interactions between the different the different operations. What we want is to look for those. That's what this lens is all about. How do these operations work together? What are they supposed to be doing? So let's go through a scenario where we're building a test that says that we can add the add-ins in any order and we'll get the same coffee at the end. So here's a one order. So for to start, we'll just start with these three. You're going to add soy, almond, and espresso in that order. And then in a different order, we're going to have almond espresso and soy. Same ones. If you added these to a coffee, they're the same coffee the customer doesn't care how you write them down. It's going to get they're going to get the same coffee. So these should be the same. Uh, so we can reduce over those add-ins, the first list of add-ins to make a coffee. And then we can reduce over the second list of add-ins to make a second coffee. And then we can assert that they're the same coffee. Right? So this is, this is a test that we could run to somewhat test that we can add these things in any order. But it's not general enough. We want it to be a little bit more useful. So we're going to change it from new coffee, which always returns like a default coffee to any coffee, calling any coffee will generate a random coffee. And then we're going to make an array of arbitrary length of any add-ins. So now it's going to have some random array. And then we're going to use that same array, but then shuffle it in some order. And now, because that's only one, we want to run it a hundred times with a hundred different random randomizations and assert that they all work. So this will test a whole bunch more. And it is a better guarantee that the property that we're looking for, this relationship that it doesn't matter what order the add-ins come in is maintained. Right? And we can do this. We can write this code without even having a data representation and without even having implementations for these operations. Like we can write this down in our code. And, you know, it wouldn't run, obviously, if we don't have the add add-in implemented. But no matter what data representation we use, we want this to be true. So it's kind of a constraint on what data modeling we use. And so we have this property that keeps the barista happy, because now he can say, I can press the buttons in whatever order I want. And I'll always get the right order. Okay, so what we did was we converted this code at the top to this code at the bottom. And if we squint, because JavaScript is not math notation, but if we squint, we can see that we've made some unknowns, right? And in math, you would use like letters like X and Y and Z to represent unknowns. They're called variables. And so this is why these are often called algebraic properties. That we're saying that something is true of this operation, add add-in, no matter what X and Y and Z are. Okay, so this is an algebraic property of our function. This algebraic property happens to have a name. A lot of the mathematical, I mean, a lot of the algebraic properties have mathematical names if they come up enough, it's called commutativity. And it means that the order of function calls doesn't matter. So it's, it's not so clear here, but let's rewrite it in a simpler way. So we got this, this for loop to do a hundred times, we're going to make any coffee, and we're just going to make two add-ins this time. And we're going to assert that the, if we add in A and then add in B, it's the same as adding in B and then adding in A. You'll notice I also did a little switcheroo where I made it that it's using the dot method notation. Just same semantics, just easier syntax for seeing this order. Otherwise you'd have like nested function calls, which are hard to read in JavaScript. Any questions at this point? Yes. So the word commutativity is usually applied to binary operators, but in this case, you're applying it to a set of functions, but you're not saying that function composition is commutative. You're saying that at this set of functions, right? Is it really the same word? Yeah, yeah, they use the same word. So it is unfortunate, and we're going to get to the binary operation one, which is basically that the argument, the order of arguments doesn't matter. This one is function calls don't matter. The functions commute, and mathematically you would write it like this, that if I apply G to F of A, it's the same as applying F to G of A. But good catch, because I'm going to get to that. That's community where the order of arguments doesn't matter. And if you wanted to, you could write it out using, what's that? I mean, using currying basically, and make the functions, you know, you autocurry the functions, and then you can change the order, right? If you had, if you had unary functions, the binary function becomes a unary function. There you go. So community commutativity, where order of arguments doesn't matter. Let's look at what this is. Uh oh, right. So let's say we have some, we have two of those add in sets, or add in maps, and we want an operation that, that combines them together, like this. And we want to say, wait, it shouldn't matter if I put the espresso one first or the soy one first, right? That's my property that I want to maintain. So I could write it like this. So I create an array of add ins, ray of add ins, okay, it shouldn't be a ray of should be a map of. Sorry about that. And then we combine them, doesn't matter if I combine them a first or b first, we should get the same add ins. And this type of commutativity is, um, is expressed like this. It's so much cleaner in math. Once you have to start, uh, writing out like long function names and variable names, it gets messy. Uh, item potance, this is an important one that we use a lot, uh, basically means doing a thing twice is the same as doing it once. So let's go through a property for that. So we've got a coffee and a size. And we're going to assert that if I set the size on the coffee, it's the same as setting the size again on this, on the coffee, right? And it's kind of unclear. Like I said, the nested function calls are not easy to read in JavaScript. But it's basically this that if I do F of a, it's equal to F of F of a. And you'll notice if you, if this is true, then it's also true of F of F of F of F of a, right, which is why similarly we could do that with commutativity, where we could only work on the binary case where you do a first and then b first instead of having an arbitrarily long, um, array of, of add-ins kind of works out that if two can be out of order, then any number can be out of order. Much simpler. Right. And so then we could also write it like this just to make it clear. Okay. Um, one more property. No, two more properties associative order of operations doesn't matter. This one, this one always screws me up because it's very similar was, it's taught like similarly to commutativity, because a lot of operations that we use are both commutative and associative, but it's order of operations. Basically think the parentheses, right? What we want to say is that if I have two sets of add-ins, or sorry, three sets of add-ins, and I want to combine them, I can combine them like this, where I combine b onto a and then c onto the result of that, or I can do the b dot combine c first, and then combine that onto a. But notice that I've maintained the order a, b, c. That's important. But what I'm doing is combining the two on the right here. It's clearer if you see in the mat. I'm saying that if I combine the two on the left first, it's the same as combining the two on the right first. The parentheses are indicating the order of operations. And this is why when you're writing out like a big sum, like a plus b plus c plus d, you don't have to put parentheses because it doesn't matter, right? They're the same. However you put parentheses, it's going to give you the same. All right, one more property. This one is the inverse. It's basically like an undo of a particular operation. So one operation can be the undo of another. So we can write out this property. So we got a coffee on an add-in, and we're asserting that if I have this coffee, it's going to be the same as adding the add-in to the coffee and then removing. And that looks like this in math. Okay, but this one, I can already tell that there's going to be a problem because my manager has said there's a maximum number of add-ins. This affects our property. He says we can't sell a coffee with more than five add-ins. So you can't have like five espresso shots in your coffee because then it's like there's no coffee in there anymore. The cup is too small. So if we put, we could do this. We could take this add add-in function and just have this guard at the beginning. It's like if it's more than five, don't add, don't add it, right? So just return the coffee unchanged. Okay, I'm just repeating the property up there so we can we can have it for reference. But I can imagine a scenario where this is this property is going to break. So I have a new coffee, so there's nothing in it. It's just no add-ins. I'm going to add the soy. I'm going to add five soy to it. Oops, forgot to close my quotes. Okay, and so then I'm going to add sixth soy, which should be a no-op because I'm guarding against if it's more than five. And then I'm going to remove the soy. When I assert that they're equal, it's going to fail, right? Because I didn't add it, but I did remove. So the coffee too is going to actually only have four soy, even though I added a, I wanted to add a sixth. So what we would say in this case is that this is a partial property. It works until you get to five, right? I think this is kind of a I looked around for a term for this. I had to make this up. But there's the idea of a total function, which is a function that has a valid return value for every valid, you know, set of arguments. Basically doesn't throw an exception. And then the partial function does throw an exception. In this case, I'm saying it's the property that's total or partial. So we want to prefer total properties because they are much cleaner, much easier to handle. But we could write this as a partial property. So we take our property and we just put an if statement around it. And we say, look, if this coffee that we just generated randomly has, it only works if this has less than five add-ins. Okay, but we don't want to do that. That is not ideal. Ideally, we'd have a total property. And so to figure out what to do with that, one thing we could do is go to the scope lens. So I'm moving into the next one. Okay, so here we have our, we have some of the operations that are related to add-ins. So we have add add-in and remove add-in. This is the notation I'm using for the function signatures because TypeScript signatures get kind of long. But you can tell what type something is just by the name of the argument. And then this is the return value at the end. So the first two return a coffee. Normalize takes a coffee and turns it into an equivalent coffee that can be compared for something like equality so that you don't have different, in case you have different representations that mean the same thing. And then is valid is telling you whether you've kind of messed up your data model in some way. And so normally we would put a coffee with more than five ingredients or add-ins as an invalid coffee or a return false. But it messes up our property because it would be really nice to be able to go above five so that we could remove it later. You know, imagine, I know the property, it's not just some abstract like, you know, ivory tower idea that we're going to keep a total property. It actually helps the barista. So if the barista accidentally presses, you know, plus soy and then wants to correct it, they can just hit minus soy. And not worry about, well, did it actually add it or not, you know, so it's much more ergonomic as well. And you can see why, like having this condition that they have to maintain in their mind, like how many soy's are in there, maybe they don't even maintain it in their mind, but they have to stop and look and see how many add-ins there are already. So the idea is let's draw a line, make a new layer. So this layer, we're just going to say, this is for all coffee shops. This is the domain of coffee shops. And then above that, we're going to put code that just relates to our coffee shop. We can justify this. We can say, hey, a different coffee shop might have a maximum of three add-ins or no maximum. I can imagine not having a maximum. Maybe the maximum is 10. You can imagine it being any number or not existing at all. So it's not, it's justifiable to say that we don't have to do that in the coffee shop domain, we could just make that our coffee shop where we put our business rules. So we can define another, oops, sorry, another function called is valid. Sorry for the name collision, but it's in a different namespace. And this is where we put the businesses, our businesses, particular business rules. Okay, so we have this add-in function. Let's comment out that guard and just always add the add-in. And then up in this is valid function, we can, you know, add this check that it's less than or equal to five. And that's a business rule. Not a domain name rule. It's a business rule. So what we're doing, what I'm trying to show is that when we're thinking about these decisions of like, do I, you know, how do I code this? Where do I put this? Often it's helpful to defer a decision to a higher layer, because it helps you eliminate a corner case or make a partial property into a total property. But you should only do it if you can justify it semantically. Like, you can't just have a million layers. And then like, well, what's that layer about? Well, this is this one little subtle thing. No, don't do that. Like, you have to have some kind of semantic justification for it. Now, similarly, you can go down a layer. And this is useful if you have some really complicated scenarios, some complicated domain logic that is hard to organize at the current layer. Sometimes you have to go down a layer. So this one is going to be hard to explain. So bear with me. But the thing I thought of for a coffee shop is they have sales and promotions. So there's all sorts of different ways promotions can work. You can have coupons. You can have like these weekly promotions that operate at the whole store level. You can have loyalty perks like those little cards that you punch. And then the discounts can work in different ways. Some of them are percentage, some of them are fixed. So the percentage would be like 10% off. Fixed would be, you know, $3 off. Then you can have stuff like buy one, get one free. And then sometimes you get 10% off of a particular kind of coffee, like they want to promote their pumpkin spice latte. So you only get 10% off of that one, not the rest of your order. And then the requirements can get really complicated. So like, you have to buy five and then you get the sixth one free. And how does that combine with it with like a loyalty perk? Sometimes it applies to any latte, but then sometimes you have a thing where no, it has to, it has to be a coffee with almond and and another coffee in the same order with soy. And then you get the discount. So it gets really complicated. And if you tried to just like say, well, I want to make a thing that's in like kind of pick and choose, it's it's not going to work so well, especially when like you combine them like what if someone goes in with a weekly promotion, when there's a weekly promotion going on and they have a coupon. And you know, it just kind of gets out of hand. So the recommendation is to go down a layer. And I'm not going to do the complete thing because it would take too long. But I want to show what that might look like. So we can make a type that is this union of these types. So it has no discount percent discount. So no discount is like, you know, you don't get any money off percent discount 10% off fixed discount $3 off both discounts is going to have two separate branches and both apply. And then highest discount is only one of them is going to apply, but we're going to give the customer the highest of the two. That's how the business likes to operate. Okay, and then we're going to have some types like these basic types like a discount. What does a discount mean? Because discounts can be so complicated, we're going to just say it's a function, it's going to take an order, and it's going to return a discount amount. Okay, and so this discount can check, you know, does it have the pumpkin spice latte? And then it's going to give a discount amount one of these things up top. And then we can say, well, we can have a higher order discount function called guard discount that is going to return a, it's going to take a discount and modify it to return a new discount that guards it in some way. So like, you only get this discount if you've ordered a pumpkin spice latte, right? Otherwise it's going to always return no discount. And then you can combine two. So you can say, well, I'm going to say, if they have a coupon and a, and a loyalty card, how do those two work together? And so you can combine the two, and it will return, you know, the highest or both or something like that. Okay, sorry, I wanted to say one more thing. So this, oof, I'm sorry. This has happened to me before. And so I want to give like one little one more illustration. I worked at a company that did voter registration. And the difficult complex scenario we were dealing with was that every state has different registration laws. And we struggled because there, there were experts on the team who knew all the laws, they like had memorized them all. And they could talk about them. But in trying to organize them, whenever we would organize it, it was like something would slip out. And then we couldn't do it. And we could find a way to like kind of have 70 to 80% of the cases work out. But then there are just so many exceptions, like it just seemed like the best we could do is just have a giant if statement. There's like, if you're in Alaska, these are your rules. If you're in California, these are your rules. But then I kind of sat on it for a while. And I realized, wait, the basic question we are trying to answer is, can this person vote in this election? Yes or no? That's a function of person and election to a Boolean. And once I had seen that, like it is a simple problem, when you look at it at the function signature level, then you can start to write rules to combine them. Like we can have a rule that's like, you know, of majority age, right, 18 or over. And it always returns false, no matter what the election is, if you're not 18. And if you are 18, it returns true, doesn't look at the election, because the election doesn't matter. And so you can have all these rules that are just small, small nuggets of rules, instead of trying to organize it by kind of like a hierarchy of like, this this kind of state, because some states are like, you don't need to register some, you need to bring an ID, some you need to register two weeks before, you know, there's all sorts of like different categories. But instead of looking at it as categories, we went down a layer and looked at it as just a bunch of, of simple functions that we can then combine using, you know, a function that combines to that says like an and of these two rules, or an or of these two rules. And just starting from that, we were able to organize it in into something that we could manage. And then of course, you still had you had the Alaska rule, which was like this big composite rule. But we were reusing a bunch of the sub rules in the other states as well. All right, before I move on to platform, are there any questions? All right, so just jump in if you have a question, I'll continue. All right, so sometimes your platform is complex enough that you need to model it as a subdomain. This is an example that I've seen a lot. I still see it. Ajax loading. So if you load a page and something like if the page is going to then do an Ajax to fetch the data that it needs from the server, sometimes if you read it real before it loads, if you read the message, it'll say something like, oh, there's no messages here or there's nothing to show. But there is, it just hasn't loaded it yet. And it's, I mean, it's a pretty clear thing like why it's broken. This is actually a picture from Slack that it'll show your direct messages like you don't have any, even though you do, they just haven't loaded yet. And really what they should show is a loading spinner, right? Like, like just just wait. Oh, and this is from Chris Jenkins blog. I couldn't find a picture myself. Like I couldn't find the bug and take a picture of it like while I was looking. So luckily he had taken a picture of it. Okay, so we're going to try to, this is like a piece of complexity of your platform. You're on the web, you got to do Ajax. Like instead of trying to get it together with like if statements and stuff, let's just model the thing. All right, so this is what is happening in this Slack client. You like initialize the messages to the empty array. And then you do a get. And then when the get succeeds, you just take the data, you grab the messages from it, and you reassign this variable to messages. And somewhere else there's a view. I'm using like a kind of react notation. If there's nothing in the messages, then you just say no messages. And otherwise, you make a list and you map over the messages in there and make list elements. And you're done, right? And this is the problem. It can't distinguish between I fetch the messages and it was empty, versus I started empty and it's still empty. And I'm still waiting for the messages to come back from the server. So Ajax is more complex. There are more than two cases. What about the when it's loading? And then what about errors? We haven't handled errors either. Let's model this. It's complicated. It's worth thinking about. All right, so this is a swim lane diagram. We got the browser and the server. The browser makes a request. And then the browser gives us a response. Very simple, but I needed to draw it out because we can see that from the browser's perspective, we don't know what's going on on the server, we're the browser. But from the browser's perspective, there are actually three different states times that we need to account for. There's this initialize time before we've sent the request. There's the loading while the request is in flight and there's success, right? So we can actually model this out with a bit of JSON. We have a status initialize status loading status success, which also comes with the value, whatever data we got from the server is stuck in there too. And then of course, it's more complicated. Like I said, there's errors. So this is actually the timeline splits here. Sometimes you get an error. So we need a fourth state where you have an error and it's instead of a value, it's going to have some error message. All right, now we can make operations that operate on that data type that we just made. So new Ajax value is just going to start initialized, right? And notice we're not giving it an empty array doesn't, it's nothing, has no value yet. Doesn't make sense to have a value. Okay, then when we request, we're going to pass in that Ajax value, but it doesn't use it yet. It's just we're going to return loading. All right, when we succeed, we're going to return this, obviously, and then the error is going to look like that pretty simple, but it doesn't really capture everything yet. Because after we do an error, we actually want to retry. So we want to go back up and load again. So what we need to do, oh, well, right, sorry. So it's it also if we get a success, we sometimes need to get a new version of it. You know, after five minutes, we want to refresh the data. We want to load it again. So what we need to do is capture these optional error and values in the loading and the error state. So if we're loading, we might have the old value still, if we were in this, this branch with this loop, y'all can see my mouse, right? Yeah, okay. So if we're in this left hand side, we got we went through success, so we have a value. So when we're loading again, we want to maintain that value, we might put a spinner, but we're still going to show the old value, because it might still be useful to the user. Likewise, if we're reloading after an error, we might show the error that happened. And then, of course, after we do the success, and we load, we come, we might come back down the error branch, right? So that could happen too. And so we want to put in the status error that we had a value from before, from that original time through the loop, we did succeed. So we want to show that too, like it still might be useful. And so I feel like this is a much more complete view of what might happen, because you can always reload. And so then that changes how we are going to write our functions. So initialize doesn't change. But this time, we are going to return a copy of Ajax value, but with the status loading. So that will keep, if there's an error in there or a value, it'll keep it in there. Now succeed is going to always wipe out any error that was in there. We don't need to keep it anymore. So it's just a literal. And then the error is going to keep, you know, a copy of the current Ajax value, but this time set the status to error and put the message. So that will keep any value that might be in there. Okay, so I put that in there because a lot of times, we, I've noticed people don't want to, they might model their domain really well, but they don't model that there's all this complexity in the platforms that they choose. They're just throwing if statements at the problem and not capturing the true complexity of it in something simple and a simple thing like this that you can kind of put in a, in a namespace and reuse. Notice value, we didn't specify a type for it could be anything. It's just, it's just there. And you can, it can compose with your domain types easily. We can model a lot of the stuff that our platform does file IO. So the files on your, that you need in your application, they are going to have some failure modes, you can build in a new type that would handle those error modes in the way that you want them to be handled instead of the way the platform handles them. Likewise, for databases, query could fail in some way. Threads and concurrency, there's always complexity there. And then exceptions like model, model your errors. Like you can, you can do a lot by like coming up with some kind of system-wide way that we're going to handle errors instead of like whatever this library we're using happens to use. All right. Any questions on that before I move on to the final, final lens? Yeah, so I think, I think I didn't quite catch what is the problem that we're solving with with this lens? Right. So the problem is people not stopping to solve the complex problems of their platform to say, hey, look, we're on the web, like that's not going to change. Ajax has its problems instead of handling it bespoke every time we do an Ajax request. Let's just sit down and think about it. Let's do some design. Right. It's like we handle it this way. On a system I worked on recently, we were doing a lot of distributed system stuff. And a lot of times we had to make operations item potent that weren't item potent. So like we had to do the get and see if, you know, and so we're like, let's just model this. How do we want to do this? How is this going to happen? Because we don't want every call to this to have to do that. We want to wrap it in something that like makes it happen every time. Right. Yeah. Okay. So that's helpful. And so one, I don't think in what I like in your podcast or in your newsletter, I don't think I've read about this lens before. This seems new to me. So or I missed it. I did change the name. I used to call it, I think I used to call it the stratified design lens. And the idea of architecture architecture. Architecture. Architecture. Okay. Yeah. Sorry. I was confusing it with the one from before. Yeah. Okay. Okay. So yeah. So I have, I have heard about it before then. But so, so one thing that I guess I'm trying to think about two things here. One is that we are dealing with platforms all the time now. And is this a way of just trying to think about the limitations, the problems of your platform? Because that seems, that seems a little more negative or snarky. It's more like the contours, the contours of your platform. Okay. Right. So Ajax, is there's always going to be the possibility that it fails. Right. And you shouldn't your approach to it as your software matures. Like I understand at first, you're just kind of hacking it together. Right. But as your software matures, you're going to want to say, how do we handle Ajax? Right. How do we handle when we, when we want to refresh a piece of data that we already have, but we want a more recent version? Like these are the kinds of things that are just distributed system problems. Like you need to call back to the server and they have like structure to them. So let's capture that structure in a space in a place and treat it uniformly. I've seen too many code bases where it's like, you know, call back hell or, you know, async await hell every time, every time there's an Ajax request, they're just like rewriting the same code over. So yeah. And obviously it still happens because you can still find these bugs out there. These Ajax bugs. All right. Thank you. Thank you. Thank you. I'm not missing any questions in the chat, am I? No. Okay. All right. Runnable specifications. This is the final lens that I'll talk about tonight. Just like a little review of the, of what, of what this process is. So we have this domain and we need to write code. So we abstract away the ideas in, in the domain into this model that we've got in our head. And then we encode that model into code. Now once we've got it into code, we can run it. We can play with it. We can just read it and see sometimes even that's enough. And we can evaluate like, is this really, does this encoding really say what is in my head? And then sometimes it does, but you're like, oh, but it doesn't work the way I thought it would. So you have to change your model. You have to go back to the domain and say, why did that break? Because the code is right. It's my model in my head. That's wrong. So this is the process we're going through. And basically what runnable specifications is about is to tighten this right hand side loop. So that we can get more information about, about our model by running it, by actually putting it into code and running it. Instead of, you know, one thing I want to avoid is like the big upfront design. I don't believe in it because you're never right. You could spend a year writing out the specification. And then after you convert it into code, you find out, oh, you forgot about this case, in that case, like you want to get it into code as quickly as possible, so that you can find that sooner. So just to pick on someone, UML, they also do modeling. That's what the M stands for. So you got this, you're here on the left hand side. That's now, and you're trying to get code into production. And what the UML approach basically says is like, well, go off in a different direction, the totally orthogonal direction, build a model out of, you know, pictures. And then you take that and you put it into production. And if it's wrong, you go all the way back here and redo the model. Okay, so it's kind of a, I'm trying to show it's like a big triangle that you're making here. Now, you know, what if the production as well? Yeah, okay. So runnable specifications is a little different. Same idea you're trying to go from now to production. In our case, you're writing the model, but it's code. It's mostly, you're encoding this thing. You know, it's function signatures that you could easily just turn into code. It's data models that run in memory. It's not production code, right? It is an encoding, but it is on the way. And then boom, then you go to production. But if you find out you can, there's a way now that you can find out that your model is wrong, much sooner, right? So you can run your model and find out and then come back. So like the feedback loop there is greater. And even though you're working on the model and not production code, it runs and it will be useful. It's kind of on the way to production code. So you get a faster feedback loop. Okay. And then of course, no design. You just go straight to production and you have to go back here. All right. So I don't know if that will last one lands very much. No design. It seems like the line should be like that. Okay. All right. So basically by runnable specifications, I mean, you're doing the design in your programming language. That's what I mean. So you, but you're making it's a model. It's not production code. It's more like a prototype. So you're doing an in memory, you know, you're doing data in memory. You're going to run tests on this model to make sure like the kinds of property tests that we wrote before, where you can test like that, you know, if I add and then remove and add in, it's the same one, that kind of thing. Then you've got this code that you can basically stepwise refactor it into a production implementation. And then you can also, because you have the model still around, you can use it as an oracle to test your implementation to make sure that it's functioning the same way. Now we'll go through each of those. So make in memory models. So just skip any platform specific stuff. Like even if it's in the browser, don't make any Ajax requests. Just just, you know, don't do any async, no database, nothing. Just remove all the complexity. You want to focus on the domain. No platform complexity. In memory data structures and functions, instead of, you know, remote procedure calls, just use like whatever the language gives you, JSON or types or objects, whatever you got. And then figure out how to tell if they're working. So like one thing you can do if it is something that's visualized, visualizable is like, show it. Like if you're working on some kind of, I mean, the idea I came up with is like, why don't we make a graphical system, right? Like if we're, if we're doing some kind of geometrical graphics, like let's just show it. And even if we're supposed to output the SVG eventually, that's a platform tail. Let's just plonk it onto a canvas in the browser. So visualize it and you can see, oh, it's working or it's not working. And then tests, of course, you got your manual tests, your automated tests. So let's talk about the tests. So you're really trying to ensure that you have the behavior and the properties that you want. So manual testing, of course, you can do that. You can write like little scenarios, you know, the barista. Was that a question? Okay. So yeah, you've got, you know, code up a little scenario. Okay, the barista adds this and this and this and then removes this. And you know, you can, you can see if it's working and use your intuition about if it's doing the right thing. Automated testing, obviously, you'd want to do that. And then the cool thing is that it's, because it's all in memory, it's like these things can run really fast. So as you're changing your model and updating it, you can just keep running those tests and get lots of really bad, fast feedback. And then because you're in memory and doing like removing all the platform complexity, there's very little investment to throw away or to fix if your problem is found. Okay, then we're going to refactor the model into the implementation. So you can stepwise convert stuff. So you can take like a JSON and just convert it into database schemas. However, you need to do that. You can take those function calls and turn them into Ajax calls. And then of course you're adding in the complexity of your platform now. So you'd have to turn it into a call that uses that Ajax that Ajax value that we made, etc. One thing that's another thing that I like to do is turn my functions into an interpreter. So instead of having, you know, functions that are opaque, you would turn it into like a piece of data that can then be interpreted. And so then you can print it out and see it. And the interpreter will run the same thing that the function would have done. And then I do want to say this, like, ideally, in a perfect world, you would not need to refactor it into an implementation. You could just run it, right? But we're not there. Like JavaScript, if you have some JSON object, it's not going to persist. You need a database. You know, you're, you're, you are going to have to make Ajax calls because the data's on the server. It would be really cool to somehow have it automatically do that, but we're not there. So you have to manually do it. And then this is the coolest thing that I think is like a big possibility, which is you have a system. You've been working on it with really fast feedback. You've tested it. It does everything you want. Now you have an oracle to test your implementation with so you can test that you're refactoring in the data to the, to the production code is actually functioning the same way. So you can run the two model, you can run the model and the final implementation with the same input. You compare the output. They should be the same. I mean, to me that this is like, it's paying for itself now, right? That little, you know, the part that where it wasn't quite lined up with that white line, the red arrow went off the line. Well, it pays for itself because you have a thing to test. You can just throw millions of tests at it and they should all come out the same. All right, my final slide. If you're interested in what I'm talking about here, I do talk about these ideas on my newsletter. I also talk about them in my podcast, but I'm not focusing on that so much right now. So get on my newsletter, ericnormand.substack.com. It's totally free. Cool. Thank you very much. All right. Thank you so much. I really appreciate this. This was really interesting. A ton of fun. There was some comments in the chat and I think we'll just open things up to discussion. What we traditionally do is we do sort of first part of the discussion, record it, and then we'll turn off the recording and continue asking questions and asking questions that we feel that we don't want to be public. I will open up the Q&A to people. Just unmute yourselves and ask your questions, please. I do have a question. Eric, have you used any property-based testing frameworks like Quick Check or FS Check? Yeah. Cool. Yeah. Because what you were saying early on was pretty much property-based testing and awareness, like stock and variance. I like the angle you approached it from. It was a nice, it was a really nice intuitive angle that you came out of from and I really dug that. So cool. Thank you. Yeah. I do, I have used property-based testing. I think it's amazing, changed my life. Didn't have to write so many, so many example-based tests. I do want to put it in my book. I think that I do have a take on property-based testing because most of the tutorials out there don't really help you come up with the properties. They're just like kind of theoretically like, isn't this great? They can generate hundreds of thousands of tests automatically. But I feel like what people need is the pre-existing algebraic properties and given to them is like, these are templates. You can take this and kind of modify it. You can generate partial properties from them. Sometimes it's communitive, sometimes it's not. But just start there. Maybe you don't want equality. You want some other comparison function. Whatever changes you need to make, just start with these as a template because people do seem to have trouble coming up with properties. My idea is that it's not my idea. It's common in the functional world. What people need to do is think about the properties first. A lot of people think about their methods on their object and they don't think about how those methods work together. So that's why I was focusing on the inverse. Some of these are going to be undoing and you probably know that already. So build that in. So can I follow up on that? First, can you turn off your slides? We can just have a discussion. I'll encourage other people to turn on their cameras. That would help. Depressing to just look at black boxes. I say this as a professor. Thank you. Can you say more about figuring out the properties? Because I was thinking about that with my own work. I think that it's really hard sometimes. It also assumes that you understand your domain and often we're figuring that out along the way. I mean, so some of the properties are things that you wouldn't even think of the test. So like one thing might be, hey, no matter what buttons I press on this UI, it has to have a price. Like it has, I have to whatever coffees I add to this order, I should be able to calculate the price. That is a property. There shouldn't be some combination of buttons that gets me into a non-priced order. So yes, you do have to know your domain. I can't help you with that. But the things that are properties, I think people just aren't used to thinking of them upfront. They're like, oh, no, wait, how did we get here? Where we have a coffee that you can't run calculate price on? It's a bug at that point. And you should be thinking about it ahead of time. Every coffee should have a price. That's a really good point. I did post a link to the F# for front-end profits. They had a list of some good properties. And you're right, trying to find the properties is challenging. And it's also interesting that these stock properties, you could often use a series of stock properties and get very close to that solution. Like, oh, yeah, well, item components plus community activity plus this. And I'm like 90% of the way to what my function should be, which is really cool when you do that. There's something else that I wasn't sure if you used any of those, like any of the model checkers, like a lot of similar line, like things like alloy. Okay, they're pretty interesting because especially like with alloy, like if you put it in your domain model as types, right? And you can put in like, keep properties that have to hold or not hold. And then you have it look for counter examples. And I think it actually uses a solver to where it actually tries to find things that disprove like the state you're telling it your coach you never be in. So instead of like the random based approach or property based testing, you know, I think I tried to do a direct idea. And it's really cool providing those kinds of states you were a bring to Galpline, you know, talking to the types of things. At the place I used to work, they use some people use TLA plus. And that doesn't exhaustive search. It basically is like every possible combination of inputs, it will test. And it is much more exhaustive than like ran only choosing 100. I've never done it. It seems like. Okay, one problem is you're now in another language, right? You're writing TLA plus for your C program or whatever. I would much prefer writing the properties in my, you know, C and using the actual data structure is the actual code that's going to go into either my model or into production. But I understand that some things are like, you know, so important, like the break system on a car that you would actually want to have a total exhaustive check. That said, like, you can always just 10x the number of tests you're running, like often it finds the problem, right? Yeah, I don't know. Yeah, the next translation step is definitely going to be an issue, right? I mean, there'll be ideals that could do it on the code you have, right? And that's, yeah, and that is kind of a bit of a weak point with some of them. It's like, yeah. Well, that's the kind of stuff that TypeScript does, right? Like the TypeScript checker is actually running a version of your code that can exhaustively check stuff. Like, so when you do a plus, it's actually using a SAT solver that represents binary notation for your floating point number, and, you know, has an adder built in, and it's a logic. I mean, that stuff is pretty cool. So you can exhaustively check, yeah, this is always going to return one or something like that. Thank you. Yeah, you're welcome. Would it be rude to ask for a preview on what the time lens is like? I can give the short thing here. The time lens is basically that when your domain model becomes sophisticated enough, it starts to incorporate time into it. So think of, think of double entry bookkeeping. You know, as banking became more sophisticated, it wasn't just enough to keep track of how much money was in each account. You actually wanted to know how did it get that way? What were the transactions in what order that moved the money around? And so you start to want to have a ledger, right? So even for something as simple as a UI, like, let's imagine you're developing the UI for someone to order a coffee on their phone. You want to have undo. So you add a coffee and add a bunch of stuff. Oh, I want to undo all that and go back to where I was. Like, that becomes a model of time, that undo, redo, because you're moving backwards in time. And so how do you do that? And basically, there's two ways to model time. One is you keep the history of all the previous states, right? The other one is you keep a history of operations that happen. So instead of having, instead of a function called, you know, set size, you would actually just transact a piece of data that was like set size to set size to gigantic or galactic. And that data would get put in an array or a list. And you could run through that list and generate the current state from it, right? As opposed to, you know, the other option, which is like you jump back to the previous state, it's like all saved there. This is saving all the operations that happen. They just reduce over it. Well, I was wondering, that's sort of related to in the earlier talk in Sweden, for it's reading, you gave a diagram that sort of looked like it would be the UI for the person taking the order. And it seemed like each of those little icons, as you touch them, would map directly to a function. And then there are two ways to build the final coffee. One is you're building the coffee or changing the size by just, oh, I tapped wrong. I just tapped the other one and it changes the size. Or you don't do anything until everything's filled out on the screen. And then you somehow turn that into one final coffee. You know, it's sort of like two ways of handling time. Sure. So let me see if I get this. You're saying that you would have some other representation for like a coffee that hadn't been like finalized? Yeah, but you were talking about undo and undo would be there's a different button. It wasn't showing on your originals. Oh, sure. Yeah, right. But I guess the other one would be submit or finalize. But in general, that sounded like a vented architectures. Aren't those really popular now? Yeah, it's like event sourcing. The event sourcing does this a lot. So, also again, I hope that this stuff isn't really new. I'm just trying to teach it better. Like I noticed someone was like, oh, yeah, free monad when I mentioned interpreter. That one popped up. Yep. This free monad. Sure. Much better to teach up without those words. Yes. You can bring it up later. Like, hey, just in case you ever hear someone say free monad. This is it. This is what you're talking about. Right. And I used to call the composition lens the algebra lens. And then people had all sorts of questions that were just distractions. So I did mention algebra, algebraic properties. But I thought coming at it from from that side was better. I do want people to learn the real words for stuff. But often, do you start with the words? They're scared. It activates like their lizard brain and they can't learn anymore. Yeah, I mean, I like your, I mean, I definitely like your approach because you sort of avoid these words until you introduce the concepts. And I think, and I think, I think with, I mean, I think one of the big downfalls with like the way functional programming is presented to many people is that a lot of these technical terms kind of come from front and center. You know, you'll get a lot of like very useful concepts, but you kind of lead from, oh, yeah, let's talk about input and output, you know, okay, well, let's talk about the above net from category theory. And you know, it's kind of like, hey, I could just go to, you know, language X and say, just do a just do a print, right? But at least by introducing and kind of showing an aid for something, you prime them. And then, and then you can sort of introduce that concept because, so I do like, I do like your approach because the way you kind of segue into the property-based testing, I thought it was really helpful. And I think that's a really good way to present it. It's something I think a lot of future programming advocates could really take a page from to really make this sort of more accessible to a lot more people or even get their interest, right? Because again, you're competing with so many other, you know, paradigms and technologies that if you leave with, you know, what sounds like jargon, you're like, you know, I think we'll take a look and see what Python's doing over here. That's right. That's right. Yeah, yeah. Thanks for, thanks for that confirmation. I do think that functional programming has a marketing problem. And also we have a teaching problem. Like, I think I talked about this last year that we tend to teach it backwards, right? So we want to teach, we're like, oh, the most important thing is monads. When it's really just the, like, the most recent cool thing that you've learned, like, it can't be the most important thing. It's not possible because monads are the super abstract things that are only applicable to certain situations. Whereas something like pure functions are necessary for monads. And also useful in a much broader context than monads are. And so they must be more important because you need them. And so, and then they're right there. They're just a step away for most people. If they already know how to write a function, it's easy to show how to make a pure function. So we teach it backwards. Like, we want to, like, start with the monads. We need to start with the more concrete stuff first. Can I ask a question? I think it was in the, in the scope section where you were talking about moving down a level. And so one was when, and I always, like, I think, I think your example of the voter registration project is super interesting and really good because it's super complicated. And it's something that, like, it's very real in a way that, like, you know, coffee shop is, I mean, coffee shops are real, but they seem to be more straightforward. And like, we can all there's an incentive on the business to stay simple enough for the customer to understand, right? That's a good point. Yes. Yes. That's a good point. There is an incentive at the state level to make voter registration as complicated as proud as possible. So sure. Or even easy for their citizens, but it doesn't, it doesn't help the person who's got to map out the whole 50 states. Well, but I live in Texas where we don't want people to vote. And I, and I put that on the recorded part. So well, to be fair, we all, we don't want certain people to vote. Anyway, I'm in Wisconsin. It's the most gerrymandered state there is. So I understand. So, so in this moving down level, I thought that there was something interesting going on that I don't know that I have a real question, but I'm hoping you can sort of say a little bit more one is that that moving down a level and writing sort of a library of small functions is very much the functional programming approach. And it's one of the places that I still, even after having done this for so long, get frustrated of like, I go to open like the documentation for a module, and it's just dozens and dozens of, you know, tiny tiny little functions that all kind of do the same thing and are slightly different. And so anyway, I just, I saw that parallel. The, the, and I'm wondering if you have anything to say about that, but then the larger sort of question that I had was in the past, you've very strongly emphasized the onion architecture. And this struck me as sort of a different approach where now, you're sort of suggesting, whereas the onion architecture is like, you start sort of small, and then you build up layers of abstraction here. It seemed like you were more emphasizing, oh, you could go up or you could go down as you sort of are teasing out these different levels of abstraction. And so wondering, yeah, I'm just wondering if you have any thoughts about those. So, my first thought is the onion architecture does not really dictate a direction of how you're building it. Right. The, I mean, I am kind of suggesting build your domain model first, right, before it's productionized. And the productionized is now you're dealing with IO, you're dealing with external servers and stuff. So then, yeah, that is kind of like adding in that layer of interaction layer that's, you know, on top of that. The thing about stratified design is that I think the more common thing is to go down a little, is to say, I can't, I can't organize it at this level. And the mistake people make is to keep trying to organize it instead of saying, this needs, you know, going back to first principles, which is basically what it is. Right. So like, you're like the classic example from stratified design literature is the thing from structure and interpretation of computer programs. It's, it's an MC Escher art language that they make in scheme. And you're at a certain level of abstraction, whatever level you're at, and you're like, I can't, I can't seem to get any leverage to draw these images better. Right. The answer is to go down. What, and look at your domain from a new perspective, like what, I'm not, it's not just this image is like this image and this every image like what is going on. Oh, I see. There's symmetry going on. There's rotation going on. There's repetition going on. And so then those have to become the building blocks that that are at a lower level that you're going to use at this level. And that's what, that's what we, that's what I was trying to express with my story. Like, what is going on? It's a simple, it's a very simple function signature of like, can this person vote in this election? It's two inputs and a Boolean output. Like this, this has a regular structure to it. That structure is only apparent if you, if you abstract away the details and you're like, what question are we trying to answer? That's at a lower level. At least that's how I see the layers going on. Similarly, what is an image? Oh, it's like a collection of pixels or whatever, whatever you want it to be. But, but you have to ask those questions that are like a level below. I think that's really interesting. In part, just, I was struck by the way you phrase of like, you're abstracting away the details and then working at a lower level. Usually when we think about abstraction, we think about moving up. Right. But, but I think you're right. Yeah, you're right. Okay, go continue. I'm going to, I'll jump in after. I'm not sure how much more I have to say. I think you're right. That's it. That's a sentence in and of itself. No, I'm thinking about my own work as an academic. And you and I have talked about it a bit before. And part of what I'm always doing is trying to, oh, I've got all these cases of whatever it is. It's companies or it's people or it's hospitals or whatever. And I've got all the details and I'm looking for the patterns. And, and trying to figure out which details are essential and which are just it is a critic incident. Yeah. And, and trying to get at something that is more fundamental so that so that is it is going down a level, but that is not how we talk about it. Right. Okay. And so this is, this is what I wanted to jump in on, but I stumbled on stopping myself. The word abstraction is really overused. In grokhing simplicity, I decided not to use it. And I talked about general and specific. And general goes down the layers. Right. So general means anybody more people can use it. Right. Because you use in more domains, use in more applications. So like you go all the way down the stack and you're at the computer, this is a universal Turing machine. Right. Like one more specific thing up is JavaScript. Right. JavaScript is more general than your application. And so you guys keep going up all the way until you've got your main function, which only does one thing it runs your software. Right. So that's the most specific thing. The thing is each of those is at an abstraction layer. You're not thinking about bytes when you're working at the, you know, main, you're not thinking about bits and NAND gates. But when you're thinking about NAND gates, you're not thinking about like, what is the particular like coffee shop that's running on this NAND gate right now? Like you're not thinking about that yet. Each one is an abstraction layer. It's just a different, a different layer. Right. And so moving up or down, you are choosing a different abstraction. What details am I ignoring right now? So yeah, that's my, my answer to that. But yeah, you're right. People think of like, Oh, I'm going to make a class. That's an abstraction on top of the four things that are fields in that, like, no, that's not an abstraction. I'm going to ask if other people have questions that they want before I stop the recording. Okay. So I want to thank you, Eric, so much for the discussion. It was great as always. And we've had a good discussion so far. I'm going to, oh, I, you know, what I wanted to ask you about Eric is, is you had talked about in the previous about the book. So apparently it's, you're publishing it for free on online. Is that the plan? That is the plan. So I'm working on getting it into like an HTML page that I could post online. And I'll have a print copy that you can buy too. Okay. Okay. Excellent. Excellent. All right. So I'm going to start, stop the recording now, and then we can continue the discussion for a little bit longer.