3 models of software development as a factory
Summary: Agile software development came from borrowing processes and ideas from manufacturing. Is software engineering like factory work? I examine three metaphors of software engineering as a factory.
Many of the processes we associate with the Agile movement, and certainly many of the ideals, were inspired by manufacturing process management---Lean Manufacturing and the Toyota Production System. Many people criticize the Agile movement on the grounds that software development is very different from manufacturing and so the metaphor might be more harmful than helpful.
Because they believe it is harmful, people have pulled back from the metaphor and only used what seems to be appropriate for teams of programmers. However, do those people know anything about what manufacturing, especially Lean Manufacturing, is even like? How do they know that software engineering on a team is not like building a car?
I've been reading books on Agile and the Toyota Manufacturing Process to really understand what's happening to my industry. My experience on development teams, and talking to other developers, is that only a few know anything about manufacturing. They only imagine what building a car must be like: turning the same screw on 1,000 different cars every day. My research tells me it's anything but. And by doing that, they're basically acting like a bad factory. Maybe it's time to reexamine this bias.
In this essay, I'd like to really dig deep into the metaphor. I'm asking the question: how *is* software development like a factory? I explore three different interpretations and their consequences.
1. Dev team as the factory
I think the most popular way to imagine a dev team as a factory is to see it as a set of features moving down an assembly line, getting gradually closer to deployment. The feature goes from conception, to design, to scheduling into the sprint, to development, through code review, then testing, and finally deployment.
Ideally, the process always flows forward as each step is completed. This way of rationalizing the process is strangely soothing, especially to a manager who would like to track progress. Number of tickets deployed, or its proxy Velocity, feels like a nice metric to get a feel for how the features are coming along.
Fleshing it out
To flesh out the metaphor, the people working the process are like factory workers. Work comes to their station, and they do it. So a feature gets to QA, QA tests the software, then hits a button for it to proceed to the next step. Each worker is a cog in a greater machine, and no one has to know more than their job.
So what are the consequences of this model? From my experience, this model makes the developers, and probably the other folks on the "assembly line", feel like cogs in a machine. Everyone understands their part, but they can't appreciate the whole. They are like so many Lucille Balls desperately trying to keep up.
People are rewarded for velocity. They don't know how each feature fits into the whole. All they know is that it was upstream from them and if they pass it on, they can push that button. And so problems get pushed down the line. I've been there and it's not fun.
The teams get divided by function. That means design is separate from dev is separate from testing, etc. Instead of working together, this metaphor can create animosity between the groups as they blame each other for problems.
2. Software as the factory
This is a metaphor that I have never really seen before, but I think it has some merit. Instead of seeing the team as a factory, you see the software as a factory. If car factories deliver cars, your accounting software delivers accounting. A smoothly running factory means smooth accounting. That means invoices move down an assembly line. Programmers build that assembly line.
Fleshing it out
If our software is the factory, then how do we correspond the various roles? Well, at the bottom, invoices and payments are the partially completed cars moving through the assembly line. As an invoice comes in, it is put through a series of steps, like review, recording, and finally payment.
The factory workers are the necessary humans in the accounting loop---usually the customers of the software, but occasionally someone from the software company will have to unjam the works. They work at their stations and review invoices or sign checks.
So what are the programmers, designers, and operations people, then? They're the process engineers, of course. They are making the machinery that the factory workers use to do accounting.
I like this model because it clearly aligns everyone to the job the customer is paying for. Think about it. It's the process engineer's job at Toyota to make sure the workers can safely and effectively deliver cars to customers. Wouldn't it be nice if everyone's job at an accounting software company was to ensure that accountants could effectively deliver accounting to customers? It's much better than the job of delivering accounting software features. Customers aren't paying for features. They want their accounting to be done.
This metaphor also means that every software engineer at the company has to understand accounting. I strongly believe that programmers should become subject matter experts themselves. Doesn't someone engineering a process for building cars have to know about cars?
Further, it means that the programmer, just like the process engineer, will sometimes go into the factory to help clear things up. This means you'll see programmers helping customers with accounting issues, like not knowing what to do with a particular invoice. If it's your job to deliver accounting and your factory's current machines can't handle it, you've got to do it by hand.
Finally, this metaphor reunites teams. Programmers are not at different stations from designers. They're all there to make this factory run smoothly and make the work of the customer better. What would an Agile team look like that took this approach?
3. Compiler as the factory
A final metaphor I'd like to explore is the compiler as the factory. In this metaphor, the various software artifacts that get deployed are the product. The compiler, the build tool, etc, those are the factory workers putting the thing together.
This metaphor was inspired by Glenn Vanderburg in a series of talks he has given.
Fleshing it out
Let's see how the various pieces correspond in this metaphor. The car, our product, is the artifact that runs the software. That could be a binary for a compiled language, or a set of files that will be interpreted by a runtime. The factory workers, and the whole factory, are the compiler and build tools that assemble that artifact. So where do programmers fit in?
Well, the programmers are engineers. They design the artifact their compiler will build using high-level design documents that they call *programs*. That's right. Coding up a solution is a blueprint for the compiler to use to know what to build. Engineers do tests and experiments to make sure their designs work. And so do programmers!
This is a very neat metaphor, which boldly lets us call ourselves *engineers*. Seriously, watch the talk to understand how programmers typically caricature the idea of engineering. I really like that we can be engineers again and not "developers".
The main problem with the metaphor is that where do the other disciplines come in? Where does the designer fit. Are they industrial designers, working alongside industrial engineers to design the product?
Another problem is how do programmers get feedback from customers about how the software can be improved? I imagine programmers and designers in an R&D lab, trying out different ideas and doing tests for how well they work with real customers. In this way, they may be more like an IDEO product design team than manual laborers.
We've assumed the manufacturing metaphor through the Agile movement, but have we really examined it? I believe the way we build software, under the Agile umbrella, is missing out on some of the best parts of modern factory work. If you look at it the wrong way, programmers are reduced to factory workers, trudging through their mindless work. Measurements of story velocity only incentivize the company to push for more features faster.
But we could be so much more! If you look through a different angle, programmers could be aligned with the business value, and measured on delivery of that, instead of delivery of features that may or may not help the customer.
I hope that exploring different ways of seeing our work can give us ideas for the ruts we're stuck in. In my experience there is lots of room for improvement. We're stuck thinking of ourselves as car assemblers when we could be designing cars---or even designing car manufacturing itself. And with these three metaphors, I wonder if there are more practices we can borrow from manufacturing.