This is a maxim from Fred Brooks’ The Mythical Man-Month. These days I’m thinking it’s the single most important lesson there is about software. It’s been brought rudely home to me by my recent work on mod_atom, whose design is terribly simple; but I still got the first cut wrong in important ways.

What I Got Wrong · mod_atom is about as simple as an AtomPub server can be; all the entries and feeds and collections are backed one-for-one by ordinary flat files, with entries scattered across a YYYY/MM/DD tree, just like here at ongoing.

I had two URI/filesystem hierarchies, rooted at atom/ and pub/, to separate the resources that exist for AtomPub, like Service Documents and Media Link Entries, from those that are there to be read, like feeds and HTML pages; the goal was to make access control easy. How could that be wrong?

What happened was, I went back, after a few months of ignoring mod_atom, to build in Meta-pubs, so you can do CRUD on whole publications, not just individual entries and media files. This ended up touching most pieces of the system, and I realized that a couple of the core ideas were leading me into all sorts of tangled-code complexity; in particular the double directory structure.

So I rewrote it with one filesystem tree routed at pub/, backing two URI spaces which were identical except for being rooted at pub/ and atom/. The mod_atom code forces all PUT and POST and DELETE requests to go through the atom/ subtree. This allowed me to slash literally hundreds of lines of code.

There was another design error, just as obvious in retrospect; its cleanup also allowed subtracting severe gnarl and great big chunks of code.

Programmers experience soaring joy when they can rip through code deleting functions and declarations, screens-full into the bit bucket, with the steady drumbeat of tests-fail-then-pass.

So maybe I didn’t build one to throw away, but I built one that needed major amputations out of the box.

And Your Point Is? · This is by most standards a simple system; at the moment, excluding various XML- and HTML-munging libraries, less than six thousand lines of code. I’m an experienced programmer, have written hundreds of thousands of lines of C code, know the internals of httpd pretty well, and understand AtomPub as well as anyone in the world.

I’m not a truly great developer (I know this is true because I’ve worked with some) but I’m not a moron either (worked with some of them too) so if I’m going to make this kind of misstep, a whole lot of other people will too.

And let me make a strong statement: I’m not sure there’s anyone living, including your five favorite programming heroes, who could tell you, without writing the code, whether one or two directory trees were going to work out better for this problem.

Given that, what hope is there for waterfall development? Or for any approach that doesn’t leave space for going back and building things right once you’ve learned what “right” is by building things once? Well, none.

But You Already Knew That · Probably true. But you know what, how about we all agree, all of us who write about writing software, to write about this once every year or so. Because there’s this terrible glaring conflict between what sensible managers want and what sensible programmers know. Managers, good managers, want a plan; they want to lock in design constraints so that work can be dealt out and progress tracked and promises kept. Programmers, good programmers, know that they’re not smart enough to get the core design choices right until they’ve built something that works.

The various techniques and disciplines gathered around the banner of “agile” are on balance more honest at facing up to this unavoidable tension. But there’s still lots more work to be done.

And the most important thing is, we all have to remind ourselves, all the time, that we’re not smart enough to get anything important right the first time.

Exceptions? · Yes, of course. If you’ve just written three driver-scheduling systems or foreign-exchange systems in a row, you’ll probably go into the fourth with a pretty good grasp of what’s important. And those kinds of systems matter. But the interesting software is by definition the stuff that isn’t the fourth iteration of anything. Perhaps it’s best to close with Brooks’ quote in full:

“Where a new system concept or new technology is used, one has to build a system to throw away, for even the best planning is not so omniscient as to get it right the first time. Hence plan to throw one away; you will, anyhow.”



Contributions

Comment feed for ongoing:Comments feed

From: John Cowan (Aug 25 2008, at 00:17)

But beware of Zerouni's Corollary to (this particular) Brooks's Law: if you *plan* to throw one away, you wind up throwing away two.

See also the Wikipedia discussion of "second system effect", also a Brooksism.

[link]

From: Dave Pawson (Aug 25 2008, at 00:32)

The next step is to look for the trigger that enables the second generation. Adding a feature in this case? Or Just the need to crawl all over the code?

If you're doing this for someone else btw, ensure you 'lose the source code' of the first iteration which then allows you to do the better job the next time round. If necessary lose the source code, you retain the ideas that worked the first time and might find the insight necessary to make the step improvements Tim talks about. The killer is taking your 'demo' system through to production use when you designed it as a knock up demonstrator! Yuk.

[link]

From: Luka Marinko (Aug 25 2008, at 01:02)

There is a big difference between rewriting parts of the code (even the core parts) and starting from scratch (throwing old code away).

It might sometimes look easier to start from scratch (especially if its not your codebase) but that means rewriting and retesting boring parts (boilerplate and glue code) as well. And in most codebases I have seen this boring parts vastly outnumber the core (interesting) parts.

So rewriting yes, but for throwing away you better have a really really good reason (ie. it was written in cobol and you can't find programmers to maintain it anymore)

[link]

From: Matthew Laird (Aug 25 2008, at 10:02)

This is the kind of article that needs to be written for CIO or some other managerial magazine, since as you said, us developers already know these lessons well.

A lot of managers I've experienced (yourself excluded of course, Tim) look at software projects in a very linear fashion. Always going forward, and when a feature is complete you close the book on that chapter and don't look back.

Let's a strong lack of desire for a "regrouping" phase where you pause and review your experiences to that point. Sadly this typically leads to code that becomes more and more spaghetti as time goes on.

I agree, the top bloggers should write such an article once a year to keep these important lessons in focus.

[link]

From: Per H. (Aug 25 2008, at 10:12)

Smalltalk & Lisp developers have been touting this for years. Incremental development; continious protyping - whatever you want to call it.

You can never anticipate the insights you'll discover after having implemented the system at least once.

[link]

From: Mark (Aug 25 2008, at 11:43)

"Lose" your source code, but keep your unit tests!

[link]

From: Daniel (Aug 25 2008, at 13:50)

<em>"Build One to Throw Away - This is a maxim from Fred Brooks' The Mythical Man-Month. These days I'm thinking it's the single most important lesson there is about software."</em>

That's rather peculiar, as Brooks himself has pretty much retracted that statement: The current edition contains a chapter <em>TMMM after 20 years</em>, which reads: "'Plan to throw one away; you will, anyhow.' This I now perceive to be wrong, not because it is too radical, but because it is too simplistic." (ISBN 0-201-83595-9, p. 265)

[link]

From: PVC (Sep 09 2008, at 11:20)

Well-told from the programmer's view. But imagine if you are an executive.

Say you can spend $5 million on R&D projects. Each of the 50 projects proposals you look through says "I don't know how much it will cost, or how long it will take until I get done. But that's ok, because Tim Bray says so in his blog."

What projects do you fund? Can you fund 5 or 15 projects? How do you get an idea of what your return will be on what you invest?

Would you hire a contractor to paint your house, if he says "I'll tell you what it will cost, and how long it will take when I'm done."

[link]

From: Stuart (Sep 09 2008, at 20:35)

Building a house and building a first of its kind structure (space station, skyscraper, medical device) are very different things.

You can estimate, but only with limited accuracy. Building the 10th skyscraper is a lot more predictable, even if it isn't the same size or shape.

which is exactly part of the point that Tim was making early on.

[link]

author · Dad
colophon · rights
picture of the day
August 22, 2008
· Technology (90 fragments)
· · Atom (91 more)
· · Coding (98 more)
· · Web (396 more)

By .

The opinions expressed here
are my own, and no other party
necessarily agrees with them.

A full disclosure of my
professional interests is
on the author page.

I’m on Mastodon!