The Regenerate Web

RollerSkateImplementation agile 9 September 2007 Reactions

A key property of agile development is figuring out how to make a system go live with a small subset of features. We build software for the business value it offers, the quicker we go live, the faster we get at least some of that business value.

My colleague Dave Leigh-Fellows told me one of my favorite examples of this kind of thinking. It came when we has working for a brokerage firm. They had a new kind of product that they wanted to get into the market. The full software support for this was a web page that the customer filled in that generated the necessary transactions against the back-end system. But Dave came up with a way to get the product into the market faster than that.

• Version 1 was a static web page that described the product and provided a telephone number to call. Some temporary staff then spoke to the customer and entered the information into the back end system.
• Version 2 was a web form that captured the data the customer filled in. However this version didn't load that data into to the back end system. Instead the web form generated a fax. They hired some more temps to get the orders from the fax machine to the people that keyed the information into the back end system. Since the fax machines were a bit of a distance away, this is where the roller-skates came in.
• Version 3 hooked the web form into the back-end system directly.

The first two versions may not have been the most elegant solutions ever conceived, but they did get the product into the market much more quickly. I've not come across any other examples of iterative development that use roller-skates, but that may be more due to lack of imagination rather than lack of need.

DoctorWho leisure 7 September 2007 Reactions

Like most Brits my age I grew up with a sci-fi Children's program on BBC called Doctor Who. (For those who know, my doctor was Jon Pertwee, although I also saw a good bit of early Tom Baker.) It was actually the longest-running sci-fi TV series in the world, running from 1963-89.

A few years ago it was revived in the UK and has become a big hit - and not just for children. Doctor Who always had themes and scripts that went beyond the children's' audience and the series developed a huge fan base that lived off books and audio series even when the TV series died. The revival follows this with shows that are written to appeal as much to adults as kids. It was really great to sit with a couple my age and their 8 and 10 year old daughters and enjoy the new series. The scripts and acting are good, the only change is that the special effects are also good now (the old special effects made classic star trek look high-tech).

At home we don't watch much telly, the last shows we watched regularly were Buffy and Angel. Cindy, being American, had never seen Doctor Who growing up, but she loves the new series. When we get a new set of DVDs there's usually several nights of "it's late, we're tired, but maybe we can do one more".

If you've never seen Doctor Who the place to start is the opening episode of the revived series: Rose. (Wikipedia has mind-bogglingly comprehensive coverage, but I won't link from here as it's naturally full of spoilers.) Rose not only introduces the set up you'll need for other episodes (who the Doctor is, what the TARDIS is) but also does a really good job of distilling the tone of Doctor Who, capturing the mix of adventure and comedy.

If you like Rose then you can either carry on with the full first season or cherry pick highlights. If you prefer the latter I'll suggest my favorites. My big favorite from the first series was the two part The Empty Child / The Doctor Dances. I rate this as better than most films I've seen, certainly better than most TV. (It won a Hugo award so it's not just me.) It was written by Steven Moffat who is also known for writing the comedy series Coupling. Almost as good is Dalek. It lacks the humor but scores due to a wonderfully intense performance from Christopher Eccleston. I also really like the final two part (Bad Wolf / The Parting of the Ways) but you really need to see the whole series to appreciate it properly. (A tip if you do watch the whole series: don't watch the trailer for Bad Wolf (it appears at the end of Boom Town) as it gives away an important part of the plot of Bad Wolf.)

The second series has a different actor playing the Doctor (they have a nice technobabble rationalization to allow them to change actors easily). It doesn't quite hit the high spot of The Empty Child but is still really good. My suggestions for cherry pickers here would by The Girl in the Fireplace (another Moffat Hugo win) and The Impossible Planet / The Satan Pit.

When I said the second series didn't quite hit the heights of The Empty Child, I'm not being very reasonable because those two episodes are far too good for any TV series to live up to. However the third series (not yet all broadcast in the US - it's good to have friends in the UK) hits those high notes twice. Human Nature / The Family of Blood is a super two-parter that threatens to take away Steven Moffat's crown of writing the best episode. Moffat's response is Blink, which is as good a 45 minutes of TV as you could ever hope for. Not just has it got a great story and some cracking humor, it also achieves Doctor Who's higher purpose. You see Doctor Who is only secondarily about entertainment, it's primary purpose was always to scare the living daylights out of small children. I may be too old now to get behind the sofa, but I do remember how much I enjoyed it.

TimeZoneUncertainty design 6 September 2007 Reactions

I was in Boston, about to fly out to our office in Calgary. I look at my calendar to see if I have a meeting. First one is at 10.30am - cool no need to rush out of bed in the morning.

I turn up and am two hours late. What happened was that I was invited to the meeting that begun at 8.30am Calgary time. Lotus Notes saw my computer was set to Boston time, and helpfully converted the time zones for the two hour shift.

You could argue that this was my fault for not paying attention. After all I know this is how Notes works and was being careless when I read my diary. I don't buy this, Donald Norman noted a while ago that we tend to blame ourselves for errors that are due to bad usability - like a door with a handle that you should push.

Time zones are particularly vulnerable to this kind of problem It's most notably a problem in calendaring applications, but you see this issue in enterprise software as well. There is a temptation to try to be clever with handling time zones, but this temptation leads to trouble if the software isn't quite clever enough, which is what happened in this case.

I'd prefer Notes ignore time zones completely. You set the time for the place the meeting occurs in and that's what it should show you. Who cares about the time zones? It's only the time on the ground that usually matters. When I look at my calendar for a day in Calgary, I want to see the Calgary times for that day - wherever I happen to be when I'm looking at it.

The exception, of course, is a phone meeting that spans time zones. But here you should do something exceptional for a phone meeting rather than complicate handling a physical one. It could be something that allows you to set a flag for a phone meeting, and then you get a different time display. Or it could be as simple as just allowing you to put the timezone on the time display, and leaving the conversion to the reader. With phone meetings you think more about time-zones than you do for physical meetings (or at least I do).

The important lesson here is to make the most common case (physical meetings) simple and only do complicated things for less common cases as exceptions (quite possibly manual exceptions). Calendaring time zones get into trouble because they make the common and simple case be more complex. The problem occurs because the designers wanted to use the same data for both the simple and phone cases - but that just gets the simple case in trouble.

Usually when I hand out a prize for worst-user experience Lotus Notes is at the top of the queue. (Indeed I find it embarrassing to admit that ThoughtWorks uses the damn thing.) But the worst time-zone experience award goes to Microsoft Office, although to be fair this was many years ago. I had recently bought a PDA (running Windows CE version 2). I had put some all day meetings into my calendar, flown to Chicago, and to get the PDA to alert me to meetings I changed the timezone of the PDA to one hour earlier.

Suddenly every one of my all day meetings shifted to a day earlier. This was due to a catalog of errors. First off they had stored an all day meeting as a meeting from midnight to midnight - that's the kind of representation error on a TimePoint that often gets people into trouble. Then it was compounded by shifting the times of meeting when I changed time zones - so all day meetings were now 11pm to 11pm. This, of course, is due to putting the time zone into the meeting so it looks like it shifted when I changed time zone on the PDA. Then to cap it off, the software was clever enough to know that an all day meeting should only show the day of the meeting, but the day it chose was the day of the start of the meeting - that was now one day earlier. That's the poor timepoint representation biting back.

CustomerLoyaltySoftware design 4 September 2007 Reactions

I was in the Calgary office last week and had a good chat with John Kordyback, one of our most trusted technical leads. He's worked on, and dug into, a number of travel loyalty software systems (frequent flyer/sleeper etc) and we talked about the nature of these kinds of things and how to think about them in a more fruitful manner.

The core of of a loyalty system is a system to keep track of points (or miles). This should allow customers to see their points and also for the company to manage the unredeemed points. Although it seems that most people don't see it this way, this is essentially an accounting system, just switching points for dollars. John's observation was that repeatedly he runs into what people see as difficult problems that are much easier to deal with once you put accounting spectacles on.

An example of this is dealing with ad-hoc changes. However good your automated rules processing is, there always cases when something odd happens and you have to intervene manually. The result for many systems is and ad hoc change to totals that is error prone and unaudited. With an accounting frame of mind, however, you look at these changes as accounting adjustments and the patterns for this are well understood.

A notable difference between a loyalty program and most accounting systems is that a loyalty program is more about managing liabilities rather than managing assets. Hence there's more focus on things like risk management, as well as common themes like taxes and revenue reporting.

Many loyalty systems have multiple kinds of points, such as such as regular miles and elite qualifying miles. This is a common point of complexity. If you use an accounting viewpoint, however, you can track these easily as multiple currencies.

An interesting twist on this is potential points. If I book a flight for next month, the airline needs to know that there are miles I will earn when I fly next month (potential miles). These potential miles affect their liabilities. However it's only when I fly that they turn into real miles. Again accounting thinking can help here, we can use multiple currencies again, or use an accounts payable notion. The mechanisms are there and well understood, we just have to apply the model to the situation.

We fleshed this out in practice where we also found it really helpful to use TestDrivenDevelopment. A group of people spent a couple of weeks trying to sort out potential miles with planned design, but the core issue was cracked in a couple of days with TDD. The crucial part of this was focusing on examples to make the problem concrete.

The accounting analogy also applies, although partly less directly, to deciding how to award miles for activity. Any program has activity rules that need to be very flexible and need to cope with constant changes to the loyalty program. We can look at this as following the model of domain events triggering accounting entries through using Agreement Dispatchers. This is a pattern John and I have used lots of times and works well to these kinds of changing rules. Essentially we have agreements that represent the overall program rules for a class of participant. Each agreement consists of a set of posting rules keyed by the type of event and a date range. When an domain event occurs (a hotel stay) we look up the agreement dispatcher for the customer, and use the event to look up the right posting rule. We then run the posting rule to create the appropriate accounting entries to represent the miles for the event. The time dating of the events allow us to change posting rules over time but still be able to handle old events and correctly do automated processing of adjustments. (Some day I'll finish writing up these patterns, but what I have on the web is hopefully enough to give you some ideas.)

The second aspect of a loyalty system is tracking the customer experience. Since the accounting requires the system to record the customer's activity, the loyalty system acts as a natural base to learn from the customer's interactions with the company. Much of this is data mining - looking for patterns in customer behavior which can lead to new products and promotions. You can also use this activity history to assess the success of promotions - if you offer a mileage bonus for flying a route what is the response like?

Like me, John is a strong proponent of using ReportingDatabases, and this is a good fit for this kind of problem. The accounting side needs a very different set of data structures and uses regular updates as activities occur. The customer experience analysis is all read only, so you can use less normalized structures with regular, but not necessarily real time, feeds from the accounting side.

Taking it further, it seems reasonable to completely decouple the accounting and customer experience systems. They are both usually lodged together as a single customer loyalty system because they track the same events. Yet since they differ so much on the inside it may make more sense to treat them as two separate systems that feed off the same event stream (the accounting side would probably generate some events for the customer experience side too).

One of the habits of customer experience tracking is frequent changes to the system to support new kinds of analysis. We speculated that we could try an approach that had a single stored event log of customer activity, and plug in relatively independent 'miners' that would transform selected information from the log into more particular data structures to do different kinds of analysis. The miners could be relatively independent of each other and thus easier to build.

As you can see, our discussion did shift from looking at John's experiences to some of our joint speculations about how a system like this could be built in the future. What's clear to us is that there is a lot of room for exploring new ideas in this space that could introduce a new set of abstractions that would lead to systems that can provide better support to this business activity. More and more attention is being paid to this these days, so this seems like a fruitful territory for us to work in.

IsChangingInterfacesRefactoring refactoring 2 September 2007 Reactions

A RefactoringBoundary.

Is changing the interface of part of the code a refactoring?

The answer to this question is pretty simple - changing an interface is a refactoring providing you change all the callers too. A great example of this is Rename Method, which is an interface changing refactoring present on pretty much all refactoring tools.

The changing of all the callers is an essential part of this refactoring. Just changing an interface declaration will break the system - and thus isn't a behavior preserving change.

Interface changing refactorings do assume that you can get hold of all the callers, which is why you have to be much more careful with PublishedInterfaces. With a published interface, the interface itself is part of the observable behavior of the system.

Dynamic languages can make these changes much more awkward. Static typing really does help here in pinning down exactly which interface is being called at various points. Reflective calls that can also make it harder to find, either by embedding method names in strings or even composing them at run-time. This is another area where tests are essential even in environments that have refactoring tools.

Updated web site: A couple of months ago I attended The Server Side's Java Symposium in Barcelona. Neal Ford paired with me on a keynote on Domain Specific Languages and related beasties and this talk is now available on video (and transcript). Neal will be joining me for a half-day tutorial on this topic at QCon in San Francisco later this year.

Updated web site: Continuous Integration is a topic that often brings people to my site. As a result it seems appropriate that I can now announce that Paul Duvall's book on the subject (part of my signature series) is now out.

OneLanguage design 28 July 2007 Reactions

Should we strive to only have one language in our development efforts?

Throughout much of the last decade the fashion in the enterprise software world has been to focus on one standard language for software development efforts. Many development organizations strive to do all their work in Java (or C#/VB).

The rationale for this is that developers find it hard to be proficient in more than one language. Sticking to a single language lowers the learning burden, particularly when hiring new people.

There's some truth to this, but also much that's missing. The programming environment is partly language, but also about languages and frameworks. Larger frameworks, Hibernate, Struts, ADO, present as much of a challenge to learn as a language even if you program them in a single host language. Often the difficulty of expressing what you need in the host language is sufficiently awkward that many frameworks resort to configuration files, which are effectively external DomainSpecificLanguages written in XML - which adds a jigger of 80 proof ugliness to them.

For many developers, the one-language notion is a sign of lack of professionalism. This is best exemplified by the Pragmatic Programmers' advice to learn a new language every year. The point here is that programming languages do affect the way you think about programming, and learning new languages can do a lot to help you think about solving problems in different ways. (It's important to learn languages that are quite different in order to get the benefit of this. Java and C# are too similar to count.)

I agree with prags' advice here, as in most things. But I also sympathize with the overhead of learning a new language. My personal scripting is pretty much all done with Ruby and I've been loath to any more than play with reasonable alternatives like Python, Groovy, or PowerShell. It's not that it's hard to use the alternatives, but with Ruby I know too much that I'd have to look up with the alternatives.

The important point here is that when I'm writing these scripts I'm not manipulating new abstractions. Much of what I do is fiddling with text, the file system, and hunks of XML or YAML. If I need to take on a sizable new abstraction, the cost of learning it as a library isn't really much less than the cost of learning a language to manipulate it. If I want to specify a directed graph structure for display, learning Graphviz's Dot language is hardly more work than learning a new Ruby library.

Using a DSL instead of a library can offer us better ways of manipulating our abstractions. This makes it easier to see what we've written and to reveal our intentions. An API is like declaring a vocabulary, a DSL adds a grammar which allows you to write coherent sentences.

This argument is strong for DSLs, but does it also apply to general purpose languages? If you are working in Java (or soon C#) does it make sense to use Ruby now it's available on your platform?

The last decade has seen the rise of memory managed C-based languages. People saw that, despite many years of skepticism, memory management makes life sufficiently better that it was worth stepping away from C and C++ in the enterprise world. A common platform and language also pulled people away from proprietary walled gardens like Powerbuilder and Delphi.

Now there is a similar question. Are modern scripting languages another step forwards? Do we prefer their well-chosen terseness? Time and time again I hear experienced Java and C# developers report they are more effective in Ruby - which is why I've been encouraging Ruby. It wouldn't surprise me if similar reports appear in the next few years about other languages too.

A decade ago I was talking to my old friend Tom Hadfield at OOPSLA 96. Java's rise was apparent and it was clear that Smalltalk's future was doomed. Despite my love of Smalltalk I was pretty sanguine. I felt that Java gave people enough of what they needed; while it wasn't quite as nice as Smalltalk it was enough of an improvement over C++, particularly with memory management, for me to be happy with it. Tom disagreed, he felt there was something fundamentally different about the expressiveness of Smalltalk, the way you could better capture the intention of what you were doing directly in your code - closing the gap between domain knowledge and programming.

In the intervening years I've come to the view that Tom was right after all. After several years in curly brace land, Ruby reminded me of what I was missing. There's a clarity to reading Ruby code that just makes it an easier medium to work with, despite the inferior tooling. I'm way more sympathetic to the Smalltalk holdouts than I felt then, even though I haven't felt inclined to open an image in anger for a long time.

So are we returning to the language cacophony of the late 80's and early 90's? I think we will see multiple languages blathering away, but there will be an important difference. In the late 80's it was hard to get languages to inter-operate closely. These days there's a lot of attention to making environments that allow different language to co-exist closely. Scripting languages have traditionally had an intimate relationship with C. There's much effort to inter-operation on the JVM and CLR platforms. Too much has been invested in libraries for a language to ignore them.

So my sense is that we will see multiple languages used in projects with people choosing a language for what it can do in the same way that people choose frameworks now. I agree with Neal that we are entering a period of Polyglot Programming.

InstallingOpenArchitectureWare dsl 27 July 2007 Reactions

There are few things more frustrating than spending hours trying to install a piece of software and then having to delete everything and start again. Today at 9.30 I began installing openArchitectureWare, I finally had it installed (I think) at 15.30. So I thought I'd write this to help someone else do it more quickly.

OpenArchitectureWare is a set of tools, based on Eclipse, to support Model Driven Development. I'm interested in exploring some of its tools that are oriented towards DomainSpecificLanguages. (Xtext - which helps you develop textual languages - is something that's specifically been pointed out to me as worth looking at.) I don't know how worthwhile these tools are yet, after all it took me most of the day just to install the dratted thing, but we'll see.

One of my problems with the installation was that I'm not an Eclipse user - my usual Java IDE is IntelliJ. To install openArchitectureWare you need to know how to deal with the plugin system in Eclipse - and I'd never done anything with Eclipse before so that was new to me.

The first step was the easiest one - install Eclipse. I installed it on my Ubuntu machine, so all I had to do was wajig install eclipse (wajig is a unified command-line for various debian packaging and sysadmin tools). Then all hell broke loose. Rather than go through my miserable morning, I'll explain what I would do now.

The trouble with OpenArchitectureWare is that it has dependencies, other eclipse plugins that need to be installed before it can work. As anyone with experience in these things knows, sorting out dependencies can be a right pain without a good tool. apt-get for Debian and gem for ruby are examples of a good tool that resolves dependencies. When I installed eclipse, apt-get knew it had to pull down a whole host of dependencies and installed them for me. The situation in Eclipse is not so good.

To install openArchitectureWare you need a bunch of plugins: EMF, UML2, ATL, and GMF. I couldn't see from the web pages exactly how to get these things, or if they had their own dependencies.

There are several ways of installing plugins in Eclipse, although I had to hunt a bit for instructions. The easiest way is a menu option in Eclipse itself. In the menus pick [Help -> Software Updates -> Find and Install] (no I don't know why it's on the help menu). With a bit of button pushing you can get it to download a list of packages - the relevant source is the Callisto Discovery Site. Once you have that list downloaded look in the Models and Model Development section and select Eclipse Modeling Framework (EMF) and Graphical Modeling Framework (GMF). You'll get an error message saying that these have an unresolved dependency. Take note of the button on the right that says 'select required'. Hit it and it will find the dependency to GEF and its dependency on Batik. If you don't see that button and hit it you'll have a frustrating time trying to find them (believe me, I know).

That gets two of openArchitectureWare's dependencies. The others, and openArchitectureWare itself need to be done the harder way. Digging around the eclipse site I found the relevant web pages for UML2 and ATL. These need to be downloaded as zip files as does openArchitectureWare itself.

When you unzip the UML2 and openArchitectureWare folders they unzip into a folder called eclipse that contains subfolders for plugins and features. You can take the contents of these folders and put them into corresponding folders on your load environment (in my case /usr/local/lib/eclipse). As that didn't work for me when I tried it first, I found another way.

The way to tell if stuff has installed properly is to go to [Help -> Software Updates -> Manage Configuration]. When you open that you have the option of "Add an Extension Location". An extension location is (almost) any directory that contains an eclipse folder with subfolder for plugins and features. I say almost because the eclipse folder also needs a file called .eclipseextension. This is just an empty file so you can create it with touch .eclipseextension. What I did is created folders in /usr/local/lib for openArchitectureWare and uml2-eclipse, moved the unzipped eclipse folders in there, did touch .eclipseextension inside each of them and then added them using "Add an Extension Location". ATL just produces a plugin directory so I copied the contents of it into the plugin directory for openArchitectureWare.

It's important that you do this after you use the Find and Install tool because if you do it first, the Find and Install tool will tell you have an unresolved dependency and refuse to do anything until you fix it. When I was all installed it tells me "UML2 End-User Features (2.1.1.v200707181556) requires plug-in "org.eclipse.emf.ecore.xmi (2.3.0)". I don't know how to fix this and I have a bunch of emf.ecore jars present in EMF. However the rest of eclipse seems to work so far, so I'm carrying on regardless.

UiPatternsReadings design 12 July 2007 Reactions

In the summer of 2006 I did a major chunk of work on UI patterns. Since then they've been very much on ice as my primary writing focus has shifted (although not very visibly) to DomainSpecificLanguages. On this page I'll keep a note of links to writings I've liked that are connected with that work.

Over the last few weeks Jeremy Miller has been writing an excellent series of posts on UI design in C#.

DslReadings dsl 13 July 2007 Reactions

(See my note on DomainSpecificLanguage for a quick intro to this topic and my terminology on it.)

Update:David Laribee has written a post contrasting what he calls ordered and unordered fluent interfaces. The distinction is that ordered fluent interfaces force a particular flow on how you compose your DSL sentence. He provides an example where he uses multiple interfaces on a single ExpressionBuilder - the same technique that's used by JMock.

Anders Norås has written two interesting articles on writing internal DSLs in C#. The first article gives a sample of the DSL and a discussion against Chromatic's cynical check-list. The second article goes into details about its implementation.

Piers Cawley makes the point that a key characteristic of DSLs is their narrow focus on a domain.

DesignStaminaHypothesis design 20 June 2007 Reactions

Is it worth the effort to design software well?

From time to time I have indirect conversations about whether good software design is a worthwhile activity. I say these conversations are indirect because I don't think I've ever come across someone saying that software design is pointless. Usually it's expressed in a form like "we really need to move fast to make our target next year so we are reducing <some design activity>".

In there is a notion that design is something you can trade off for greater speed. Indeed I've come across the impression a couple of times that design effort is tolerated to keep the programmers happy even though it reduces speed.

If it were the case that putting effort into design reduced the effectiveness of programming I would be against it. In fact I think most software developers would be against design if that were the case. Developers may disagree on what exactly is good design, but they are in favor of whatever brand of good design they favor because they believe it improves productivity. (And by "design" here I mean either up-front design or agile's approach, ie planned or evolutionary design.)

Design activities certainly do take up time and effort, but they payoff because they make it easier to evolve the software into the future. You can save short-term time by neglecting design, but this accumulates TechnicalDebt which will slow your productivity later. Putting effort into to the design of your software improves the stamina of your project, allowing you to go faster for longer.

One way of visualizing this is the following pseudo-graph.

The pseudo-graph plots delivered functionality (cumulative) versus time for two imaginary stereotypical projects: one with good design and one with no design. The project that does no design expends no effort on design activities, whether they be up front design or agile techniques. Because there's no effort spent on these activities this project produces function faster initially.

The problem with no-design, is that by not putting effort into the design, the code base deteriorates and becomes harder to modify, which lowers the productivity, which is the gradient of the line. Good design keeps its productivity more constant so at some point (the design payoff line) it overtakes the cumulative functionality of the no-design project and will continue to do better.

I call this a hypothesis because it is a conjecture, there is no objective proof that this phenomenon actually occurs. In scientific terms it's not a very good hypothesis because it's hard to test. We CannotMeasureProductivity nor can we measure design quality.

But despite it being only a hypothesis, it's also an axiom for many people, including myself. We may not have objective proof that this effect occurs but many of us feel that this explains what we see we see qualitatively in the field. It's an axiom for me as it's the assumption that underpins my entire career as a writer about software design. If design doesn't actually improve productivity in some way, most of my writings are worthless.

I'm sure it sounds strange to many people to treat a hypothesis as an axiom, but it's a common thing to do. I look at it that I use my judgment to assess that the hypothesis is true, but can do so without ignoring the objective weakness of the hypothesis. I'd love to find a way to to prove it and almost as much to refute it.

The hypothesis has a corollary, which comes from the the design payoff line. If the functionality for your initial release is below the design payoff line, then it may be worth trading off design quality for speed; but if it's above the line then the trade-off is illusory. When your delivery is above the design payoff line neglecting design always makes you ship later. In technical debt terms it's like taking out a loan but not using the principal for so long that by the time you use it you've paid out more in interest payments.

This raises the question of where that line is. Even with people who accept the design stamina hypothesis there is substantial, and important, differences over where the payoff line sits. I take the view that it's much lower than most people think: usually weeks not months. But again this can only be a judgment call.

This leads to a consequence for Technical Debt. Technical Debt is a fantastic analogy and I use it frequently. But the design payoff line reminds us that taking out a Technical Debt is only worth doing up to a certain point. Not just do we have to consider whether delivered value is greater than the interest payments, we also have to judge whether the delivery is above the payoff line in the first place.

DuplexBook writing 13 June 2007 Reactions

Last week I got the newest book in my signature series: xUnit Test Patterns by Gerard Meszaros. I've been working with Gerard with it on and off for a couple of years, so I'm fairly familiar with its contents, but somehow seeing the physical copy gave me quite a shock. Somehow it hadn't dawned on me how big the book was - 883 pages, easily the biggest book in my series.

On the whole I'm not keen on big books, I was very proud of keeping UML Distilled so small. A book this big scares me, how will I ever get time to read it?

But xUnit Test Patters isn't as scary as it looks, because it's actually two books in one set of covers. In this it follows a style I also used in P of EAA. The first book is a narrative book, designed to be read "cover to cover". The narrative book is something small enough to be digestible, in xUnit Test Patterns it's 181 pages, 106 in P of EAA. The second book is reference material, which is designed not be be read cover to cover (although some people do) but instead to be dipped into when needed. The idea is that you read the narrative book to get a good understanding of the field, then put it on your shelf so it's handy to grab when you need to delve into the reference section.

I read a lot of history books and I've often wished that the author had written a duplex book. History books often need detail to expand upon a particular topic or to describe the evidence for a point of view. The result, however can be a long book. One example is a favorite book of mine: Guns, Germs and Steel. I'm really glad I read it, and I do recommend it. But it did feel long, and I wonder how it would have worked in duplex.

How to Read a Book suggests that when you read a book for the first time you should deliberately skip-read it, unhesitatingly skipping over detail sections. I read very fast, which helps me, but although fast or skip reading is a plus, I'd rather the book was designed to help. My personal rule of thumb is that 200 pages is the limit for a narrative cover-to-cover technical book. If it goes over that you need some way to allow people to get the core information with less bulk. A duplex book isn't the only way to it, such as selected bolded paragraphs in the Timeless Way of Building, which worked pretty well for me. I'm sure there are other techniques that I haven't run into yet, but the duplex is currently top of my list.

A duplex book is really a specific case of a more general principle to organize a book in gradually increasing sections. The Pick Axe is a good example of this. The first two chapters are a quick overview of ruby in 24 pages. Then you have 280 pages of tutorial, followed by 500 odd pages of reference material. The first few chapters of the Enterprise Integration Patterns book is an overview (95 pages) with rest (~550 pages) reference. Books like this often don't make these successive layers of revelation as clear as they might, however.

An interesting question is whether we can do more with the duplex book format. A lot people won't realize that xUnit Test Patterns is a duplex book because it looks like one thick book (the back cover actually claims it's three books by considering the reference section as two separate sections). Could we physically split the books, perhaps by packaging them as two volumes in a slip case? I don't think actually selling the books separately would make sense as they are too closely coupled together.

Naturally we think about online access. Reference material often works better on the web, so perhaps we could make the reference book web only (maybe with a CD or something in the physical book) and sell only the narrative book. What would that structure do to sales?

There's interesting questions here, but the final message is that I'd urge authors of big books to think more about how the book is structured in a way that helps someone who wants something within that 200 page limit.

UpcomingTalks writing 6 September 2007 Reactions

Updates: I'll be in Europe for the two weeks before JAOO and will be around a few conferency like events.

I'll be giving a couple of talks with my regular co-talker Dan North. Both are part of ThoughtWorks's technology briefing series. Our topic this time is "selling agile to your organization". As I detest selling anything to anyone it will be interesting to see how this talk works out.

I'll also be attending RailsConf Europe. I'm not speaking (thank goodness) but I will be around.

After I return to the US I shall be at the ALT.NET conference. As indicated in a previous post, I'm a keen advocate of a broader based .NET community. Lots of people at this conference are people whose blogs I regularly read, so it will be fun to meet them face to face.

I shall be taking my regular trip out to JAOO again. As regular readers know, this has become my favorite conference and about the only one that I make a regular effort to get to. I'll be doing a tutorial on Test Driven Development with my colleague Erik Doernenberg. We did this one together at Øredev last year and would have done it at QCon London this year had I not gone down with lurgy before my flight out. I'll also be taking part the closing panel.

In October I'll be back at OOPSLA - another long running favorite conference. I'll be on a panel "No Silver Bullet Reloaded", which looks back at twenty years since Fred Brooks's famous paper. The panel line up includes The Man himself, which is rather intimidating, although nowhere near as intimidating as being on the same panel as 'Bedarra' Dave Thomas.

JAOO and InfoQ are collaborating on another QCon conference, this time in San Francisco. For this show I'm intending to do my first shot at a half-day tutorial on the material I've been working on for Domain Specific Languages. I'll be ably assisted by Neal Ford.

One thing I'm hoping won't repeat is a further fare-up of my back problems. I've gone twice thought the cycle of seeing the back problem go and come back with a vengeance (the last time wrecking my vacation). So I'm being cautious about making too many promises.