September 16, 2016

Inflection points in incremental design

Hello, developer friend!

I hope you're having a great weekend. Before I start my own, I wanted to share a little thought experiment that you might find interesting.

I've always been a huge fan of working in tiny increments when it comes to software development, and really, with pretty much everything. I'm a terrible top-down planner, and so I guess over the years I've compensated by being halfway decent at improvisation.

The number one question I end up getting from people who aren't used to doing incremental design is "how do you know where you'll end up?" -- and the answer to that is one that often disappoints: I don't know where I'll end up, whenever I build anything. And even for me that's a little scary, because it means that I can't predict what will happen along the path ahead of me.

But that question on its own is a sign of a bit of a muddled concept of what incremental design really is. Incremental design is about aligning with a purpose rather than a plan. As long as each incremental step in a project gets it closer to serving its purpose, it doesn't matter if you end up in a totally different place than you might have imagined at the outset. In fact, in nearly every incremental design, if you end up exactly where you expected, it's a sign you have not been trying out enough variations along the way.

When people get past the "does the finished product match the original vision?" question, the next thing that comes up is what happens if you hit a dead end?.

And to be fair, it can happen. There are some situations in incremental design in which you try out an idea and then you realize that it was the complete wrong plan -- and at that point you have to choice but to eat the cost and go back to the drawing board. Top-down planning is one way to mitigate (if not totally eliminate) this sort of risk, and so this is often perceived as a weakness of incremental design by those who prefer to know where they're heading before they get in the boat.

But there is a critical thing that many top-down planners miss about incremental design, and that is how to leverage inflection points. In mathematics, an inflection point is an area where there is a sign change on a graph... in the context of design it's a point where you can sort of twist your position within a problem domain to approach things from a different angle.

To work with inflection points in a system, you take the parts that are in common between multiple different (possibly even conflicting needs) and you group those things together to form a bounded context. In doing this, you connect together the shared parts that are likely to change at the same time or not change at all, while separating the parts out that would otherwise be in tension with one another into their own distinct areas of a model.

Whenever you feel like you've hit a dead end in an incremental design, it's usually a sign that you should be looking for a bounded context to extract and formalize (if not several of them) rather than throwing everything out and starting from scratch. This lets you bust up parts that are struggling to do too many things at once, and can clean up connection points so that a change that initially seemed impossible or overwhelming can become small. This is the easy part, spotting the opportunities to apply the pattern is more challenging.

To get better at making use of inflection points, it helps to spend some time doing design as a form of pure play. I've always been a fan of the game Nomic for this reason, which is played by incrementally modifying the rules of the game itself. A single newly introduced rule can completely change the meaning of all the previously defined rules in the game, if you design it cleverly enough.

Another game I've enjoyed playing with a musician friend of mine is to start with some particular song, and then incrementally evolve a playlist that flows naturally but ends up nowhere near where it started. He is much better than I am at this, and even at one point managed to put together a massive (120+ track) playlist that connected a long-arc history of Jazz, spanning an entire century of music. When you listened through the whole thing, it played seamlessly even though nearly every detail about the style of music had changed along the way, and even though he had explored several different styles even within a single time period.

I'm not as good at that stuff as my friend is (I don't have any formal music background), but I still dabble in it from time to time. For example, a few days ago I put together a playlist that begins with a pure vocal hymn that blends into singer-songwriter acoustic music, and then drags you through an incremental path of various love songs from different genres. But somewhere along the way, the music goes from English to French, and then French and Russian combined, then Russian only, then to pure instrumental Balkan brass band music.

By the time you get to that point, the focal point of the music is no longer love songs at all, and it's the polar opposite style of a hymn. I had created an inflection point by changing the languages while still staying on the "love song" theme, and then went along a new incremental path where the style gradually shifted towards brass bands. Once I got *there* I twisted things again from traditional brass style to a song from a Russian rock group which although technically played with brass instruments is a through and through modern rock song. That was my second inflection point, which allowed me to round out the rest of the playlist with songs in English that were a bizarre medley of experimental rock and antifolk music that *kind of* reflected the starting point but with a totally different feel.

In this particular playlist, it only took two degrees of freedom to connect several completely different worlds of music, the rest was all done through very straightforward incremental changes.

This is the kind of stuff that keeps me up at night, and it's what I like to do when I'm not coding or working on business projects. I'm sharing this story because I think that this kind of creative play can be really valuable for improving the way we think about our work, and so that it might inspire you to think about something weird and wonderful this weekend.

Hope you enjoyed the random brain dump, and have fun out there!
-greg