Velocity is an attitude

Can I Haz Speed?

(Like this article? Read more Wednesday Wisdom!)

Here is something you rarely hear: "Wow, you people delivered this software project on time." If anything, news of delayed software projects abound and sometimes even makes the national news. In Holland, they tried to automate the internal administration of the court system and that turned out to be a terrible disaster that missed all delivery dates, went over budget, and was eventually canceled with nothing to show for it. 

The speed with which IT departments can deliver on projects is a worry for IT executives all over. It has become so common that it prompted a very cavalier attitude towards deadlines, as evidenced by many project managers just moving the deadline whenever they run into trouble, seemingly without repercussions.

I love deadlines, I love the whooshing noise they make as they go by.

In short: We have a problem with velocity.

When I speak about velocity I am not talking about speed: Velocity is speed with a direction; it's a vector. If you run around in circles very fast, you have lots of speed, but you are not going anywhere anytime soon. This is important to realize because our overall lack of velocity is not because people are not working hard enough. To the contrary, most people I know work very hard. It's not necessarily speed that is lacking.

Then what are we lacking, if not speed?

I think we have a problem with velocity because of six issues that go to the heart of what we do and how we do it:

1. The world is complex.

2. We are too ambitious.

3. Programming is hard.

4. It is hard to get agreement among a group of people.

5. We are risk averse.

6. We are bad at planning.

These six factors conspire to ensure that we talk a long time to make ambitious plans that we are then slow to execute on and ultimately deliver only with delays, if at all.

Low velocity is death by a thousand cuts. There is no silver bullet that will magically solve the issue by killing the problem at the core of it. This is because there is no problem at the core of it; the only thing we have is dozens of bigger and smaller problems that make everything difficult and slow.

As I wrote before, it is not true that there is no such thing as a silver bullet. In fact, you can buy one at Amazon. I did this, so that I can wave it around in meetings when someone says that there is no silver bullet: “Yes there is, I got one, here it is!”

The problems that slow us down boil down to all sorts of friction and inertia. Fortunately we know from physics how to overcome friction and inertia: By the constant input of energy. So that's what we need to do to make sure that we deliver anything on time: Relentlessly invest energy in countering the problems mentioned above:

1. Make things simpler.

2. Do not make the system more complicated than it needs to be.

3. Be modest in our goals.

4. Minimize the need for communication.

5. Consciously onboard manageable amounts of risk.

6. Plan well.

This will not make you the most popular person in the room, but as my dear old father used to say: “If I give you two quarters, you can phone all of your friends” (this was in the time of public pay phones). If you already have two friends you are good to go and you can afford to annoy some people 🙂.

There are always good reasons to go slower: The world is complex, so doing another round of analysis to get an even better handle on the requirements is always good, isn't it? And programming is hard, so investigating a new storage technology that promises to solve some problems we might have in the future is a worthwhile investment, right? And it's important that everyone is on the same page, so a weekly status meeting with everyone involved is good, isn’t it? And of course we need to meet our customer’s high expectations, right?

The answer is: It depends, and sometimes it is, but "no" is a good default answer.

At Google Zürich around 2010, the Facilities team had come up with the idea of naming all the corridors, with street signs and all. I conspired with the facilities manager to have our corridor named “No Way”. Whenever someone came in to ask for something, I would point them to the sign.

Let's start by looking at the stunning complexity of the world. Pretty much every business problem worth solving is a fractal of complexity with new details waiting to be uncovered in the next layer. The deeper you dive, the more of that complexity you uncover and the less obvious it is how it all should work. That complexity also means that for most business domains, it is impossible for a single person to oversee all of it. This leads to multiple people being involved in defining what needs to happen. This then leads to more communication (slow), the need for resolving conflicting statements (slower), and prioritization (difficult, and hence, slow).

Uncertainty about how things work also causes us to make things more complicated than they need to be. For example I regularly hear untrue statements pertaining to complicated things being "legally required." 

It is true that some laws make it hard to do some things, but my estimate is that less than half the things people say are “legally required” are actually mandated by law. It’s usually people parroting other people and/or being very risk averse.

Velocity demands that at some point we stop defining requirements, write down what we know, and implement that. Unfortunately, that means onboarding risk because we might build something that is not 100% correct. And because we are risk averse this creates a strong tendency to research more before committing to what we need to build. Input of energy can overcome this, but it requires you to put your foot down, which may be hard.

I wrote about (some of) the causes of risk averseness here, but I promise to write about it more in an upcoming episode.

Then, finally, once we figure out what we need to do, we can start programming.

But, before we get there: Inside that statement hides another cause of low velocity: Not parallelizing enough. It is often perfectly possible to start coding while we are still figuring out what we are going to do. Indeed, that is one of the big advantages of Agile methodologies! I see way too many teams using a waterfall method for getting to the requirements and then plan the coding as a series of six-week sprints! This is terrible! The whole point of Agile methodologies is that we use the short cycle of delivering something and then talking to the customer again, to figure out the right requirements in turbulent and changing environments. You might learn that some requirements are not required; you might find out that something doesn’t work the way you thought it did; or your might discover that you didn’t understand the customer all that well. All of these are velocity wins if you find them out early, which means that you have to start early.

If you don’t yet know what to do, you better start coding, because that forces you to confront the hard questions sooner.

Once we start programming, yet another cause of low velocity kicks in, which is that programming is hard. Hence, it takes a long time to do something new and interesting. Each piece of software is a bespoke work of art, a one-off, built to unique requirements that require unique and never before seen code. And because this is the first time we are building this application, we don’t quite know yet how to do it.

Remember the first time you had sex? That’s about the level of skill and expertise you have the first time you build a specific application.

Unfortunately, it is typically also the only time we are building this application, so we stumble through it all, make mistakes, and start refactoring even before the code is out of the door for the first time.

But what about tools? Tools make programming easier, don’t they?

Our tools don’t help much. That is to say, they don’t help much with writing complex business logic. Most of the tools we have today exist to make it easier to type in the source code (which was never a problem to begin with) or they address non-functional requirements. It is easier now than it ever was to write a secure, distributed, and scalable “Hello World” application, complete with APIs, SSL, unit testing, continuous integration testing, continuous deployment, automatic failover, observability, and alerting. However, we get no help in building correct business logic, other than maybe the most basic libraries.

And even these are hard to use. I recently debugged some weird runtime exception that was caused, basically, by Java’s libraries for manipulating dates, times, and durations being a complete mess.

Our industry is fortunately not alone in being slow in the building phase. In 1997 I bought a new house and the construction company managed to miss the deadline for completing it by so much that they forfeited a penalty. Here in the US, Boston's Big Dig ran over so spectacularly that it became a meme.

But it looks totally amazing now!

One of the problems that impacts all phases of software development is communication. I mentioned already that it is slow and difficult, but on top of that it suffers from the fact that the number of communication channels in a network grows O(n²). If you structure your project so that everyone needs to talk to everyone, all the time, about everything, you will do nothing but talk. The answer is obvious: Structure your project so that that is not necessary, for instance by clearly defining responsibilities and only having the leads of the various work streams talk to each other. Unfortunately, that runs into problems because people feel left out if they do not get to attend the meeting of the important people, and hence the overhead grows.

In the 90s, the then new CEO of the Dutch electronics giant Philips famously had chairs removed from the boardroom. When people came in and found they didn't have a place to sit, he said: “The message is clear: There are too many people here.”

The last thing that I want to touch upon is that we suck at planning. Really, if I am going to see one more plan that consists of a spreadsheet with a linear list of action items, I am going to explode. The weird thing is that we do know how to plan, we just don't do it. In 1987 I took an Operations Research class in college and there they explained (among other things) the Critical Path Method. The idea is simple: Your plan is a DAG of action items, and each item has a planned duration range (low/high) and a list of actions that it depends on. If you know the start date of the project, you can then calculate the earliest and latest start of each action item as well the earliest and latest end of the project. The critical path is that path through the DAG that visits the action items that, if they are delayed, delay the entire project.

All of this made perfect sense to me and I have never seen it be used in practice, even though professional project management software typically supports it.

Knowledge of the critical path is useful because it helps you respond correctly to delays by reassigning people to action items that are on the critical path and at risk of delay. My whole work method is driven by the critical path: I try never to be on the critical path, and if I am, the first thing I do is get out of it by a spurt of activity on the actions that hinge on me.

If you don’t employ the critical path method, you won’t recognize the impact of delays in individual action items until they percolate down the graph to a point where it is harder to address and leads to delay of the entire project..

Low velocity is death by a thousand cuts and the only way to address it is by the persistent input of energy to overcome the myriad of in itself plausible reasons to slow down. Doing that work won’t make you very popular because you will kick people out of meetings, tell teams not to use some unproven (but cool) technology, force the start of building even though the requirements are not 105% clear yet, and generally be seen as a single minded curmudgeon that is always willing to cut corners to meet the deadline.

All in all, high velocity is an attitude, and maybe pushing for it comes more naturally to people who have an attitude already :-)