Noblesse oblige

Why we need to be nice to non-techies…

(Like this article? Read more Wednesday Wisdom!)

Recently there was a bit of an uproar on LinkedIn, apparently the latest and greatest vector of outrage on the Internet, about a recent report by McKinsey (compulsory book link) on how to measure programmer productivity. I don't have a real opinion about their "method" because I don’t know enough about it, but I suspect it to be dumb and naive. I do understand the need for a mechanism for measuring programmer productivity though. However, the difference between McKinsey and me is that I admit that I don't know how to do it.

While discussing promotions we are regularly faced with the question how productive a software engineer is. Literally every time I am in a promotion committee I find it incredibly hard to opine on that topic using the “evidence” that I get (the “packet”). The only way for me to have a good view on someone’s productivity is if I have worked with them directly for an extended period of time. Getting a good insight into a software engineer’s productivity using written collateral such as is represented in promotion documentation is just not enough, not even if you understand in-depth what it is software engineers actually do.

Aside: We probably don't have to discuss measuring programmer productivity by "Lines of Code" here, but let's recognize that this is an awfully seductive metric because it's a number and it’s somewhat easy to determine. In many environment it is also one of the few clear metrics they have…

Bill Gates famously said that measuring programmer productivity by lines of code is the same as measuring the progress of building an aircraft by its weight.

I have been coming across the root cause of the "programmer productivity" problem all my life: Software engineers differ tremendously in quality and an excellent engineer is not only more productive than an average engineer, they are exponentially more productive! For non-software engineers it is impossible to figure this out because they don’t understand what software engineers actually do…

Last week I was at an offsite and as luck would have it I was sitting next to our executive admin. Since what was going on at the front of the room was not terribly interesting (for her or for me), I was hacking at a Java client library to extract some data from AWS Athena. She looked at my screen and asked me what I was doing. "I'm coding", I said. "Oh, is that what that looks like?", she replied :-)

For a non-techie, hard- and software are black magic and it is impossible to guess how difficult or easy something is. When I look at a shed and then look at a skyscraper, I get a good idea about the differences in scale and complexity, and I can guess correctly that one is much harder to build than the other. These visual cues are absent in our virtual world. No matter how simple or complicated something is, it all manifests on a small screen and sometimes a report with a single number on it is harder to produce than a report with a million lines of data.

Years ago I was working for a well-known video sharing web site. One fine day during an on-call week, I was talking to one of our directors and I told him I needed to run off in a minute to launch our website in Turkey. He asked me what I would have to do. I told him that it was really simple: I would edit a file with changes to our DNS records to map "www.website.tr" to the IP addresses of our Internet gateways, merge that file into our configuration repository, and we were launched.

Of course the process of launching in Turkey is much more difficult; over the course of the project we translated the site in Turkish, created auto-curated channels with content targeted at Turkey, got Turkish advertisers on board, obtained all necessary legal approvals, trained our AIs to generate subtitles in Turkish, and whatnot. This was the work of dozens of people who had taken months to get all of that done. None of these people had any clue how their work would eventually make it to the Internet. All of their work culminated in me editing a single file to enable "www.website.tr" and we were live. As a gimmick, I took a video of me pressing the enter button on the merge command, which was the culmination of months of their work, and sent it to the entire project team.

As problem statements go, “Launch the site in Turkey” is a one liner on par with "Build a moon base." Most of us will probably have some clue that building a moon base is hard, whereas, without deep insight into the technical, commercial, and legal domains, it is impossible to estimate how hard "Launch the site in Turkey" is.

I recently came across a super funny video where an unsuspecting product manager asks about the progress of displaying the user's date of birth on the settings page. An exasperated software engineer then launches into an explanation of how hard this is in their architecture, which consists of umpteen micro-services, a complex authentication solution, and various (non-authoritative) data stores. I felt right at home, since this is the kind of talk I have to give whenever someone asks me something similar which seems easy. I mean, really, how hard can it be to get a user’s date of birth?

Answer: Sometimes that is really hard, and it is equally hard to explain why.

To make matters worse, we regularly come up with crazy stuff which makes it even harder in the future to do simple things. This then makes it even harder to explain why something is difficult and takes a lot of time.

My darling wife, who works at a bank, once asked me if it is possible to store an 11-digit account number in a database field that is 9 characters wide. I first investigated where the 9-digit field size limit came from but apparently that was a law of nature and could not be changed. Then I asked where the 11-digit account numbers came from and she answered that they were merging two systems with different account number lengths. Then I asked if there was some sort of pattern to the account numbers that we could use to devise some crafty encoding scheme, but apparently there wasn't. Then I asked what the data type of the column was and, after some research, she said: I don't know what that means, but my tech contact told me it it something called "var char nine". 

So I explained number base 36 to her. She looked at me and said: "This is brilliant, but I cannot explain this to any of my colleagues." I have no idea what they ended up doing, but I totally understand that these are the kind of problems that make it impossible to guess for a non-software engineer how easy or hard something is, how much work something is going to be, and if a programmer is productive at all or not while doing it. Depending on your architecture a simple database change (like encoding your account numbers in base 36 to make it fit a database field size limit) might need months of hunting through the code to find all places where this field is interpreted or set. It’s often impossible to explain to the layperson why this is the case. 

It also works the other way round: I am regularly in discussions where something that I know to be very simple (like sending an email) is treated like a difficult problem that needs a design and a plan and a team. It has led to a situation where many project managers automatically make every customer request a big deal because most things are complicated and take a lot of time and even the things that seem simple are often more difficult than was initially imagined.

All of this creates an obligation on us to help the non-techies in our projects as much as possible with understanding what is actually going on. That requires excellent communication skills, which is an incredible force multiplier for a senior engineer.

People are often saying things like: “Are you really an engineer? You communicate so well!” My darling wife, who disagrees on the topic of my communication skills, expressed her general feelings about engineers when she described one of her colleagues as “a tech person, well, not really a tech person, because she is normal…”

When in a project I never assume that any of the non-tech people know what is going on when it comes to the complexity of our system and why something is easy or hard and So I usually offer my help explaining things or taking care of chores that require technical understanding. We have weaved such an intricate web of fractal complexity that, as the high priests of the church of complex IT solutions, it behooves us to show empathy to the non-initiated.

It's not just that non-techies often have a hard time understanding how the system works, even my technical colleagues are sometimes suffering from this. Recently I saw a cry for help from one of our technical account managers who wanted to write some Go but who had gotten lost in our internal build system. He obviously knew Go very well, but had failed to make sense of the hyper-complicated tools we built to meet the extreme demands of our environment. I scheduled a video chat and walked him through the basic setup, which, hidden among other things, requires you to move your code directories to a volume on your MacOS laptop that supports case sensitive file names because, of course, the build system requires this. That is so counter-intuitive that even I fell into that trap when I started using the system.

"Noblesse oblige", the English say (at least the do if they took A-levels in French). It refers to the idea that people with noble ancestry or high social status have a moral obligation to act honorably and generously towards others. As the wizards of high tech, that obligation falls onto us. And since we're throwing foreign sayings around, let me end by referring to the Dutch proverb "Wie goed doet, goed ontmoet": If you do good things, you'll encounter good things. 

Give it a try!