Did you know that Wednesday Wisdom is also a podcast! Find it on Apple Podcasts or on Spotify. Did you know that there is also a custom GPT called Midweek Muse that has access to all of Wednesday Wisdom? Did you know that all Wednesday Wisdom videos are also available on YouTube?
When I was nineteen, my father gave me his Honda CB 750 motorcycle to get to college. It came with an owner’s manual, which was unfortunately quite light on Zen and the art of motorcycle maintenance, so I got myself an additional manual that described how to do most maintenance tasks yourself. Chapter 8, on the topic of brakes, started as follows: “Now is the time to be honest with yourself about how good a mechanic you really are. Are you the kind of mechanic that usually gets everything right on the first try? Or are you the kind of mechanic that regularly makes mistakes?” I had to admit to myself that I am the latter and so I left brake maintenance to the shop, which is quite possibly why I am still alive today.
Fast forward twenty-odd years. I was in a project at a bank that was developing applications for large international customers. These applications had to support fine-grained authorization so that the treasurers of our customers could define the roles and responsibilities of their colleagues. The design of the user administration module was in the hands of a genial and brilliant colleague who created a data model that was so complicated that few people understood it, let alone could implement or query it correctly. I wrote an internal email called “The intergalactic data model” that explained why this design was not the right one for us. Yes, it could do everything any customer could ever ask for, but I could not see us implement, query, maintain, and troubleshoot the system correctly because of the sheer complexity of the model and the overall knowledge and skills of our software engineers.
I was mostly ignored, but for a few years after sending that email, I would regularly receive requests to re-send a copy of it, probably to support some internal argument. The model was implemented, but the equally genial and brilliant tech leads of the team I worked with, wrote a cron job that denormalized the information in the user administration into a data model that was much easier to understand (both for developers and for Oracle; critical for performance) and query.
Years before this episode, I was teaching MC/ServiceGuard courses at Hewlett-Packard (HP). MC/ServiceGuard was a product that enabled you to run services with “high” availability on a cluster of machines. When a machine failed, the services running on that machine would move to another node in the cluster, taking its disks and IP addresses with it. This might seem trivial today, but believe me, in the 1990s this was a highly advanced use case that required specialized hard- and software support.
Running an MC/ServiceGuard cluster was not for the timid, as it required a lot of knowledge of, and experience with, HP-UX, the HP Logical Volume Manager, TCP/IP, and SCSI. My colleagues and I used to joke that, given the expertise of the average HP-UX system administrator, a standalone machine was probably more reliable than an MC/ServiceGuard cluster, as a single mistake could easily bring down the entire cluster or maybe erase a mirrored logical volume.
During one course, I was joined by a few system administrators from the Rotterdam public transport system, who were moving from their proprietary HP3000 (with the MPE/iX operating system) to “open” systems (meaning: HP9000s and HP-UX).
Fun fact: The later models of the HP3000 and HP9000 were exactly the same hardware architecture (PA-RISC based), but HP3000s were way more expensive, because HP3000 customers had nowhere else to go and HP9000 customers could relatively easily move to other Unix systems, for instance ones offered by Sun, Digital, or IBM.
The students from Rotterdam were unique, because they were attending the course before the decision was made to invest in an MC/ServiceGuard cluster (most students came after the deal had been done, to learn how to use the stuff they had already bought). At the end of the five day course, I was talking to the team lead and he said that, based on what they experienced in the course, they would not be moving forward with MC/ServiceGuard, as they had figured out that they did not have enough basic HP-UX knowledge to manage their cluster reliably. Great self-knowledge, leading to a great decision.
When doing something, anything, you have to be aware of who you are and what that means for what you are doing. I am not a great mechanic, so I should not fix the brakes of my motorcycle. The team I was in at the bank did not consist of great software engineers, so we should not develop an application that was based on a hyper-complicated data model. The system administrators of the Rotterdam public transport system were new to HP-UX, so they should not install and run an MC/ServiceGuard cluster. Technical decisions must be grounded in a correct assessment of what the people responsible for execution are capable of.
It’s not just about knowledge and skills, knowing your company culture matters just as much.
While at Google Flights, I worked on a system for processing airline data and building the indexes that were then pushed to the flight search engine. For many reasons, all outside the scope of this article, this is a complex problem and the solution we had was starting to show its age. I set off to design a new and complicated parallel processing engine that would significantly speed up building these indexes using the latest and greatest Google technology available at that time. This project took over a year to complete its first deliverable, which given Google’s internal culture (which I once described as being the “Germany of software engineering”) was not out of the ordinary. We wrote a large and complicated piece of software, but Google’s culture was receptive to that approach, so no problem at all.
To be honest, there were regular questions from managers about how much time this would still take, but at the end of the day, they allowed us to continue and eventually the project was finished and went into production.
Compare this with the following experience: I once worked for a startup with incredible velocity and a culture that valued shipping above anything else. Through company shenanigans outside of my control, I was asked to join a team that worked on a very fundamental and very secure solution for the problem of machine and workload identity. This is a complex problem and, much like the Google flights “fareload” building system, we set out to design and build a complicated solution that would fundamentally solve this problem. This project also took a long time; much longer in fact than our startup had an appetite for. We designed and built a great system, but it took a long time and before we could finish it and roll it out, the organization had lost focus, executives had moved on, and our project was disbanded. Great solution, beautiful design, but all things considered, we were not the kind of organization that could build and run something like this. Not for lack of technical chops, we had oodles of that, but for cultural reasons.
You cannot build something that takes longer than the attention span of the company.
I did put that experience to good use by counseling new colleagues on what to work on and how to deliver: No big-ass projects that would take months to complete, but smaller scoped targeted solutions that could be released iteratively and show value along the way.
The adage “Know yourself” even applies at a national level. As you may or may not know, I am an active member of a Dutch political party. Within the party, we have a committee on digital affairs, which is currently mostly concerning itself with AI and digital autonomy. Within our Signal group, I regularly read far-reaching proposals on how to fix our government’s dependency on American tech companies. Most of these proposals could only work in a hierarchical “command and control” organization where the executives declare a new plan and the lower reaches of the organization faithfully execute it. But that is not who we are, and thus these proposals are meaningless.
The Netherlands is a negotiation-based society: Kids negotiate with their parents about bedtime and where they will go on vacation this year, students negotiate with teachers about grades, the police negotiate with the (equivalent of the) district attorneys which laws they will enforce. The Dutch form a large, complicated, and pluralistic society where nobody is in charge and everybody negotiates with everybody about everything all the time. Our prime minister is not even the leader of the cabinet, but a “primus inter pares” instead (the first among equals).
In a country like that, you cannot get the government off of Microsoft Outlook by making a decision and ruthlessly executing it from the top (hint: There is no top). You will need to create an interdepartmental platform, propose a change, seek consensus, expect resistance, do research and a pilot, expand the pilot, and generally talk until the cows come home. It is a slow process, which annoys the hell out of the nerds who think that the clock has already struck midnight on our digital dependence, and who think that big things need to be done now!
As a comparison: The reform of the Dutch pension system that went into law in 2023 and that will take effect in 2026-2027 was the result of over fifteen years of discussion between successive governments, unions of employees, unions of employers, pension funds, and regulators. That’s how we do things…
Every plan and technical design you propose needs to match the skills of the people who are going to build it and the culture of the organization that is funding it. If you don’t understand Java very well, don’t use Websphere but maybe consider Apache Tomcat (not a random example). If you don’t understand HP-UX very well, don’t use MC/ServiceGuard, but instead buy a Stratus Continuum fault tolerant server. If you are a high-velocity startup, don’t propose a big-ass design that will take a year or more to build.
To say it with Charlie Munger: Knowing what you cannot do is more useful than being brilliant.
