(Like this article? Read more Wednesday Wisdom! No time to read? No worries! This article is also available as a podcast). You can also ask your questions to our specially trained GPT!)
I love getting reader feedback because without it, I feel like I am shouting into an unbounded void, never even hearing the echoes. Reader feedback gives me the warm and fuzzy feeling that someone is actually reading my weekly ramblings, which is very satisfying indeed. Sometimes, the readers who contact me have questions and I love answering these. It is an incredible honor to be asked for advice, even though taking advice is a dangerous thing because, in essence, all advice is autobiographical and you are not me, I am not you, and we should both be happy about that.
Sometimes, the reader’s questions are so intricate that they become the topic of a Wednesday Wisdom article if and when I think that the answer would be of interest to more people than the audacious question asker. Recently, I received this question: “ One topic which I would be very interested in is how to grow from being a good engineer, successful in tackling relatively small/precise problems, to leading engineering challenges with a large scope. I personally feel comfortable with the former but feel more overwhelmed with the latter. I have the impression that some qualities of a good engineer at a tactical level such as “attention to details” might not be a quality for a broader task.”
Excellent question, and fortunately for the person asking the question, the only reason I amount to anything is that I already made most mistakes, which everyone can then learn from.
A year into my first job, our team grew to a somewhat unmanageable size, so we got split up into subteams with tech leads overseeing each of these subteams. Despite my very young age and lack of experience, I was asked to become one of these leads. This was an audacious move by our manager because, really, I had no business leading anything at that time, green as grass as I was. Fortunately for everyone involved, the responsibilities were very limited and I also managed to stop myself from behaving as if I were the Almighty’s gift to IBM mainframe systems programming. Because of this, things mostly worked out.
My next stint at leadership (conveniently ignoring “leading” a small company I co-founded) was at Google where I grew into the tech lead, and then later the manager, of an SRE team that was taking care of Google’s first foray into social networking: Orkut. Bit of a strange name for a social networking site really, but as we soon learned, that product was named after the product manager who initiated it.
Leadership tip #1: Don’t name things after yourself! It makes people laugh at you behind your back and no one will take you seriously. That’s why the operating system that I am designing is not called J/OS or something similarly inane.
How did I get ready to take on the TL ship of a team and then later the TLM role? The answer: Slowly…
Leadership tip #2: Take your time. There is more to life than speeding it up!
In my philosophy, leadership is a verb; it is something you do, not a title that is bestowed upon you. It sure can be and then maybe you are formally a leader, but what I think you should be aspiring to is becoming a leader in the material sense of the word.
A few years ago, during COVID, when we were all working from home, the luscious Mrs. Wednesday Wisdom was in the position to observe the meetings I was in, courtesy of the fact that I was working from our dining table which abutted the kitchen. After one of the first meetings she witnessed, she asked me: “Do these people report to you?” “Nope”, I answered: “I have zero reports.” “But you were telling them what to do”, she continued. “Yes, I did”, I said. “But why should they listen to you? You are not their manager!” My answer: “They listen to me because I am the guy with a clue.”
Leadership tip #3: To become a leader, get a clue.
When I am talking about “clue”, I mean that you need to combine your existing deep technical knowledge with a holistic vision about the solutions needed for badly specified or yet unrecognized engineering problems in your organization. As a more junior engineer, you typically have enough clue to code up a working solution for a well-defined smaller scale problem. The clue needed for that often boils down to knowing how to use the tools available to solve the problem at hand. For the bigger problems, for the yet unknown problems, for the problems that nobody realizes need to be solved, you need more, bigger, and better clue.
Very simple example: In my second year at Google I wrote a highly optimized HTTP server for serving static content for the product I was supporting. That is not a huge problem, but it does require mastery of a bunch of technology to do this right. This is something I would expect every junior engineer to be able to pull off. However, the “leadership” aspect of this particular project came not from the fact that I could write this server and roll it out. Instead, the leadership happened when I had identified the problem, namely high latency and considerable resource usage by the instances of the Java application server that we were using for serving static content. After bringing that problem to the attention of everyone, I then convinced people that this was a problem worth solving. It is a small example, but it does show all the steps: Identify a yet unknown/unowned problem, figure out it can be solved, convince people that it needs to be solved, and then solve it.
When I used to be in promotion discussions I often used the following simple definitions of the various job ladder levels that we had:
Level 3: Able to solve a well-defined small-medium task, but needs regular checkins to validate that they are still on the right track and not blocked.
Level 4: Able to solve a well-defined medium-large task without regular followup and knows (how) to escalate when blocked.
Level 5: Same as level 4, but never blocked 🙂
Level 6: Able to identify problems worth solving, to convince people that these problems are worth solving, to sketch out a design for the solution, and to lead a small team of engineers to realize the solution.
The kind of leadership that the reader whose question prompted this week’s article starts happening around level 6 in this scale.
Warning: Given the sad fact of level inflation, your mileage may vary. Much to my chagrin, one company I was working for started hiring newgrads as “Senior Software Engineer”. Most recently I heard that this company also stopped doing regular performance review cycles. On the other hand, they are being acquired by IBM, so maybe performance isn’t required anymore, as they can all look forward to their jobs being outsourced to India.
The ability to identify problems worth solving requires a holistic view of the things you are working on and their position in the overall system that you are supporting. There are always a million things wrong with whatever pieces of software you are working on, but is it worthwhile fixing them? To make that holistic assessment, you need to understand what is required, how well your current solutions meet these requirements, and if, all things considered, it is worth doing the work and if so, what work and how?
Getting that holistic overview requires getting a good understanding of the organization and systems around you. How is everything working together to serve the business? How is the business going to change? What is needed to support these changes? One of the things that I typically do in the companies I work for is attend as many Q&A sessions with company executives as I reasonably can, because that gives me information about where the company is moving to and what the challenges of tomorrow are. Leadership is the act of adding to the situation what it needs and to do that you need to know what the situation is and what the needs are.
Building this global view of your world not only takes time and effort, it also requires an interest in this. Investing that time and effort should be your personal choice. I worked with many engineers who were totally happy sitting in a corner, being told by other people what needed to be done, and then executing on that with a speed and technical excellence that was simply breathtaking. In my view, these people are the backbone of your engineering organization and when one company I worked for formulated an “up or out” strategy, I campaigned vigorously to make sure that these people would not be let go for showing no interest in becoming tech leads. You cannot have all generals and no soldiers; every healthy team needs a balance between people with a more strategic mandate who are interested in figuring out what to do at a higher level and people with a tactical or operational mission. Both are important, so don’t let anyone tell you that you should aspire to become a more strategic leader.
If you have that interest, you need to start building your knowledge and network of people: How do other systems work? Who works on what? Why were these things built the way they were built? What is going to change for them if and when the business changes? This requires a desire to “pull your head out of the bucket”, so as to speak and to look at the world from a higher level.
I am a bit of a fan of “adaptive leadership”, a framework for leadership that distinguishes between “technical problems” that can just be solved with a one-time solution and “adaptive challenges” that require significant changes, for instance in behaviors and beliefs. When teaching about adaptive leadership I often use the metaphor of the dance floor during a silent disco. When you are on the dance floor, right in the middle of it all, you see nothing but individuals moving in seemingly random patterns, without much rhyme or reason. But if you leave the dance floor and move up to the balcony so that you can oversee the entire floor, you start seeing the patterns and you can identify clusters of people that are tuned to the same channel.
Building a leadership muscle requires being able to differentiate between these technical and adaptive challenges, a willingness to delegate the technical challenges and the ability to build a platform to solve the adaptive challenges.
That last one is crucial. Engineering is a team sport and in order to be successful you need the help and support of others. This is a difficult process, especially for engineers who are taking their first steps in the wonderful world of technical leadership. In the leadership program that I cofacilitated, we started with a quote from one of our SVPs of engineering: “Engineering is easy, people are hard.” You might think learning Rust is hard, but believe me, compared to cat-herding a group of engineers to build anything that is more intricate than a “Hello, world!” program, Rust is a piece of cake.
Many engineers are of the mistaken opinion that it is the quality of their ideas that should convince other people to go along with them. They are wrong on two accounts. First of all, ideas are cheap; I have five ideas before breakfast every day. What matters is execution. On the whole, I prefer a mediocre idea brilliantly executed over a great idea badly executed.
Here I would like to ask you to ponder the following question: Is Kubernetes a bad idea that is extremely well executed, or is it a great idea that is very badly executed?
The second account on which these engineers are wrong is that your great idea that you are able to execute is only your entry ticket to the negotiation table where scarce people and resources are allocated. This negotiation is a game of give and take, and to get support from others, you will need to understand their problems and offer ways to help them reach their goals. As I wrote before in an article called “Parameters for success”, you can only be successful with help from others in the organization and why should they give you that help?
With all of this work ahead of you, you might be tempted to forget that your credibility mostly depends on being a good engineer in the first place. As a technical leader, I have gotten tremendous mileage from the fact that no one I ever talked to was in any doubt that I understood the technology and that I would be able to build myself whatever I proposed. This stands in stark contrast with many of the “architects” that I used to work with who couldn’t code themselves out of a wet paper bag.
Leadership tip #4: Do not lose your technical chops.
This experience led me to write an article (all the way back in 2004!) called “How to design and implement unmanageable and insecure systems”.
The ability to attend to technical detail is not, as the reader wondered, a drawback, but it is important that you learn when to pay attention to details and which details to pay attention to.
Obviously, to get there requires years of practice. But remember leadership tip #2? Take your time!
Getting back to the original question: How to grow from being a good engineer, successful in tackling relatively small/precise problems, to leading engineering challenges with a larger scope?
This is a muscle that can be trained. When you think you are ready, you should start by asking for a project that is badly defined and that requires agreement among multiple teams. This will force you to first understand the problem and then create a consensus around your problem definition. This requires talking to people and maybe writing a few documents that seek to represent the shared understanding.
Leadership tip #5: The ability to write well is a career amplifier. Another one is to be able to run a productive meeting (upcoming WW topic).
Drive this new project from its bad specification to an agreed upon solution. Or not; the successful outcome of taking on such an idea might also be a shared (and documented) understanding that this is something that can not or should not be solved.
In one of my more successful projects I researched whether it would be possible to replace a jungle of point-to-point fibre channel connections from machines to tape drives with a star topology and then using iSCSI to talk to the tape drives. Answer: No, that was not possible. I then wrote an extensive document that outlines why this was not possible. For years, people kept coming back to me for that document when the idea came round again.
So there it is, your pathway to becoming more senior and more of a technical leader. Of course, you don’t have to take my word for it. You could also read “The Staff Engineer’s Path” by Tanya Reilly, published by O’Reilly, so clearly an inside job. She might agree with me, I don’t know, because I must admit that I haven’t read the book. Not because of lack of interest mind you, but because of lack of time, as I am currently in two book clubs and racing to finish “Katabasis” by Rebecca Kuang as well as some other book, the title of which eludes me at the moment.
