A Tale of Two Knowledges

It was the best of knowledge, it was the ...

(Like this article? Read more Wednesday Wisdom!)

One of the things that really amazes me is the sheer amount of people in the world today with great knowledge about computers.

When I was a little Jos, toying with my first computer (a Sinclair ZX-80 if you must know, with one whole kilobyte of RAM to hold the system variables, the BASIC program and the screen buffer) there was not a lot of knowledge to be had anywhere. There were a few English (and thus hard to get) hobby magazines and the best book store in Rotterdam had a small section in the back with a few expensive books on computers and computer science (most of which were academic and way out of my league). One book in particular fascinated me. It was pinkish, had a large dragon on the cover, and was on the topic of compiler construction. I knew nothing about anything but I was fascinated by the topic and hence the book.

With my limited amount of cash I invested in these and other books. In the meantime I had upgraded to a BBC Microcomputer and my two favorite books for that computer where the "Advanced User Guide", which contained in-depth description of the hardware and the operating system, and the "BBC Micro Compendium" which contained, among other things, a complete commented listing of the BBC’s BASIC ROM (8 Kb of tightly packed machine code).

I was hungry for knowledge and devoured any and all information I could get my hands on.

Because of this incessant reading and experimenting I was in no time the most knowledgeable guy in my county when it came to computers. At least, that's what I like to think, because I knew nobody who knew as much about computers as I did and neither did anyone I knew (though this was partly because my class mate Alex de V. moved to England, clearing the playing field).

At that time Computer Science was a fledgling field of study and there were only a handful of colleges offering programs. When I enrolled into one of these programs I got in touch with more people that knew something about computers, but only a few of them had the extensive background that I had, also obtained by misspending most of their childhood toying with computers.

In the meantime, computers were starting to appear on the desks of people in offices. There was however an incredible shortage of people who knew something, anything, about these computers. One of the effects of this was that IT people were in high demand. Now remember, the eighties were a time of crisis, with lots of lay-offs, which unnerved everyone.

In the eighties in Holland, people were actually discussing how to give a meaningful existence to the many people who, it was thought, would never ever have a job. Things were that bad.

I didn’t face unemployment though. Instead there were career fairs for IT people where employers were trying to bag scarce talent. Even baby-faced 22 year olds like me were treated like, if not like royalty, at least like gentry. I was happy as a clam at high tide to be in a field that was in a high demand and to be very knowledgeable in that field.

For quite a long time I thought that knowing technical things that other people didn't know was the essence of my economic value.

For my first job I joined a group at a large Dutch bank that administered the IBM mainframes that ran the core banking software. I had no clue what the job was actually about, but apparently as long as we kept the computers running that was fine by them. There were no OKRs, no SLOs, no KPIs, and no plans. We were the magicians that could control the big calculators in the air-conditioned buildings and every morning we got in and basically did whatever we thought was necessary. Such was the power of knowledge.

Because I thought that knowledge was the key to success, I kept amassing knowledge. I read books, took every opportunity to attend courses, read manuals, and experimented with lots of different things.

Throughout this stage of my career, knowing things was incredibly helpful to me. For one thing it allowed me to be very effective in my job. Having deep and wide knowledge enabled me to work on widely different projects for very different customers. One of my customers was the services organization of a large American computer vendor. They used external consultants like me when their (small) internal pool of people was exhausted. The manager of that group maintained a list of these external consultants which included their areas of expertise. The entries for the company that my friend Erik and I had started said: "Can do everything". It was a matter of considerable pride for us.

One of the things we did a lot of in the nineties and early 2000s was teaching courses. Because knowledge was our trademark, we had some intense discussions on whether that was a smart thing to do. Weren't we giving away our precious knowledge too cheaply? Weren't we just training our future competitors? Shouldn't we keep our hard-earned knowledge secret?

Pretty soon it dawned upon us that there is no secret knowledge anymore. By then the Internet was there and even in its infancy it contained everything there was to know about most topics. Apparently our value was no longer that we knew things; it was what we could do with that knowledge.

Of course this was always our value, we just hadn't realized it...

In the training discussion we came down on the side of continuing teaching courses because a) it was good money for little effort and b) we thought that if we would ever have to compete with one of our former students we could always tell the prospective customer that we had taught them everything they knew :-)

Sheer width and depth of technical knowledge helped me pass the Google interview in 2006. That really was a bit of a miracle because I was completely unprepared for these interviews. Of course this was very early days for professional interviewing and there wasn't a lot of information available, but at the same time I was seriously underprepared for their style of interviewing and the things they were going to ask.

I am pretty certain that the only reason I got hired was a quip I made during one of the interviews. An interviewer asked me the "Two Robots on a Plane'' question. I came up with a solution that would require an infinite amount of memory; when that was pointed out to me I said: "Well, then I will first invent infinite memory, which I can do because I have infinite time 🙂." The interviewer cracked up and admitted that he had never heard that particular response before…

At Google I was in a completely new technical universe and hence there were completely new things to learn: Distributed systems, load balancing, observability, containerization, global file systems. All yummy, all sweet.

One of the things that changed quite significantly when I was at Google is that I was no longer the only person in the room who knew a lot about computers. Quite the opposite actually, because Google at that time only hired absolute geniuses (and me, for some reason). Pretty quickly I went from the smartest person in the room to the dumbest person in the building. Strangely enough I was okay with that, though for quite a lot of people this was really hard to deal with.

During one interview with a fairly young candidate from a small South American country I asked a Linux warm-up question of the nature "How do you remove a file called '-f'". When this candidate answered the question correctly he looked at me triumphantly and said: “Did you ever meet someone who at 26 years old knew as much about Linux as I do?” My answer: Yes: Everyone in this building…”

(Another reason this candidate was legen… wait for it …dary, was that he tried to pick up the receptionist on his way out of the building after his interviews :-)

So I moved from a situation where it was no longer what I knew that defined my value, but how I could use that knowledge to get things done together with other people.

Yes, all of us here can write a C++ compiler and I know some of you are working on that right now, but can we please agree on how we are going to use the compiler we already have to solve this problem over here?

Of course getting things done with other people is a knowledge area of its own, so my strategy for improving continued to work: Buy books, read them, think, try to apply that knowledge, rinse and repeat. But now the books no longer had cute dragons on them and neither were they filled with lists of system calls or explanations on how to turn a grammar into a DFA using an LALR(1) parser generator. Instead these were books on emotional intelligence, conflict resolution, giving effective presentations, influencing without authority, managing your energy, and related topics.

This was a very timely change for me because at the same time I noticed that the candidates I was interviewing were getting smarter. Not just a little bit smarter, but a lot smarter. Really most L3 new grad candidates today could easily out-interview the 2006 me (with 24 years of experience at that time).

To be honest, I probably still know more about Unix than most candidates even today, but that’s most likely because that is fast becoming obsolete knowledge that only a few hacks will ever have to worry about.

The reason for that enormous knowledge leap is that the industry has made an equally enormous tech jump in the last 20 years and by now pretty much everybody has caught on. By 2010 I was among a happy few that knew how to develop distributed systems with NoSQL database systems, Map Reduce, distributed lock managers, and similar technologies. By now, interns know this stuff.

A lot of the knowledge that I used to have and pride myself on has become outdated or has become commodity. And for these commodities there is now a large number of people who are writing blogs, developing open source code, producing books, and answering questions on Stack Overflow. And the rate is still increasing! Look at AI and machine learning: Only ten years ago that was a rare and dark magic, but right now people with Excel and a data science micro-degree from an online program can whip up and run a model in a few hours.

Over the years I found that only the rarest of use cases required the most complex and intricate elements of my knowledge. Yes, I know how a mark & sweep garbage collector actually works, how OS/2 assigns drive letters, and how Linux sorts the list of outstanding I/O requests on a block device for optimal traversal of magnetic disks, but guess how often I have actually used that knowledge? And guess how often I have used the core rules I learned from "Influencing without Authority". Quick guess: More or less often? :-)

As you get more senior the second type of knowledge becomes more important and more valuable.

It's definitely good to keep your tech knowledge up to date, especially about big new trends and game changing technologies, but don't fret about whether you are still the most knowledgeable person in the room when it comes down to the nitty gritty. More often than not it doesn't matter, and if it does, I am certain an hour of study is all it takes to get you up to speed about whatever is going on.

Hands-on tech knowledge matters, but as your career progresses and your environment changes, other types of knowledge start mattering more. Don't ignore that second kind and don't worry that the kids are getting smarter every year.

Remember: In high school you were taught math and physics that would have mystified the certified geniuses of not too long ago. Why would it be different in our field?