It's the community, stupid!

Don't choose a language, choose an ecosystem...

(Like this article? Read more Wednesday Wisdom)

A few years ago I was working on a team that was maintaining a large critical service written in Python3. I love Python, but I don’t think it is an amazing language for developing large systems. We were plagued by a number of problems that are specific to Python and Pythonesque languages, most prominently performance problems (due to Python’s essentially single-threaded nature) and low developer velocity because of typing issues which made refactoring a piece of code a right regular hassle.

There is a well-known story among old-time Googlers that when Guido van Rossum joined Google, he had a series of conversations with Googlers about what they thought should be added to Python. When asked, Mike Burrows, one of the principal authors of Google’s Chubby lock service, answered: “Strong, mandatory, typing.” 🙂

The team had used all the tricks of the trade (within reason) to deal with these problems, including extensive use of Python3’s system for asynchronous execution and through the equally extensive use of  Python3 solutions for type hints and type checks. Unfortunately, performance was still a problem and developer velocity kept on decreasing as these tools added complexity and are incomplete and imperfect.

It was time to consider rewriting the system in another language. But which one?

Language discussions are akin to religious discussions and usually about equally useful. To top it off, lots of seemingly rational discussions about languages are not in fact rational but instead rationalizations where people are reasoning backwards from the preferred solution to reasons why this is the right solution to begin with.

I am a language fanatic, but not an acolyte for a specific language…

I love English for its flexibility and capacity for puns. I love German for its structural beauty and capacity for specifying an exact meaning (it is not surprising that God speaks German whenever it’s heavy business).I love Swiss German for its melodic pronunciation and of course for being context-free (as proven by its use of cross-serial dependencies). And I love Dutch because it’s also context-free and it’s my native language so I can express myself effortlessly in it.

I am equally flexible when it comes to programming languages. With the exception of Javascript and PHP I typically find nice things to say about all languages.

Sorry but not sorry: If you need an === operator, you are effed up.

For instance I appreciate the expressive power and speed of C++, the no-nonsense get-things-done feature set of Go, the tight architecture of Forth, the recursive beauty of LISP, and the intricate way in which Java gets even extremely badly written code to work.

So back to the original problem: Which programming language to choose?

I often get asked what my favorite programming language is. My answer is invariably: The one that gets the job done. There are always some bad language choices for a given problem and mutatis mutandis there are also a number of very good ones. It is rare to come across a problem that really can only be solved well in a single language. And if that is the case it is often a language that was custom developed for that specific problem. 

Whenever making a choice for a programming language I am typically guided by an insight that was amazingly well worded by my friend and former colleague Toni Timonen: You don’t choose a language, you choose a community.

In the olden days writing software was a battle between you and the machine, usually with the help of a BASIC interpreter, maybe a Pascal compiler, and sometimes an assembler. That is no longer the case. Whenever I write software, I write some thousands of lines of code and then combine them with millions of other lines of code to create a system. In order to write working software I no longer need just a compiler and an editor. I need an IDE (preferable based on vim), libraries, a testing framework, a protocol buffer compiler, Thrift bindings, a load testing tool, integration with the continuous build and deployment system, a profiler, static analysis tools, a configuration system, code generators, a debugger, and what have you.

I am quite productive, but there is obviously no way that I am going to cobble all of that together on my own. Instead, I depend on a large community of engineers using the same language in the same context, and each of us builds a little piece of this entire edifice in order to make the language usable for all. That’s the thing that I choose. I don’t choose the language, I choose the community!

For the system that I started this article with we quite quickly settled on Rust, even though all other things being equal I thought that Go provided the better tradeoffs in complexity, speed of development, ease of use, and power. However Go was not in widespread use at the company and hence there were all sorts of limitations like an absence of Go client libraries for infrastructural services and limited support for Go in the entire development ecosystem. Rust on the other hand was supported by a very active internal working group which were busy wrapping C++ libraries, providing Thrift bindings, integrating the Rust compiler with the build system, and all sorts of other goodness.

We didn’t choose just a language. We chose an entire ecosystem. The community made up for what I deemed to be the problems of Rust: An unstable language with a high rate of change and an incredibly steep learning curve.

To paraphrase former President Bill Clinton: It’s the community, stupid!