Rust should be your next language...

Be a fool with many tools…

(Like this article? Read more Wednesday Wisdom!)

I regularly run into people who are devout followers of a single programming language. The underlying thought seems to be that the entirety of the world’s problems can be optimally modeled and solved with that particular programming language and really there is no need for any other one. 

These people are, of course, sadly mistaken.

There are many reasons why there is room for different programming languages. That doesn’t mean that all programming languages make sense. Personally I wonder why Netscape invented Javascript and I honestly don’t get the point of Ruby in a world that already has Python. But many programming languages, even newer ones, make some sort of sense to me.

My interest in programming languages probably comes from the fact that I grew up programming in sad BASIC dialects that were all clearly suboptimal for anything but the toy problems at hand. Edsger Dijkstra famously said that “It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration.” Personally I think that Dijkstra was not entirely on point here and that he mangled two problems into one: It is a) very hard to teach good programming to students (full stop) and b) many of the students that he tried to teach had some exposure to BASIC. A lot of high school students today seem to learn some Java (God knows why) and were Dijkstra alive today, I am quite certain he would have made a similar remark about this programming language.

Unless of course I got to him first to explain the error in his thinking. By the way, I don’t hate Java, but I do hate most Java programmers.

The computers of yore were seriously underpowered and came with a fairly limited BASIC.

Exception: The Jupiter Ace; it was seriously underpowered, but it came with Forth!

These computers did however have one aspect that proved to be critical for an entire generation of computer scientists: They afforded a complete understanding of the entire computer, from the hardware to the boot process, of the “operating system”, and of the (builtin) programming language. This turned out to be crucial as it gave budding nerds a good understanding of many different aspects of the computer. As a contrast: Yesterday I wondered about the boot process of the Raspberry Pi and before I knew it I was lost in GPUs reading SD cards and waking up the ARM core. That’s quite different from my trusty 6502, which upon boot would read a 16-bit address from address 0xFFFC and proceed running code from there.

Getting back to programming languages: One of the inevitable aspects of learning new programming languages is that you are not yet tuned in to the optimal patterns of the language and therefore reuse your old coding patterns. Sometimes this doesn’t work out too badly; one memorable day I was patching up some Perl code when an associate of mine looked over my shoulder and cried out: “Wait, this is Perl, and I understand what it is doing!” Clearly I hadn’t fully embraced the coding style of Perl yet… 

I have recently been coding a lot in Rust and that has been an interesting ride.

I think I first became aware of Rust in the mid-2010s when a colleague of mine (nicknamed “The German McDreamy” :-) mentioned Rust and its emphasis on keeping track of ownership of data. Fast forward half a decade and I found myself in a team that was developing a continuous deployment (CD) solution that was used for the deployment of code on a variety of platforms, including individual bare metal machines, cloud jobs, and even for the reinstallation of MacOS on a farm of Mac Minis. For reasons that made sense at the time, the system was written in Python. Unfortunately, over time, the Python code base had become a huge mess, leading to ever decreasing developer velocity. Two big (and obvious) problems were raw performance (due to Python’s essentially single-threaded nature caused by the global interpreter lock) and the lack of a strong typing system that is integral to the language, making any code changes or refactoring a huge hassle.

The team had used all the tricks of the trade to deal with these problems, most prominently through unit testing, the extensive use of Python’s system for asynchronous execution, and through the equally extensive use of  Python solutions for type hints and type checks. Unfortunately, developer velocity kept on decreasing as these tools added complexity and were incomplete and imperfect.

There is a well-known legend that when Python inventor Guido van Rossum joined Google, he had a series of conversations with Googlers about what they thought should be added to Python. Mike Burrows, one of the principal authors of Google’s Chubby lock service, apparently answered: “Strong, mandatory, typing.” 🙂

We were clearly in the mood for a programming language change and after looking at the alternatives, we settled on Rust.

At that time Rust was an up and coming language at the company, with a very active internal group of people working on making it a first-class citizen, including integration with the developer environment, wrapping existing (C/C++) APIs, and improving RPC bindings. On the downsides, Rust is complex and has a very steep learning curve. However, all things considered, we thought that, despite the complexity and learning curve, a rewrite in Rust would lead to faster and more reliable code and therefore would result in an increase in developer velocity. We turned out to be right.

Another argument I brought to the table was that Rust was/is considered “cool” and I predicted that this would help us attract talent.

Since then I have been coding Rust on and off, and it has been a wild ride. Of course no language is perfect, but looking at the entire playing field, I think that Rust is currently the best low-level language for high performance systems where reliability is paramount. Beter even than the workhorse in that space: C++.

For those of you who have never looked at Rust: The language’s most innovative and important feature is that it keeps track of where data goes and who potentially holds pointers to it (even across threads). This eliminates the most important source of bugs: Unexpected modification of memory by someone else that you forgot existed. Because Rust always knows who owns the data and who has a pointer to it, it can prevent you from making mistakes where a pointer escapes and later wreaks havoc.

The annoying thing about this level of control by the compiler is that there are loads of situations where you might know that a particular order of events is safe, but the Rust compiler cannot figure it out and therefore blocks you from doing something that is ultimately correct. That is a hassle and a half, but be honest for a second: Have you ever been exposed to bugs that were the result of you thinking that a particular order of events was safe but turned out not to be? Rust prevents these situations. The upshot of this is that it sometimes takes a lot of time to get your code to compile as you are fighting the so-called “borrow checker”, but when it compiles, it runs (bar logic errors)! No null pointer exceptions, no weird race conditions, no inadvertent memory leaks (only advertent memory leaks :-), nothing!

This works especially well in multi-threaded code. Rust’s deep insight into what you are doing makes sure that if you have a writable pointer to some data, you are the only one who does! Data structures that are shared widely between various elements of the code need to be protected by appropriate mechanisms, not just because you think it is a good idea, but because without these, you do not get to read or write that data! Rust even protected me from releasing a mutex in another thread than the one that acquired it because the underlying operating system might not support this!

One of the things Rust threw in my face time and time again is that many of the coding practices that I brought along from C and C++ are inherently unsafe and really only work under very strict conditions that the programmer needs to be aware of. I recently picked up an old open source project of mine (in C), compiled/ran it using address sanitization, and found dozens of bugs that made me wonder why the “original” ever worked at all. Over time, C and C++ added features and tools to improve things (most prominently const and valgrind), but integrating with existing libraries that do not consistently use or support these features make that an uphill battle.

For me the fun of learning a new programming language is that it affords me the opportunity to understand existing things in a new light and the ability to express thoughts using a new idiom. Well thought out new programming languages like Rust and Go offer opportunities to write reliable code faster because they have a feature set that was designed based on both decades of understanding software engineering and the current capabilities of computers. C is a magnificent language and I love it dearly, but it hails back to a past where we needed something that was easier to code than assembly language but still possible to compile using the hardware and software we had back then. For me, Rust represents the next major step forward because it uses the fast CPUs and huge amounts of memory we have today to create a deep insight into the program under compilation to make sure we are not inadvertently breaking things.

Oh, and here is a boss tip: When starting out with any new language: Use CoPilot or a similar AI coding assistant because it will help you get accustomed to the patterns of the language by suggesting snippets of code learnt from analyzing other people’s working code.

Try Rust. You might not be disappointed.