Python to Rust: Breaking Down 3 Big Obstacles

Python pro to Rust rookie — a data scientist’s transition tale

Everybody around me knows that I am a big fan of 🐍 Python. I started using Python about 15 years ago when I was fed up with Mathworks Matlab. While the idea of Matlab seemed nice, after mastering Python I never looked back. I even became a sort of evangelist of Python at my University and “spread the word”.

The ability to code does not make you a software developer.

At my current employer TenneT, a large transmission system operator in the Netherlands and Germany, we are building a document parsing and validation solution with a team of about 10 people. Building such a solution, especially in a team, is much harder than I thought. This also made me more interested in proper paradigms of software engineering. I always thought that my code is not too bad, but after looking at working from my software engineer friends: man there is so much to learn!

As I learned about topics like strong typing, SOLID principles, and general programming architectures, I also glanced at other languages and how they solved the problem. Especially Rust caught my eye as I often saw Python packages that were based on Rust (e.g. Polars).

To get a proper introduction to Rust I followed the official Rustlings course which is a local Git repository with 96 small coding problems. While it was quite doable, Rust is very different than Python. The Rust compiler is a very strict fellow that does not take maybe for an answer. Below are my three major differences between Rust and Python.

Disclaimer: while I am quite proficient with Python, my other languages are a bit Rusty (pun intended). I am still learning Rust and I may have understood parts incorrectly.

1. Ownership, Borrowing, and lifetimes

Ownership and borrowing are probably the most fundamental aspect of the Rust programming language. It is designed to ensure memory safety without the need of a so called garbage collector. This is a unique concept to Rust and I have not yet seen it in other languages.

Lets start with an example where we declare a value 42 to the variable answer_of_life. Rust will now allocate some space in the memory (this is a bit more complex but lets keep it simple for now) and attach “ownership” to this variable. It is important to know that there can only be one owner at a time. Some operations “transfer ownership”, making the previous variable references invalid. This ensures memory safety by preventing issues such as double-freeing memory, data races, and dangling references.

A term that is also used in other languages is scope. This is can be seen as a sort of area in which a part of code “lives”. Every time when the code leaves a scope all variables that have ownership are deallocated. This is something that is fundamentally different in Python. Python uses a garbage collector that deallocates variables when there are no references to it. In the example in source 1 the transfer ownership from variable s1 to s2 and thereafter, variable s1 is not usable anymore.

As a Python user ownership can be confusing as is a real struggle in the beginning.

In the example in source 1 is a bit simplistic. Rust enforces you to think where a variable is made and how it should be transferred. For example, when you use a parameter to a function ownership can be transferred as seen in Source 2:

Only transferring ownership can be cumbersome and maybe for some use-cases even unworkable so Rust came up with a so called borrowing system. Instead of transferring ownership, a variable agrees to borrow the variable while the original variable stays owner. By default a borrow variable is immutable, i.e. read only, but by adding the mut keyword, a borrow can even be mutable. While there can be unlimited immutable borrows, only a single mutable borrow is allowed. In Source 3 I show an example of two immutable borrows and one mutable borrow. All variables will be removed when the function goes out of scope.

Lifetimes is a concept in Rust that is related to borrowing and ownership and helps the compiler to enforce rules on how long references can be valid. You can get into a situation that you create a structure or a function that is build using two borrows. This means that now the result of the function or the structure might depend on the previous inputs. To make this more explicit, we can express relationships by annotating lifetimes. See an example in source 4:

Ownership, borrowing, and lifetimes is not easy to cope but definitely forces you to create better code. At least, when you can get past the compiler (-:

2. Rust does not take None for an answer

Something that is very common in Python is not possible in Rust: having a value that is set to None. It is a deliberate design choice that aligns with Rust’s goals of safety, predictability, and zero-cost abstractions.

The safety aspect is similar to the ownership, borrowing, and lifetimes aspect of rust: prevent the possibility of references pointing to unallocated memory. By not giving the possibility to return None will result in more predictability as it forces the developer to explicitly handle cases where a number might be absent. Due to memory safety and predictable behavior Rust can achieve all its high-level language features without sacrificing performance.

None shall not pass — Gandalf the Grey

Just denying None would make Rust a terrible language, therefore, the creators come with a nice alternative: the Enums Option and Result. With these Enums we can explicitly represent the presence of absence of a value. It also makes the error handling very elegant. Lets consider Source 5 for an example of Option.

Wait a minute! Didn’t you say that there was no None? This was also something that tricked me the first time, but None here is a special Enum struct that does not take a parameter. Also Some is a special struct, but that can take a parameter. Our function divide() returns one of these possible Enum values and we can later check what it is and act accordingly.

Without None and forcing a return value makes Rust very predictable.

The main function uses a match construct to do the result handling which is very convenient. It is somewhat similar to a switch/case construct in other languages except Python (see Figure 2 for Guido’s response). The match checks if it is Enum Some or Enum None and performs the associated action.

The Option Enum is a special structure for functions that can return a value or not. For functions that can return a value or an error, Rust has an even more explicit Enum called Result. The idea is exactly the same, with the main difference that Option has a default “error” value None set, while Result needs an explicit “error” type. This type can be a simple string or a more explicit Struct to identify the error. In Source 6 the divide function is rewritten using Result.

The Rust developers saw that the match construct can sometimes be a bit tedious and therefore, added if let and while let operators. These operators are similar to match but give some nice syntactic sugar with juicy icing. There is even a very cool ? operator (not shown here) which even adds a cherry on top of the juicy icing!

Using Python I learned to use the Optional keyword to type a result for being a value or None. But I have to agree that Rust has solved this part very neatly. I can imagine that the Python community will also move more towards this style, similar to what is going on with strong(er) typing.

3. Where are the classes?

Both Python and Rust can be used in the two programming paradigms: functional programming (FP) and object oriented programming (OOP). However Rust takes a different flavor on how it implements these so called objects. In Python we have a typical class object for which we can associate variables and methods. As with many other languages such as Java, we can now use this method as a base and expand functionality by creating new objects that inherit methods and variables from their parent.

In Rust, there is no class keyword and objects are fundamentally different from Python. Rust uses a trait system for code reuse and polymorphism which can give the same benefits as multiple inheritance, but without the problems associated with multiple inheritance. Multiple inheritance is often used to combine or share various functionality with multiple classes but it can make the code complex and ambiguous. A famous problem is the so called diamond problem shown in Source 8:

While I think we can easily work around this problem, if I would create a new language, I would also try to do this differently. For multiple inheritance, the goal was mostly to share similar functionality with other objects. This is done much more elegantly in Rust using the Trait system. This method is not unique to Rust as similar systems are used in Scala, Kotlin, and Haskell.

Classes in Rust are created from Enums and Structs. On its own, these are only data structures but we can add functionality to these classes. We could do this directly, however, by using traits this functionality can be shared with multiple “classes”. A big benefit for using traits is that we can check beforehand if a certain trait is implemented. See the following example:

In this example, we have a Speaker trait representing characters that can speak. We implement this trait for two types: Jedi and Droid. Each type provides its own implementation of the speak method.

The introduce function takes any type that implements the Speaker trait and calls the speak method. In the main function, we create instances of Jedi (Obi-Wan Kenobi) and Droid (R2-D2) and pass them to the introduce function, showcasing polymorphism.

For me, as a Pythonista 🐍, the Rust trait system was very confusing. It took me a while to appreciate the elegance of the syntax.

Wrap up

Rust is a very cool language but is definitely not an easy one to learn. The Rustlings course showed me a bit the basics, but I am by far not yet proficient enough to pick up big projects. But I really like how Rust is forcing you to write better and safer code.

Python will still be my daily driver. At work, our document pipe-line is built fully in Python and also in the machine learning world I do not see everything change to another language. Python is just too easy to learn and even if you are a terrible developer (not me of course (-;) you can get the job done.

There is however a small momentum towards Rust. Of course, some packages like Polars and Pydantic are built using Rust, but also HuggingFace has released their own first version of a Machine Learning framework built in Rust called Candle. So I think it is not a bad idea to learn a bit of Rust!

My next (or actually current) journey for Rust is participating in the Advent of Code 2023 using Rust. I am also looking into Leptos and plan to create a profile website. Still a lot to learn!

Please let me know if you have any comments! Feel free to connect on LinkedIn.